Machine Learning Product Manager

In this post, I will talk more about some patterns in the life of a Product Manager who leads a Machine Learning team to build customer value.

Structure:

• What is ML PMing?
• How does it work? How is it different regular PMing?
• How do I go from good to great at ML PMing?

One thing I can’t stress nearly enough based on my experience is that the best value a PM can add to an ML/AI product is to identify how best to solve the problem non intelligently first and then define exactly why it needs AI.

Just because you can AI, don’t AI. It can give back so much time to your team!

What is ML PMing?

At a very high level, it’s product management on products or services that deploy machine intelligence to solve human and business problems in a way that the system learns over time. Examples of some ML “products” around you:

• The way Netflix predicts what you may like to suggest you options — The Netflix recommendation engine, the most beaten to death example of all :)
• The way an e-commerce portal may customise the homepage to maximise your engagement — That’s a recommendation engine that tracks your activity to personalise results for you.
• The way a bank may computationally adjudicate fraudulent transactions — That’s a fraud classifier model
• When Google photos can auto-tag your ex in your photos, so you can remove him — That’s vision ML. The phone has learnt to “see” like you.

How is it different regular PMing?

In my experience, these are the key differences I’ve identified:

Granular goal setting:
I find it crucial to cut sharply to the business problem and setup a granular validation critieria for science team to track model performance against. I’ve often seen ML crews celebrating increased higher accuracy without an handle on business value.

Co-create, Start testing early:
As an AI PM, getting your hands dirty early and often helps, especially in the vision space. Keep testing WIP models using a demo app on an as-real-as-possible set to predict how well the model will solve the original problem in production. It can often happen that the goal seek itself needs an edit and that’s gap is better caught early.

Designing for recovery:
This is specific to user-facing AI products. In a lot products, AI provides an optional enhancement, such as most recommendation engines. However, in the computer vision space, which is where I work — AI is often the central value and a core experience rests on it. For eg. An AI processed video stream. Now in such products, it’s really important to design for graceful recovery for when it fails! Which it will!

This, by, no means is exhaustive since I haven’t had a chance to work on all classes of ML products.

How do I go from good to great at ML PMing?

I often get asked “How do I become a great ML PM”? My intuitive answer to that has always been:

• Become a great PM — I’m always learning and getting better on the job.
• Pick up ML projects organically at work — I’ve observed some patterns with ML products that I’ve worked on.

In the next post, we can will take the example of a few projects and deep drive into exact metrics and day to day decisioning ops of building an ML product.

Leaving you with a handy visual summary below from a talk I attended last month organised by Institute of Product Leadership with Vamsi Krishna from Amazon AWS Recognition Team.