How to build ML-first products

Problems companies face in adopting ML and how to solve them

In this article, we explore top 5 problems companies face in adopting machine learning (ML) and how to solve them:

#1 Sprinkling ML late in the product does not work well.

#2 Expecting too much too soon from ML.

#3 ML requires changes in habits.

#4 ML team should jointly own user-problems and not act as consultants to product teams.

#5 Building in-house ML infra is expensive and risky.

This is relevant to product managers and eng managers who are considering applying ML to a user problem core to their company. I have also shared a blueprint in this presentation: How to build ML-first products.

Problems

#1 Sprinkling ML late in the product does not work well.

“The problem is that ML takes time and iteration to do well and some product teams continue to think of ML as the last 20% in the 80/20 or the Run in Crawl/Walk/Run and this usually means they are not investing in ML till very late in the product development cycle.”

- CTO, XXB company

a) By applying ML late, one ends up building lots of affordances i.e. piecemeal solutions that do not necessarily solve the core user-problem.

b) The UX by this time is inherently not setup to learn from feedback. There isn’t as much focus on “feedback architecture” (i.e. how to enable users to give specific and timely feedback) as on “information architecture” (i.e. how to give the right information to the user). Both are equally needed in ML-first products.

#2 Expecting too much too soon from ML.

a) ML takes time due to its high dependency on data, computation and iterative nature of learning. At times gains from this “disruptive technology” are “unknowable” in advance (Innovator’s Dilemma reference expanded later in the article.)

b) Typically product stakeholders can “bootstrap” rules based solutions to enhance their intuition and get a better understanding of what business objectives / user-inputs would be critical. Product should provide this business intuition to incorporate into the ML solution for a better user experience and higher velocity of development.

The learning curve with ML will initially appear to be slower. ML engineers need to invest heavily into incorporating product intuition perhaps at the cost of delaying making progress on the ML model.

#3 ML requires changes in habits.

a) Work is less about creatively coming up with if-then-else conditions that might work, and more about providing clear objectives, launch criteria and a feedback architecture to learn from.

#4 ML team should own user-problems and not act as consultants to product teams.

a) In this article we are advocating using ML in products that need ML to deliver a 10X performance. Setting up ML as consultants/platform teams hurts the product and makes ML engineers feel less empowered.

Successful outcomes come from examples where teams own user-problems and work together across product and ML expertise to solve them as one team.

b) Successful examples in the industry that have done this:

1. Youtube Home team (ML/backend SWEs + UI SWEs in same team),

2. Other examples: TikTok, FB News feed, Twitter Notifications, High Frequency Trading

Andrew Carr @andrew_n_carr

Large companies with strong ML groups have been somewhat quietly moving those groups under product orgs. It's game time for applied ML making its way even more into products. Exciting time to be an applied scientist

#5 Building in-house ML infra is expensive and risky.

a) Today, there is state of the art ML infrastructure as a service available at low costs (e.g. serving, searching, embedding search, feature store, model training pipelines, recommender systems)

b) Some companies who have built infra in house are facing the same transformation that teams inside Google, MSFT, Twitter went through over the last 5 years. Most had built their own infra but chose to sunset them and switch over to centralized solutions.

c) Building infra in house is not only costly but incurs the organizational headwind that makes #4 hard. Internal ML infra will force an org design where ML teams are centralized. They will have to set up ML-enabled services a certain way.

d) For most companies, I recommend using external ML infra and building only what is uniquely value-adding to your company.

These problems are not new. They happen to most disruptive technologies. In Innovator’s Dilemma, Clayton Christensen summarizes them into five principles.

Disclaimer: These are the personal opinions of the author. Any assumptions, opinions stated here are theirs and not representative of their current or any prior employer(s).