This is an early thought experiment, but it was sufficiently compelling that I couldn't help but share.
If you're an executive with deep domain expertise but no ML background, and you're trying to understand if ML will be useful in your space, it can be useful to think of an ML system as a type of factory. In particular, it's a "decision factory".
A physical factory (in the manufacturing sense), is an expensive piece of infrastructure that can be worthwhile if you need to affordably and scalably spit out a lot of identical (or at least fairly similar) widgets. Your automobile factory can be made to spit out all sorts of vehicles, but it can't, for example, spit out t-shirts -- that's a different factory.
Similarly, your ML system, is an expensive piece of infrastructure that can be worthwhile if you need to affordably and scalably spit out a lot of identical (or at least fairly similar) decisions. Your facial tracking system can be extended to classify gender, age, emotion, etc, but it can't, for example, navigate a self-driving car -- that's a different ML system.
The Gmail product needs a lot (like millions) of affordable "is this email spam or not?" decisions. Their spam-classification ML system is a factory for those types of decisions. The Gmail product team, on the other hand, does not need millions of "what feature should we build next?" decisions, so (I presume) their sprint prioritization process is still manual. (Needless to say, I know nothing factual about the Gmail product team's internal deliberations.)
The Netflix product needs a lot of affordable "what content will user X like?" decisions. Their recommendation ML system spits out a lot of those decisions affordably, and there goes my Sunday afternoon.
The FAANG hiring teams are hiring a lot of software developers, but even at their crazy scale, they don't need millions of "should we hire X?" decisions. The final call on hiring isn't automated by an ML system. Conversely, for each position, they get many thousands of resumes, and they DO need a lot of "should we look at this resume more closely?" decisions. This is why so many applicant tracking systems are trying (with questionable, often problematic results) to find ways to usefully incorporate ML into the applicant tracking process.
For your organization, what decision do you need a lot of?
Once you've found that decision -- your DecisionWidget, as it were -- you can apply the same logic you would for building a regular physical factory:
- Do you need a lot of DecisionWidget? If not, probably better to do it by hand the few times a year you need it.
- Have you made DecisionWidgets successfully before (even if manually), and just want to scale production? If not, probably need to do some learning to figure out how to do it correctly first. This will lead to (literally) 'training data'
- Is your business constrained in some meaningful way by lack of DecisionWidget? The level of the constraint would likely dictate how much of an investment is appropriate to unlock growth.
- Is there a similar DecisionWidget already widely available in the market that you can just buy? For common decisions, there may be existing DecisionWidget factories that you can buy product from directly (i.e. hosted ML services at AWS, etc).
- Is it strategic to own your own DecisionWidget factory? Existing DecisionWidget factories may need to be replicated internally for strategic reasons, even if you start by using the same technology and processes. You may need to control growth, scale, direction, or the need may be so critical to your business that you need to ensure capacity.
- Do you have the resources to make an investment for a DecisionWidget factory? If you're not sure you're able to invest the time and money to build a factory, work through issues, and get it online, you may need to lease or contract decision manufacturing services with specialized manufacturers with the relevant experience.
This is just an early thought experiment, and I'm not yet fully confident of this metaphor. I've tried it out in some early conversations, and it's been surprisingly helpful in shedding light onto the decision-making process, so I thought I'd share.
Are there other metaphors that are a better fit? Where does this one fall down? Let me know: @AnkurHop
If you're not sure if you've identified the right decision, or if it makes sense to build a decision factory for your organization, I'm happy to share some tricks we've learned along the way. Feel free to reach out.
Thanks to Chris W, Kelly D, Scott Y, Michael B, Sarah A, and many others for feedback on earlier drafts of this article.
— Ankur Kalra, Founder/CEO @ Hop