LLMs aren’t yet widely used as an architectural component in production — the core issue is reliability. Not knowing how to engage with the reliability challenge limits the success of most teams building LLM-powered applications. In our projects at Hop, we’ve developed a relatively uncommon perspective on how to effectively engage with this reliability challenge.

It’s now possible to train complicated models while understanding very little about how they work. There’s a widespread attitude that it’s enough to know how to code up architectures in PyTorch and correct obscure bugs, and that the math is someone else’s problem. We at Hop put ML models into production, and we’re here to tell you that the math is not someone else’s problem.

Research (and especially applied research of the type that Hop does) is a type of multi-armed bandit problem — one that tries to balance new approaches (exploration) with successful approaches (exploitation). The code quality/technical debt conversation becomes a bit easier to think about if you articulate where on the exploration/exploitation spectrum you currently are.

If you're an executive with deep domain expertise but no ML background, and you're trying to understand if ML will be useful in your space, it can be useful to think of an ML system as a type of factory. In particular, it's a "decision factory".

A question we often get from executives exploring machine learning and AI for their organizations is: "What is the minimum viable ML/AI team?" There are likely many right answers, and some industries have unique constraints. However, in our experience, a minimal-but-effective ML/AI team requires a few specific roles to be filled.