Frontier AI labs are making massive leaps. There’s an explosion of research, accelerating in every industry. Then you look around at your own AI org, and progress feels... underwhelming. It’s easy to lay the blame on bad data, small budgets, or the wrong headcount, but the real problem is your org’s AI maturity — whether or not you understand the level you’ve reached, and what you need to work on next. We've built a tool to give you a clear picture of exactly these things — the AI Org Maturity Assessment.

ML infrastructure is painfully finicky, and using it entails managing a dozen irrelevant technical details just to do basic research. There are ideas that researchers don’t let themselves have, because engaging with them would mean weeks of infrastructure-wrangling before anyone gets to learn a thing. Sooner or later, your infrastructure starts setting your research agenda. We measure this as Research Agility; and it might be the most important metric in your organization.

An uncomfortable claim: In pharma, biotech, and any serious applied AI, projects almost never fail because of the model. They fail because of the data. Everyone agrees that data matters, but not everyone is interested in funding the work needed to keep it useful. Data work has to run as a standing program, owned, scheduled, and budgeted, not as a one-time cleanup at the start of a project.

For any model tackling any problem, the hardest part of evaluation is keeping the metric aligned with reality. An evaluation is just a measurement tool, and tools can be miscalibrated, biased, or simply pointed at the wrong thing. The urgent question is less: “Are my evals lying to me?” (They are, or they soon will be.) Instead, it’s: “When my evals do lie, how will I know — and what can I do about it?”

While large language models (LLMs) have made many routine tasks easier, they have made hiring much harder. How can we discern the authentic candidates from the AI-generated ones?

Just like success in business often comes down to fundamentals, there are a handful of AI techniques that power most modern business applications.

As an executive, you're likely facing the challenge of creating an AI strategy that drives real value for your organization. We've identified two levels of questions that can serve as the pillars of a robust AI strategy.

Before building your AI strategy, it's helpful to consider some foundational truths. These are some things we believe about AI strategies that are not necessarily widely understood.

I'm all for reasonable disagreements, but I find a lot of the current conversation around generative AI relatively unproductive. Here are some of the phrases that people use almost axiomatically that I don't think are actually true.

Agile methodology is widely used and well understood in software engineering. However, there are some unique considerations when it comes to employing Agile within ML engineering specifically.

Technology offers many benefits in our day-to-day lives, including social opportunities not possible without it. However, awareness of its limitations, as well as our fundamental need for real-life connection, is key to staying healthy.

Prompt engineering, fine tuning, and LLM chaining are most commonly considered in guiding LLM behavior. In this post, we cover three more tools, from de rigueur for complex LLM applications to speculative techniques that may not be production-ready for some time.

When creating LLM applications, a lot of emphasis is on the foundation model underpinning the app. Most likely you’ll want to modify the behavior of that model; here are some techniques for achieving that behavior modification, from well-trod to exploratory.

Picture this: two users need advice on a health issue. One employs an AI text interface, resulting in a plan that leaves them feeling informed and empowered. A voice interface leads the other through a back-and-forth conversation with the AI; they feel cared for, supported. Same need, two very different experiences. All because of the interface.

The general nature of LLMs makes them inherently powerful but notoriously difficult to control. Operators must defend against unintended and potentially risky interactions. Our team has investigated many of the relatively nascent solutions out there for this issue; we share what we’ve learned in this post.

We love chatbots – ChatGPT and others in its class are amazing tools – but, as an AI consultancy with a long history of projects in the space before the current mania, we’re sensitive to the conflation of LLMs and chatbots. Many of the most exciting potential uses for LLMs have little to do with the chatbot interface, and we think those should get more attention.

Treating the process of your work as important as the result will improve the quality of your results. A lot of focus gets put on building the right thing for customers, and rightfully so, but it’s important to remember that we have to first build our workbench. Whether we do that haphazardly or intentionally can have an enormous impact on the quality of our results.

The process of making an LLM app reliable is mostly trial and error, involving spot-checking by the developer, reviews by product owners, and auto-evaluation. At Hop, we feel there’s a rich set of design decisions when it comes to auto-evaluation that aren’t regularly discussed; we share those in this post.

Think back to your last telehealth visit with a doctor. Perhaps your kid had a persistently high fever, or you had worrying chest pain. Are you sure you were interacting with a human? What makes you sure?

Machine learning researchers often don’t write tests for their code. They’re not software engineers, and their code needs only to train a model or prove out an experiment. However, at Hop, we’ve found that adding certain kinds of tests can actually accelerate research and increase confidence in results through improving code quality and encouraging reuse.