Streamlining Data Extraction to Accelerate Research
Pharmaceutical Industry
SUMMARY
A number of teams within a global-scale pharmaceutical company were facing the same challenge: how to convert unstructured data into structured tables.
They struggled to determine their best choice among various off-the-shelf tools or developing proprietary technology to address this issue.
For a prototype knowledge extraction tool our client had already built, Hop developed an evaluation strategy and constituent datasets, then assessed performance.
Our client now has an initial version of a tool that addresses their challenge.
Routine information extraction processes are executed in a fraction of the time it takes to do it manually, making a literature review 3-5x faster.
Our team improved the performance of this tool to the point where users could have high confidence in the results – above 95% accuracy for some tasks of interest.
THE COMPANY
Our client is one of the oldest and largest pharmaceutical companies in the world, focused on oncology, neuroscience, gastroenterology, immunology, and plasma products. A leader in the industry, they look to leverage the latest in technology throughout their organization, to support their aims to improve the patient experience and advance a new frontier of treatment.
THE CHALLENGE
Across large organizations, it’s often the case that many teams are facing similar problems at the same time, and often independently developing their own solutions in parallel. A number of teams in this particular organization were facing the same challenge: how to convert unstructured data into structured tables. Significant operational resources were being deployed toward this problem, without significantly helpful outcomes for the organization overall. Clinical trials reports, dataset metadata, audio of patient interviews, and other data sources were processed by different teams into different structured formats.
There are many vendors who offer tools that address this challenge, but our client struggled to balance the tradeoffs. Buying a pre-built external tool may work out of the box, but could expose the organization to the risks of startup failure and divergent product roadmaps. Investing in their own development of purpose-built, proprietary technology would avoid such risks, but would likely be a longer and more expensive approach. Which was the right choice?
THE APPROACH
Hop worked with our client’s AI research group to develop an alpha version of the product they needed, to explicitly demonstrate the possibilities and limitations of this technology for their context. Working with a prototype knowledge extraction tool they had previously built, we developed an evaluation strategy and constituent datasets, then assessed performance of the prototype tool.
“Hop’s unique strength is that you bring AI research expertise, which other vendors don’t typically offer. And I appreciate that you can jump into a project and get on board quickly.”
Working with minimal parameters for this project, it was especially important that we collaborated closely with our client counterparts to ensure that work aligned with their goals. When the project lead on their end needed to offboard mid-project, this approach allowed the work to keep moving. Our collaboration also carried over to other contractors on the project – though our domain was the back end of this tool, we knew integration of our work with the front end experience was crucial to ultimately addressing the client’s needs, which was our primary concern.
THE RESULTS
Our client now has an initial version of a tool that addresses the challenge of converting unstructured data into structured tables. Users are able to define arbitrary data models, and extract that data from arbitrary documents. Routine information extraction processes are executed in a fraction of the time it takes to do it manually, making a literature review 3-5x faster using this tool. Extracted information is explicitly grounded in document excerpts, which prevents hallucinations entirely.
Hop was able to improve the performance of this tool to the point where users could have high confidence in the generated results – above 95% accuracy for some tasks of interest, unlocking an opportunity for our client to use this tool across workflows to save time and money.
Where performance fell short of that bar, we characterized the types of tasks in this category, for future iterations and improvement. We purposely built this tool in such a way that, as the underlying LLM technology advances, our client is positioned to continue to assess its suitability for their own needs, to know when to apply it and when to defer.
“I would recommend Hop for organizations that don’t have their own ML research capability. I think this is a great way to test things out, see what they could bring.”
Aligning and optimizing workflows isn’t the work that makes headlines, but it’s crucial to an organization’s progress. Hop is glad to contribute operational improvements that will accelerate our client’s momentum in creating better health for people and a brighter future for the world.
Struggling to make the most of your org’s data? Contact us to learn how Hop can help.