Webinar Recap: Adelphi Group and Excerpta Medica on Leveraging AI-Enabled Classifiers for Faster, More Efficient Systematic Literature Reviews

by | Jan 25, 2024

Webinar Recap: Adelphi Group and Excerpta Medica on Leveraging AI-Enabled Classifiers for Faster, More Efficient Systematic Literature Reviews

In a recent webinar, DistillerSR customers Rosie Morland, AI and Data Science Specialist at Excerpta Medica and Remon van den Broek, Head of AI Strategy and Data Science at Adelphi Group were joined by Mark Priatel, VP of Software Development at DistillerSR. They shared their journey with AI Classifiers and how it significantly expedited and improved the accuracy of their systematic literature reviews.

Q: Can you share your experience with using classifiers in literature review projects, Remon?
A: About 10 years ago, I did a large systematic review with about 20,000 records. It was a challenge, and when we finally completed the project, the evidence we gathered ended up being out of date. We then started developing algorithms with IBM Watson, but moved to DistillerSR for its flexibility in training classifiers. This allowed us to adapt classifiers based on specific project needs.

Q: What was your experience, Rosie?
A: Our first project using DistillerSR and trained classifiers was delivered four months early. We screened about four times as many references. The process of training classifiers is integrated into the project, which is a key advantage. We’ve also developed over 100 classifiers, tracking them in Excel to understand their evolution and suitability for different questions.

Q: Remon, have you seen time savings with the implementation of classifiers?
A: Yes, there’s significant time savings. We created a live database pulling data from PubMed and Embase, which gets automatically classified. This allows for multiple projects to be integrated into the same database, saving a lot of setup time.

Q: Rosie, what has been your experience?
A: Absolutely, we’ve seen time savings and efficiency. Classifiers allow us to focus on specific questions and streamline the screening process significantly.

Q: How do you decide on and create classifiers for your projects, Rosie?
A: The classifiers are based on need and project requirements. We started with disease-agnostic classifiers and then moved to more bespoke ones for specific needs. We prioritize questions that are straightforward for both people and classifiers.

Q: What about you, Remon?
A: It depends on the project’s output and the questions we need to answer. Some classifiers need thousands of abstracts for training, while others need much less. It’s often a trial-and-error process to find what works best.

Q: What are your thoughts on Large Language Models (LLMs) compared to traditional classifiers, Remon?
A: There’s plenty of interest in LLMs like ChatGPT, but the reliability issues cannot be underestimated. LLMs are a black box and can’t provide the transparent explanations needed in systematic reviews. AI-enabled Classifiers, however, offer transparency and more reliable results.

Q: Your thoughts on the subject, Rosie?
A: LLMs are exciting but volatile for systematic reviews. Classifiers are much more reliable for specific data extraction tasks. LLMs can often produce overwhelming amounts of data that still need sifting.

Q: Any best practices or recommendations for those starting with classifiers, Remon?
A: Make sure the question for the classifier is simple and clear. Multiple reviewers should be on the same page regarding the question. It’s important to ensure that the classifier’s training is transparent and the output is trustworthy.

Q: Rosie, any recommendations to get started with classifiers?
A: Pay attention to the data used for training classifiers. Broader training sets can be beneficial, especially for disease-agnostic classifiers. It’s crucial to have clarity on what you want from the output.

Watch the recording of the webinar for all the insights, including an introduction to AI Classifiers by DistillerSR Machine Learning Senior Engineer, Rushdi Shams.

DistillerSR
  • Vivian MacAdden

    Vivian MacAdden is DistillerSR's Senior Manager, Industry Marketing - Medical Devices. Throughout her career, she has accumulated 20 years of strategic marketing experience in various industries in Canada and international markets such as Brazil, China, Singapore, Jordan and Japan. A problem solver at heart and forever an optimist (and karaoke lover), she is passionate about telling great stories that make a positive impact on the world.

    View all posts

Stay in Touch with Our Quarterly Newsletter

Recent Posts