AI-powered pre-screening requires more than a single prompt

Introduction

Pre-screening for clinical trials can be approached in several ways, including rule-based algorithms, embedding matching, or prompting an LLM. LLM-approaches have taken a leading position in solving this problem as we wrote in our previous blog.

The most obvious approach is to provide an LLM with a complete medical record and the study protocol, then ask whether the patient is eligible. This can produce convincing results in a proof of concept. However, for a production system, this approach has significant limitations:

• inconsistent results due to the probabilistic nature of LLMs;

• limited control over how the model reaches a decision;

• limited explainability;

• difficult scaling to thousands of patients.

In this article, we describe how Monsana addresses these challenges through a structured, production-grade agentic workflow.

From a Monolithic LLM Approach to an Agentic Workflow

A monolithic LLM approach attempts to solve one complex problem in a single step. One large prompt contains all available information and asks the model to reach a final decision. The quality of this approach has improved significantly in recent years thanks to reasoning tokens.

An agentic workflow divides the same problem into a series of predefined steps. Each step resolves a specific sub-problem, after which its result is passed to the next step. Some steps use LLM calls through an API; others may, for example, be performed using rule-based logic.

The goal is to make the success of the application less dependent on one good prompt, and more dependent on control, evaluation, and system discipline.

One or more prompts can often yield good results during a manual assessment. The user can review the outcome, steer it, and try again. In a production system, however, that approach is not sufficient. Simply splitting an assessment into individual LLM calls per criterion, as described in the paper Zero-Shot Clinical Trial Patient Matching with LLMs , does not in itself create an agentic workflow.

A consistent AI workflow should also account for additional functionality:

• Evaluation: intermediate results are checked for accuracy, usefulness, and risk, keeping the system on its predetermined path.

• Guardrails: permitted and prohibited actions are defined. Both input and output can be validated at any point.

• Error handling: the workflow accounts for time-outs, limited retries, fallbacks, human escalation, and loop breakers.

• Latency and cost: at scale, unnecessary context, complex decision paths, and excessive token consumption must be avoided.

  
  

Monsana has developed an approach in which LLMs are used for specific sub-problems within a controlled workflow. This creates a balance between the flexibility of LLMs and the predictability of conventional software.

Monsana’s Implementation

Preparing Medical Records

The workflow starts by structuring and cleaning medical records. Not every document or text fragment in a patient record is relevant when assessing trial eligibility. The system first determines which source information may be relevant. This can include relevant medical reports, diagnoses and treatments, laboratory results, imaging, medication, dates and timelines, while leaving out sections that are irrelevant or outdated.

The main objective is to reduce noise. Keeping irrelevant information out of the analysis reduces the likelihood that it will influence the decision. It also limits the volume of text, reducing costs and processing time.

This step remains sensitive to errors. Relevant information may be filtered out incorrectly, or information from different documents may be placed incorrectly on the timeline. This preparation therefore forms an important foundation for the reliability of all subsequent steps.

From Protocol to LLM-Optimised, Readable Criteria

The inclusion and exclusion criteria are then extracted from the study protocol. However, study protocols are not always written in a form that an LLM can interpret efficiently. A single criterion may contain multiple conditions, specify exceptions, or use a writing style that differs significantly from that of a clinical record. A criterion may appear clear to a physician while remaining ambiguous for an automated system.

Recent research, [add reference], also highlights the importance of preparatory rewriting steps before the actual screening process. Monsana has therefore developed a module that converts complex criteria into simpler, more manageable sub-criteria. This may involve logical AND/OR structures, in which a composite criterion is divided into individual, verifiable conditions.

For example, the sentence “The patient has diagnosis X, received treatment Y, and has no contraindication Z” can be divided into the following questions: Is diagnosis X present? Has the patient received treatment Y? Is contraindication Z absent? Each condition can then be examined separately.

This decomposition makes the assessment more transparent and reduces the likelihood that the model overlooks part of a composite criterion.

Classifying Criteria

The criteria and any sub-criteria are then classified according to types that were defined by Monsana during development.

This classification determines which assessment strategy will be used later. Not every criterion requires the same type of analysis. Some criteria can be evaluated directly using rules, while others require context, temporal relationships, or medical interpretation.

The complexity of a criterion determines the most appropriate solution: for example, a rule-based evaluation, a smaller and faster LLM, or a larger model for more complex interpretation.

Rule-based methods have the advantage of being highly consistent. Smaller models often offer lower latency and require a smaller token budget. Larger models, on the other hand, can handle complex language and medical context more effectively.

By classifying criteria in advance, Monsana gains greater control over the workflow and can select a proportionate approach for each sub-problem.

Linking Criteria to Relevant Record Information

Once the criteria have been prepared and classified, they are linked to relevant passages in the medical record.

The workflow no longer searches the entire record for everything that might be important. Instead, it specifically retrieves the information needed to assess each individual criterion.

This linkage is important for both efficiency and explainability. An assessment is stronger when it is clear which concrete record information it is based on. Moreover, the next step only needs to process the relevant context, rather than analysing the full record repeatedly.

Assessment at Criterion Level

Each criterion is then assigned to a separate mini-evaluator, or sub-agent, to use a popular term, based on the previous classification.

For each criterion, the result is a structured output containing at least:

• whether the patient does or does not meet the criterion, or whether the outcome is uncertain because the available information is insufficient or unclear;

• the reasoning that led to the decision;

• the source information on which the decision is based.

This increases auditability and makes human review easier. A reviewer can more quickly determine whether the correct information was used, whether the interpretation is correct, and which records require additional attention.

We have already described the benefits and inner workings of these sub-agents in our Technical Deep Dive blog.

Reflection and Validation

The initial assessment is followed by a reflection and validation step.

Its purpose is to reduce hallucinations, inconsistencies, and other detected issues. The evaluator’s output is checked against predefined validation rules. When a problem is identified, a revision agent can reassess and correct the evaluation.

Reflection adds processing and can therefore increase latency and cost. In return, it can improve the reliability and verifiability of the final results.

Aggregation into a Final Eligibility Score

Once all inclusion and exclusion criteria have been assessed individually, the results are combined. Monsana uses its own aggregation formula, developed and refined during validation projects. The formula takes into account, among other factors:

• the type of criterion;

• the extent to which a criterion determines trial eligibility;

• the assessment outcome.

  
  

The individual criteria therefore do not necessarily receive the same weight. For example, a decisive exclusion condition may have a greater impact on the final score than a more supportive criterion.

The weighted results are combined into a final score between 0 and 1. This score is then mapped to four eligibility categories: unlikely, uncertain, likely, and highly likely. These categories support the decision on which records should receive further review.

Configurable Review Thresholds

An important advantage of a scoring system is that the threshold for human review can be adjusted.

When a hospital wants to minimise the risk of missing eligible patients, it may choose to review not only patients in the highly likely category, but also those in the likely and uncertain categories.

This increases the likelihood that potential candidates are included. At the same time, it also increases the number of records reviewed manually that ultimately prove to be ineligible.

A stricter selection reduces the review burden, but increases the risk that eligible patients are not identified. Each site must choose the right setting based on:

• available review capacity;

• the desired sensitivity;

• the acceptable review burden;

• the priority the hospital assigns to completeness versus efficiency.

AI Technology Is Only the Beginning

While technology provides an important foundation, it is only one element of a successful clinical workflow. To make a real impact, other components are also needed:

• secure access to relevant patient data, in line with regulations such as the GDPR;

• user-friendly tools for human review;

• robust infrastructure with encryption, authentication, and scalable performance;

• clear and actionable reporting;

• processes that help teams deploy the workflow strategically and effectively.

  
  

In addition, a network of strong hospital and pharmaceutical partners is essential. They understand the practical pain points of day-to-day work and can provide feedback on what is truly needed to make AI practically useful.

Today, the greatest challenge no longer lies exclusively in the technology. While further innovation remains welcome, the current version is ready to create real impact. The next step is action: organisations must enable technology, processes, and people to evolve together. Only by integrating solutions into daily practice and actively involving staff can innovation become lasting value.

Conclusion

Monsana uses LLM technology within a structured, production-grade agentic workflow. The technology is not deployed as one general evaluator of complete medical records, but as part of a sequence of controlled steps.

By preparing records, rewriting and classifying criteria, linking relevant source information, validating assessments, and aggregating results using weighted logic, the process can become more efficient and scalable.

At the same time, the outcome remains verifiable: every criterion assessment includes a decision, reasoning, and the source information used. This makes human review more targeted and supports a better balance between sensitivity, review burden, and operational efficiency.

This approach combines state-of-the-art LLMs with the consistency of a predefined algorithm. The technology is ready. The next step is to integrate it carefully into clinical practice.