From Cardiology to Oncology: expanding Monsana’s AI assistant with promising results at Ziekenhuis aan de Stroom (ZAS)

Building on the success of our pilot with AZ Delta’s cardiology department, we are excited to share the results of our latest collaboration with ZAS Hospital, this time in oncology. This milestone highlights Monsana’s ability to deliver strong performance not only across different hospital environments but also across diverse therapeutic areas.

In this pilot, Monsana’s AI assistant was used by the oncology team to pre-screen medical records to identify potential trial candidates within the center. Compared to estimations of traditional manual screening, our solution achieved nearly a 2.5-fold increase in patient identification. Additionally, for every four recommendations made to physicians, an average of three patients were potential trial candidates.

Why oncology matching is so challenging

Timing is crucial in oncology trial recruitment since patients often need to start treatment quickly, leaving little room for manual screening. Oncology trials often have complex eligibility criteria, targeting smaller patient groups than more common conditions like diabetes. With numerous trials and complex criteria, doctors struggle to track them all, meaning patient eligibility often goes unnoticed at the critical moment. Monsana changes that by continuously screening patient data and alerting clinical teams in near-real time, helping more patients access the right trials at the right moment.

To validate our AI technology in Oncology, we screened 1,500 patient-trial pairs, consisting of 250 breast cancer patients across 6 clinical trials, at ZAS. To ensure a comprehensive evaluation, we also manually reviewed each patients' eligibility status.

Input data

The dataset consisted of Multidisciplinary Oncology Consultation (MOC) reports for 250 breast cancer patients, generated in a real-world workflow by physicians, in an unstructured format. The patients were treated at ZAS and randomly selected. 1 to 2 medical records were included for each patient. On average, a single report contained around 941 words. The collected records were pseudonymized in-house  at ZAS before interpretation by Monsana.

The trial protocol data consists of the inclusion and exclusion criteria for 6 ongoing clinical trials. Each study contained on average 42 in-and exclusion criteria.

AI assistant performance

The Monsana algorithm was assessed based on its sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV).

	All trials
Sensitivity	85% (75.00% -100.00%)
Specificity	98% (91.67% -100.00%)
Positive Predictive Value	73%
Negative Predictive Value	99%

Table 1 

This table presents the key performance metrics for the Monsana AI screening across six clinical trials combined. The metrics include: (1) Sensitivity: A measure of the AI's ability to correctly identify patients who are eligible for the trial (TP/(TP+FN)), (2) Specificity: A measure of the AI's ability to accurately exclude patients who are ineligible for the trial (TN/(TN+FP)), (3) Positive Predictive Value (PPV): A measure of the proportion of patients flagged as eligible by the AI who are truly eligible. (TP/(TP+FP)), (4) Negative Predictive Value (NPV): A measure of the proportion of patients flagged as ineligible by the AI who are truly ineligible (TN/(TN+FN)).

In a practical hospital workflow, this means that +/-85% of potential trial candidates are identified by Monsana AI. Comparing this to estimations, where only 33% of eligible patients are identified, this means almost a 2,5x increase in patient identification. Furthermore, for every 4 recommendations made to a doctor, on average 3 patients could be eligible for a study. This ensures that the additional workload for both doctors and clinical trial coordinators remains minimal.

When calculating the actual impact in the clinical workflow in absolute numbers, we saw that during the screening period, i.e. the time when patients were actively being screened for potential participation in the clinical trial, the Monsana.AI algorithm flagged 45 out of 250 patients as potentially eligible. Manual validation revealed that one of these was incorrectly flagged, as the patient had metastatic disease and did not meet the criteria. Additionally, the algorithm identified 3 patients who were not yet eligible but were likely to become eligible soon. For example, in cases where the protocol required chemotherapy to be completed, the model preemptively flagged patients who were still in treatment but approaching completion. After accounting for these cases, the absolute number of eligible patients correctly identified by the algorithm was 41.

Recognizing the need for flexibility in patient eligibility assessment

During the study, it became clear that a binary approach to assessing patient eligibility was not always practical. In many cases, certain information, such as a patient’s Mammaprint status, which is important for risk stratification, was not yet available at the time of screening. This highlighted the need for a more nuanced system. Some users might prefer to still include these patients as a reminder to complete the necessary testing, while others might choose to exclude them to limit the number of cases requiring review and avoiding increased workload.

To accommodate this need for flexibility, the Monsana tool is designed to present screening results on a scale rather than in a simple binary format. Patients are categorized as highly likely, likely, uncertain, or unlikely to be eligible. This approach enables users to tailor their review process, for example, choosing to include or exclude patients in the uncertain category depending on the time and resources available.

To assess the impact of Monsana's flexible screening approach, we evaluated the performance using the two strategies:

1. Strict evaluation, where only patients labeled as highly likely or likely were considered positive.

2. Broad evaluation, which also included patients marked as uncertain.

	Strict Evaluation	Broad Evaluation
Sensitivity	69%	85%
Specificity	97%	92%
PPV	85%	73%
NPV	93%	96%

These findings highlight how users can tailor the AI tool’s behaviour depending on their goals, whether they prefer a more focused list of patients with higher precision, or a broader set to ensure fewer eligible patients are missed. This flexibility ensures Monsana can support different workflows and resource settings across trial sites.

Conclusion

The Monsana algorithm demonstrated robust performance in identifying eligible oncology patients for clinical trials, with a sensitivity of +/-85% and a positive predictive value of +/-74%. Comparing this to estimations for manual recruitment,  Monsana showed almost a 2,5x increase in patient identification. Furthermore, for every 4 recommendations made to a doctor, on average 3 patients could be eligible for a study. 

Importantly, the study highlighted the need for flexibility in eligibility assessment. Monsana’s graded output, from highly likely to unlikely, allows teams to adapt their review process based on resources and priorities, supporting both strict and broad screening strategies while reducing the risk of missing eligible patients.

Interested in staying updated on our progress? Reach out to the Monsana team or follow us on LinkedIn to learn more.