Our partners from the Computer Science Department at the University of Cambridge, led by Prof. Pietro Liò, have taken important steps toward incorporating interpretability and explainability into their AI models developed for the CHARM project. These efforts aim to ensure the AI predictions are not only accurate but also understandable and trustworthy for end-users, particularly pathologists. We talked with Dr. Tiago Azevedo, the Research Associate leading this interpretability initiative.
Making AI Decisions Visible
The Cambridge group is currently implementing prototype-based neural networks that go beyond traditional “black box” AI systems. These innovative models, adapted from those proposed by Chen et al., don’t just provide predictions—they visually highlight the specific regions of medical images that influenced their diagnostic decisions. This capability addresses a critical need in medical AI, where understanding the reasoning behind predictions is as important as accuracy itself.
“We’re essentially giving pathologists a window into the AI’s decision-making process,” explains Tiago Azevedo. “When the model identifies potential abnormalities, it shows exactly which tissue patterns contributed to that conclusion.”
Balancing Transparency with Performance
The team’s initial experiments with fully interpretable models have yielded encouraging results. The AI successfully identified key tissue patterns that align closely with expert pathologist annotations, demonstrating its ability to focus on clinically relevant features. However, these transparent models currently achieve lower overall accuracy compared to conventional deep learning approaches.
Dr. Azevedo views this as an expected trade-off in early-stage research. “The complexity of pathological diagnosis, combined with current data limitations, presents significant challenges. But we’ve proven that building interpretability directly into the model architecture is feasible—that’s the crucial first step.”
The Cambridge team is now working to bridge the gap between interpretability and accuracy. Their roadmap includes expanding their training datasets, refining the prototype-based architecture, and exploring hybrid approaches that could deliver both transparency and high performance.
This research represents a significant step toward transparent and reliable AI decision-making tools within medical diagnostics, where medical professionals can not only rely on AI predictions but also understand and validate the reasoning behind them. As the CHARM project progresses, these interpretable models could become essential tools for building trust between AI systems and the medical professionals who depend on them for patient care.