Earning clinical trust for an AI diagnostic tool
An AI diagnostic tool pathologists didn’t trust became one they rated 4 or 5 out of 5 on every trust dimension. The client productionized the recommendations, expanded their model offering, and revised their go-to-market on the strength of the findings.
The problem
SlideFlow Labs, a clinical AI team within UChicago Medicine, builds AI-powered diagnostic software for pathologists, using specialized models to identify computational biomarkers that eliminate the need for expensive, time-consuming external testing. Their first model ready for clinical validation addressed thyroid pathology, distinguishing PTC from NIFTP, a clinically meaningful difference with diagnostic and prognostic implications. The product existed but had never been tested outside the team that built it.
An unvalidated AI diagnostic tool in a clinical setting is a liability. The client needed to know whether pathologists would trust, accept, and use the product before investing further. I led the research and product validation effort, managing 2 designers and coordinating across ML engineers, physician scientists, and 12+ pathologists across 3 hospitals.
Will pathologists trust, accept, and use this tool?
Constraints
| Timeline | Pathologist sessions were scarce and non-recoverable. Every interaction had to be designed to extract maximum insight. |
|---|---|
| Stakeholders | 12+ thyroid pathologists across 3 hospitals, among the hardest clinical users to access. |
| Technical / domain | Findings had to survive clinical review; every claim needed to hold up to a physician scientist. |
| Business need | An unvalidated clinical AI tool is a liability. The client needed proof of trust before further development. |
Key decisions
With pathologist access this scarce, the wrong research approach was expensive in a way the schedule could not absorb. Five decisions shaped what we tested and, in turn, what shipped.
The temptation was to minimize user research given how hard pathologist time was to secure. We committed to it and designed every session to extract maximum insight per interaction, because a validation effort without the actual users is not validation.
Contextual inquiry showed that what “transparency,” “trust,” and “credibility” mean for an AI tool is entirely use-case dependent. We scoped to thyroid pathology only; generalizing would have made every finding weaker.
Rather than asking “does this feature work,” we asked “does this tool earn the right to be in a pathologist’s workflow.” That reframe determined what we tested, and it is why the result was a glass-box interface instead of a better black box.
We investigated whether the product was solving the right problem at all, and found that some pathologists had more pressing workflow concerns the tool could address more immediately than the diagnostic use case it was built for.
The instinct was to surface everything the model knew. When we tested that assumption, it failed. The flood of information confused experts instead of reassuring them. What pathologists needed was the right information at the right moment in their workflow.
The solution
The product moved from a black-box output to a glass-box interface. Instead of a bare result, pathologists see the model’s reasoning surfaced at the point in their workflow where it matters, in terminology validated through usability testing rather than raw technical specs. The design separated two distinct user modes, pathologist and technician, based on actual workflow mapping instead of assumed roles.
It also surfaced digital readiness as an adoption variable that differed by hospital type, and packaged market positioning intelligence the client did not previously have, including how pathologists actually perceive the product relative to larger competitors.
Outcomes
Trust was measured with a structured rating across trust, clarity, and credibility, with fresh pathologists from all three hospitals in each round, rated only after hands-on sessions. The client productionized our recommendations, implemented specific elements of the glass-box interface, expanded their model offering to additional use cases to position as a full-stack platform, and revised their go-to-market after discovering their original value proposition was already being promised by competitors.