Measuring the human impact of AI during medical visits
Context
Hospital Sant Joan de Déu piloted LAIA, an AI transcript agent, for healthcare visits.
My main goal, as part of the Patient Experience area, was to assess whether AI improved patient-family interactions and the overall visit quality.
This project lasted six months. Three different medical areas (haematology, rheumatology, and general paediatrics) participated.
To measure the AI's impact on healthcare visit quality, this research consisted of three stages.
In-depth interviews with families about their perception of AI usage
The interviews explored families’ opinions about AI during attention processes, compared their perceptions, and considered their concerns about data privacy, while using AI-generated storyboards as vivid examples.
Defining quality parameters to measure AI's impact during observations
Those parameters included dedicated time for each segment of the visit process (case history, exploration, results, next steps), family and patient comments, visual contact between professionals and families, and professional interactions with the computer.
Performing observations and training medical professionals to observe the defined quality parameters
On-field observations during medical visits were conducted to compare quality parameters with and without AI and understand the qualitative implications of this technology.
Impact
Considering the previously defined quality parameters, the pilot test was a success. When comparing 30 observations with, and 30 without LAIA, families were aware of the AI use and they embraced it (30 out of 30.)
Families felt more listened and comfortable with healthcare professionals when using LAIA. Therefore, the medical visit lasted an average of 4 minutes longer (25.7 min. vs 21.1 min.) The number of interactions with the computer was also lower when LAIA was used (95.7% vs 78.9%.)
Future
Despite the encouraging patient and family engagement, there were technical issues to be addressed. AI sometimes struggles to understand both specific and generic vocabulary, requiring professional review. Apart from that, there was also professional resistance to this technology, since the vocabulary mistakes increased their hesitance.
While patients and families welcomed LAIA, the key to widespread adoption lies with professionals. Strengthening LAIA’s vocabulary and comprehension will reduce professionals' hesitance and will encourage its use during the medical visits.
