AI workshop highlights Friday’s QCOR lineup
When it comes to putting knowledge into practice, the Quality of Care and Outcomes Research (QCOR) Scientific Sessions — an annual tradition that kicks off at Scientific Sessions — is a great place to start.
QCOR committee member Emily O’Brien, PhD, FAHA, said QCOR adds value for Scientific Sessions attendees by bringing them information they can use in their day-to-day practices.
“QCOR’s ‘secret sauce’ is the application of rigorous methods to highly relevant questions in real-world practice,” said O’Brien, associate professor in population health sciences at Duke University School of Medicine in Durham, North Carolina. “We love to talk about innovations in data science, but we also want to show people how they can use those innovations in their own work.”
Committee member Modele Ogunniyi, MD, MPH, FACC, FACP, FAHA, said QCOR fills a critical gap by focusing on implementation science, health systems and outcomes research.
“These are areas that directly impact patient care quality and health equity,” said Ogunniyi, professor of medicine and master physician at Emory University School of Medicine in Atlanta. “QCOR addresses how to translate evidence into practice, optimize workflows and improve population health outcomes. This aligns with the American Heart Association’s mission to advance cardiovascular health for all.”
One of the innovations featured in Friday’s QCOR program was “AI in Action: A Hands-On Workshop on LLM-Powered Quality Assessment and Improvement in the EHR,” organized by the Yale Cardiovascular Data Science (CarDS) Lab. The session was co-led by Rohan Khera, MD, MS, assistant professor at Yale School of Medicine in New Haven, Connecticut, and director of the CarDS Lab, and Aline Pedroso, PhD, scientific operations lead at the lab.
Participants had access to a secure workspace with de-identified clinical data. Using a point-and-click interface, they interacted directly with a large language model (LLM) to learn how to use it to assess quality of care.
Attendees learned how to query structured electronic health record (EHR) fields such as lab values and medications and unstructured fields like clinical notes. They also learned about retriever-augmented generation (RAG), a way of making LLMs smarter and more trustworthy.
“On their own, LLMs rely mostly on the information they were trained on, which may not always be up to date or specific enough,” Pedroso said, while the RAG process “involves a user query being converted into a vector — a similarity search performed in a vector database to find matching contextual information — and that retrieved data being added to the LLM’s prompt.”
“RAG enables LLMs to access up-to-date, domain-specific data without retraining, providing verifiable sources for the generated content,” she said.
Khera said RAG-enhanced LLMs can also integrate narrative clinical documentation, such as physician notes, imaging reports and discharge summaries, where much of the clinically relevant data relates to the structured data.
“This enables a more comprehensive and accurate assessment of care delivery, capturing quality measures that might otherwise be missed if only structured fields were used,” he said. “A key advantage of RAG-LLMs is that they provide explainable outputs tied to source documentation. Rather than returning ‘black box’ results, the model delivers traceable outputs anchored in the underlying text or guideline reference that informed the assessment.”
“For example, a patient with atrial fibrillation may not be prescribed a blood thinner despite clear recommendations, or a person at high risk for heart failure may not be receiving all guideline-directed therapies even when there are no contraindications,” she said. “LLM-powered quality measurement can identify these situations more accurately because it looks not just at check boxes in the EHR, but also at the doctor’s notes, test results and discharge summaries where important details are often hidden.”
Once those gaps are identified, Khera said, novel AI tools embedded in the EHR can flag, for example, the fact that a patient sitting in front of the doctor today needs a preventive treatment or a follow-up test.
“This represents a shift in how the EHR functions, from being primarily a record-keeping system to becoming an active participant in improving care quality,” he said. “By embedding these interventions in workflows, the goal is to ensure timely, guideline-based care.”
A panel discussion during the second half of the session included Faraz Ahmad, MD, assistant professor of cardiology and preventive medicine at Northwestern University’s Feinberg School of Medicine in Chicago, and Bobak Mortazavi, PhD, associate professor of computer science and engineering at Texas A&M University in College Station. The panel explored the opportunities and challenges of bringing AI into health systems.
“On the opportunity side, we highlighted how these tools can promote equity by ensuring that patients, whether treated in a small community clinic or a large academic hospital, consistently receive evidence-based care,” Khera said. “AI tools also have the advantage of scalability, because they are not tied to a single infrastructure. Once validated, they can be deployed across hospitals or even entire health systems without the need for major redesign.”
On the other hand, Pedroso said the challenges facing AI systems include workflow integration and regulatory hurdles.
“Clinicians already face heavy demands, so AI solutions must be designed to fit seamlessly into their routines rather than add extra steps or disruptions,” she said. “Another challenge is regulatory considerations, as health systems and policymakers must ensure that these technologies are safe, reliable and protective of patient privacy before widespread adoption.”
