AI in Medical Education: Personalizing Learning and Supporting the Next Generation of Clinicians

Podcast Transcript

Dr. James K. Stoller:

Hello, and welcome to MedEd Thread, a Cleveland Clinic education podcast that explores the latest innovations in medical education and amplifies the tremendous work of our educators across the enterprise.

Dr. Tony Tizzano:

Hello, welcome to today's episode of MedEd Thread, an educational podcast exploring how artificial intelligence is transforming medical education by providing personalized learning, immersive simulations, and efficient administrative support, while also becoming a crucial subject of study itself. I'm your host, Dr. Tony Tizano, Director of Student and Learner Health here at Cleveland Clinic, Cleveland, Ohio. Today, I'm very pleased to have Dr. Rick Rejeleene, a special fellow for artificial intelligence and education at Cleveland Clinic here to join us. Rick, welcome to today's podcast.

Dr. Rick Rejeleene:

Tony, thank you for the invitation and thank you for the opportunity.

Dr. Tony Tizzano:

Our pleasure. Rick, to get us started, could you give us a little bit of background on yourself, your education, what brought you to Cleveland and your role here at Cleveland Clinic?

Dr. Rick Rejeleene:

Yeah. So my education background is computer science and engineering. So my bachelor's was on computer science engineering, my master's, and also my PhD. All my degrees were in computer science engineering. And in my specialization, my focus has been on artificial intelligence or machine learning. And in my career path, I've worked on various roles from computer science teacher where I've taught computer programming like Python, and then I've worked as a software engineer, data scientist, and, and then finally, I ended up being a machine learning researcher.

Dr. Tony Tizzano:

So a very wide breadth of experience. So in today's episode, we'll explore how integration of artificial intelligence aims to enhance the efficiency and quality of medical education while preparing trainees for a technology-driven healthcare landscape.

So Rick, if you would, could you frame today's topic by providing our listeners with some context around the importance of AI across all aspects of medical education?

Dr. Rick Rejeleene:

Oh, yeah, Tony. So in today's context of medical education, artificial intelligence is coming up in three big reasons. First, the scale and then the complexity, and it's also becoming clinical practice. So a lot of the education programs are managing lots of learners, and there's lots of competency to test, lots of documentation, giving feedback to the learners, but this comes with a challenge that faculty don't have a lot of time to give feedback.

Now, AI can really help in this situation by organizing information, drafting feedback, identifying patterns, and also helping with repetitive task. And then AI has already become part of clinical practice. Medical education has to prepare the residents and trainees for a world where AI-assisted tools are existing already, whether it's basically documentation support, imaging support, patient communication tools. So we're not just teaching medicine. We're teaching medicine in an environment where decision support and AI tools are already present and we want residents to know how to effectively use these tools.

Dr. Tony Tizzano:

You know, your point is, gee, it's not the future. It's already here. It's in the room. And the question is, how are we going to integrate this in the best way possible?

Dr. Rick Rejeleene:

There are, like, really promising applications of AI and medical education, and I'll give, like, four big categories. So one is personalized and adapting learning. Next is virtual patient simulation. The third is automated assessment and feedback, and the last is administrative efficiency.

So the first is personalized and adaptive learning. So if you see in the traditional education, the teacher is in the middle and, you know, they give feedback. With AI-assisted systems, the learners can get support earlier within a classroom or, or training. Basically, the AI systems can help to identify gaps in performance patterns of the residents or learners. And all, it can also help us to adjust the pace of their learning and really give, like, personalized feedback and also even content.

The main point is the goal isn't to replace a teacher, but to give every learner an extra support or, like, a coach that reinforces concepts and they can practice questions and that can guide them through the right level of challenge.

Dr. Tony Tizzano:

Well, those are excellent points. You know, historically, it was hard, you know, in the 19th century to even have books, let alone have hands-on experience. And many trainees, if you wanna even call them that then, barely got to see patients. And this is a way that it sounds like we're much more able to immerse the learner in a way where they can be observed in a entirely synthesized environment, non-threatening to anybody and, and learn and, and get weighed and measured.

Dr. Rick Rejeleene:

That's correct. And the goal is to personalize and align with the curriculum goals with evidence and, you know, good quality. So that's the goal of the personalized and adaptive learning.

And the second, as I shared, is virtual patient simulation. Basically, what's exciting with the AI tools is that it can simulate a real life patient, and then the learners can practice difficult co- conversations, end of life discussions, even like angry patient encounters in basically a psychologically safe environment, and repeat it as many times until, like, they master the skills and gain feedback. And this really matters because, you know, as they repeat, the skills become more reliable and it becomes automatic. So AI is actually, like, helping them to have, like, a structural, simulated rehearsal.

And the next is basically assessment and feedback, especially in areas where it, it's really time-consuming. For example, narrative feedback and reflective writing and even clinical reasoning write-ups, professional notes.

So in this domain, AI can basically summarize the learner's portfolio and draft a personalized feedback that faculty can review and edit, and also give them highlighting missing elements, giving what's insufficient evidence, and can also point out unclear reasoning, lack of reflection depth, and support consistency across graders. So the important point is that there is a human as part of the system, which we call human in the loop. So AI is not the final judge. It's, it's a very good tool that is helping us do automated assessment and feedback.

Dr. Tony Tizzano:

So this is a really very commensalistic type of relationship. I mean, some of maybe the initial heavy lifting is done by making a framework, and then the instructor can go look through this and polish it, tweak it this way at that, and save a tremendous amount of time, and, and it sounds like it really adds value.

Dr. Rick Rejeleene:

Yeah, that's correct. And just wanted to share the last one. Administrative support, as you know, like, a lot of time, you know, within the healthcare education is spent on administrative support, scheduling, you know, document, routing, drafting routine communication, when also organizing accreditation ma- materials. Even if you're like a faculty, you would direct some of your time into this. And AI can really help to reduce the repetitive effort and faculty can focus more on human to human teaching. These are the ways AI has been extremely helpful. It's already part of these domains.

Dr. Tony Tizzano:

And Rick, in consideration of all the things you've said, it sounds like the educator can perhaps better tailor the training because we all learn at different rates. And sometimes we get tripped up by certain things, other people get it right away, and this is a way that by saving time, there's perhaps even more time to devote to areas where someone really has a need.

Dr. Rick Rejeleene:

Yeah, that is correct. So faculty can give more attention and time on human-to-human interaction than, you know, all other tasks, and AI can help them to augment already existing skills they have.

Dr. Tony Tizzano:

Yeah. So I look at the generation of our trainees right now, and they are, for the most part, they're Gen Z. I mean, they have grown up with the internet from day one, and they are very familiar. So here we are, many of us, myself included as teachers who are from a long ways back, and this is not overly familiar to us.

You know, what has been the buy-in from educators and students regarding artificial intelligence facilitated assessment, and what does the feedback look like?

Dr. Rick Rejeleene:

A lot of faculty are already using it, and there's lots of excitement, and both from, you know, the learner perspective and, you know, the faculty perspective. And I want to share a couple of projects that can give the audience concrete examples of how this is already being used and, and benefits.

So for students, personalized learning paths, practice, you know, faster and frequent feedback, you know, for educators, they have more time, less burnout from repetitive task. For, you know, patient and administration, you know, consistent training, quality, better access because there's more education support. So these are some of the benefits for both the learners and educators.

Dr. Tony Tizzano:

And are you seeing this gain traction?

Dr. Rick Rejeleene:

Oh, definitely. It's already being used.

Dr. Tony Tizzano:

So when you look at your projects that you're involved in, you know, in what ways are you leveraging artificial intelligence and simulations in things that you're currently doing?

Dr. Rick Rejeleene:

Yeah. So I want to share one exciting project that we've been working diligently with Dr. Neil Mehta. So one area is an artificially intelligent, augmented assessment of learner reflections. So a lot of the residents have portfolios and reflective writing is one of the important identity formation and part of their training, but scoring and giving narrative feedback to all of the learners, say, like, 40 or 50 or even sometimes more is really time-consuming and can vary across faculty. So we're actually exploring a AI tool we're building. We have a prototype that can basically map the portfolio text to a rubric, and then it can identify evidence of the reflection in the portfolio and give them growth feedback and at the end, like, give, like, a, a great score and how they can personally improve in their portfolios and in their reflections. So using AI, specifically large language models, this has been a great tool, and it's already, like, we're, we're already excited and we're hoping that we would deploy.

A second area is simulation and communication rehearsal. There's a lot, a lot of chat-based agents where you can simulate communication situations or virtual patient, and the learner can actually, like, practice repeatedly and control, uh, difficult conversations when patient can be, you know, neutral, confused, uh, anxious or angry, and they can practice deescalation or empathy skills, which is essential for a physician.

A third area is the research and scholarship around AI that we're collaborating in anatomy and other education contexts. So we're building all these tools and implementing AI, but we want to know exactly does the feedback improve quality of the training? Are learners engaging more using these tools? Are the outcomes better than non-AI approaches?

Dr. Tony Tizzano:

Do you have evidence for that now, even, or is that in the works?

Dr. Rick Rejeleene:

It's in the works. And at present, we are seeing the large language model is actually better than human feedback in some situations, which is quite surprising.

Dr. Tony Tizzano:

I've heard that when a physician writes a note after an encounter with a patient and AI writes the note, and someone who doesn't know who did which reads that the AI-generated note contains the same information, but sounds more empathetic at times, which is really remarkable.

But with regards to the, the timing, so is the feedback then more real-time since you have AI, is it able to get to the student more quickly?

Dr. Rick Rejeleene:

It's not yet real-time, I would say. Again, a human has to perform the feedback or control the tool, and the human is the one who is giving the feedback at the very end. The AI is just part of the workflow that's helping both the learners and also the faculty.

Dr. Tony Tizzano:

So if it's a time saver, though, is it possible that the instructor, the educator will have more time to get to a student more quickly? Because I can think in my training, you'd go through a course for several weeks, and at the very end, you finally get some evaluation, and you found that, gee, maybe I should have done this, I should have done that, or I needed to improve here. And had you known that earlier, you could have focused a bit more. Do you think there's an opportunity there?

Dr. Rick Rejeleene:

Yeah, there is an opportunity. To give you a concrete example, say you have 60 portfolios and you use this AI tool to give reflective writing feedback, can pretty much do it within a day. It won't take more than five to 10 minutes, so that's a lot of time saver.

Dr. Tony Tizzano:

Yeah. But this is absolutely fascinating. So, you know, Rick, one might surmise that the effective and responsible integration of artificial intelligence into medical education requires, if not demands, a balanced human-centric approach by ensuring that technology compliments rather than replaces a human expertise.

Would you respond to this assertion and what do you see as challenges or perhaps ethical considerations as AI is finding a home in more and more aspects of healthcare education?

Dr. Rick Rejeleene:

Yeah, Tony, that's a very good question. I would agree, but I view AI as a very good tool and you want to use the tool in a really good way to help the learners. One startling thing is we are noticing de- skilling, which is concerning. So when students are reliably using these tools for all their, you know, competency training or exams or homeworks, without reflective or without taking the time to learn, they're finding that it's diminishing their critical thinking, memory, and creativity over time.

There was a study from a lab from a research scientist at MIT, from Nataliya Kos'myna, and basically her study was the alarms are making writing easier, but when people are too dependent on it, all these skills and all these, uh, de- skilling is being experienced by students, which is very concerning. So in our work with Dr. Neil Mehta, they've come up with, like, five-step approach to mitigating cognitive de- scaling.

So when you have a student or a learner in your class, the first step is first think about what you already know, note it down when you're learning, and then engage in Socratic interaction with LLM. Don't just ask the question and then copy it down, and then the third is to verify if it's reliable as LLMs are known to be highly unreliable and also hallucinate. So you want to make sure that you verify the accuracy of, of the information. And then fourth is reflect. This is the very important thing. What did you learn and what do you still need to learn? What are the gaps? And then take a week off using AI and then try to summarize on your own. This is very essential as people have found that using, you know, ChatGPT has decreased their critical thinking skills from this study.

Dr. Tony Tizzano:

I don't think medical education is alone in that realm because I was reading a commentary from a journalism professor at Boston University and he said he has had to, in his introductory courses, especially, you do your homework in class with your phone in your pocket. So he wants to see how you can write all by yourself without this augmentation, which, you know, once you're skilled and can use it to speed things along is one thing. But, you know, it's, it's amazing how these various technologies ... I know in my career, there was a point in time where we dictated and did not write notes. And what I found after doing this for over a decade is I just about forgot how to spell because I didn't have to spell anything. I just had to say it.

Dr. Rick Rejeleene:

Oh, wow.

Dr. Tony Tizzano:

And so your spelling of difficult words became challenging and then you go back to the computer and now you have to type. And so you had to spell correctly, and so it was a bit of a little curve there that I hadn't thought about.

Well, Neil Mehta, working with him is always a pleasure. I'm sure the two of you just loved getting together on these topics. So as you look at the things that you came to us with from industry, which you have this broad experience, were there any surprises over the course of your fellowship work thus far in the educational milieu?

Dr. Rick Rejeleene:

Yeah. So one surprise is basically when people go from, oh, they're not sure about AI, they're skeptical to, "Oh, curiosity, like, oh, how does this work? Can it do this for my task or can it do this in education?" When they actually see the tool solving a real pain point. So it's just not just, you know, all the hype, there is actually a good workflow improvement in healthcare education. The hard part is implementation and aligning stakeholders, you know, in the reflective writing, defining rubrics, uh, deciding how the outcomes would be used and also building trust to use these tools and designing the human in the loop process so it fits actual education practice.

Dr. Tony Tizzano:

Sure. Well, I've been doing podcasts for about four years and I, we had some early discussions around ChatGPT and so forth when it was just coming out and everyone was wide-eyed, I can't even begin to imagine the progress we're going to make and the speed at which it's going to occur. I mean, it is occurring so quickly, which doesn't usually happen in medicine. I mean, every now and then, there is a eureka moment and a disruptive technology, but this is moving along so quickly and it's being refined so carefully. I mean, these things are being identified. Everyone's looking at it with such a critical eye. I think it's very intriguing and I, I can only see benefit in this, but with persons like you steering the ship, that's the thing. And individuals like Neil Mehta, I mean, these are, you know, to have people who are critical and in the know.

So as you look down the road, what do you see on the horizon for students, educators, and patients as AI becomes more integrated in all aspects of medicine?

Dr. Rick Rejeleene:

Yeah. So I think we'll see growth in personalized education. Learning support is individualized timely and competency line. So this could mean, you know, other detection of struggling learners, more tailored recommendations, also like giving feedback and more frequent practice opportunities. And we'll also see all these tools becoming having a good core professional competency of the learners. So the learners would not just need fluency, but in using AI, how to evaluate it, and also being aware of the limitation, uncertainty, and ethical boundaries of these tools.

Dr. Tony Tizzano:

Well, this is very, very intriguing. And, uh, I hope for our listeners, if you're not familiar with this, you can begin to see, you know, there's light at the end of this tunnel for all of us, you know, as this becomes more commonplace and more easily understood as we become familiar.

Rick, is there anything that we haven't discussed that you feel is important for our listeners to know?

Dr. Rick Rejeleene:

Yeah, I, I want to emphasize that success of AI in medical education is all dependent on intentional integration of these tools. AI is just not a plugin that automatically improves education. We need, like, clear education goals of how these tools would be used within the workflow and then valid rubrics and outcomes and for faculty development, and then transparent policies for learners, and then ongoing evaluation methods. So if we do that, you know, AI can genuinely increase what we want more of, you know, meaningful feedback, a really good coaching from faculty to learners, and really better prepare clinicians of the future.

Dr. Tony Tizzano:

Well, it's been exciting to watch it just this far, and, uh, I just really look forward to, to seeing this evolve. Well, Rick, I wanna thank you so much. This has been a very thought-provoking and wonderfully insightful podcast.

To our listeners, if you would like to suggest an education topic to us or comment on an episode, please email us at education@ccf.org. Thank you very much for joining, and we look forward to seeing you on our next podcast. Have a wonderful day.

Dr. James K. Stoller:

This concludes this episode of MedEd Thread, a Cleveland Clinic education podcast. Be sure to subscribe to hear new episodes via iTunes, Google Play, Stitcher, Spotify, or wherever you get your podcasts. Until next time, thanks for listening to MedEd Thread, and please join us again soon.