Health Misinformation in the AI Era: Risks, Realities, and the Role of Librarians

Podcast Transcript

Dr. James K. Stoller:

Hello and welcome to MedEd Thread, a Cleveland Clinic education podcast that explores the latest innovations in medical education and amplifies the tremendous work of our educators across the enterprise.

Dr. Tony Tizzano:

Hello, welcome to today's episode of MedEd Thread, an education podcast exploring the growing problem of health misinformation for health sciences learners and potential untoward consequences for consumers of healthcare services. I'm your host, Dr. Tony Cesano, Director of Student and Learner Health here at Cleveland Clinic in Cleveland, Ohio. Today, I'm very pleased to have with us Michelle Craft, Director of Library Services for the Cleveland Clinic Health System here to join us. Michelle, welcome to today's podcast.

Michelle Kraft:

Thanks for having me.

Dr. Tony Tizzano:

Also joining us, Olivia Davis, librarian for the Cleveland Clinic Health System. Olivia, thank you for joining us today.

Olivia Davis:

Thanks for having me.

Dr. Tony Tizzano:

To get us started, Michelle and Olivia, would you please tell us a little bit about yourselves, your educational backgrounds, what brought you to Cleveland, and your respective roles here within the Cleveland Clinic Health System?

Michelle Kraft:

Sure, I'll get started. I am the Director of the Library System and Services for the Cleveland Clinic Health System in the United States. I oversee operations of services and resources for all the Cleveland Clinic locations and help my library staff provide information to all of the caregivers at Cleveland Clinic.

Dr. Tony Tizzano:

Fabulous, a much-needed service, I might add. Olivia.

Olivia Davis:

I'm a medical librarian at Cleveland Clinic. I do the interlibrary loan, which is, you know, ordering articles that people request. I got my MLS right in 2023 from the University of Washington. So I'm happy to be back in Cleveland, which is my hometown.

Dr. Tony Tizzano:

Fabulous. Well, welcome and glad to have you both. So in today's segment, we want to explore the impact of health misinformation on medical education and the consequences of this misleading information, not only on consumers of healthcare services and products, also on our learners and persons doing research. So Olivia, before we dive into this, these terms can be confusing. Can you help clarify the differences between health misinformation and health disinformation?

Michelle Kraft:

Sure.

Olivia Davis:

According to the Office of the Surgeon General, misinformation is the unintentional spreading of information that is false, inaccurate, or misleading according to the best available evidence at the time. On the other hand, health disinformation is intentionally created and shared falsehoods with full knowledge intended to deceive, cause harm, or achieve some economic, ideological, or sociopolitical gain. Both are harmful in their own ways.

Dr. Tony Tizzano:

So this might even be, you know, talking with family members and friends, what we think might be the better way to go. And particularly if you're in healthcare, people have a tendency to listen. We don't always have the facts.

Michelle Kraft:

Absolutely. And information comes in so many forms and ways, you know, In the old school, it used to be like newspaper and books. And I hate saying it was old school when you think of books, but you know, that was back then. And now we're seeing it on the internet and YouTube and all other ways and people to people.

Dr. Tony Tizzano:

Well, you know, we used to have a library in our office when I first began practice. And whenever I had an issue or I was stumped by something a patient was presented, I would say, "Give me a minute." And I would head into the library and look it up. And as we got new folks coming in in the last decade, they'd see me heading towards the library, they go, "What do you need?" And their phone would come outta their pocket, da-da-da-da, thumbs would go flying, and pretty soon they gave me everything I wanted to know, and I just was a bit of a dinosaur. So things are changing. So in framing today's topic, Michelle, give our listeners some context around the challenges faced by medical librarians as they endeavor to vet artificial intelligence-generated information for accuracy, so that professionals and learners that require your assistance have assurance of some level of truth?

Michelle Kraft:

Well, you said it earlier, you used to go and look up the information in the book or in a journal article. And librarians have been vetting that information even before things went online and before things went to AI. So what we are doing is just reusing our knowledge and our critical evaluation techniques to appraise and vet AI information or online information. So using the same techniques that we used even back during print and when things got into print, we are using them for the online resources and AI. So it's like lateral reading, checking whether information is trusted and whether they're using independent sources.

Olivia Davis:

Mm-hmm.

Michelle Kraft:

Verifying the information to see if the citation that AI is bringing up really does exist. And I like to think of it as like we're sifting for information. And why I use the word SIFT is because it's an acronym too. S for stop. You wanna pause before you accept the information that the AI just generated. You wanna investigate, you wanna evaluate whether the original source information is correct. It depends upon the AI and different versions of the AI, but there's studies that have seen that as many as 20% or higher of the sources that an AI puts out as the source of the information are completely fabricated, what we call AI hallucinations. They're completely made up, yet they look legitimate. We also wanna find information to find out if there's better coverage. So the F is find. Find the better coverage. Did the AI give you good information, but it's too general? Is it on the topic or a little loosely related to it that you have to find more in-depth information? And then T, we trace the claims. And this is stuff that librarians and researchers have been doing since I think we've been librarians and researchers.

Michelle Kraft:

We go back to the origin of the information to find the original quote in the context. Many sources, that the AI produces are real papers, but they might not be on the topic. They might actually mention a drug as being effective or AI misinterprets it or reads the different algorithm of the words and says it's not effective. So by looking at the source material to see, A, is it real? And B, is it saying what the AI says it says?

Dr. Tony Tizzano:

Mm-hmm.

Michelle Kraft:

Is the most important thing.

Dr. Tony Tizzano:

So these are kind of like accidents within the interpretation by AI, not purposeful, but just the way things are framed and the fact that it works on algorithms and what it's seen before, it makes these errors.

Michelle Kraft:

Well, I think you said it exactly in the sense that people think it's interpreting, and that's exactly it. It's not interpreting, it's not evaluating, it's an algorithm, and it's predicting the next logical word. I like to think of it as your autocorrect on your phone on steroids.

Dr. Tony Tizzano:

Oh boy, yeah. And if it's like the autocorrect for my language when I try to speak into my phone and dictate something, God only knows the way I enunciate things, it can come out with things that I'd be embarrassed to even mention. So Olivia, how prevalent is health misinformation and what are some of the ways that it spreads?

Olivia Davis:

Yeah, it's more prevalent than ever before. I feel like social media and generative AI being some of the main purposes perpetrators. We're living in what the World Health Organization calls an infodemic, which is an overabundance of information, both accurate and not. And it spreads for a multitude of reasons, some of them being people tend to believe information if they encounter it repeatedly. Information that elicits an emotional response increases the likelihood of sharing content online. And people share information that aligns with identity, reinforces a group sense of connection, or offers a sense of control and meaning. Mm-hmm. And then anybody and everyone are susceptible to misinformation.

Dr. Tony Tizzano:

Well, I think that is really well said. You know, we all have a tendency when we hear something that is in keeping with our perspective, it reinforces our perspective, even if it's not accurate. And pretty soon we gain some level of momentum and it's almost like we have to editorialize. And I think, I feel like librarians do that more and more where we used to editorialize a book or a manuscript. Now you have to editorialize the information coming out of AI so that persons using it have something of substance.

Michelle Kraft:

Well, and to give a non-medical example, if you think about it, living in the bubble of perspective, long, long ago, we all thought the world was flat, you know? And it took people and explorers sailing to prove that the world was round. And we're living in those bubbles thinking about those things that we thought were true but we're not.

Dr. Tony Tizzano:

Absolutely. But with the technology we had and looking offshore, the world was flat. The horizon was flat. We couldn't see it from space. Yeah, that's an excellent analogy. I really like that. How about social media? What role does that play in causing these issues for us?

Olivia Davis:

Anybody can share anything on social media. So I think that has a lot of impact on things people see and hear. A lot of times you have a grandparent sharing something that they see, they just read a headline and they say, "I believe this." So they share it and it gets around. I mean, it's an issue 'cause they're not opening the articles, they're not reading the full thing, they're seeing a headline, they're sharing.

Dr. Tony Tizzano:

They're not looking at the references, they're not diving deeper.

Michelle Kraft:

Exactly. Well, exactly. And to touch on what Olivia was saying, Social media also goes by number of clicks. People are rewarded by the number of clicks and how viral and how popular their message is. So there is somewhat of an incentive for people to be really outlandish. And even in the newspaper days, the headlines were something that grabbed you so you could sell more newspapers. And social media has that same thing. So misinformation can easily spread, especially if they wanna make it very shocking.

Dr. Tony Tizzano:

So the more sensational, the more its ability to capture our attention and for us to want to think, oh my gosh, look what just happened and we need to believe this and talk about it. So, you know, I imagine, Michelle, that the ultimate impact on misinformation on public health is difficult to really pin down. What are your thoughts around where misinformation led consumers related to the COVID-19 pandemic? You know, we all talk about this as an example that's fresh in the minds of really most everyone.

Michelle Kraft:

Well, I think exactly that. That is probably one of the larger, most recent examples of where information was just all over the place and misinformation was running rampant and conflicting with good information. And you also had good information that changed because the science and health and technology told us that's no longer up to date and it became old. So we had people who were hesitant about the vaccines, declined vaccinations, which put them at risk. We had people who dismissed the public health measures. They didn't want to wear masks. And people even more so were looking at alternative treatments that were completely unproven and they disregarded the proven ones. So all of those contributed to a greater morbidity and mortality than probably would have happened if we had a better way of evaluating information as a society. But it also not only injured the people who were believing that information, it also led to, I think, mental health and violence that was outside of what you think of information. Like, okay, somebody doesn't get a vaccine, that's harm to themselves or harm to their immediate family members.

Olivia Davis:

Mm-hmm.

Michelle Kraft:

But it led to harassment and violence against healthcare workers, airline staff, other frontline workers when they were communicating their policies or their public health measures. So it exploded to be more than just what I would say is a medical or health issue and became more of a society, mental health, and violence and safety problem.

Dr. Tony Tizzano:

Yeah, that's really interesting. And I think we have to credit The manner in which information is spread online, it used to be that it would have to come out in a newspaper or you'd have to read it in a book if it was from a source that people believed. It was National Geographic or Science Magazine, sources that we kind of all thought were credible. It took time for things to get there. Now anybody can say anything at the moment and it can catch fire online, and pretty soon by the end of the day, everybody knows it. And that is such a difficult thing. And it's, it's led perhaps to us not trusting science in the way that we used to, good or bad, not to say that science doesn't make mistakes. So Olivia, I listen to all this and I have to imagine that the role of health sciences librarians has really evolved in the service of communities and learners related to AI and this kind of misinformation. What kind of changes have you seen? I mean, you've recently recently trained? What did it used to look like and what's it like now?

Olivia Davis:

Yeah, I think librarians have had a huge learning curve in the last few years. We've been at the forefront of combating misinformation in many ways. I know that many of my colleagues have had to assume roles in teaching critical evaluation skills, utilizing strategies such as pre-bunking, which is warning about misinformation trends, debunking, correcting false claims with facts, and helping patrons navigate the infodemic and identifying trustworthy sources. And with AI, it's it's gotten even more difficult to do so. But librarians such as myself have been reminding people everywhere that the strongest tool that you have at your disposal is your own brain.

Dr. Tony Tizzano:

That's true. You know, it's everything quick and let's get it over with and looking at it. If you can just click a button, cut and paste, that's just not enough. I mean, I think it can point us in a direction, but it can't validate what we're trying to say. And we need to spend that extra time. So I just think about this, Michelle. Throughout history, our libraries have provided the informational backbone for advancing medical research and education. I think there's no question about that. What are the risks to learners and researchers who fail to recognize the potential for this AI-associated misinformation in pursuit of their research and education and how they're viewed by journals and what have you?

Michelle Kraft:

I think that's a good question. People need to understand the consequences to the integrity of the healthcare research and education. We need to be aware of the data corruption and bias propagation. AI models are trained on certain sets of data and they will have biases. They will perpetuate the existing gaps in healthcare access and quality. So for example, they may be trained on images for white people and not have any training or less training on images of people of color.

Olivia Davis:

Mm-hmm.

Michelle Kraft:

And that will continue to generate information that is biased or misleading. And that will continue to perpetuate the differences in health outcomes that happen from that kind of bias. It leads to inadequate and inequitable clinical decision-making. So if you don't have all of the information because you're not getting all of the information, you don't have the best information to create a good clinical decision. Mm-hmm. It also will fabricate the evidence. And so we need to remember the chatbots, as I mentioned, or the LLMs, they are just your autocorrect on your phone on steroids. And so they are predicting the next set of words that are gonna come or that are in those results and things like that that match with the question that you had. So a lot of times there's hallucinations. And so they fabricate studies. They mistakenly cite the research or incorrectly cite the research, which will undermine the scientific integrity and the evidence of the researcher. It's a really bad idea if you say, "This paper says this, this, and this," and the paper doesn't exist, or the paper exists, but it says exactly opposite of what you're trying to say. You didn't read it.

Michelle Kraft:

You didn't do your due diligence. How could you be an expert in writing on it if you didn't even read the stuff that other people are writing on it? Mm-hmm. So that really undermines somebody's credibility if they're not looking at the evidence that's there. And then there's safety risks associated with data poisoning, leading to masking around certain adverse events, and it compromises patient safety and clinical trial outcomes. So like some of the data is not there or the data is incorrect, which actually can lead to events where it can cause harm to patients.

Olivia Davis:

Mm-hmm.

Michelle Kraft:

All those examples were really healthcare and treating patients or doing research. With medical education, we're starting to see evidence of skill degradation, and it's associated with overreliance on AI for clinical decision-making. People are not thinking through the problems that they were once given. They're going straight to the AI. And I think the easiest comparison I can give Is that people were concerned back when the calculator became easy for everybody to have. There was a lot of concern about how people's math skills were happening and whether they could do that. It took a while for people to understand the calculator is a tool. You still have to think through the math problem to come up with the correct answer, 'cause if you put in the wrong numbers, you're still gonna get the wrong answer. So we have to remind students Don't rely on the AI to completely do the decision-making. You have to think through the process. You have to develop your critical thinking skills. And then we are starting to also see a little bit of atrophy in humanistic skills. And I think we were starting to see this a little bit before AI came out with people just with their nose buried in their cell phones, people staying in and really only watching the—

Olivia Davis:

Watching the news.

Michelle Kraft:

Online shows and things like that, we're starting to see people who are having problems with empathy because they're used to communicating with themselves or with the computer.

Dr. Tony Tizzano:

Yes, yes. But you know, it's interesting because if you have AI listening to an encounter with a patient and the physician summarizes that encounter in a note and AI summarizes that encounter in a note, someone who doesn't know who did what will sometimes look at the AI and feel it's a more empathetic accounting of that encounter than what the physician wrote. And if you're a dinosaur like me, who if I was any worse with the computer, I'd have a tail, I'll get a phone message from MyChart and AI has generated a response. Well, at first I thought my staff was generating that response. And so I'd start to read that and I thought, "Oh my gosh, they really, they've took some time here, but I don't quite agree with this." And then finally at the end I'm saying, This is AI. And I'm like, oh my gosh, if you're not careful, you can't just click and say, okay, that's good enough. You definitely have to read those things because some things can easily be taken out of context. I wanna ask a question though. You mentioned LLM. What is LLM?

Michelle Kraft:

Large language models. It's a type of AI in which it's using algorithms to predict language and match how people speak. And its job is to generate a natural language experience of talking to somebody. In a way, it anthropomorphizes the actual computer. And I found myself, even when I'm looking for directions or best restaurant reviews and trying to get some information, I find myself asking the AI, "Please, can you give me this information?" Or, "Rewrite this, please." I mean— When have we asked a computer, "Please"? But yet I'm finding like my natural language, like I'm talking to somebody. So the LLMs are doing a very good job of almost creating that social engagement, which is in a way detrimental 'cause I'm not engaging or we as a society are not engaging with humans as much.

Dr. Tony Tizzano:

And it's getting better. You know, Olivia, you mentioned hallucinations and I get that, that inadvertent mistake accidents by AI tools, but there's also this thing called deepfakes. Can you expand on that concept of the deepfake?

Olivia Davis:

They are created by humans through AI to deceive people. They're deepfakes of other people. You can make them say whatever you want, really.

Dr. Tony Tizzano:

Boy, that is really disconcerting. How do you tease that out? From everything that we're trying to accomplish.

Michelle Kraft:

It's getting more and more difficult. I mean, some that I think that are meant more for satire, you can definitely tell it's a deepfake because you wouldn't expect that celebrity to say those kinds of things.

Olivia Davis:

But— It can be really harmful though, if you have somebody saying something that they wouldn't say and people take it at face value, they might believe it and say that could maybe ruin that person's career. Yeah, it's getting worse.

Dr. Tony Tizzano:

That's a problem. So for both of you, considering how critical the nature of this issue is of misinformation and the magnitude of concerns, particularly around health misinformation, are we making progress? Do you think we're getting better? Is it being recognized?

Olivia Davis:

You know, honestly, I don't know. Generative AI is making it difficult to parse out what information is true and what is false nowadays. I mean, within our library alone, I've seen many citations requested by library users that are simply not real and hallucinated. And so that's the concern for me as a librarian. And in general, I'd like to see more people becoming more critical of the information they come across on the internet and definitely implementing that SIFT method that Michelle had mentioned earlier going forward.

Dr. Tony Tizzano:

Yeah, I think that's a key.

Michelle Kraft:

Yeah, I think for me, it's uneven. I love seeing people using AI or those kind of tools to do things like sorting data and doing things that really make a lot of sense. But with better tools, we need more awareness. And I think that's where we're lacking as a society is more awareness. People are looking for the quick fix. So we need more awareness, not less. Yes, these are tools to help us speed up or improve efficiencies, but we do need to pump the brakes a little bit and slow down to do it right, 'cause it will take longer if you have to do it over again. And if you have to do it over again, depending upon what it is, it could ruin your reputation or hurt somebody in the course of medical information.

Dr. Tony Tizzano:

And I have a feeling that when we see it and it's congruent with what we already think, we're saying, "Okay, this is right." We don't take time to sort that out. So again, for both of you, what's on the horizon? What are we looking at as next steps for optimizing the way library science approaches misinformation in health science education?

Olivia Davis:

I mean, I think just as librarians keeping people informed about our ever-changing digital world, updating them on how they can stay vigilant when it comes to health education and misinformation. I think that's what's most important, at least to me.

Michelle Kraft:

And to kind of partner onto that, I think it's less about finding the information and more about teaching people on how to think about information. Not better tools, but creating smarter users, building the skills early and teaching how to recognize the evidence or recognize the lack of evidence when you're talking about bias and things like that.

Dr. Tony Tizzano:

Yeah, very well said. For both of you, is there anything that I should have asked and didn't ask that you think is important for our listeners to know?

Michelle Kraft:

For me, I want to just go on where I say misinformation has been around since the dawn of time. We as humans, when we've been communicating, we had misinformation. So it's not an AI thing. It is, at the end of the day, the tools will always be changing, but the need for good judgment won't.

Dr. Tony Tizzano:

That's an excellent point. And as my aunt, who's 91 years old, would say, this AI, what's happening? But it's always been there. It's Great point. It's just, it's a just a different platform for getting this information. Well, I'd like to thank you both so much, Michelle and Olivia. This was an intriguing and a wonderfully insightful podcast. To our listeners, if you would like to suggest an education topic to us or comment on an episode, please email us at education@ccf.org. Thank you very much for joining and we look forward to seeing you on our next podcast. Have a wonderful day.

Olivia Davis:

Thank you.

Dr. Tony Tizzano:

This concludes this episode of MedEd Thread, a Cleveland Clinic education podcast. Be sure to subscribe to hear new episodes via iTunes, Google Play, Stitcher, Spotify, or wherever you get your podcasts.

Olivia Davis:

Until next time, thanks for listening to MedEd Thread, and please join us again soon.