[This transcript is auto-generated and unedited.]
- How is Gen AI catalyzing a paradigm shift in medical education? What will it mean for AI to assist in medical school applications or clinical teaching? Can AI be harnessed to elevate the skills and acumen of clinicians while also allowing them to connect more deeply with their humanity in encounters with patients? I'm Dr. Kirsten Bibbins-Domingo, and I'm the Editor-in-Chief of "JAMA" and the JAMA Network. This conversation is part of a series of videos and podcasts hosted by "JAMA" in which we explore the rapidly evolving intersection of AI and medicine. Today, I'm joined by Dr. Bernard Chang, a neurologist and the Dean of Medical Education at Harvard Medical School, where he is also the Daniel D. Federman M.D. Professor of Neurology. Welcome Dr. Chang.
- Thank you, Kirsten, and thank you for the opportunity to talk about AI in medicine.
- Oh, so great, yes, and let's do first names. So you are a new dean, congratulations.
- Well, it is an interesting time for medical schools, and you recently wrote an editorial that accompanied what "JAMA" publishes every year, the demographics of the students in medical schools in the US. Your editorial focused on how AI is going to transform medical education, and I wonder if you could just give our listeners and viewers a sense for what that editorial is about.
- Yeah, and Kirsten, I think this is about to be a major inflection point in medical education, akin to what we experienced when the internet and internet search engines became available. The internet was around when I was in medical school in the mid-1990s, but it wasn't really a source of medical information. Certainly, as students, we didn't go to the internet to help in our courses or help to learn material. But when search engines became available, it became clear that access to knowledge was so much easier and cheaper and simpler than it had been before, and what many medical schools around that time did was evolve their curriculum from one that emphasized lectures, which, of course, are very efficient ways of transferring facts and knowledge, to small group discussion formats, which are more beneficial toward knowledge integration and knowledge analysis and interpretation and help students with oral presentation skills.
- So it's not as much that we have to memorize everything and just have all of that knowledge transferred to us, it's more the medical education's focused on the higher order skills of integrating that knowledge to try to make a clinical decision.
- Exactly, exactly, and I think coming up in these next few years, we're gonna have a similar transformation, which is that what generative artificial intelligence, these AI tools, like ChatGPT, can do very effectively is summarize and even analyze and even make probabilistic decisions for us using data that we provide to it. And so, like what happened years ago, I think in undergraduate medical education, we need to similarly evolve our curricula to reflect this new era in which our students, of course, still need to learn the fundamentals, that will never change, the fundamentals of how to be a doctor, but more quickly, we can move our students toward doing even higher levels of cognitive analysis, higher levels of understanding the individual patient nuance, which I think might still be difficult for AI to handle, higher levels of compassionate and culturally competent communication, which we know AI might have some difficulty with, and returning students to the primacy of the physical exam, which as far as I know, chatbots are not gonna be replacing in the next few years. So in other words, what I point out in my viewpoint is that we need our students to be even more human in their doctoring skills than ever before, working at the highest levels of cognitive analysis, the most personally nuanced forms of communication, and remembering the importance of the actual laying on of hands.
- So wonderful. I like that's how you end this editorial, to be even more human. Let's take those three elements together. So we already know that things like ChatGPT take tests pretty well, but you're challenging us that the medical student then of the future will need to be not just about building this fund of knowledge, but have to be even better at integrating and figuring out how to synthesize that knowledge for the types of clinical decisions that are being made. So they might rely on ChatGPT for a first order, but they have to be even better using their human skills at figuring out how to integrate that for the patient. ChatGPT also does, it communicates, it seems to be very good at being empathetic, but you're challenging us to be more human in our communication skills that we're teaching medical students and then the physical exam, which presumably, these gen AI tools are not gonna ever be able to do.
- Right, right, exactly. So as an example, we know that ChatGPT, when given a set of signs and symptoms, can produce a fairly good differential diagnosis, and that's something that we still need to teach our students, but maybe more quickly than before, we can move our students to a level at which they're working with that differential diagnosis to make it individualized to their particular patient, to take into account some of the particular nuances and specifics of their patient's history or their patient's lived experience, that ChatGPT really can't take into account. And that's where their role, as medical students and future doctors, can be most useful. We wanna train our students to be the physicians of the future, who are going to be AI enabled physicians. Artificial intelligence is not gonna replace physicians, right, but physicians who use artificial intelligence are really gonna be working at the top of their game in clinical medicine. So we want to train our medical students to be those physicians, to be the physicians, who in a clinical visit, can really focus on that interpersonal interaction, to really get to know their patients as human beings, to be that compassionate provider and to do the most incisive levels of clinical decision making, while AI is presumably running in the background and doing some of the lower level tasks that otherwise would've occupied those physicians in the past. So, it's not that we don't need to start from the basics. Of course, medical students need to start from the basics, but I believe they can move more quickly from the basics to more advanced levels of reasoning and communication, knowing that they'll be supported by AI in the future to do the fundamentals of decision analysis and communication.
- Right, it just seems, what I like about your piece so much is it seems like this natural evolution. We used to never think we would have calculators in exams, or we would never have sort of these other types of catalogs of medical knowledge as a part of our exams, and now we routinely accept them because we want our clinicians to be functioning at much higher levels and that higher order thing. So it resonates so much for me what you've written. Now, how do we do that? How are you gonna do that at Harvard Medical School?
- So our students learn to take a history, they learn to perform a physical exam, and then they learn to write a proper clinical note, and then we ask them to do it over and over and over again, to basically instill that into their minds and show us with confidence that they can automatically generate solid clinical notes that are interpretable by other providers. Well, we know that ChatGPT writes pretty good clinical notes, if given the right inputs, so I'd love for our students to learn the basics of history and physical exam and writing clinical notes, but instead of spending as much time as we do now writing notes over and over and over again, we can move our students to higher levels of analysis and interpretation earlier in their medical education.
- So you're not talking about replacing the building blocks, you're talking about moving more quickly through those to get to the point that we're actually focused on, still the big gap between what we learn in medical education and what we need as a practicing physician, to get to that point more quickly. How do we do that? Where do you see the changes are gonna be in the first few years of medical education? Are they gonna be in the later years? Do you have a sense? I recognize you're a new dean, but do you have a sense of where, what type of things will look different as we go into the future?
- Honestly, I think every phase of medical education is gonna look different, and every phase of undergraduate medical education, from pre-clerkship curriculum to the clinical clerkships to the post clerkship phase, I think everything is gonna look different. In the pre-clerkship curriculum, for example, in the classroom, in the basic science and social science courses, AI is gonna be present as an educational tool, right. It's already happening. Our students are coming in now, the entering class of first year students, many of them were college seniors when ChatGPT became available last academic year, and they are now going to be using ChatGPT, GPT four, and other tools like that to help them learn, to help them preview the material in our flipped classroom environment, to help them consolidate the knowledge afterward and to study the content for exams. So, that's already present. They're using it as an educational tool. They can use it to serve as a self tutor because you can ask ChatGPT to serve as a tutor, to test you with certain questions, to alter the level of difficulty of the questions based on your responses. So already, that's right from the beginning of medical school. In our clinical skills course, where we're preparing our students to be able to have the fundamental building blocks to go onto the clerkships, again, I think we're gonna be able to use the fact that ChatGPT is available to move our students to those higher level clinical skills earlier on. And then I think after the clerkships, as students are exploring sub internships and electives and thinking about applying to residencies, of course, this is gonna be two or three years from now for our entering students, we'll have two or three years of generative AI under our belts, and I think it's gonna play a very large role in our students' experience, as they actually, as sub interns, lead the care of their own inpatients on teams, on the wards. We're already seeing that among our house officers, who after all, are some of the primary teachers of our students. Among our residents and fellows, ChatGPT is being used to help refine a differential diagnosis, to make sure that nothing is missed, to help with some of the difficult or rare conditions, that we know that AI can be very helpful for and that sometimes humans have cognitive biases against. And so, as our students see this being used by residents and fellows and attendings on the wards, that's gonna be part of their education. because AI is gonna be part of the future of clinical medicine. And just like any aspect of clinical medicine, as medical school leaders, we need to adapt what we're teaching and how we're teaching to prepare our students for clinical medicine in the future.
- So when these types of generative AI tools first came into prominence or awareness, educators, whatever level of education they were involved with, had to scramble because their students were using them, and they were figuring out how to put the right types of guardrails, set the right types of rules. Are there rules right now that you're thinking for or are there danger zones that you're worried about having this, embracing AI as a part of medical education, that you want to make sure that mm, we're not moving into these danger zones or that we're not replacing students actually thinking through material themselves?
- Absolutely, and I think there's quite a number of these, and this is a focus that we're embarking on right now, because as exciting as the future is and as much potential as these generative AI tools have, there are also dangers and there are also concerns that we have to address. One of them is helping our students, who like all of us, are still new to this within the past year to understand the limitations of these tools. Now, these tools are gonna get better year after year after year, but right now, the tools are still prone to hallucinations or basically making up facts that aren't really true, and yet, saying them with confidence. And our students need to recognize why it is that these tools might come up with those hallucinations, to try to learn how to recognize them, and to basically be on guard for the fact that just because ChatGPT is giving you a very confident answer, it doesn't mean it's the right answer. And in medicine, of course, that's very, very important. And so that's one, just the accuracy and the validity of the content that comes out. As I wrote about in my viewpoint, the way that these tools work is basically a very fancy form of autocomplete, right. It is essentially using a probabilistic prediction of what the next word is going to be, and so there's no separate validity or confirmation of the factual material. And that's something that we need to make sure that our students understand. The other thing is to address the fact that these tools may inherently be structurally biased. Now, why would that be? Well, as we know, ChatGPT and these other large language models are trained on the world's internet, so to speak, right. They're trained on the non-copyrighted corpus of material that's out there on the web. And to the extent that that corpus of material was generated by human beings, who in their postings and their writings exhibit bias in one way or the other, whether intentionally or not, that's the corpus on which these LLMs are trained. So it only goes to, it only makes sense, that when we use these tools, these tools are going to exhibit potentially evidence of bias, and so we need our students to be very aware of that. As we have worked to reduce the effects of systematic bias in our curriculum and in our clinical sphere, we need to recognize that as we introduce this new tool, this will be another potential source of bias. You alluded to the fact that we need to help our students understand what the right use of these tools is, so is it okay to use these tools to write an assignment? Is it okay to use these tools to help yourself study? Is it okay to use these tools to draft that clinical note, that writeup on the patient that you just saw in your doctoring course? Well, we need to establish a set of educational policies and so forth to make sure that our students are still learning the basics and are using these tools as aids to their education and aids to their work. You use the calculator analogy before. A student who started with a calculator right at the beginning might not ever learn basic arithmetic, but once you've learned basic arithmetic, we want you to be able to get on to learn more advanced forms of mathematics by using a calculator to help you take care of the basic arithmetic more quickly and without as many errors. And that's what we need to tell our students about for ChatGPT. We still need you to learn those fundamental doctoring skills, but once they are learned, these tools will help you to move on to more advanced skills more quickly.
- Yeah, I like that. I mean, it does strike me that even language models that are not trained on all the world's information on the internet but are just trained on healthcare data, the challenge is that it risks learning the patterns of bias and not optimal care that we in medicine have delivered for our patients. One of the things that I've loved being part of an educational institution, and I'm sure, I suspect the same is for you, is teaching sort of the next generation of clinicians to imagine the world different than what we have currently created. It's good, but we also want it to be better, and that's part of the ambition, I think, for medical schools. And so, it feels like that's another, placing these tools in context as one of the tools, but not something that substitutes from imagining the world differently than currently exists.
- Absolutely, and I think actually that it seems a little bit counterintuitive to be talking about how we may be able to strengthen and return to more of that humanistic patient physician encounter in the face of a computational revolution here, but I really believe it. I'd love to see in the future our medical students and future physicians be able to spend more time at the bedside, more time looking at the patient and less time typing over on the keyboard, more time concentrating on the individual nuanced communication and making sure that everything is being understood and less time worrying about whether this or the other fact is gonna be recalled. And I'm imagining that after a clinical encounter, which is gonna be more face-to-face time and more effective and compassionate communication, then the physician will be able to turn to the screen, aided by AI, and have a note drafted, have a differential diagnosis listed, have laboratory results from the online medical record that might be relevant to the conversation, pulled up. Things that might have otherwise taken the physician quite a lotta time to search for in the past, maybe have letters to consultants already drafted, and knowing that that is running in the background should give me, as the physician, more opportunity and more leeway to spend time talking with my patient the way I really ought to have been all along.
- Yeah, I love it, and that is the way you end your editorial, with making us as physicians and those who train the physicians of the future to doing this, to connect back with our humanity, and to do this in a more human way in the face of this new technology. Okay, I have one more question for you. We have all of these wonderful applicants and entering students who have lived in this AI world, and then they're still being taught by the faculty, who , how are you gonna train us faculty members to be the faculty of the future?
- This is a great point, and this is a major challenge for us, but it's also a great opportunity. So, our faculty are excited to learn. They need to learn. They are not native to this. They need to learn, but they want to learn, because they see the potential for what this can mean for the education of our students. First of all, they see, frankly, the potential that this can make their work a little bit quicker and a little bit easier in terms of generating new content, generating assessment questions, finding ways to instruct the students and also to test the students on their knowledge. So in some ways, this is a tool that will make our faculty's job easier, but what we need to do in medical school is work on faculty development, and we're gonna spend a lot of this next year doing that. First of all, right now anyway, there's no better way to learn how these tools work than to try them, and that's been our mantra to our faculty, which is, just try it. Try it, try it using your current course materials, try it with what you're doing on the clerkships right now or what you're doing in the sub internships right now and don't just rely on the first response, right. And so, we've had a couple of different messages for our faculty. Number one is that generative is the key word in generative AI. This is not just a fancy Google search. Don't use it just to try to find some more obscure fact than you are gonna find on Google. This generates content for you. It writes human sounding text for you, so it plays its most important role at times when you need to generate, when you need to write, when you need to create text content for the page. And that's one thing we're telling our faculty. Another thing we're telling our faculty is the importance of what's now known as prompt engineering, which is knowing what questions to ask. It's funny because that's an old fashioned thing we tell our students on the wards, right. That when students say, "Well, that patient was a poor historian." Well, in fact, perhaps it's because you didn't ask the right questions, and it's the same thing with these generative AI tools. The quality of the prompt that you give it is proportional to the quality of the response that you're gonna get, and so we have to become better at generating specific prompts that actually are we know are gonna elicit the kinds of responses that we need that are gonna be most helpful. And I always tell our faculty, "Don't just stop with the first response." The first answer that you get back is just a first draft, and you would never accept a first draft as your final version. So if there's something that you don't like about what came back, if there was an error, or if it was not quite what you were looking for, the beauty of ChatGPT, here's the chat part of it, is that you can converse. You can say, "Actually, "that's not quite what I was looking for. "I was expecting X and Y and please don't do Z again." And then it'll come back with another response, and in my experience, after just two or three iterations, you have something that's much closer to what you intended than maybe the first try. So these are some of the simple mantras that we've been spreading to our faculty as they're learning, and we're all learning how to use these tools in our educational program.
- Well, you are so enthusiastic and so clear in your expression of the potential for these tools and just the reality for these tools, but medical school faculty are a hard bunch, so I hope you can get them on board.
- It's been a real pleasure, Bernard, thank you so much for joining me today.
- Thank you so much, Kirsten, I really appreciate it.
- And to our audience, thank you for watching and listening, as well as giving us feedback on this series. We also welcome submissions in response to JAMA's AI in Medicine call for papers. Until next time, stay informed and stay inspired. We hope you'll join us for future episodes of the AI and Clinical Practice series. For more videos and podcasts, subscribe to the JAMA Network YouTube channel, and follow JAMA Network Podcasts wherever you get your podcasts.