General Surgery Residency: Reimagining the Selection Process

Podcast Transcript

Dr. James K. Stoller:

Hello and welcome to MedEd Thread, a Cleveland Clinic Education Institute podcast that explores the latest innovations in medical education and amplifies the tremendous work of our educators across the enterprise.

Dr. Tony Tizzano:

Hello, welcome to today's episode of MedEd Thread, an education podcast exploring what's new in the selection process for graduate medical trainees. I'm your host, Dr. Tony Tizzano, Director of Student and Learner Health here at Cleveland Clinic in Cleveland, Ohio. Today, I'm very pleased to have with me Dr. Jeremy Lipman, a colorectal surgeon and director of our Graduate Medical Education Program at Cleveland Clinic. He also serves as our designated institutional official responsible for oversight of all of our accredited residency and fellowship programs. Jeremy, welcome to the podcast.

Dr. Jeremy Lipman:

Thanks very much. Happy to be here.

Dr. Tony Tizzano:

Also joining is Dr. Judith French. Judith is our educational specialist in the department and Vice Chair of Education at the Department of General Surgery. Judith, thank you for coming.

Dr. Judith French:

Thank you very much for having me.

Dr. Tony Tizzano:

And finally, Dr. Pooja Varman. She is one of our general surgery residents who is currently a surgical education research fellow joining us as well. Pooja, we appreciate you coming. Thank you so much.

Dr. Pooja Varman:

Thank you.

Dr. Tony Tizzano:

So to get us started, would each of you tell us a little bit about yourselves, your educational background, what brought you to Cleveland, and your respective roles here at Cleveland Clinic? Jeremy, we can start with you.

Dr. Jeremy Lipman:

Sure. I am a colorectal surgeon, as you mentioned, and I started in education as the clerkship director for the general surgery core clerkship when I was working over at Metro. And then just as I'd made a life decision that I was going to pursue undergraduate medical education as a career plan, I got a call from the Chair of Surgery here that they were looking for a Program Director. And I got to have lunch with Dr. French and that was it. I was hooked and came over here in 2016 to be Program Director for the general surgery residency.

Did that for six years, really enjoyed it. And then, when Elias Trabolsi retired as designated institutional official and director of GME, I applied for and was thrilled to get his position, which is what I do now. But still get to work with all my general surgery friends from time to time.

Dr. Tony Tizzano:

Fabulous. And we're lucky to have you. I will say that much. Judith, what about you?

Dr. Judith French:

It was a heck of a lunch that we had with Jeremy there. So I was a high school biology and anatomy teacher in the Detroit area before I got into medical education. I went on to get my master's and PhD in science education. So my background is actually in teaching people how to teach science. And I was looking for a postdoc after I finished my PhD. I was living in Lexington, Kentucky, and I saw that there was an opening in the Surgical Education Office at the University of Kentucky, and I was like, "Let me give this a try." And sure enough, I was hooked and I ended up seeing a position open at Cleveland Clinic, and I started here back in 2013.

Dr. Tony Tizzano:

It's amazing to me how teaching has evolved. And you speak to that, the fact that you have an entire degree formed around, how can we do this in the best way possible? And I think it's really important in medicine where we have a tendency to do things in the way we've always done them. And so thank you for what you put into our efforts. And, Pooja, please.

Dr. Pooja Varman:

Hi. So I am a general surgery resident, like you mentioned. I matched at Cleveland Clinic back in 2022. I was always really interested in education, but didn't have formal background in it. I just took opportunities to teach when I could. And when I found out there was a surgical education research fellowship that's offered as a two-year fellowship in the middle of our general surgery training, I was really excited to apply for that. And so I have been in that role for the last two years. And during that time have been doing research projects, recording episodes with Behind the Knife's Education series with Dr. Lipman and Dr. French, and pursuing my master's in health professions education.

Dr. Tony Tizzano:

Excellent. Well, to all, we are so happy to have you. So to get us started, Pooja, if you might help frame today's topic by providing our listeners with some context around the challenges of trainee evaluation and selection, how the process occurred in the past, what are the changes that are being considered to ensure the best fit for trainees into programs anywhere?

Dr. Pooja Varman:

So the process for trainee evaluation and selection for most programs, including general surgery, falls under the umbrella of the National Residency Match Program, or the NRMP. So they created a match algorithm back in the 1950s to help manage the basically numbers problem of, how do we get medical students who are about to graduate, how do we maximize the matches between students looking for jobs and programs looking for PGY1 residents? So they created this algorithm that has worked very well. It's a very efficient system to give everyone the best match that's possible. Over the years, some of the challenges that have arisen stem I think in part from how many students there are, how many programs there are, the volume has really increased, the competitiveness of applicants and programs has increased.

In 2019, when we all started experiencing COVID and everything went virtual, there was a big shift to this whole process going virtual and an explosion in applications that has happened in the years since. And so the numbers problem I think has gotten even more difficult and there's changes in the kinds of metrics that we actually use to evaluate trainees. So there's been a trend to take a more holistic approach, to consider more than just numbers that an applicant is submitting on their application, and more so the broader list of experiences and background that they bring, as well as shifts in education more broadly in terms of competency-based education. And so there's a lot of things that go into the match and it's something that we are continuing to evolve how we address training physicians.

Dr. Tony Tizzano:

Well, I think evolution of this is the word. I think there has been tremendous change. Jeremy, what inspires and compels you and other GME programs to consider a more holistic approach, perhaps a more novel approach to resident recruitment and selection?

Dr. Jeremy Lipman:

The volume of applicants to programs is really enormous and it requires a very thoughtful way of filtering out who are the right people to be looking at for the right spots. I think a common trap that programs and applicants get into is this idea that there is a perfect applicant or a perfect program, and it's one that's right for everyone, and everyone just needs to try to get there and to get it, looking at it from both sides. But the reality is that each program has its strengths and weaknesses, just like every applicant does. Their learning preferences, their teaching styles, all these things need to be considered to really make a great match between the applicants and the programs.

And if we're only looking at things from one lens like standardized exams, like USMLE, we're missing out on a lot of those other characteristics that are really critical to providing great training and for a resident or fellow to be getting great education. So it's imperative that the programs take this opportunity to reflect on, what are they good at? What do they really offer to their trainees? How do they teach? Where do they thrive? And what are the things that they're just not able to do? And to be honest with that in thinking about who are the applicants that they want to look at that are going to mesh well with what they can do. And same thing on the other side.

Dr. Tony Tizzano:

So you said the numbers are daunting. What kind of numbers are we talking about?

Dr. Jeremy Lipman:

So if you look at nationally for all specialties, this past year, in 2026, there were over 50,000 applicants who submitted a total of four million applications to different programs.

Dr. Tony Tizzano:

Oh my God. That's unbelievable.

Dr. Jeremy Lipman:

Each applicant is applying more than once, obviously, to different programs.

Dr. Tony Tizzano:

And since these are being done virtually now since COVID perhaps, has that increased the number because you don't have the travel expense and so on?

Dr. Jeremy Lipman:

Yeah. I mean, not everyone is still doing virtual. During the virtual times of COVID, it definitely made it easier for applicants to expand where they're applying and for programs to offer more interview days. There's not much expense with setting up a Zoom meeting and having everyone come. It can be very flexible. You can do it at different times for the different coasts. Takes a ton of expense out of the applicants, so they have the opportunity to apply more broadly and to interview more broadly.

Dr. Tony Tizzano:

Well, that begins to let us all see just the magnitude of the problem. So Judith, listening to your background, is there a science behind the contention that a more holistic approach might help identify and deliver a cohort of more diverse trainees who perhaps possess greater empathy and readiness or ability?

Dr. Judith French:

So holistic review is not something that's completely new to education. This is something that's been going on for a really long time. Medical education, however, obviously, during COVID and those times after picked up on it. But in the other specialties, in the other areas, holistic review has shown to be able to filter the candidates that are from all backgrounds, the candidates that do match with those characteristics that specialties are looking for. As Jeremy was mentioning, we have to start out with the programs. Know thyself. What is it that you want from your candidates? What can you offer your candidates? So, for example, if you're a community program and you're screening people based off of their research that they have completed in medical school or even before, what are you doing? Because you know that that's not going to be something that you're going to be able to help them with and to focus on. It's not going to be a focus for your program.

So we really need to think about that initial step for all the programs, is understanding who they are and what they're going to be offering and be able to then use that to identify their candidates. If you can start with that, then the science behind the holistic reviews really starts to fall into place. And the other thing to think about, before we started doing holistic reviews, it's not like the metrics that we were using had a ton of scientific evidence behind them to say that, "Oh, this is great. You need to look at USMLE step one scores." Or, "Hey, the personal statement can really tell you about your candidates. You really need to put a lot of weight into that."

So the things that we were using before we got into holistic review, there wasn't a lot of evidence to solely use one thing over the other. You have to look at the broad application and all the different pieces and parts to be able to truly get a sense for who these candidates are, what they're bringing to your program, and what you can offer them as a program.

Dr. Tony Tizzano:

Is that similar to what we've perhaps seen with the MCAT where while it may predict what you do in your first and second year of school performance with basic and clinical sciences, once you get to clinical experience in your clerkships, that all of a sudden takes lesser importance? And is it still not looking at this cohort when you initially put them in front of you and say, "Okay, if they don't have at least this score, we're not going to look at them"? Does that still happen?

Dr. Judith French:

Now we don't have step one scores anymore, now we have step two scores. So they can't look at step one except for pass/fail, but there are some programs out there that will still utilize step two. But the problem is like what you were going back to with the MCAT, you're looking at a knowledge base like medical knowledge. Okay, you can measure that on a multiple choice question type exam, but yet you're also asking people in their clinical years to work as a team, to communicate with your teammates, to work with patients, and to understand the system. You're going beyond.

Start thinking about those ACGME six core competencies. Maybe you're covering one of them with your multiple choice test, but all of those other ones you're missing out on and it becomes exceedingly difficult to be able to say from that multiple choice exam, "Oh yeah, they're going to be great. They're going to be a great doctor in their third and fourth year."

Dr. Tony Tizzano:

So, Pooja, you're in the midst of this and you're close to the time when you were applying. And students will often ask, "What is the value of me doing an audition rotation? I really would like to go to this program. I'd like to learn more about it and get an up close and personal look." But that of course goes both ways. You get to look at them, they get to look at you. What do you think the value of that is and what would your recommendations be to someone who would ask you that question?

Dr. Pooja Varman:

So I think if you had asked me this when I was a student applying to the audition rotations, it just seemed like a no-brainer. If you want to go to a program, you should try to do an OA rotation with them. I think that having a few years of experience as a resident now, I think that I see it more as a high risk, high reward type of endeavor where I think that we have students who rotate with us on general surgery who are total stars. They impress all of the attendings that they work with. They impress all of the residents that they work with. And when application and interview season comes around, they really stand out in people's minds as this person was very clinically on top of things, they were great to work with, we really want them here. And I think that that's awesome. That's an argument to do the audition rotations.

I think that what people maybe don't consider, is what happens if you're not a total star? What happens if you're neutral? What happens if you're a fly on the wall, you don't really stand out in a really positive way, people don't really remember your name or your face after you spent a whole month rotating with them? And the worst end of that spectrum is, what if you stood out in a bad way? And people didn't like working with you. They thought you were condescending. They thought you couldn't answer questions in the OR. So I think that's a lot of time to spend with a program that you're really interested in and it only helps if you are making a positive impression, and things that are neutral to negative can actually really hurt an applicant, I think.

Dr. Tony Tizzano:

Boy, that is such an interesting perspective. And you could also be a superstar and all of a sudden have something in your personal life, a family member becomes ill, or something else that's distracting you and you're not able to give the attention you normally would. And all of a sudden, well, gosh, they weren't on time today. Well, I was talking to my mom, was in the hospital, or something like that, that, "Oh yeah, I understand. Thank you very much, but they weren't here." And you remember that they weren't here.

So I think that's important for individuals who are considering it to really think, "Okay, I have to bring my A-game and be everything I can be, and it can go both ways." That's interesting. So, Jeremy, despite all these innovations aimed at enhancing the selection process, how do programs maintain the rigorous due diligence necessary to bring really strong candidates and get the very best that we can in the end game of good patient care?

Dr. Jeremy Lipman:

Yeah, it's a complicated situation. And even in the case where someone has done an audition rotation and maybe really shined, as Pooja was alluding to, it's possible that they just were able to sprint for that month. And the reality is that's not who they are. When you're talking about long residencies, as all of them are, but specifically surgery and some of the procedural specialties, that can fizzle out. So it's tough to know. We rely heavily on the medical schools to tell us how the students are doing. And unfortunately, much of the information we receive has some bias and is designed to favor the students. The schools are under enormous pressure to help their students to succeed and to get into the residencies that they want. And so there's not a lot of incentive to share information that might detract from their application.

Dr. Tony Tizzano:

I can see that because schools want to say, "Look, we get into these very competitive programs. This is our percentages getting in." So there is an intrinsic push to say, "Boy, I've got to be able to write a good letter." And students, of course, do anything to get that.

Dr. Jeremy Lipman:

Yeah. And the letters are almost universally favorable. And when they offer even very mild detraction, it's taken as a red flag because all of the other letters are so glowing. There was a paper some years ago that, When Good Isn't Great, I believe was the title, that if someone is listed as being good or being in progress with something, which is exactly what you'd expect for a student who's developing their skills, I mean, forget it.

Dr. Tony Tizzano:

Interesting. Interesting. So Judith, have you had any experience with this idea of virtual evaluation tools in the trainee vetting process? And what exactly is that?

Dr. Judith French:

So we have looked at utilizing AI for some of the application reviews, specifically using it for personal statements. And there's been other research out there looking at letters of recommendation. And in terms of AI use, you have to be really specific with your prompt, for sure. Outside of that, there have been, at the medical school level, for years they were doing the MMIs, was it the multimodal interview type thing where you bring your candidates into a room and you give them a scenario and they have to try and act it out. I believe one of the ones I remember was you are going to the airport with a colleague and you show up at their apartment to pick up your colleague and they're afraid to fly and they won't come out of their apartment. And you have to figure out how to try to get this individual to get in the car so you guys can go to the airport and go for your research presentation, or whatever it was.

So I mean, I think that there is something there, but the problem is that, what validity evidence do we have behind those things? So if you're going to do that, if you're going to collect that data, if you're going to use that information, what do you have to show that it's truly testing the thing that you want to test and that it's not something else? So yes, in that instance, in a simulated environment, you could see that someone is able to communicate with their colleagues. Is that going to translate though? Is that going to have some sort of impact on a communications outcome with a patient? So I would love to see some of these things, but as the PhD, show me the evidence. Show me the validity behind it. I want to know that you're measuring what you say that you're measuring.

Dr. Tony Tizzano:

So this is a work in progress, for sure. So for Jeremy, Judith, and Pooja, any of you, in an earlier conversation, we made mention of this recent work being done around signaling. What exactly is this and how are you and other specialties looking at it and regarding it?

Dr. Pooja Varman:

So signaling is something that was implemented by the AAMC a few years ago, basically giving applicants a packet of stickers. In general surgery, at least at the beginning, they had five gold stars that they could hand to the five programs that they're most interested in matching at. And this was supposed to basically help to flag those applicants who have particular interest in a program. Out of the 20 to 100 applications that they may have submitted, they want to say, "I really want to highlight myself for these particular programs." This was supposed to help programs also to basically interpret who has real interest in our program and who's sending out a vast net to see what programs they can get interviews at and is supposed to help in both directions to really discern genuine interest.

Dr. Tony Tizzano:

Now, do other programs get to know that they've starred them and perhaps other programs as well? So a program who they didn't star, would say, "Oh, well, they starred somebody in Cleveland Clinic, but they didn't star me." Do they know that?

Dr. Pooja Varman:

So the programs won't know the other programs that receive signals from that applicant, but for each applicant, they'll just know, "Did that applicant send me a signal or not?"

Dr. Tony Tizzano:

Gotcha.

Dr. Pooja Varman:

And knowing that applicants have access to now 15 in general surgery, if they didn't get a signal, then they know I'm not in the top 15 programs for that applicant.

Dr. Tony Tizzano:

Gotcha. Judith and Jeremy, when you see that star, does it make you take a harder look?

Dr. Judith French:

The first time that we did this, when we saw that, we didn't know what to do with it and we're like, "Oh, well, they signaled this. Okay. That's nice. What do I interpret from that?" So the first year we really didn't do too much with it. Now that we have the 15, we have a few more years under our belt, we do consider it. We do take a look, especially for people that are on the cusp of being offered an interview. We don't take it any further than that. But for people that are borderline, we will consider it.

Dr. Tony Tizzano:

Okay. So at the end of the day, this is for all of you, how does one weigh and measure the success of your selection process? How do you look at the way you're doing things? And we did well this round or we didn't do well, how do you even look at that?

Dr. Jeremy Lipman:

I think the outcomes take a long time to see. We don't really know until they're here, but most of the people that are here do well. And we don't really know the true outcome until they're out in practice and doing the work. Were they the right type of learner to come into our learning environment to get what they need to go out and be an incredible independent provider, which is what we want? So we survey our graduates to see how they're doing and that becomes our metric of success, but that's five, 10, 15 years from when we pick a spot for them on our rank list.

Dr. Tony Tizzano:

And a subjective, if it's coming from the person who was in the job. So that's interesting. Do you ever go to the person who's employing that individual in the cases where you're employed to see how someone's doing? Is there a way to look at that? It would be a different [inaudible].

Dr. Jeremy Lipman:

Well, we try to look at some of the objective data such as for programs that are academically oriented like most of ours are, do they have publications? What academic rank are they holding? Are they teaching? Are they leading research? Do they have grants? Those types of things. Do they have leadership positions in their departments? So all of those things. And we ask them subjectively, "Are you in the job you want? Are you happy with where you landed? Is this where you thought you'd be? Are you practicing in the specialty you thought you would?" So all of those.

If someone's happy with where they are and objectively succeeding in the academic community, I think that's a pretty good marker that they're doing well.

Dr. Tony Tizzano:

Gotcha. I hope among our listeners there are persons who are looking at applying for residencies and fellowships and thinking, what is it like to look at this from the other side? So for all of you, what lies on the horizon? What are next steps in the evolution of graduate medical education selection processes?

Dr. Judith French:

I think one of the things that I've already mentioned is, how can we utilize AI to help? Because when you have 2,000 applications, you do want to go through with them and give everybody an equal opportunity. But right now it is difficult to do that with getting AI to understand the nuances of the language that it's used in the applications.

Dr. Tony Tizzano:

So it's got more to learn. But it's learning, isn't it?

Dr. Judith French:

Yeah. I mean, as each iteration of these AI models come out, they are getting better and better. Still not quite where we need it to be, but I think hopefully before I retire, we'll have it figured out.

Dr. Tony Tizzano:

Yeah. I'm astonished even on my end when a patient calls and asks a question that when I go to answer it, there's already an AI-generated potential answer to the question, which initially I didn't realize was coming from AI. We thought one of the nurses was writing this and I go, "Well, wait a minute. Yeah, I agree with some of this, but not all of it." But it's getting closer. It's getting closer. Are there questions? Anyone else? Pooja, would you add to that? Or Jeremy, the horizon down the road?

Dr. Pooja Varman:

I would ditto the answer about AI. I think across probably all of our sectors we're thinking about how AI is going to transform our field.

Dr. Tony Tizzano:

It's making me take a second look, for sure.

Dr. Pooja Varman:

Mm-hmm.

Dr. Jeremy Lipman:

I think we're starting to see programs becoming more introspective and really understanding their capabilities and capacity and being more thoughtful about what makes a good applicant for their program rather than what is a good applicant overall. And that is probably the place where I most see the AI fitting in, is helping to see the things that perhaps the programs don't see about themselves and the applicants don't see about themselves to link them together and really try to get to a great connection.

Dr. Tony Tizzano:

I think it's great to see that we're taking a different look, whatever that look is, to consider other pieces that may really be important parts of this puzzle. And I remember, and I won't identify the institution, but I know it's one that all of us would hear and respect, and the dean of the medical school said, "We actually take a really hard look at blue collar backgrounds because we believe that these individuals not only come to our institution with a likelihood that they may stay in this region, but also that they have a resilience and resolve that we don't always otherwise see, which how do you weigh and measure that?" I just thought that was such an interesting statement.

Well, are there questions I should have asked that I didn't that you feel are important for our listeners to know?

Dr. Pooja Varman:

It's not a question, but I think something that I've reflected on, I think especially through the conversations I've had these last couple years with Dr. Lipman and Dr. French is just how much of this process is so dynamic and seeing the perfect fit thought as a fallacy and that really people change a lot over time. And medical students who are applying right now and might have a really strict definition of what program is totally perfect for them, I think can be transformed even after they get to their program by their clinical interests can change, their career goals can change, their family goals can change. And I think that having a more holistic view of this and to let go of our grasp of there's a place that's totally perfect for me could probably help people to think more open-mindedly about a place that they're going to grow for the next three to eight plus years.

Dr. Tony Tizzano:

Well, I thank you all for your transparency around this. I often say, when I look at persons in healthcare, I really don't really concern myself right off the bat with how much they know, but how much they care. And it's clear in your responses that you really care about this process and I hope that comes through to our audience. And I want to thank you each so much, Pooja, Judith, Jeremy. This has been enlightening. It's been a wonderfully insightful podcast.

To our listeners, if you'd like to suggest an education topic to us or comment on an episode, please email us at education@ccf.org. Thank you very much for joining and we look forward to seeing you on our next podcast. Have a wonderful day.

Dr. James K. Stoller:

This concludes this episode of MedEd Thread, a Cleveland Clinic Education Institute podcast. Be sure to subscribe to hear new episodes via iTunes, Google Play, SoundCloud, Stitcher, Spotify, or wherever you get your podcasts. Until next time, thanks for listening to MedEd Thread, and please join us again soon.