Today was the day that I finally cleared out some of the Sheriff of Sodium’s e-mail, which has been somewhat voluminous lately. Although I try to respond to all messages individually, some questions have come up frequently enough that I decided to answer them publicly.
To maintain confidentiality, I removed any identifying details from these messages, and I took the liberty of editing for brevity and clarity. Still, the questions here are all based on real e-mails from real readers. So with no further ado, I present to you the inaugural Sheriff of Sodium Mailbag.
–
Today I received an e-mail from my dream program inviting me to interview. When I clicked on the link 20 minutes later, all of the program’s interview slots had been filled. Somehow this is more disappointing than if I hadn’t been offered an interview in the first place. Why do programs do this?
Dude, I don’t know.
I’ll tell you what I do know: there is no good reason for this practice; program directors would never stand to be treated this way themselves; and when I gave a talk to a group of PDs last spring, I urged them to stop doing it.
From YouTube.
Disappointing as this must be, I think this is a time when applicants really need to take the long view and consider what the program is trying to tell you when they do this kind of thing.
Let me translate.
They’re telling you that you’re completely replaceable; that there is nothing special about you personally that makes you a better fit for their program than some rando with better Wi-Fi; that the years of work that went into the accomplishments noted in your application matter less than having immediate access to e-mail; and that they have no issue with instilling panic in applicants every time their e-mail notification goes off so as long as it allows them to fill their interview slate a few minutes sooner than they otherwise would have.
If a program director treats you like this now – how do you think they’ll treat you when you’re actually in their program?
–
I’m having a hard time getting a feel for what a residency programs are really like based on just a virtual interview. Do you have any suggestions or advice?
This is a tough one – but yeah, I do. And it boils down to this:
Don’t tell me – show me.
Let me explain what I mean.
If you ask whether residents and faculty get along, or if residents’ schedule requests get met, or there are adequate opportunities for procedures, or anything else, whoever is interviewing you knows that the correct answer to your question is “yes.” You can’t just let them tell you – you have to ask them to show you examples.
You say your program has good teachers? Then I want to see their lecture – is it online? Better yet, show me how the institution prioritizes and rewards good resident teaching. Because if faculty with NIH research funding get promoted, while faculty who teach get a thank you certificate at the end of the year awards ceremony, guess what the faculty will focus on?
You say your residents pass the boards. Then show me the question banks and prep resources the program provides. If you struggle to come up with these things, I’m going to wonder whether that high pass rate actually reflects added value – or if you’re just slapping yourselves on the back for selecting residents whose USMLE scores predicted a high probability of board passage regardless of their program experience.
You say your program has a commitment to diversity and inclusion. These days, everybody does – according to their website, at least. So show me your residents. Show me your program leadership and division chiefs and department chairs and deans. If all I see is a bunch of old white dudes, your commitment to diversity is either newfound or insincere.
You say your institution prioritizes good patient care. Sounds great. So show me examples of when your hospital has placed its dollars behind something that benefitted patients more than the hospital’s bottom line. If instead I see residents under pressure to rush patients out of the hospital in order to meet ‘discharge-by-noon’ initiatives, it’s clear your focus is on patient care quantity, not quality.
You say the hospital serves the community. Well, that’s the kind of place I want to work – so show me how you’ve invested in services to meet the needs of people in your area. Because when I see your hospital’s logo advertised on every throwaway journal in my mailbox, it kinda looks like your priority is improving your U.S. News & World Report ranking more than serving the people in your community, and I might even start to wonder how many patients you could have served with the money you spent on consultants and glossy annual reports.
You say you support residents. I hope so. So it shouldn’t be hard for you to show me examples of instances when you’ve helped a resident who was struggling. And I don’t even want to hear about how your residents don’t struggle – because a real residency training program takes the word ‘training’ seriously, and takes pride in their ability to build out weaknesses, develop, and even remediate a physician who needs extra support.
I think you get the idea.
My point is, don’t settle for blanket platitudes – dig deeper. You may be surprised by what you’ll find.
–
I applied to 70 programs and received 3 interview offers. A classmate applied to 50 programs, got interview offers everywhere, and now has 42 interviews scheduled. What can be done about interview hoarding? Will you please advocate for interview caps?
Folks, don’t say I didn’t warn you.
Remember, even before COVID-19, we had a distributional problem with interviews, with 12% of applicants taking 50% of interview slots in specialties like internal medicine or general surgery. So back in May, just after the virtual interview season was announced, I predicted that the lower opportunity/financial costs of virtual interviews mixed with the uncertainty of the COVID-19 pandemic would lead to highly-qualified applicants choosing to “overinterview.”
Obviously, I wasn’t the only person to predict this. Back in July, the National Resident Matching Program (NRMP) – an organization with a knowledgeable leadership team and a Nobel laureate economist on retainer – quietly added an extra round to this year’s Supplemental Offer and Acceptance Program (SOAP). They didn’t do it for funsies – they did it because they thought we were gonna need it.
Meanwhile, many other students, residents, and faculty tried to sound the alarms and get people thinking about the possibility of capping interviews.
One clever study used data from last year’s OB/GYN Match to predict how virtual interviews might change the number of interviews that various groups applicants received this season.
To do this, they divided last year’s applicants into two groups – those who received 12 or more interview offers, and those who received fewer than 12. Then, they modeled what would happen if those applicants were interviewing this season. They assumed that, unlike last year’s Match, when travel costs and the realities of the time-space continuum provided a natural limit on the number of interviews that an applicant would accept, applicants this year would choose to accept all of the virtual interviews they were offered.
The results were striking.
Without interview caps, the type of applicant who completed 12+ interviews in 2019-2020 was predicted to complete around 19 interviews in 2020-2021.
But when “top” applicants fill more interview positions, fewer are left over for applicants who don’t look as good on paper. Without any changes to the system, the model predicted that the average applicant who completed <12 interviews last year would receive an average of less than one interview apiece this year.
Interestingly, under the modeling assumptions the authors used, encouraging programs to offer 20% more interview slots didn’t help. Under this scenario, the top applicants completed an average of 22.9 interviews, and the less polished received still received just 0.8 interviews apiece.
Only capping interviews resulted in a more equitable distribution of interview offers. And if interviews were capped at the same time that programs increased the number of interview offers available, then both groups of applicants received an average of 12 interview offers apiece.
When interviews go virtual, the rich get richer and the poor get poorer. From Morgan HK et al, J Surg Educ 2020. PubMed
Capping interviews is technically feasible: some authors have already proposed workable solutions, like an “interview ticket system” to prevent interview hoarding. What’s less clear to me is whether interview caps are legal.
If a hard cap were imposed on the number of interviews an applicant could accept, someone who went unmatched might reasonably claim that this policy imposed an inappropriate restraint of trade and sue. Of course, just because someone files suit doesn’t mean that their claim would be upheld by the courts – but the mere threat might be enough to deter an organization from implementing such a policy.
From my standpoint, capping interviews is a step in the right direction – but it’s still a half-measure. I still say we need application caps, and we should have the fortitude to articulate the reasons why in court if need be.
Unfortunately, since we chose to pursue neither interview nor application caps this season, we’re all stuck riding this out. And yet, we may still have some hope… Zoom fatigue is real, and I have a feeling that many of the applicants snapped up 42 interview slots will end up cancelling many of them as the season wears on.
–
I’m a first year medical student, and some of my classmates are completely focused on USMLE Step 1 preparation. It’s freaking me out because I thought the test would be pass/fail by the time we took it – but my classmates say there is a petition circulating among students and program directors demanding a delay to pass/fail implementation. Is the USMLE really going to begin pass/fail score reporting in January 2022?
Relax. All signs point to a pass/fail Step 1 for the Class of 2024.
Since their initial announcement in February, the USMLE’s messaging on the transition to pass/fail scoring has been entirely consistent regarding the January 2022 target date. In fact, earlier this month, they reiterated that “pass/fail score reporting is on track to be implemented in January 2022.”
Pass/fail score reporting for the USMLE Step 1 exam remains on track to begin in January 2022.
Whether Step 1 results are reported as a three digit score or a P/F has implications for many people whose opinion the NBME cares about. If they intend to delay the start of pass/fail score reporting, they’ll begin to signal that soon in their monthly updates, so that everyone can plan ahead.
Notice my use of the term “people whose opinion the NBME cares about.” This includes a large and heterogenous swath of stakeholders (mainly in the state boards and ‘alphabet soup’ governance organizations) but notably does not include any group of aggrieved students or program directors circulating petitions arguing to delay pass/fail scoring at the eleventh hour. (It doesn’t include students or deans arguing for earlier pass/fail implementation due to COVID-19, either.)
The bottom line – barring some major snag like a state medical board that needs more time to amend their administrative code to accept pass/fail scoring for licensure – Step 1 will become pass/fail in January 2022 as planned.
My advice to current first year students is to study hard and learn as much as you possibly can – just do it for your future patients, not for Step 1.
–
I’m an MD/PhD student who took a scored Step 1, but I won’t apply for the Match until after the test goes pass/fail. How will programs evaluate applicants like me who have a score versus those who just have a P?
To answer this question, you have to understand how the Step 1 exam became the King of Residency Selection in the first place.
Did you ever wonder why we chose to prioritize scores from Step 1 – a test of basic science – over those from Step 2 CK, which tests material more directly relevant to the care of human patients?
It’s not because careful scientific study demonstrated that immediate factual recall of G-protein subunits or the various forms of RNA polymerase were necessary prerequisites for effective functioning in a prestigious residency programs. It’s just because of convenience.
Historically, most students took Step 2 CK in the late summer or early fall of their fourth year – which meant that they might or might not have scores available when ERAS opened in September.
Of course, if not all applicants have Step 2 CK scores available at the time you’re reviewing their applications, then you can’t easily deploy Step 2 CK score filters to weed out applications. The only thing you could use as a universal filter was the Step 1 score – and use it we did.
In other words, our idolatry of the Step 1 score stems from its utility as a screening metric – not its intrinsic value. And as soon as not every single applicant has a numeric score, the Step 1 spell is broken. Once programs can’t use Step 1 as an automated filter, it goes from being the golden ticket to just another metric that some applicants have and others don’t.
Applicants who have high Step 1 scores will continue to benefit from them, even after new scores are reported pass/fail. Meanwhile, applicants with low scores will still struggle to get certain programs to look past them. But the moment new results are reported pass/fail, the value of anyone’s score decreases substantially.
–
I’m an international medical graduate (IMG). To compete with US medical graduates, I *need* to have a Step 1 score, so I’m rushing to schedule my exam before the test goes pass/fail. If I take a scored test, will programs still be able to see my score even after other examinee’s results are reported pass/fail?
Despite their vagueness on this point for the first few months after the original pass/fail announcement, the USMLE has now clearly stated that Step 1 results will remain ‘as is.’
That is, if you receive a score when you take Step 1, that’s what programs will see when they review your transcript. If you receive a P or an F, that’s what programs will see, too.
USMLE Step 1 results will be reported to residency programs the same way they were reported to you.
So yes, if you take a scored Step 1, your score will be reported to residency programs in perpetuity. (Which makes it even more important that applicants carefully consider whether rushing to take the test is more likely to result in a score that will benefit – or dog – their application in the years to come.)
–
YOU MIGHT ALSO LIKE:
Virtual Interviews: Winners and Losers Edition
The Residency Selection Arms Race, Part 1: On Genghis Khan, Racing Trophies, and USMLE Score Creep
The Sheriff of Sodium 