Whenever we emerge from the COVID-19 pandemic, we’re all gonna have at least a few horrible memories burned into our minds. This thing is going to take something from everyone – and from some of us, it’s going to take everything.
The gravity of the crisis makes most of the things I’ve been writing about lately seem trivial – which is why I’ve been slow to put out some of the pieces I’d intended to write this spring. But social distancing leaves a lot of time for thinking.
And it occurred to me that, when we look back on the pandemic in the future, one the more benign images likely to become a permanent part of the historical record is something like this:
Shelves of toilet paper, empty, cleared out as consumers descended on big-box retailers like a horde of locusts.
Weeks before SARS-CoV-2 arrived in my community, toilet paper became a scarce commodity. (A few days ago, when my wife finally scavenged a few rolls, she presented them to the family with the confident, self-satisfied look of a lioness with a fresh kill.)
The sudden scarcity of toilet paper isn’t because folks in my area suddenly needed twice as much of it. (Thank God.)
And it isn’t because there was catastrophic disruption to the toilet paper supply chain. (At least, not yet.)
Instead, the depletion of toilet paper has to do with human psychology.
It all started when a few consumers bought a little extra TP. Because the news was making this coronavirus thing sound kinda bad. But then, when others made their usual toilet paper run, they had trouble finding this necessary product… so when they finally did, they bought extra, just to be sure they had enough for their family in these uncertain times.
Of course, as shelves emptied out, the panic spread. Soon, we all had to buy extra toilet paper – if for no other reason than because everyone else was buying extra toilet paper.
From a societal standpoint, we’d all be better off if everyone purchased only enough toilet paper to keep our individual bottoms neat and tidy. Then, shelves could re-fill, and anyone in need of toilet paper could buy it.
But from an individual standpoint, if toilet paper is scarce, you’re a fool not to buy as much of it as you can. Who cares if your neighbor’s butt stinks? You’re not going to be quarantined in a house with them!
In the absence of an individual incentive to ration appropriately, an external actor has to impose limits. And so, in response to the Great Toilet Paper Run of 2020, many retailers stopped allowing customers to buy a pallet of toilet paper. Instead, individuals could only purchase a smaller amount.
This kind of rationing doesn’t usually sit well with Americans. This is the land of the free and the home of capitalism, and we hold it as a self-evident truth that a person should be allowed to have as much of a thing as he or she has money to pay for it.
And yet, there are times when enforced rationing is the only way to make a marketplace work effectively again.
World War II rationing poster, from the U.S. Office of Price Administration, 1943.
I’m talking, of course, about residency application caps.
I’ve discussed how Application Fever is hurting both programs and applicants. And I’ve argued repeatedly (not only on this site, but on Twitter, YouTube, and various and sundry public forums) that application caps are the only way to get application inflation under control.
Thing is, when I talk about it, I often hear the same misinformed counterarguments. So I thought I’d take a momentary break from COVID-19 social distancing, and for everyone not on the front lines, let’s separate myth from reality.
_
MYTH #1: If you implement application caps, the Match rate will plummet!
REALITY: No, it won’t.
Last year, the average U.S. senior medical student submitted 65 residency applications. And in 2020, the Match rate for U.S. seniors was 93.7%.
So what would happen if fourth year MD students decided instead to apply to half as many programs? What would the match rate be?
Turns out, you don’t have to speculate – because Application Fever has created a natural experiment for us.
Each year, residency applicants apply to more and more programs.
Go back to 2007, when the average U.S. senior submitted only 32 applications – and the Match rate was 93.4%.
Matter of fact, the Match rate for U.S. seniors has been steady at 93-94% for decades, even as the average number of applications submitted has increased each year.
The point is, the overall Match rate is a function of the number of positions available and the number of applicants for those positions – not the number of applications that each applicant submits. Unless an application cap were so restrictive that it prevented applicants and programs from finding each other – which would probably require limiting applications to <5-10 – the Match rate ain’t gonna budge.
_
MYTH #2: But I don’t go to a prestigious medical school! I NEED to apply to XXX programs or I won’t Match!
REALITY: No, you don’t.
Let’s think back to the guy buying 400 rolls of toilet paper at Costco. He doesn’t need 400 rolls. (At least, let’s hope.) The only reason he has to hoard toilet paper is because everyone else is hoarding toilet paper. If he doesn’t – and everyone else does – then he’ll be wiping his butt with a stack of leftover fast food napkins. And no one wants that.
Today’s residency applicants are stuck in a similar situation. The main reason they have to overapply is because everyone else is overapplying.
Overapplying does not improve overall match rates. But it does increase the likelihood that an individual student will Match. Trouble is, once a few students decide to overapply, everyone else has to overapply just to keep up.
Residency application can be modeled as a Prisoner’s Dilemma, in which overapplication is a dominant strategy. (PubMed)
It’s not fair to blame students for overapplying – they’re making a rational decision given the incentives involved.
But it’s also unrealistic to expect students to choose against their own self-interest and just stop overapplying. They’re stuck in an arms race – and application caps are the only way to end it.
–
MYTH #3: But I’m an international medical graduate (IMG)! I NEED to apply to XXX programs or I won’t Match!
REALITY: No, really, you don’t.
Suppose there’s a guy who needs some toilet paper. He goes to a store – but they’re sold out. So he goes to another, and another. He drives all around town, to twenty different stores, and finally comes home with a pack of Charmin.
Which of the following statements about this person’s experience is more true?
A. He had to go to twenty stores, or else he wouldn’t have come home with any toilet paper.
B. He could have gone to just one store and achieved the same outcome – especially if he knew which store was most likely to have toilet paper.
When I make the example about toilet paper, almost everyone would agree that B is the more correct answer. But if we’re talking about residency selection, I usually hear everyone endorse A.
On Twitter, at least, the argument usually takes the form of a personal anecdote. “I’m an IMG, and I applied to 200 programs and got 3 interviews. Caps won’t work! IMGs like me have to apply to lots of programs or we won’t match.”
Really? Do you think the three programs that offered an interview did so because they knew that you applied to 200 programs? Of course not.
All else being equal, which of these high schoolers is more likely to get into college? The girl in orange applies to more schools, but the girl in yellow applies to one school where her application will be very competitive.
Just like colleges, residency programs differ in their competitiveness. A solid applicant could have a 100% chance of getting an interview at a community internal medicine program – and yet have no chance of being interviewed at one of the ‘top’ academic IM programs. It’s foolish to act as if all programs have the same likelihood of granting an interview, or that the number of applications submitted is more important than the nature of the programs to which the applicant applied.
The real issue here is that many applicants – especially IMGs – do not have enough information to know where their application will be competitive. But the solution to this is to demand transparency from programs – not continue to allow applicants to waste their money applying to programs where their application sits unread in some ERAS hell.
–
MYTH #4: But I’m applying to a competitive specialty! I NEED to apply to XXX programs or I won’t Match!
REALITY: No, you really don’t.
Sigh.
I guess the discussion above wasn’t enough – so I’m gonna try coming at this in a slightly different way.
For the moment, let’s assume that programs do not provide any more information than they do now to help applicants understand where they might be competitive.
All the average residency applicant wants is to apply to enough programs that they’ll end up with a job, right? So the real question is, how many applications do you need to submit to maximize your chances of matching?
And please, don’t get me started on the horribly biased Apply Smart analyses from the AAMC. As I’ve noted before, the relationship between applications submitted and the probability of matching is variable – but the relationship between the number of interviews completed and the probability of matching is quite predictable.
Below, I used the number of contiguous ranks submitted as a proxy for the number of interviews completed. (And yes, not everyone ranks every single program at which they interview – but according to the NRMP Applicant Survey, the median number of interviews completed and the median number of programs ranked is the same.)
For almost every specialty, a U.S. senior who completes 10 interviews (and ranks 10 programs) has a 90% or better likelihood of successfully matching.
As you can see above, the average MD student applying in orthopedics submits 81.5 applications – but ranking 12 programs is enough to achieve a 90% match rate.
Diagnostic radiology applicants, apply to 58 programs if you want – the average student does – but all you need is to interview at and rank 7 to have nearly maximized your probability of matching.
And for family medicine? Go on and apply to 42 programs if you’ve got money to burn – but know that applicants who rank 5 programs match at a better than 90% clip.
And yes, I realize that most applicants would never settle for “just” a 90% probability of Matching. But last year, there were 1.40 applicants for every position in orthopedics; 1.49 for every position in radiology, and 1.54 for every position in family medicine. Let every applicant apply to every program if you want – but there is no way to give everyone a 100% chance of matching.
If you consider residency selection from the perspective of an individual applicant, it’s a zero sum game. If you “win”, it’s because someone else loses. And if you lose, it’s because someone else wins. The best we can do is make the competition fair.
_
MYTH #5: Capping applications is unfair.
REALITY: Allowing residency positions to be assigned by willingness to pay for applications is unfair.
Until or unless the number of residency positions available perfectly matches the number of applicants (and their specialty preferences), some people are always going to be unhappy on Match Day.
The real issue is, how should we decide who the winners will be?
Students overapply because it benefits them to do so. Hold everything else constant, and the student who applies to 100 otolaryngology programs has a greater chance of becoming an otolaryngologist than a student who applies to 30.
We live in a country that prizes the idea, at least, of meritocracy. So what type of merit does this reward? I can’t think of any. Instead, it provides a relative advantage to applicants with more money – or greater willingness to incur debt. Of all the ways we could select applicants for residency, is this really how we want to do it?
I say we level the playing field.
Let all applicants pay the same price and apply to the same number of programs, and let program directors separate the most meritorious in a system in which they can actually read the number of applications they receive.
_
MYTH #6: Capping applications only benefits applicants from elite schools.
REALITY: Capping applications benefits anyone whose merit isn’t captured by convenience metrics.
Again, let’s think through this one with some real numbers. For sake of example, let’s use general surgery.
In 2020, there were 1,536 categorical general surgery positions available at 328 programs. That means that the average program was trying to fill only 4.7 positions.
However, according to the 2018 NRMP Program Director Survey, the average general surgery program received 742 applications – or 158 applications for every position they’re trying to fill.
I want you to think about that number for a moment.
Suppose that the program director spends just 10 minutes reading every application. It would take her 124 hours to get through them all.
Maybe that’d be okay if PDs were like admissions officers, who spend 100% of their time evaluating applications and doing admissions stuff. But being a PD is a part-time gig – and much of the time allocated to being a PD has to be spent taking care of the residents who are already in the program.
The average general surgery program offers interviews to just 85 applicants – 11% of all the applications they receive. So if you’re the PD – given the realities of time – how are you gonna decide who makes the cut?
You’re gonna use a convenience screening metric.
Maybe you exclude applicants with a USMLE score below an arbitrary threshold. Or maybe you use ERAS to filter out osteopathic students, or IMGs, or applicants with a gap since their graduation year, or anyone outside a small number of prestigious feeder schools. You know that none of those factors are good predictors of an applicant’s capability to perform surgery, and all of them probably leave talent on the table – but you’ve just got to get through the pile.
So what do you think happens when all PDs use these convenience metrics? They end up inviting the same applicants over and over.
Again, more real numbers: among the U.S. seniors who applied in general surgery, 16% were AOA, 28% attended one of the top 40 medical schools (by NIH funding), and 16% had a USMLE Step 1 score >250. If these applicants apply to 66 programs, how many interview invitations do you think they’re going to get?
I’m guessing 66 – or darn close to it.
Think for a moment about the downstream consequences.
At at least one paper attempted to quantify the effect that over-invitation has on other applicants. Using NRMP data, the authors estimated that just 12% of general surgery applicants consume half of all interview spots.
Just 12% of applicants in general surgery consume 50% of all interview spots – and the interview distribution in some other fields is even more disparate. (Source)
When elite applicants overapply, they get overinvited to interview – which wastes the applicant’s money, the program director’s time, and diminishes the opportunity for other applicants.
Now, it’s easy for non-elite applicants to look at these data and say, “See! That’s why I have to apply to more programs!” But is that the only possible conclusion? Maybe the better answer is for everyone to apply less.
If you’re an applicant, and you want a program director to be able to look at you as a whole person – something more than your USMLE score or the prestige of your medical school or your citizenship status – then application caps will help you. Because without caps, everyone is going to be reduced down to whatever variable can be filtered in ERAS.
_
MYTH #7: Some program directors aren’t gonna read applications, even if you do cap applications. I should be allowed to apply to as many programs as I want!
REALITY: Uncontrolled application makes things more chaotic for everyone.
The premise here is true. Even with caps, programs are still going to receive a lot of applications, and many PDs will still choose to rely upon convenience metrics.
That’s okay.
My goal here isn’t to dictate how programs should evaluate applicants to interview. It’s to make the residency application process work better… and right now, the congestion in residency application isn’t just hurting programs, it’s hurting applicants, too.
Wanna see what I mean? Check out the data below.
In 2019, the number of U.S. seniors who matched to their first-choice program was 47.1% – the lowest on record – while 23.1% got their fourth or lower choice (an all-time high).
Although the overall match rate is unchanged as applications increased, there has been a slow erosion in the number of U.S. seniors who Match at their first-choice program.
This is a predictable consequence of a congested market that lacks preference signaling – and without intervention, it may soon lead to a preventable increase in the number of good applicants who fall through the cracks.
(Keen-eyed readers may have noticed that the cost of congestion was built into the Prisoner’s Dilemma matrix above. When both applicants applied to an ideal number of programs, they each had a 5% chance of going unmatched. But when they both overapplied, that probability increased to 6%.)
–
MYTH #8: If you cap applications, an East Coast applicant will never get into a West Coast program (or vice versa).
REALITY: Capping applications gives PDs an incentive to take your application seriously.
When applicants overapply, PDs lose their ability to separate applicants who are truly interested in their program from those whose shotgun approach just happened to send some buckshot in their direction. So when a West Coast program gets an application from an East Coast applicant, it’s easy to dismiss it.
But forget about geography. The inability of PDs to discern true interest is causing bigger problems.
Last year, I spoke to a general surgery program director at a respectable program who actually screens out applicants with high USMLE scores – because he’s learned that these applicants almost never end up matching at his program. Honestly, how bad have things gotten when sending an application to a program isn’t a credible signal that the applicant is actually interested in going there?
The calculus changes when you cap applications. Students will have an incentive to apply only to programs where they have serious interest – and program directors will have an incentive to take those applications seriously.
_
MYTH #9: We don’t need to cap applications – just cap the number of interviews an applicant can accept instead!
REALITY: That’s fine for applicants… but won’t help program directors.
Capping interview acceptances might curtail interview hoarding and address some of the problems noted above. But it also has some likely unintended consequences that I’ve discussed previously.
More importantly, capping the number of interviews an applicant can accept does almost nothing to help program directors.
Program directors – even those in “noncompetitive” specialties – are impossibly burdened by Application Fever.
_
MYTH #10: Capping applications limits “upward mobility” for students from less-prestigious medical schools.
REALITY: Application caps reward thoughtful applications.
I don’t quite follow the logic here.
I mean, if your application doesn’t stand out when a prestigious program receives 20 applications for every position, it wasn’t gonna stand out when they were receiving 200 applications for every spot.
What is true is that capping applications will place new emphasis on deciding where to apply. And this is the real issue that we have to confront for caps to be a workable solution.
Right now, programs provide very little information to help students decide where they’re a good fit. Program websites are so hopelessly generic that spamming all of programs in a given geographic area is a totally rational strategy. (And from the program’s standpoint, what’s the point in explaining how you pick applicants if you’re just going to get spammed with applications regardless?)
I’ve written before that it wouldn’t be that hard for programs to provide higher-quality information to applicants. This is a mandatory pre-requisite to instituting caps. But to me, it seems like a fair trade off. If PDs want to get fewer applications, they have to provide some transparency and clarity to applicants.
_
MYTH #11: Caps are un-American! We don’t limit the number of applications you can submit in any other field.
REALITY: The residency Match is already unlike any other field.
It’s true that when you apply to medical school, or when you apply for jobs as a physician, there are no caps. You can apply to as many places as you damn well please.
But comparing these situations to the residency match is kind of silly. Residency selection isn’t a free market. It’s designed market. It was created at a time when both programs and applicants were not being served by a free market system. And whether you like it or not, we’re stuck with it now: in 2004, Congress granted specific antitrust protection to the National Resident Matching Program (NRMP) to conduct the Match.
But the cool thing about a designed market is that we get to decide the rules for how it works. There’s no reason to cling to some false notion of ideology when we can impose some simple boundaries to make the system work better.
_
MYTH #12: But some applicants apply to multiple specialties! Caps will only work if you let them have a higher cap.
REALITY: Give applicants who apply to multiple specialties extra applications – and suddenly you’re gonna have a lot more people applying to multiple specialties.
Here’s one of the insidious things about Application Fever: applicants aren’t just applying to more programs within the same specialty. They’re applying to more specialties, too.
ERAS cross specialty applicant data. (Source)
In the table above, medical specialties are listed in the columns and again in the rows. So for instance, there were 193 applicants who applied in both radiation oncology and dermatology.
Now, perhaps these 193 individuals are truly undecided between a career treating cancer with radiation and a career treating cancer caused by radiation. Or perhaps – just perhaps – they’re applying to one specialty as a backup.
Look, there’s nothing wrong with being undecided. (Back in the day, I applied to residency programs in internal medicine, med/peds, and pediatrics.)
And there’s nothing wrong with applying to a backup specialty. This year, only 78% of U.S. seniors who preferred dermatology successfully matched. It’s a good idea to have a backup plan.
But are we obligated to allow indecision and backup plans to gum up the system for everyone else? I say no.
I say cap applications, and give everyone the freedom to decide how to spend them. If you want go to high-risk, high-reward, and use all 20 of your applications on prestigious programs or in a selective specialty, have at it. If you want to save some of them for a backup specialty, I’m fine with that, too. (And if this results in fewer applicants choosing to pursue highly selective specialties, I’m not so sure that would be a bad thing for society.)
More importantly, if you cap applications – but allow those applying in multiple fields to apply to extra programs – you can guarantee the table above will become even more ridiculous. (And let’s be honest: it’s pretty ridiculous already. Note that the only combination of specialties that garnered zero cross applications last year was otolaryngology and nuclear medicine.)
Applicants will always have an incentive to overapply. But in fixing that, we need to be careful not to create incentives that work against what we’re trying to accomplish.
_
MYTH #13: Application caps are illegal.
REALITY: Probably not.
Those who make this claim usually assert that application caps would run afoul of restraint of trade laws. The idea here is that, in America, we value the free market – and we generally allow people to buy and sell their wares without interference. Thus, imposing an arbitrary limit on the number of residency programs to which an applicant can apply may unreasonably restrain their ability to “sell” their services and conduct their business.
But the key word is unreasonable. Because we restrain trade all the time. Why can’t I hire a employee for less than minimum wage, or corner the market on a particular good, or sell you one of my kidneys? For various reasons, we regulate and restrain trade all of the time.
Look, I’m not an attorney. I’m not even a layperson who is especially well-informed on matters of the law. But even cursory research shows that restraints of trade may be legal if the restraints are reasonable. Generally speaking, a reasonable restraint of trade
- Serves a legitimate interest
- Is limited to that particular interest
- Does not run contrary to the public interest
Do application caps fit that standard? I say yes. In the aggregate, over-application does not help PDs or applicants, and it certainly does not serve the public interest.
However, as the corporate sponsor of the Electronic Residency Application System (ERAS), Application Fever does serve the AAMC’s interest.
In 2019, the AAMC took in $94,153,946 in revenue from ERAS. Growth in ERAS has outpaced growth in the AAMC’s other revenue streams.
So although I suspect caps can be configured to withstand legal review, this is America, where you can sue anyone for anything – and the AAMC would have to be willing to defend caps in court. Given the chart above, would they?
What seems more likely is that the AAMC would use the threat of a lawsuit as a way to justify dragging their feet in considering caps. After all, they have 94 million reasons to be free market enthusiasts.
Of course, if real attorneys are sincerely concerned about the legality of caps, there’s a ready-made end run: the early residency application program (ERAP) being studied in OB/GYN (with support from the AMA).
The ERAP consists of a two-round residency application system in which the first round is capped at 5 applications (and 3 interivews). Applicants who do not match in the first round can apply to an unlimited number of programs in the second round.
Critically, however, participation in the first round is optional. You can participate or not – which makes it hard to claim that your trade is being restrained.
_
MYTH #14: Capping applications is arbitrary and not evidence-based.
REALITY: We have the data to pick a sensible cap.
My gut says that giving every student 15 applications will be more than enough. But I’d be the first to tell you not to listen to my gut.
If the AAMC and the NRMP are willing to share data, we have all the information we need to build high-quality models of what might happen if we capped applications at various levels. (And even if they refuse, we could still take a pretty good stab at it.)
In all likelihood, we wouldn’t want to go from the current situation to a tight cap in one step. More likely, we’d ratchet the cap down each year till we hit our goal, studying outcomes all the while.
The point is, as much as I like the idea of application caps, it’s not something we should jump into headlong. And I’m okay with that.
But let me tell you what I’m not okay with: our current inertia.
Application Fever is hurting us.
We’re allowing medical students to be plundered for application fees – then shrugging our shoulders as their unread applications sit in ERAS purgatory because they didn’t chase enough empty brass rings to pass some automated filter.
We’re sitting idly as PDs get buried beneath applications, then waste time rolling out the red carpet for applicants who don’t really give a rip about their program (and may not even want to match in their specialty).
We’re allowing the narrow self-interests of a few to supersede the consideration of better policy for the many.
It’s time to stop acting like Application Fever is a problem that’s gonna get better on its own. It won’t. Tomorrow’s medical students will apply to more programs than today’s – because the marginal cost of a few more applications will always be lower than the cost of going unmatched.
So when we finally get through this public health crisis, I hope we realize that COVID-19 isn’t the only curve that needs flattening.
[ADDENDUM: A video based on this post is available on the Sheriff of Sodium YouTube channel.]
YOU MIGHT ALSO LIKE:
What’s the antipyretic for Application Fever?
Applying Smarter, Part I: Breaking Down the AAMC’s Apply Smart Campaign
The Sheriff of Sodium 