In Part 3 of this series, we covered Dr. Kevin Jon Williams’ decades-long fight for a student-optimal match algorithm. Left unanswered, though, was why the National Resident Matching Program (NRMP) initially resisted using an algorithm that favored students.
What did it matter to them? After all, the NRMP is a neutral party, and they could have easily used the mirror image of the Gale-Shapley algorithm to optimize results for students instead of programs. Why did they care so much which version of the algorithm got used?
In this post, we’ll answer that question – and in so doing, review a critical policy initiative that consolidated the NRMP’s power.
–
On market design and “thickness”
The residency match is a designed market. Doctors and hospitals do not contract with each other naturally like they do in the post-residency open market. Instead, the NRMP imposes rules and boundary conditions with the goal of making the transactions in the market more efficient.
But there is a problem with designed matching markets. They tend to unravel.
Just like a ball of yarn that can only be neatly wound one way, disrupting anything about a designed market can encourage participants to pursue their self-interests outside the market – and make the whole thing fall apart.
A necessary requirement for an effective market is what economists call “thickness.” Thickness means that the market includes a large enough proportion of all the buyers and sellers that market participants can get what they want by participating in the market.
If you can’t find what you’re looking for in the market – or if you can get a better deal on it elsewhere – you’ll go outside it. And when you do, others may follow.
If even a few participants make their own arrangements, they undermine the benefits of the clearinghouse, because those who opt out – either by refusing to participate or by failing to honor their commitments – may very well have been best matched with others who stay in. Soon the market unravels, and participants once again experience the problems the clearinghouse was meant to solve.
Roth AE, Harvard Business Review, October 2007.
–
The Match Marketplace – Then and Now
Back in 1951, when the Match began, around 5800 medical students were available to fill around 10,500 intern positions.
In other words, there too few medical students to fill the available residency positions.
Hospitals were starving for residents – and if they could sate their hunger outside the Match, they would. Therefore, to provide adequate thickness, it was essential that every single medical student participate in the Match.
(This, of course, is why the Harvard students’ threat to pull out of the match carried so much weight. If even a few students had refused to participate, residency programs would have chased them – and soon, the whole thing might have unraveled.)
Of course, things have changed a lot since 1951.
In 2019, there were 44,959 applicants for just 37,256 residency positions. (Graphic from the NRMP Main Residency Match Report.)
Until the mid-1970s, there were still more residency positions than there were applicants. But by 1981 – the same year that Dr. Williams’ original article appeared in the New England Journal of Medicine – things were beginning to change, and since the mid-1980s, the situation has been completely reversed, with a substantial excess of applicants to program positions.
This changed the calculus for the NRMP.
Unlike in 1950s, when it was imperative to keep all the students in the Match, in the 1980s it became more important for the NRMP to keep programs happy.
See, when you have way more applicants than positions, it’s really no big deal if some students choose to not participate in the Match. Have at it! Hospitals will still have no trouble filling their positions from the pool of students still participating in the Match.
On the other hand, if some programs pulled out of the Match, those program directors might discover that they could find capable applicants without going through the hassle of the Match. And if they did, other programs would be tempted to follow suit.
This is why the NRMP ignored Williams.
Changing to a student-optimal algorithm would have been nice for students – but it might not have sounded so good to program directors. And if it scared even a few programs out of the match, it could pull a thread that would make the whole operation unravel.
Ultimately, that worry proved to be unfounded. In the furor that followed Williams’ 1995 Academic Medicine article, most program directors sided with the students and called upon the NRMP to study the issue.
With the assistance of Nobel prize winning economist Alvin Roth, the NRMP changed to the student-optimal Roth-Peranson algorithm and finally published an analysis of rank order list data that demonstrated that outcomes were very similar, regardless of whether the hospital-optimal or student-optimal method were used.
–
Unraveling: A Case Study
Even with the new algorithm, the NRMP’s worries that the Match might unravel persisted. And these concerns took on new importance after they watched the gastroenterology fellowship match unravel before their eyes.
The trouble started in 1996. That’s when a workforce analysis in JAMA suggested that the United States was overstocked with gastroenterologists – with twice as many GI doctors per capita as some health maintenance organizations thought we needed.
In response, national organizations recommended phasing in a 50% reduction in the number of GI fellowship positions over a five year period, as well as lengthening the fellowship from 2 to 3 years.
Those measures turned out to be unnecessary: applicants were already spooked. It’s one thing to go through a fellowship if there’s a lucrative career on the other side, but the idea of spending several years maneuvering a scope through incompletely prepped bowels only to struggle to find a job afterward was an unappealing prospect indeed. So even with the planned reduction in fellowship positions, the number of applicants plummeted even further. In 1992, there were 1.75 applicants for every GI fellowship position. In 1996, that figure fell to 0.9.
Suddenly, programs were in the uncomfortable position of having to compete for fellows – or go unfilled.
Accordingly, some programs began offering interviews earlier – and filling positions outside the Match to lock up qualified candidates before they had a chance to interview elsewhere. Other programs raced to catch up, and by 1999, there were only 14 positions offered in the match – all the others were filled outside of it. In 2000, even these stragglers abandoned the Match.
In the wake of the implosion of the gastroenterology fellowship match, the NRMP looked critically at their organizational vulnerabilities – and how they could patch them up.
–
Going “All In”
In particular, the NRMP was concerned by programs who offered some of their positions outside of the Match. Often, these were programs that relied upon international medical graduates (IMGs) to fill their allotment.
For these programs, offering outside of the match was helpful for two reasons. For one, it gave these programs the opportunity to “cherry pick” the best IMGs before they got snapped up by other programs. For another, it gave the program director more time to arrange work visas and ensure that their new interns were ready to start on July 1. (Waiting till Match Day in mid-March often imposed an insurmountable administrative burden.)
These kind of early arrangements often worked well for applicants, too. Many IMGs were eager for the opportunity to secure a U.S. residency position. Especially in situations in which a program and applicant strongly preferred each other, it gave them a way to solidify that commitment and avoid the uncertainty of the Match. Why wait?
Despite the potential advantages for certain applicants and programs, the NRMP had always regarded these outside-the-Match dealings warily. The unraveling of the GI fellowship match was proof that their designed marketplace could fall apart quickly if too many deals were struck outside the matching market. It was imperative to keep as many programs in the Match as possible – so the NRMP began to push forward a policy that would make it nearly impossible for programs to offer positions outside the Match. They called it the “All In” policy.
Put simply, the All In policy says that if you offer any of your residency positions in the Match, you have to offer all of them. No more holding out a few positions to lock up good candidates. No more withdrawing positions to make deals outside the Match. By keeping positions in the Match, the market would maintain enough thickness to function efficiently.
In 2007, the NRMP proposed an All In rule for institutions. If the University of Such-and-Such wanted to fill its OB/GYN or surgery positions in the Match, it would have had to fill its psychiatry and internal medicine positions there, too.
Institutions bucked. Institutional officials worried that they would not be able to get all of their residency programs on board. At the time, around a quarter of all internal medicine and preliminary surgery positions were offered outside of the Match – often to IMGs. Unfortunately, the post-9/11 immigration fallout had made visa processing slower and more unpredictable. Program directors in specialties that depend on IMGs (like internal medicine, family medicine, pathology, psychiatry, and pediatrics) warned that having to offer all of their positions in the Match might result in having a substantial number of residents who were unable to start work in July.
Rather than forcing the issue – and risking that some institutions might choose to pull all their residency programs out of the Match – the NRMP quietly retreated.
In 2011, the proposal was changed. Now, instead of institutions, a individual program had to offer all of its slots. This put the onus squarely on program directors, who now faced a choice: would they be better off filling all their spots through the NRMP – or none of them?
Despite the objections of 35% of internal medicine program directors, this version of the All In Policy was implemented in 2013 and remains in effect today.
The All In rule makes it much less likely that the Match would unravel. But it’s not impossible, as I discussed in this video.
–
Wait… isn’t that illegal?
It sounds like it might be, right?
I mean, in any other area of American business, it might be considered an impermissibly anticompetitive business practice for a company to corner the market on a particular service and then demand that all their customers exclusively use only their service.
But thanks to an epic legal battle two decades ago, the NRMP needn’t worry about the threat of antitrust lawsuits. We’ll cover that story in Part Five before considering the big question – would we be better off without a Match? – in Part Six.
–
You may also like:
The Match, Part 1: Why do we have a Match?
The Match, Part 2: Getting under the hood – how does the Match work?
The Match, Part 3: On proposals and the fight for a student-optimal Match
The Sheriff of Sodium 