Symbiosis: The USMLE, ERAS, and medical education

From time to time, hunters exploring the depths of the Amazon rainforest stumble upon an eerie sight. Nestled amidst the almost-impenetrable jungle are well-circumscribed areas where the forest has been almost completely cleared out.

Instead of the usual lush vegetation extending from the groundcover to the tree canopy above, these clearings contain only a single species of tree: Duroia hirsuta. Locals termed these areas “Devil’s Gardens,” believing that some evil spirit must have killed all of the usual vegetation and planted the D. hirsuta trees in their place.

In 2005, scientists from Stanford unraveled the mystery of the Devil’s Gardens.

Turns out that, in these areas of the Amazon, there lives a species of ant called the lemon ant (Myrmelachista schumanni).

Lemon ants like to live inside the Duroia hirsuta tree. The tree’s hollow stems make perfect nesting sites, and even better, they house nectar-like food bodies for the ants to eat.

In fact, lemon ants like D. hirsuta trees so much that they aggressively attack any pests that try to feed on their tees. They even swarm and attack other plants that try to grow near the trees, injecting an invader’s roots and leaves with formic acid until it withers away and dies.

The relationship between lemon ants and their tree is an example of mutualism, a symbiotic relationship where both organisms benefit. From the tree, the ants get food and shelter; from the ants, the tree gets unobstructed sunlight and space for its roots to spread.

Together, they create a perfect little ecosystem for themselves – but without the other, each struggles to survive.

Uhh… what does this have to do with medical education or residency selection?

I’m getting there – stay with me.

Unless this is your first time on this site, you know that I’ve been writing a lot about “Step 1 Mania” – how our obsession with using USMLE Step 1 scores for residency selection is hurting medical education. I’m on record as supporting a pass/fail score reporting policy, and strongly believe such a step is necessary to get out of the mess we’re in.

Some of you out there are unconvinced. I’ve gotten a lot of feedback about why changing Step 1 to pass/fail would be detrimental. By far, the two most common objections are these:

From Program Directors: We get XXXX applications for just X positions. How can I triage applications, or compare candidates from different backgrounds, without some broad, easy-to-use metric? We need Step 1 scores.
From Medical Students (especially IMGs and DOs): Step 1 scores are the only truly objective measure in a residency application. It’s our only chance to distinguish ourselves and get into a “top residency.” We need Step 1 scores.

I disagree with their conclusion – but the issues raised are important ones and deserve a closer look.

Why do some applicants and program directors feel so strongly that they need Step 1 scores?

This. This is why.

As I’ve recently discussed, we are facing an epidemic of Application Fever, and there’s no sign of it breaking anytime soon.

At the same time, and in parallel with the increasing number of applications per student, scores on USMLE Step 1 are rising.

And far from their intended purpose in informing licensure decisions, Step 1 scores are now primarily used to screen candidates for residency . According to the 2018 NRMP Residency Program Directors’ Survey (the source of the graphic on right in the Tweet below), 64% of all residency programs use specific Step 1 target scores (i.e., cutoffs) for considering candidates to interview.

Application Fever and Step 1 Mania: a chance association?

Correlation doesn’t mean causation… but do you really think that the epidemic of Application Fever and the rise of Step 1 Mania are separate phenomena?

I don’t.

Consider this: if we lived in a world without Step 1 scores, do you think Application Fever could have gotten as far as it has? Or do you think the situation would have become unmanageable long ago? Step 1 scores make Application Fever possible.

At the same time, Application Fever feeds Step 1 Mania. A candidate may have all kinds of noteworthy achievements tucked away in his or her ERAS application – but if the PD is trying to cull the pile from 1000 applications to 100, only one of them may matter.

Just like the lemon ant and the Duroia hirsuta tree, Step 1 Mania and Application Fever need each other.

The benefits of symbiosis

Application Fever and Step 1 Mania have benefitted two groups very clearly: the AAMC (sponsor of the Electronic Residency Application System, or ERAS) and the NBME (sponsor of the USMLE Step 1).

Here again are the AAMC’s revenues since 2008, broken down by program. ERAS is their cash cow, with revenues that have more than doubled over the past decade.

Business has been booming for the NBME as well. Overall revenue growth is shown below, and as I’ve shown previously, much of their recent growth has been driven by Step 1 Mania.

The NBME and AAMC benefit richly from the growth of the other in this Devil’s Garden. Maybe the NBME is the lemon ant, and the AAMC is the tree. Or maybe it’s the other way around.

Or maybe I’ve got it all wrong. Maybe this isn’t mutualism. Maybe it’s parasitism. Because the millions of dollars of revenues pushing up the bars in the charts above all come from medical students – who don’t seem to be benefitting at all. Maybe instead of lemon ants and trees, we should be talking about leaches or fleas.

Look, the AAMC and NBME provide useful public goods. We need a licensure exam. And having an electronic residency applications is awfully nice, if not absolutely necessary. The real question is, are these programs delivering a social good commensurate with their cost?

I say no. If Match rates are exactly the same today as they were in 2007, when students applied to far fewer programs, then could we please go back to spending just $36 million a year on ERAS? What, exactly, are we getting for that extra $49 million we’re paying nowadays?

Similarly, if students today are spending more time on Step 1 prep than any other course, clerkship, or rotation during medical school, shouldn’t we demand some evidence that such an investment is providing a meaningful return?

These are the questions we don’t ask ourselves enough. But if we leave the decision-making on residency selection and USMLE policy to organizations who have a financial interest in maintaining the status quo, we shouldn’t be surprised when that’s what they choose to maintain for us.

How to grow flowers in the Devil’s Garden

So how do we move forward from here?

First, make Step 1 pass/fail. We do not need a scored Step 1. We need a licensure exam, and we need fair and meaningful ways for students to distinguish themselves to programs. They don’t have to be the same thing.

Yet we also must understand that by itself, making Step 1 pass/fail accomplishes little. It’s like planting flowers in the Duroia hirsuta grove, and expecting they will just fill in the empty space. There is more going on than there appears at first glance, and simplistic solutions will fail. Step 1 Mania and Application Fever are closely related problems, and we should accept that we can’t tackle one without the other.

So anyone who supports a pass/fail Step 1 must also commit to addressing the issues that led to its rise in the first place.

Program directors need fewer applications; a better articulated vision of who they’re trying to Match; and better tools for finding candidates who fit their values and mission. Students need – and deserve – unbiased and relevant ways to show their capabilities, and a system that rewards achievements and skills that actually matter.

We absolutely can make such a system – but it will take participation from all of us with an interest in the outcome (not just those with a financial one).