Welcome to the second part of our trilogy on the USMLE Step 2 Clinical Skills (CS) exam.
In Part 1, I reviewed how the Step 2 CS examination came to be. We traced the history of the test from its roots in the early NBME comprehensive exams through its initial demise and eventual resurrection (both motivated by financial circumstances). It was an epic journey. But it was also a lot of prose.
In Part 2, we’re gonna cover student opposition to the test. And this, too, requires a lot of prose. So to freshen things up, we’re gonna hit this with the Sheriff of Sodium’s now familiar Q&A format.
Cool! I love the question and answer format!
That wasn’t a question.
Before I ask about Step 2 CS opposition, can I ask a random question?
Sure.
Step 2 CS was the fourth USMLE examination, right?
Yes, it was.
Well, why wasn’t it named Step 4?
That’s actually a very, very important point. Hold that thought… we’ll get there.
Fine. So how did U.S. medical students react to the announcement of the Step 2 CS exam?
Poorly.
Remember, the idea of a clinical skills examination had been kicking around for a while. Medical student liaisons from the AMA and AAMC had been attending meetings with the NBME for several years, and had been told that the new clinical skills exam would be offered at their medical schools.
So it came as quite a surprise when the NBME announced that the new clinical skills exam could be taken in only one of five test centers… and oh yeah, it would cost $975.
Medical students felt betrayed, and mobilized in earnest. The medical student sections of the AMA, AMSA, and AAMC all issued statements in opposition, and urged students and faculty to lobby their state medical boards to oppose or at least delay implementation.
Wow. Were all U.S. medical students opposed to Step 2 CS?
No.
At least one medical student was very much in favor of it, and wrote a letter directly to the NBME to say so. Extended quotations were recorded for posterity in the 2002 NBME Examiner.
An excerpt from the 2002 NBME Examiner.
(I don’t know about you, but I can’t read the quotes without hearing them in the voice of Martin Prince from The Simpsons. “Although our cost of education may increase, the potential benefits are astounding!”)
What were the arguments against the USMLE Clinical Skills exam?
Most students were angry about the test’s high cost and limited test-center availability – which also raised questions of equitable access. Unfortunately, these arguments didn’t carry much weight.
The NBME cared little about the test’s costs. After all, they owned and operated the test-centers and directly benefitted from the increased revenue produced by the CS exam.
(How much did they benefit? Well, in 2003 – the year before CS went live – the NBME reported an annual program service revenue of $56 million. Not bad. But by 2005 – the first full year of Step 2 CS – that figure surged to $82 million, an increase of 46%.)
The cost of USMLE Step 2 CS is substantial and keeps rising… though fees have increased less less during and immediately after periods of more intense student opposition to the exam.
But to be honest, cost-based arguments didn’t resonate with physicians in practice or the general public, either.
Because doctors are well compensated, the idea that medical students must to take an expensive licensing exam just doesn’t stir the passions of the public. (Not long ago, after I pointed out another price increase for ERAS applications, a medical professional direct messaged me to encourage me to stop my whining and “consider it your entry fee to the 1%.” Unfortunately, this mindset is exactly what has allows multiple organizations to extract money from medical students with impunity.)
Did the CS opposition have any other arguments?
They did.
CS opponents pointed out that the ECFMG’s Clinical Skills Assessment had a pass rate of 96.9% – and 80% of those who failed did so because of inadequate spoken English proficiency and interpersonal skills. What evidence was there that U.S. medical students had similar deficiencies?
To this, the NBME responded that communication mattered to patients, and that communication problems were the root of most patient medical malpractice suits and complaints to state medical boards. Therefore, they needed to test future physicians’ communication skills with USMLE Step 2 CS.
Both students and the AMA pointed out that this last part didn’t follow. Communication is good, sure – but was there any data showing that the new exam would do what it claimed it would and identify tomorrow’s problem practitioner?
The NBME’s retort was printed in the New England Journal of Medicine:
There is, however, limited evidence that any examination, including the existing components of the USMLE, predicts long-term outcomes. . . [T]here is no precedent for requiring proof that a test predicts the future performance of a student, and the public has little patience for such an intellectual debate.
Did you catch that?
Yeah, so maybe there’s not any evidence that passing Step 2 CS predicts good physicians. But there’s no evidence that Step 1 or Step 2 does, either – and since you accepted them, now you have to accept this!
This curious line of reasoning was followed by a logical red herring – “and the public has little patience for such an intellectual debate” – as the NBME positions itself as the protector of the public, those impatient masses just yearning for their physicians to be tested with a clinical skills exam (even if said exam had no evidence that it was effective).
I highlight this exchange because it’s emblematic of the type of Step 2 CS debate that often continues today. The two sides argue past each other.
The AMA resolution recommended that assessment of clinical skills was “best performed using a rigorous and consistent examination administered by the medical school,” and “should not be used in evaluation for licensure of graduates of LCME- and AOA-accredited medical schools.”
Note that no one was arguing that clinical skills weren’t useful, or that patient communication was unimportant. At issue was not whether a clinical skills evaluation was a good thing to have – it was who should be responsible for providing that evaluation, and how those skills should be assessed. In the opinion of the students and the AMA, the teaching and evaluation of clinical skills could be rigorously performed by accredited U.S. medical schools.
Of course, the NBME didn’t want to engage on that front. They preferred to present their argument for a clinical skills exam as if the opposition were arguing against doctors having good clinical skills and patient communication. They build up the straw man, then righteously tear him down in the name of patient safety.
So did any of the early opposition to USMLE Step 2 CS change anything?
No.
Except for a few furrowed brows and thoughtful beard stroking, the massive student resistance (and significant faculty opposition) to the new exam had precisely zero measurable impact on the NBME’s plan to roll out the CS exam.
And so, despite the angry resolutions, hard-hitting op-eds, and spewed invective, the USMLE Step 2 CS exam went live on June 14, 2004 as a requirement for all medical students.
How was the NBME able to make their new test a requirement for licensure? Aren’t licensing requirements determined by individual state medical boards?
Yes, they are.
And though the NBME had the support of the FSMB in introducing Step 2 CS, it still wouldn’t have been easy to get the legislature in every single state to amend their medical practice acts and require the CS exam.
So they didn’t. They did something far more clever.
The NBME/FSMB simply required passing Step 2 CS in order to register for USMLE Step 3. Because existing state regulations typically required completion of Step 3 for licensure – poof! – just like that, Step 2 CS became mandatory for all medical students.
(This, of course, is the answer to the previous question about why the clinical skills exam was numbered to be part of Step 2 and didn’t become Step 4.)
In reviewing this historical tidbit, I hope that all readers feel two distinct emotions.
The first is admiration – because regardless of how you feel about the NBME, forcing Step 2 CS on students in this way was frickin’ genius.
But the second should be fear.
Why? Because this blueprint would work just as well today as it did in 2004. If the NBME decided tomorrow to introduce a new Step exam, they could, simply by linking its passage with registration for a required future exam. And there wouldn’t be a damn thing anyone could do about it.
What happened next? After Step 2 CS was implemented, did opposition die down?
Sort of.
I can supplement this part of the story with my own memory. See, I took CS in 2006. And by then, student anger seemed to be giving way to a begruding acceptance. Step 2 CS was starting to become just another item on a long list of things that medical students don’t like but have to do.
But on the bright side, the test wasn’t that hard. I remember our class being counseled by our dean that Step 2 CS was just a test of spoken English proficiency, and that students didn’t need to study beyond what they could accomplish on the train ride to Philadelphia.
Was that really true?
It wasn’t far off. In 2006, 98% of test-takers from MD-granting medical schools passed Step 2 CS on their first attempt.
Of course, having a pass rate that high was problematic.
Why?
If you make your living selling an expensive test, a high pass rate is bad for business. Because if everyone passes, pretty soon, your customers start questioning why they have to take it in the first place.
The high pass rate presented another problem, too. The NBME had pitched Step 2 CS as a way to protect the public by screening out incompetent practitioners. But if 97-98% of test-takers pass… and the overwhelming majority of those who initially fail pass on re-take without any additional preparation… then are you really screening out anyone?
In 2013, a couple of residents made this very point in a stinging New England Medical Journal editorial. With some simple calculations, they showed that finding a single examinee who failed Step 2 CS twice cost $1.1 million.
If USMLE Step 2 CS does “weed out” incompetent practitioners, it isn’t cheap to find one.
The authors concluded, “Given the pass rates among examinees and the exam’s costs, we believe that Step 2 CS provides a poor return on investment and little appreciable value to the U.S. health care system – and should therefore be eliminated.”
What was the NBME’s response?
The presidents of the NBME and FSMB wrote a counter piece, in which they indignantly noted that “literally hundreds” of U.S. medical students fail CS each year, and that approximately 3400 had failed CS since its inception in 2004. (Naturally, they were silent on whether any of these students actually gave up on practicing medicine after failing CS – or whether they just had to pay another four-figure exam exam fee before carrying on their merry way.)
They also began to advance a new argument – that the value of the CS exam wasn’t even that it screened out incompetent practitioners. It was that the CS exam improved the curriculum at medical schools. (In fact, several others wrote letters to the editor to make this same point.)
Wait – is it the NBME’s job to improve medical school curricula?
No. But it was a convenient way to move the goalposts in response to the students’ objections about the test’s lack of value.
For the record, medical schools did increase clinical skills teaching in response to the implementation of USMLE Step 2 CS. That’s a good thing. But demanding that medical students cough up a four-figure examination fee is hardly the only way to effect such a change.
See, medical school deans like it when their students to pass the boards, and they’ll change the curriculum to help them do so. But they really like it when their school doesn’t lose its accreditation. Make clinical skills education a priority for LCME accreditation, and watch as schools fall into compliance like dominoes.
Of course, the NBME wasn’t about to hear any of that. Their authors noted that anyone who thinks accredited medical schools could capably assess clinical skills (and allow those test takers to sidestep the Step 2 CS exam) “shows a lack of concern about patient safety and the quality of health care.”
Are you sensing a theme here? You can’t oppose the CS exam – for any reason – without being against patient safety.
Wow. It sounds like the debate over CS was escalating.
It really was. Student resentment toward USMLE Step 2 CS had been simmering for years. All it took was a hashtag and a social media savvy group of medical students to make it boil over.
The student organizers of the End Step 2 CS campaign.
In 2016, a group of Harvard medical students got tired of just complaining about CS and launched #EndStep2CS. What began as a Facebook campaign rapidly galvanized into a multi-pronged political movement.
The primary goal was to present a petition demanding the end of the Step 2 exam to the NBME. Within just a few months, they had gathered over 17,000 signatures – and momentum was still building. Soon, students had a rapidly growing coalition of students and faculty supporting their efforts, with op-eds published in the Los Angeles Times and USA Today and interviews with multiple media outlets.
Other groups started to take notice. Some state medical societies – like those in Massachusetts and Michigan – even drafted resolutions in opposition to Step 2 CS.
It was clear that there there was a large and diverse group who were vehemently opposed to the very existence of the USMLE Step 2 CS exam. Eventually, the NBME couldn’t ignore their voices – and gave the End Step 2 CS organizers an audience with some of their top executives.
Wow! What happened then?
Nothing.
What do you mean?
I mean, after that, the End Step 2 CS campaign kind of slowly faded away.
Why?
Residency.
It’s remarkable how far students got with the End Step 2 CS campaign. I can only imagine the amount of time and effort that the organizers put in. But it’s one thing to lead a grassroots social movement as a fourth year medical student. It’s another to do it when you’re working 80 hours a week as a resident.
Ultimately, the NBME simply held the line, engaged in more thoughtful beard stroking and serious-faced head nodding, and watched the clock.
See, the students couldn’t force the NBME to do anything. Neither could faculty, or deans, or program directors, or state medical societies. Because of the way Step 2 CS was configured – with passage as a registration requirement for Step 3 – the only groups who could change anything were the very ones who had a vested financial interest in maintaining the status quo.
Did End Step 2 CS change anything?
Unfortunately, yes.
In the aftermath of the End Step 2 CS campaign, the USMLE Management Committee met in July 2017. Tired of hearing about how the exam was unnecessary because everyone passed it, they increased the minimum passing standard for Step 2 CS. Now, around 1 in 20 U.S. medical students fails CS on their first attempt.
So there’s no happily ever after, huh?
Nope.
And with that, we come to the end of our story of Step 2 CS resistance. For medical students, its a story of persistent resentment; occasional action; and recurrent futility.
Do you think it’s possible to get rid of USMLE Step 2 CS?
It’s possible – but it’s very unlikely. If you want to know what I think should be done about Step 2 CS, come back soon for Part 3.
The Sheriff of Sodium 