Origins: On Netflix, Instagram, and a pass/fail USMLE Step 1

In large part, this blog was started to archive some recent discussions on social media. As you probably know, from time to time I tweet (@jbcarmody). While the immediacy of Twitter is incredible, it’s easy for more substantive content to get swept up in the current and disappear. Lately, I’ve been a part of a few discussions that I felt should be archived in a more easily-locatable way – which is what I hope to do here.

This all got started in the final days of 2018. For the past couple of years, I’ve taught the kidney module to second year medical students, and was reviewing some of the course feedback. In response to a request for how a certain clinically-focused component of the module could be improved, one student wrote this:

The sentiment here is extremely common, but this student framed the issues so starkly that it gave me pause.

Ever since I got involved with undergraduate medical education, I’ve recognized that we have a major problem with the way that USMLE Step 1 scores are being used. By using Step 1 scores in residency selection, we have taken a test that was intended to serve a useful purpose – to make a binary decision on a physician’s appropriateness for licensure – and turned it into a high-stakes medical trivia competition. The increasing focus on Step 1 crowds out more important content from both the medical school curriculum and the thoughtspace of our medical students. It’s hurting the quality of undergraduate medical education, and by extension, may hamper graduate medical education and even patient care. And worst of all, it’s getting worse, and there’s no natural end in sight.

I’m hardly the first to point any of this out. In fact, a group of medical students recently wrote an article in which they described the deleterious effects of the “Step 1 Climate” in medical education.

So why not make the USMLE Pass/Fail?

This is a natural question, right? If excessive student focus on USMLE Step 1 is producing diminishing (or negative) educational returns, why not report scores as pass/fail? After all, this is the primary purpose of the test.

The USMLE is jointly sponsored by the National Board of Medical Examiners (NBME) and Federation of State Medical Boards (FSMB). So what do their presidents have to say about a pass/fail Step 1?

This.

The statement speaks for itself. However, to ensure that it is read in the appropriate context, two additional things should be pointed out.

This was published in a peer-reviewed academic journal. This statement was apparently not found objectionable by the authors, the referees, or the editor.
This was published in direct response to the article mentioned above expressing medical student concerns with the “Step 1 climate.”

Naturally, when I pointed this out, it generated a vigorous response on Twitter, from both medical students and faculty. Shannon McNamara (@ShannonOMac) was particularly articulate and vocal in spreading the word and holding the authors accountable.

Within hours, Dr. Katsufrakis made a statement of “mea culpa” on Twitter, and later, the NBME and FSMB issued a joint apology.

So this is how things got started. But this is not where it ends. I feel strongly that we’ve reached a tipping point. Until now, I fretted and complained about how “Step 1 Mania” was hurting medical education – but I thought that the best thing I could do as an educator was help my students succeed in the gauntlet that we created for them.

Now I’m not so sure. See, the more I’ve looked into things – like the financial conflict of interest for those most vocal about maintaining the status quo – the more deeply convinced I’ve become that the whole system is rotten and we have to do more.

In posts to come, I’m going to try to dig into these issues in a substantive and data-driven manner, and I hope you’ll come along with me. But every story has to start somewhere, right?