A Non-MBA’s Guide to NBME Revenue in 9 Simple Charts

A few weeks ago, I commented on how financial conflict of interest may prevent the NBME from considering a pass/fail USMLE Step 1.

Look, I’m on record as supporting a pass/fail Step 1. But even if you don’t, the lack of transparency in the NBME’s defense of maintaining a scored test should bother you a little bit.

In a recent invited commentary, the NBME President argued that a scored Step 1 was necessary for residency selection for reasons including:

test scores might be correlated with things that correlate with physician quality (but probably not)
it levels the playing field for applicants from non-elite medical schools (unlikely)
the test’s objectivity may serve as “an antidote to implicit bias” (almost certainly untrue)
patient safety might suffer due to excessive use of Netflix and Instagram (yes, they really said this)

Left unmentioned in all of this hand-waving was how a change to a pass/fail test might impact the NBME’s finances.

It’s understandable that the NBME executives would prefer to cast themselves as disinterested arbiters of test policy rather than financially-interested oligarchs sitting atop a sprawling business enterprise that has benefitted from the rise in “Step 1 Mania”. But how we score our licensing exam is an important question for all of us: physicians, students, educators, licensing bodies, and the general public. It’s a topic that deserves a vigorous and transparent debate.

Toward that end, I will present here a simple, nine-chart overview of the NBME’s revenue.

1. NBME program revenue has grown rapidly since 2001

And no, this rise is not explained by inflation. This is real growth in real money taken in by the Board. Revenue derived from the NBME’s programs has more than tripled from 2001 ($47.5 million) to 2017 ($153.9 million).

2. The NBME’s growth has been good for its executives

This chart shows the total compensation for former NBME President, Donald Melnick. Along with the tripling of revenues at the NBME, his compensation tripled, to $1.2 million in 2016.

(Also, in case you’re thinking of applying for the job, beyond the compensation and benefits, the NBME President also gets free first-class airfare for himself and his travel partner, as well as a membership to a Philadelphia social club that offers $45 tequila shots and dancing at eye-level with the city skyline.)

However, it’s not just the NBME President that has benefitted. According to the 2017 Form 990, Schedule J, two lower executives received total compensation over $700,000; another two over $600,000; another three receiving over $500,000; and another 6 receiving over $400,000. All told, in 2017, the NBME payroll included 207 persons receiving more than $100,000/year – a figure that has more than doubled since 2008.

So where is all this money coming from?

3. The NBME derives revenue from four service lines

Clearly, the USMLE program is the primary revenue generator for the NBME, accounting for 72% of total revenue in 2017.

But take note of the orange wedge – the Medical School and Student Services program line – which accounted for 20% of revenues in 2017. This has become an extremely important source of revenue for the NBME in recent years because…

4. The USMLE ‘pipeline’ is not growing

The total number of test-takers for the USMLE Step 1 has been essentially unchanged for the past 10-15 years.

This creates an interesting business environment if you’re trying to achieve growth. How do you increase your revenue when you have a fixed number of customers? Well, you have to squeeze more money out of each one of them.

So how can you do that?

You could increase the price of your mandatory tests. The NBME has done this a little bit. For a U.S. medical student, Step 1 registration now costs $630, vs. $525 in 2010. That’s a 20% increase – which buys you a few extra $45 tequila shots while you’re dancing at eye-level with the city skyline, but isn’t going to triple your organization’s revenue.

You could also add another mandatory test. The NBME did that in 2004 with the introduction of Step 2 CS. The registration fee for Step 2 CS is currently $1290. That ought to fund an executive or two.

But the problem with both of these strategies is that students complain when you raise the price on a mandatory product or add another mandatory test. And if the complaints get too loud, or too persistent, someone may try to do something about it. In a worst-case scenario, a state medical board might be persuaded into offering or accepting an alternative, lower-priced licensing exam – which would disrupt the NBME’s monopoly and risk toppling their financial house of cards altogether.

A better strategy is to sell customers products that they choose to pay for. You get much less complaining that way. And this is where the Medical School and Student Services programs come in.

The NBME’s Medical School and Student Services programs include a variety of non-mandatory testing products that medical schools or medical students can purchase. Here, let’s focus on just two specific products.

5. Students are buying more NBME self-assessment exams

The Comprehensive Basic Science Self Assessment (CBSSA) is a practice test of Step 1-type questions that has been offered by the NBME since 2003.

The NBME advertised the CBSSA in Academic Medicine, noting that the correlation between CBSSA and USMLE Step 1 scores was strong (R2 = 0.67). The manuscript includes this quote (my emphasis added):

Although Step 1 is designed to be part of the licensing process, it also has important secondary uses for U.S. students that make the pressure to obtain a passing score on the first attempt extremely high. Two common secondary uses are the requirement of a passing score for promotion to the third year at many medical schools and the importance of a high score for acceptance into competitive residency training programs.

Would you like to be promoted to the third year of medical school, or gain acceptance to a competitive residency training program? Sure, we all do!

(As a sidenote, this paper – written by NBME authors – included no financial disclosure statement or mention of potential financial conflict of interest, either.)

You can purchase as many CBSSA exams as you like. They cost $60 apiece – so with sales of 128,934 in 2016, the NBME cleared over $7.7 million.

6. Medical schools are buying more NBME tests to give to their students

Customized Assessment Services (CAS) is a program started in 2007 that allows medical schools to create their own USMLE-style exams using questions from the NBME. We use it at my medical school to make the midterm and final exams for every preclinical unit. And we’re not alone – 99 medical schools subscribe to CAS (including 16 international medical schools).

It’s not exactly cheap. Schools pay $1,500/year for a subscription, then a per-test fee depending on the number of test items. If we assume that all of the 102,135 tests administered in 2017 were in the middle price range (likely a conservative estimate), then CAS generated $3.1 million in revenue in 2017.

I should mention that, although the medical school pays for CAS, I doubt many do it out of charity. These costs get rolled up in tuition – so students are pay for it, even if they do so indirectly.

7. Medical School and Student Services program revenue is climbing rapidly

Riding the wave of USMLE Step 1 mania, revenue growth has been robust in the Medical School and Student Services program.

Note, however, that the orange bars here represent the NBME’s total revenue. And you don’t have to have an MBA to know that that’s just one side of the equation. All of the NBME’s programs have costs – so one thing we may want to know is how much revenue these programs generate revenue in excess of their costs. (It is technically appropriate to call this “profit” – but to avoid confusion, I’ll try to use the term “excess revenue” since the NBME is a non-profit organization.)

And this is where things get very interesting.

8. In 2017, the NBME derived more excess revenue from Medical School and Student Services than from the USMLE

As I showed before, the USMLE program generates a lot of revenue… but it’s also an expensive program to administer. In 2017, it generated $110.5 million in revenue – but carried $99.6 million in expenses.

On the other hand, most of the Medical School and Student Services products are comparatively cheap. There are some costs associated with maintaining the computer platform, ensuring test security, etc. – but the real thing you’re selling is the questions. And those are cheap. Most of them were written by volunteers, and have been retired or passed over for use on the USMLE. Without the CBSSA or CAS to monetize them, they’d just be sitting around in a file drawer in Philadelphia gathering dust. The bottom line: in 2017, Medical School and Student Services generated $30.5 million in revenue with a program cost of just $15 million.

9. In fact, Medical School and Student Services generates most of the NBME’s profits

Note the difference between this pie chart and the first one I presented above. The grey and yellow slices disappeared, because those programs cost more to administer than they generated in revenue. (These programs, which include the Health Profession Services and all of the minor programs run by the NBME – including its recent foray into the Health & Wellness Coaching certification market – had expenses that exceeded their revenues by $6.7 million overall.)

In other words, in fiscal year 2017, the Medical School and Student Services program was responsible for 59% of the excess revenue (profit) received by the NBME.

Since the NBME is a 501(c)(3) nonprofit organization, it can’t distribute these excess revenues to shareholders. However, since these monies were derived from activities related to the organization’s nonprofit mission, they are completely within their rights to reinvest these funds in the organization.

They can, for instance, expand existing testing programs or come up with new ones. (USMLE Step 4, anyone?). Or they could – and hey, I’m just spitballing here – give their executives a big fat raise for doing such good work. It’s really up to them, so long as the spending can be justified as going toward the organization’s overall mission.

—

So what do you think? Should the NBME disclose a financial conflict of interest when they discuss whether Step 1 should be pass/fail or scored? Can you see any reason why the NBME may wish to advocate for maintaining the status quo – a reason beyond their impassioned advocacy for “other stakeholder needs”?

If you’re still undecided, then ponder these questions:

If USMLE Step 1 became pass/fail, would you expect the NBME to sell more CBSSA and CAS exams, or fewer?
And if they sold fewer, would you expect the organization’s profit/excess revenue to increase, or decrease?
And if it decreased, would their executives get larger raises, or smaller ones? Would they have more job security, or less?
If you were in their shoes, what would you do?

—

Notes

Want to check my figures? Please do. All of these data are publicly available from the NBME’s IRS Form 990 tax returns or their Annual Reports. You don’t have to agree with my conclusions, but all the data presented should be accurately reported. I want to get it right – so if you find a mistake, please let me know.
Lost revenue from ancillary products may be only part of the story. There may be another reason that the NBME opposes a pass/fail test… one that would threaten their business interests in a different but fundamental way. Find out more here.

ADDENDUM

For anyone who needs a more authoritative reference than “I saw it in some guy’s blog”, the data presented here are also now published in Academic Medicine. (The article was published alongside an independent commentary describing the current “crisis of trust” due to the NBME’s financial COI.)