#ERASGATE

It was around a week ago that I first heard that there might be a problem with the Electronic Residency Application Service (ERAS).

I got a message from a well-connected faculty member at another institution asking if I’d heard from any students who had problems with ERAS not delivering certain portions of their application to the residency programs to which they’d applied.

No, I said. I hadn’t.

But within a couple of days, I began to see applicants sharing their stories (like this or this) on Twitter. Several other students who were afraid to speak up publicly began to message me directly.

Their reports shared a common form.

An applicant asks a faculty member for a letter of recommendation (LoR). The faculty member writes the letter and uploads it to ERAS.

The applicant then assigns the letter in ERAS to certain residency programs to which to which they applied.

But then, the application season opens, and the applicant receives no interview offers. Or maybe they fail to receive an interview offer from a program from which they expected they would. Upon inquiring, the applicant learns that their application was incomplete, because the program did not receive the LoRs.

In another common variation, the the applicant receives an interview offer, but then gets a message from the program coordinator reminding them that their application is still incomplete and asking when the applicant plans to have their LoRs uploaded.

In either situation, the applicant believes very strongly that they assigned all the letters appropriately, and may have a distinct recollection of double- or even triple-checking to be sure that all letters were assigned and their application was complete.

A sampling of some of the messages I received on Twitter.

After a few days of increasing attention on social media, ERAS responded, stating that they could find no evidence of systems-related errors. In other words, the only LoRs that were not assigned were because applicants did not assign them.

Official statement from the AAMC on October 29, 2021.

–

The accounts of the applicants and the statement from ERAS are not compatible. So who should we believe?

In this corner…

…we have the applicants, a group of type A overachievers who have reached this point in their lives in no small part due to their attention to detail, who were using ERAS to engage in the high-stakes, one-shot-do-not-miss-your-chance-to-blow-this-opportunity-comes-once -in-a-lifetime game of residency selection.

In the other corner…

…we have the AAMC’s Electronic Residency Application Service, a usually-functional but hardly cutting edge website that tends to crash during periods of high traffic (see: SOAP most years) and seems to require unusually frequent and prolonged periods of system downtime for ‘scheduled maintenance’ throughout the application season just to maintain functionality.

Hmmm…

So yes, it’s possible that all of these applicants simply forgot to assign their LORs, thus fumbling the ball on the 1-yard line of their decades-long journey to becoming a doctor. But any fair-minded observer would have to acknowledge that ERAS does not have a confidence-inspiring record of IT prowess, and the idea that there could be an ERAS glitch is not at all far-fetched.

ERAS does not have a confidence-inspiring record of IT prowess.

So let’s walk through a few possibilities here.

–

1. ERAS is right.

Maybe ERAS is working as designed, and all letters were delivered or not, exactly as assigned by the applicants.

I mean, it’s possible.

But if you believe this, you have to wonder why this seems to be a new problem this year. Did the Class of 2022 just forget how to use ERAS? Did this generation of digital natives who grew up with the internet suddenly lose their ability to fill out a web form when previous generations of students seemingly did this without difficulty?

(Or maybe, like one dean who e-mailed me, you believe that this isn’t a new problem, and a small number of applicants fail to assign their letters each year. Maybe this year’s applicants are just more willing to speak up, or maybe social media allows concerns that once would have fallen on deaf ears to be amplified.)

After all, even though these stories are highly visible, they still seem to affect only a tiny fraction of applicants: according to the official statistics, over 46,000 applicants have used ERAS to apply to residency programs this year.

So maybe ERAS is right, and there really is nothing to see here.

Maybe.

Or maybe…

–

2. There’s a coverup.

Maybe there was a glitch – and now there’s a cover up.

Maybe ERAS inadvertently unassigned some letters that were properly assigned. And maybe the auditors discovered this, but instead of acknowledging it, chose instead to claim that it did not occur. Maybe the plan is to stonewall and blame the applicants until things blow over.

The Electronic Residency Application Service (ERAS) is the AAMC’s cash cow, with **$100 million** in revenue for fiscal year 2020.

Remember, ERAS is the AAMC’s biggest revenue generator, and a mistake like this could be financially devastating. Not only might it urge programs to stop using ERAS, but what if that the affected applicants made a legal case that their potential earnings as a physician were impacted because they failed to match for a year because of applications that were erroneously listed as incomplete? If such a case could be made, the exposure is potentially enormous.

And in this scenario, if there were a glitch, why not cover it up? Who’s gonna contradict your story that the software is working? No one can fact check you: what happens in ERAS stays in ERAS.

This, too, is possible.

But to believe it requires assuming that a significant number of people at the AAMC would be willing to go along with such a cover up. And I frankly don’t believe that to be the case.

Look, I have my doubts about the AAMC’s motives in certain situations, and I’ve been pretty public about airing them. But the idea that a bunch of people at the AAMC – not just highly-paid executives, but regular folks just doing their job – would engage in behavior that’s not only blackhearted but outright criminal strains credulity.

But there is, of course, another possibility – one that better reconciles the experiences of the applicants with the statement from ERAS.

–

3. There’s a glitch.

And the nature of that glitch is such that certain LoRs got unassigned soon after applicants assigned them… so that when auditors look now, it appears as though they were never assigned by the applicants.

This, to me, seems like the likeliest possibility.

These reports from applicants are credible – and they’re troubling in their frequency and consistency. And a glitch like this would also square with the ERAS statement that “all letters of recommendation were delivered as assigned” – because if the system erroneously registered them as being unassigned, it would appear as though they were never assigned by the applicants in the first place, wouldn’t it?

So what should happen next?

If I were the CEO of the AAMC, this is what I’d do.

1. Acknowledge that this is a real problem.

For applicants, the implications of an ERAS glitch are potentially career-altering. A three-paragraph statement with a link to your instructions for assigning letters does not begin to acknowledge the gravity of the situation for the applicants potentially affected.

The tone of the ERAS response doesn’t fit the seriousness of the allegations, as the world’s favorite physician comedian pointed out.

–

2. Run a real investigation.

This is not an issue that can be authoritatively resolved from a computer terminal. What the AAMC needs to do is investigate these complaints the same way that a police detective would – by talking to the parties involved and finding out what they saw and when. Eyewitness testimony should be taken at face value, and the system should be tested to see if a similar problem could be provoked.

Better yet, the AAMC should hire an outside auditor to review the electronic trail, or an independent law firm to investigate the credibility of the complaints.

Does that suggestion sound naive? Not to me. The AAMC should understand that if they allow this to mushroom, they will be facing a real crisis of confidence – if they aren’t already.

I’ve gotten multiple messages from panicked students wondering how they would be able to tell if their letters weren’t received. Meanwhile, student affairs deans are sending out e-mails like the one below urging their fourth years to triple-check ERAS.

Students and schools are spooked about the possibility of an ERAS glitch.

The trouble is, you never know why you don’t get something.

Maybe you didn’t get an interview because your USMLE score was too low, or because the program director thought your personal statement was corny, or because your LoRs were uploaded – and they weren’t as flattering as you’d hoped. You probably never know – you just know that you didn’t get an interview.

Until or unless these concerns are resolved, any applicant who doesn’t get an interview will be left wondering: did they actually get my application?

So if you’re concerned with long-range strategic planning, this loss of confidence is a huge threat to your operation.

Remember, that residency programs should use ERAS to process applications is not a commandment that was spoken out of a burning bush or handed down on stone tablets from heaven. The service ERAS provides is neither unique nor irreplaceable, and it wouldn’t take much for an entrepreneur to realize they could disrupt this marketplace by building a better mousetrap.

–

3. Transparently report the investigation’s results.

Again, a statement on ERAS letterhead saying that the allegations are “unsubstantiated” is inadequate.

If the unassigned letters were really due to applicant error, then you need to convince your stakeholders that you’ve done your due diligence by sharing with them exactly what the investigation looked like and why it would have uncovered any accidentally unassigned letters. (Then you need to fix the software so that future applicants can’t make this mistake: applying to programs without assigning letters should get a hard stop in the system, just like if I tried to order 1000 mg of morphine for a patient in the EMR.)

The AAMC is recruiting a technical support specialist to work on, among other things, investigating, identifying, and resolving “inaccurate data/files.”

And if the investigation should discover that a mistake was made, be prepared to be held accountable.

This won’t be pretty.

The residency interview season moves fast, and there are no backsies. Programs are awash in applications; many won’t review an application unless it is filtered as being complete – i.e., with the applicant’s information accompanied by the requisite number of LoRs. Most residency programs have already offered the bulk of their interviews, so if applicants have been affected by an ERAS glitch, there is precious little time for them to receive a remedy for it this year. But a personal message to programs from the AAMC leadership saying that such-and-such application may not have received full consideration because of our mistake might help a few applicants salvage an interview offer or two – and would probably place the organization on sounder footing if applicants choose to pursue other remedies in the future.

Look, the AAMC is a not-for-profit corporation whose mission is to “lead and serve the academic medicine community.” They’ve got an opportunity to showcase that leadership and service right now.

Will they take it?

–