A new scoring approach introduced in 2009 was supposed to diminish bias during the National Institutes of Health (NIH) Enhanced Peer Review process, but Black researchers applying for the agency’s prestigious and highly competitive R01 grants consistently receive lower scores than White applicants in the first and critical phase of consideration, a new study reveals.
Carol J. Lee (left) and Elena A. Erosheva. (Photo: University of Washington)
Those preliminary and worse scores largely account for why Black investigators receive fewer NIH grants, said University of Washington researchers Elena A. Erosheva, a statistics and social work professor, and Carole J. Lee, an associate professor of philosophy who specializes in peer review. Erosheva and Lee, with four other colleagues, prepared the report, released this month, under an NIH contract to study the Black-White racial gap in the peer review process for the years 2014 to 2016.
Their findings focused on an early phase of the R01 review process. When NIH-assigned peer reviewers receive grant applications, they read and score them based on five criteria — significance, innovation, approach of the grant application, investigator(s) and research environment. They then weigh those scores to derive a “preliminary overall impact score,” which is supposed to reflect a project’s potential for major impact, said Erosheva and Lee. The better the overall impact or preliminary score applicants receive, the more likely they are to advance to the next round and ultimately receive an NIH research grant.
After evaluating preliminary scores from nearly 55,000 NIH grant applications, the study’s authors found that Black researchers receive worse scores on all five criteria — even after controlling for “key application and applicant characteristics,” such as their career stage, gender, the type of degree they earned and their area of science.
The authors demonstrated that these differences in preliminary scores entirely account for Black-White disparities in funding. “This was surprising to us,” said Erosheva. They found that from 2014 to 2016, the overall award probability for Black applicants was “55% of that for White applicants — [or] 10.2% compared to 18.5%.”
That a racial gap exists in grants awarded by the NIH has been known for a while. In 2011, a study published in Science found that the Black-White funding gap was wide. A Black investigator’s chance of winning a sought-after NIH funding award was 10 percentage points lower than that of a White investigator.
While Lee and Erosheva cautioned against making comparisons to work done before the NIH introduced the five criteria scoring process in 2009 to mitigate bias and improve transparency in funding decisions, their new findings align with those of researchers before them.
The report’s authors told Diverse that their new study did not seek the reason for why Black applicants consistently score lower than White applicants. But like other groundbreaking studies, Lee and Erosheva identified substantial racial disparities in the NIH R01 review system.
Lee pointed to a 2019 study in the journal Science Advances. This NIH study showed that research topic contributes to the persistent gap in the agency’s funding awards to Black scientists, accounting for more than “20% of the gap” after controlling for a host of factors, including the applicant’s prior achievements. Black applicants disproportionately proposed health disparities research, which is less likely to receive NIH funding.
At the time, NIH Director Francis S. Collins, M.D., Ph.D. called the results surprising.
“We need to understand whether there is an intrinsic bias against such topics by reviewers, or whether the methodologies used in those fields of research need an upgrade,” Collins said in a 2019 statement.
And while Erosheva and Lee’s new study stopped short of drawing causal conclusions, the authors suggested that investigating “choice of research topic, implicit racial preferences, and differences in mentorship, coauthor networks,” and other factors may determine why the longstanding Black-White disparity in NIH-peer review exists.
“My hope is that, by bringing attention to these matters, we are getting closer to seeing real change happen in the future,” said Erosheva.