When Areeb Khan tried to prepare for the bar with a mock exam, the virtual proctoring program administering the test couldn’t identify his face. The software, from the company ExamSoft, alerted him that he was sitting in “poor lighting,” even though he was in a well-lit room.
Areeb Khan“Starting to think it has nothing to do with lighting,” Khan tweeted. “Pretty sure we all predicted their facial recognition software wouldn’t work for people of color.”
Khan isn’t alone. As COVID-19 popularized virtual proctoring, students have increasingly reported incidents where the technology failed to register their faces, locked them out of their tests or overzealously flagged them for cheating – particularly students of color and students with disabilities.
Two weeks ago, a group of six Democratic senators, led by U.S. Sen. Richard Blumenthal, showed they shared students’ concerns. They sent letters to three exam proctoring companies – ExamSoft, ProctorU and Proctorio – calling on them to answer a series of questions about their practices and share next steps for preserving students’ privacy and removing potential biases. The letters urge the companies to respond by Dec. 17.
“As we have seen far too often, students have run head on into the shortcomings of these technologies—shortcomings that fall heavily on vulnerable communities and perpetuate discriminatory biases,” the letter to ExamSoft reads. “It is critical that bias, including racial and gender disparities, be addressed expeditiously to ensure that our students of color are not facing additional barriers in their fields.”
The letters were signed by U.S. Senators Elizabeth Warren, Cory Booker, Chris Van Hollen, Tina Smith and Ron Wyden. They point to accounts by people of color, like Khan, who struggled to access their testing materials because of the facial recognition technology – noting that some Muslim students wearing hijab faced similar trouble. They also highlight stories from students with disabilities who were asked to show documentation for their accommodations during test time or flagged as potential cheaters by the software for facial tic disorders or involuntary muscle movements.
Bob Schaeffer“It is our policy to not only make approved accommodations but investigate and make changes any time a test-taker or testing institution feels those accommodations have not been met completely,” ProctorU CEO Scott McFarland said in a statement in The New York Times. “There is no excuse for doing otherwise. When an error does occur, we apologize and pledge to not only fix it but do better in making remote testing and assessment the best possible experience for everyone.”
Since the pandemic began, the use of virtual proctoring companies shot up. More than half of institutions were using the technology in April, according to an Educause poll. And Proctorio CEO Mike Oleson told The Verge that the company proctored over 16 million exams this year, up from an estimated six million last year.
In terms of standardized testing, the SAT and ACT aren’t currently offered at home. But the Law School Admission Council, which administers the LSAT, and Educational Testing Service (ETS), which offers the GRE, use software from ProctorU.
“ETS has no comment on this complaint at this time as we are still learning from these Senators about their concerns,” wrote an ETS spokesperson in an email to Diverse. “As a mission-driven organization, ETS is constantly working with policy makers to help inform policy by offering our assessment and education research expertise. This situation is no different.”
Lawmakers are right to ask for transparency from these companies, said Bob Schaeffer, executive director of the nonprofit FairTest, which advocates for colleges to adopt test optional policies.
While these companies are obligated to monitor cheating, they also need to be able to answer, “What is the evidence that this technology is fair and accurate in identifying test takers who are crossing the ethical line? And does that technology reflect the build-in biases of the people who created it who are subjective human beings?” Schaeffer asked.
He thinks virtual proctoring companies suffer from a lack of oversight.
“The problem is, there’s no test you have to pass to become a proctoring company,” Schaeffer said. “There’s no outside agency monitoring them. They’re just businesses selling product. Who will examine the examiners?”
In general, he sees it as an “oddity” that the testing industry – and the industries surrounding it – remain unregulated, despite their high stakes for students.
Meanwhile, concerns about virtual proctoring are a “lens” into broader equity problems with online testing, Schaeffer added. FairTest is one of the plaintiffs in an ongoing lawsuit against the College Board for “access and technology failures” during this year’s computerized Advanced Placement (AP) exams, which cites a host of issues, including disparate access to Internet.
Even pet food is overseen by the Food and Drug Administration, he said, “but the tests we give our kids aren’t regulated by any industry.”
Sara Weissman can be reached at [email protected].
