Sergine Beaubrun never thought she would have to grab her laptop and stand on a table, putting her face right next to the ceiling light, just to take her mock bar exam. She said that ExamSoft, which is often used to remotely proctor the single most important test a law student will take, had difficulty registering — or even detecting — her face.
Sergine Beaubrun, at her home in Brooklyn, New York, tells other students who have to use ExamSoft not to be discouraged if virtual proctoring software can’t see them. (Photo: Scott Heins/YR Media)
On the day of her actual bar exam, Beaubrun said that she sought help from an attorney she found on Twitter, who suggested that Beaubrun avoid backlighting in order to increase the contrast between her and her background, so that ExamSoft would register her face. The tip worked.
But her technical difficulties left Beaubrun bewildered. If these were the hurdles she had to jump through just to take her bar exam, what would it be like once she started her career?
“It's just like, was this profession really meant for me? The software isn't. It shouldn't have taken me jumping on a table and going this close to light, just so you guys could recognize me,” Beaubrun said. “I hope that this profession recognizes me as a person when I enter it.”
These kinds of questions are what students of color and others are asking when proctoring software fails to work for them or interferes with their ability to perform at their best. It’s that serious. So how did we get here?
The Rise of Virtual Proctoring in the Pandemic
Attending virtual classes — and taking virtual tests — isn’t ideal for anyone. All through the pandemic, faculty and students alike have struggled to create the genuine connections and rich dialogues that happen when everyone’s in a room together. Technical challenges range from random computer glitches to missing out on important bits of the lecture when the WiFi fades. Zoom fatigue spares no one. And with everything on the computer, professors need new tools to make sure their learning environments are fair, which requires an ability for those who use testing to prevent and catch cheating.
Some university administrators believed they’d found a solution to this last concern by contracting with virtual proctoring companies like, say, ExamSoft, which was founded in 1998. Since then, it has administered 75,000,000 exams, according to the company website.
Jack Schutte, a first-year student at UC Santa Cruz, shared his experience over text with YR Media during his first time taking an exam with ProctorU. (Photo courtesy of Zoe Harwood.)
ExamSoft isn’t the only such product on the market. In May 2020, Proctorio's Chief Executive Mike Olsen told the New York Times that after campuses started going remote because of the pandemic, his virtual proctoring company grew 900 percent. Another large-scale virtual proctoring service is ProctorU, which reported proctoring over 5,000,000 exams. Jarrod Morgan, founder and chief strategy officer of ProctorU, told YR Media: “We feel very strongly that we should protect the education industry and the integrity of exams.”
“The pandemic put everyone in a situation where they had to use this,” Morgan said of virtual proctoring. “Nothing really changed about our service, but the fact that we were not really an option in 2020: we were a requirement to continue to work.”
I’m a freshman virtually attending classes at University of California at Santa Cruz, and I’ve heard lots of stories from students using proctoring software for tests. By and large, the chatter amongst my friends hasn’t been good. I spent several months looking into the rise of various virtual proctoring companies, the effects they’re having on students, and whether, in the eyes of students, the benefits outweigh the costs.
Students I spoke with reported that using proctoring software added significant anxiety to the already-stressful situation of taking a high-stakes test, and that the anxiety could affect their performance. Many also expressed concern that these companies violated their rights in one way or another. Some raised issues related to digital privacy, others about racial and other forms of bias within the technology. Our data analysis of 79 Change.org petitions against virtual proctoring software in the United States echoed these themes. Petitions were launched across 28 states. Privacy — mentioned in 72 of the 79 petitions — was the issue cited the most. Over half of the petitions raised concerns that virtual proctoring drove inequality and damaged mental health:
How Virtual Proctoring Works
The variety of online proctoring services on the market today is vast. For the sake of this investigation, we focused on three types: 1) automated tools using artificial intelligence (AI) in place of live proctors, 2) those involving live proctors, and 3) tools that lock down browsers and prevent students from switching tabs, without a proctor involved.
ExamSoft technology goes back to the days of floppy disks. In 1999, the company filed a patent that outlined plans to develop technology that secures exams in computer environments. Today, the company offers two features that use facial authentication and AI for automated proctoring. When students sit down to take a test, ExamID uses “facematching analysis” to verify that they are who they claim to be. ExamMonitor, a second product, records video and audio of students all through the test-taking experience and uses artificial intelligence (from a third-party company) to analyze data including gaze, eye movements and sound to flag anomalies. A (human) proctor then sifts through behaviors to note any unusual behavior, “giving the institution the final review,” according to the ExamSoft website. Third parties that ExamSoft works with also “use the exam videos to refine their artificial intelligence to improve the accuracy of their virtual proctoring software,” according to a letter from ExamSoft that addresses concerns with the software. ExamSoft says, after one year, it “automatically deletes an exam-taker’s video.”
ProctorU combines automated and live proctoring. The company collects biometric data and then uses third-party tools to verify student identities and track keystrokes and other data points during test-taking, according to ProctorU's 2019 Technical Security Overview. But that’s just the automated aspect of its service. With ProctorU, real, actual people sit down with you (well, via your screen, not in the room) and watch you take your test. While the proctors monitor students through their webcams, the students YR Media interviewed said that they couldn’t see the proctors at all, and could only talk to a disembodied voice. ProctorU founder Morgan said live proctors can make active interventions. For example, if a test-taker is “constantly looking out the window,” Morgan said, the proctor can remind the student: “’Hey, I need you to keep your focus here.’ It could be [the student] picking up their cell phone or could be something as flagrant as picking up an answer key.”
ProctorU’s statistics on academic integrity, from February 2021. Out of all the exams they proctored, only .72% involved confirmed “breaches.” As far as we can tell, ProctorU has removed this graphic from its website.
Finally, Respondus, while not involving any proctors, purports to preserve academic integrity by shutting down students’ ability to switch tabs, ostensibly preventing them from looking up answers online. According to the company’s website, 100,000,000 exams are proctored through their service each year, and it also reports that third-party services use Respondus in their assessment programs. This tool is the least complicated out of the three. However, students still report problems with it.
Areeb Been Khan, a law graduate based in New York City, grew anxious when ExamSoft virtual proctoring software failed to detect his face before a mock exam. (Photo: Suzannah Hoover/YR Media)
When Software Can’t See You
Bias and other concerns related to AI are nothing new, as YR Media’s own newsroom has reported through stories featuring leading voices in AI, explorations of deepfakes in music and beyond, and an interactive investigation into facial recognition systems.
We know that tech giants including IBM, Microsoft, and Amazon all have a track record for creating artificial intelligence that carries a gender and racial bias, according to Joy Buolamwini, an AI researcher working at MIT who has studied this phenomenon extensively. Buolamwini found that because faces used to test the software often leave out women and people of color, the software is only “optimized for a small portion of the world.”
When YR asked if ExamSoft had difficulty registering faces of students of color, Nici Sandberg, the company’s associate director of marketing content and communications, said that ExamSoft "does not have any data that supports the claim of any significant problems with the software detecting faces of students of color."
But our interviews with students show that Sergine Beaubrun was not alone in her struggle to be seen. Areeb Been Khan also had trouble with his mock bar exam. He told YR Media that it took five business days before Examsoft detected his face at all. “That was anxiety inducing and it made it difficult to study information that was going to impact my livelihood,” Been Khan said.
Been Khan gave YR Media the emails he received from ExamSoft. The emails suggested he could lose his registration status to take the exam. (Photo courtesy of Areeb Been Khan.)
Been Khan said he emailed ExamSoft on a Monday, got a written response on a Wednesday, and when that wasn’t sufficient, he called customer support Friday morning but couldn’t get through because of a long wait. After selecting the option of an automated call back, he said he finally received help Friday evening.
Been Khan said that when customer service called him back, an associate made changes that eventually permitted him to take the test. “He said straight up … sometimes our program just needs, like, an updated data set … like a baseline for your face,’ which proves to me that they didn’t have a baseline for my face,” Been Khan said. “There's racism in technology, and that's kind of just what it comes down to, at least for me.”
ExamSoft confirmed to YR Media that it was aware of Been Kahn’s experience and wrote that after he called the ExamSoft support team, “his issue was resolved within minutes of speaking with a representative.”
As more allegations similar to Been Khan’s and Beaubrun’s came to light during the pandemic, the U.S. government took notice. Six U.S. Senators wrote a letter to Sebastian Vos, the Chief Executive Officer of ExamSoft, in December 2020. The lawmakers asked ExamSoft to outline plans that safeguard the civil rights of students of color and students with disabilities. The letter cites numerous media reports of proctoring apps erroneously flagging students for possible cheating.
ExamSoft responded to the senators’ letter more or less by saying: that facial recognition technology that we use... it isn’t ours. It denied that there is any bias in its products and said that the more common difficulties for exam-takers are internet connectivity, poor picture quality, and lighting issues. Responding to concerns about how the software could unfairly flag students with disabilities, ExamSoft said it was not involved with granting accommodations and that it’s up to institutions to make those accommodations when using the software.
The ExamSoft letter signed by Vos said, “We have investigated whether there have been any systemic issues with ExamID due to individual exam-takers’ race or gender identification. It would be a key priority to resolve any such issues, but we found none.”
Vos' letter added that ExamSoft was making "Deferred ID" mandatory for a majority of its institutions in 2021. The "Deferred ID" feature allows exam takers to start their exam even if there is a problem with photo matching, said Sandberg, the ExamSoft spokesperson. She did not confirm whether making the feature mandatory was a response to stories like Sergine's or Areeb's.
Jack Schutte, a first-year student at UC Santa Cruz, said he felt forced to give up his privacy while studying remotely at his home in El Cerrito, Calif. He said a virtual proctor asked for a 360 view of his room multiple times to ensure Schutte wasn’t cheating during his exam. (Photo: Marlena Sloss/YR Media)
The Cost of Virtual Proctoring
Jack Schutte watched with stunned amazement as his mouse and keyboard moved all on their own — or not quite on their own, but guided by an invisible proctor on the other end of his computer.
Schutte, like me, attends UCSC. He used ProctorU to take an exam for his calculus class. He and I are close friends.
The first thing Schutte had to do before he could take his test was download a ProctorU extension on Chrome. Schutte became uneasy as soon as he read the extension’s permissions, which included the ability during the test for ProctorU to “read and change all your data on the websites you visit,” “read and modify data you copy and paste,” “detect your physical location,” and “capture the content of your screen.”
“Between taking control of someone’s computer [and] reading and accessing all of your data ... it feels like someone [could] exploit the university’s use of ProctorU for nefarious ends,” Schutte said.
Because Schutte was using a product that included live proctoring, unlike Beaubrun and Been Khan, he could actually talk to the person who had control over his entire computer during the exam time. When the proctor asked for permission to take control, Schutte said he felt nervous.
Having an “outside proctor to monitor you, in your own living space... if ... there was an actual person that came and sat in your home and watched you take your test ... You feel like they're expecting you to cheat. They just want someone there to catch it when it happens,” Schutte said. But he went along with it anyway. “It was my exams,” he said. “I couldn’t exactly refuse at that point.”
UCSC covered the cost of Schutte’s exam, but that’s not the case at all universities. The University of Illinois Urbana-Champaign, for example, listed ProctorU costing between $16 and $38 per exam, depending on length, with late fees that could run between $8 and $12 if you don’t reserve your ProctorU appointment at least three days in advance. Jaylyn Stubbs, a teaching assistant from Louisiana State University, expressed concern for low-income students already struggling to stay afloat. “It's kind of unfair,” Stubbs said. “For the students that can't really afford it, I don't feel like it's fair that they have to come out of pocket to take a test.”
When asked for comment on Stubb’s concern, LSU told YR Media that faculty members are required to communicate costs to students in syllabuses at the beginning of the semester, and that the university offers “other primary assessment options.” Professors opt for ProctorU when “they feel that this is the best and only virtual route to assess the student during these times,” Alison Lee Satake, the Associate Director of Research at Louisiana State University, wrote in an email.
Universities that have added or extended use of these services during the pandemic have also done so at a cost — literally. Take my own university. In the years prior to the pandemic, UCSC’s agreements with ProctorU were largely with the university’s off-campus UC Extension programming, based on purchase orders YR Media obtained through public record requests. Between 2016-2019, these records showed the university approved $67,600 in fees to ProctorU.
When the coronavirus hit, UCSC’s Vice Provost for Academic Affairs Herbie Lee authorized spending on ProctorU for the main campus during the 2020-21 school year, and for a lot more money: $200,000.
Lee couldn’t tell us which part of UCSC’s budget was used to cover the software costs and said that he could not directly provide invoices from ProctorU that would confirm how much UCSC has spent to date on the software. “There was a lot of allocation that was done very quickly in response to the conversion to emergency remote instruction. A lot of things happen behind the scenes,” Lee said. “I wasn’t a part of exactly which type of funding was used for this.”
YR Media Journalist Zoe Harwood placed a FOIA request asking all UCs and CSUs for their vendor contracts with virtual proctoring companies. UC Santa Cruz had the largest purchase order of all the universities that responded.
Compared with some other universities in the UC system, UCSC’s $200,000 in authorized spending on ProctorU was a lot of money. UC Merced authorized spending of $20,997 on Respondus, and UC Santa Barbara, which is larger than UCSC, authorized spending around $60,000 on ProctorU from March 2020 to June 2021, according to public records requests.
Given the problems with proctoring software, plus the anxiety it can cause, I began to wonder: what if some of that $200,000 had gone towards programs that better provide mental health services to students struggling during the pandemic? Based on YR Media’s prior reporting for NPR, we know that even before the coronavirus hit, universities were grappling with how to provide vital mental health services to students. During the pandemic, students experienced mental health issues at even higher rates. With limited budgets, administrators are forced to make tough choices about what services to pay for. We can learn a lot about their priorities from where they put their money.
ProctorU wouldn’t confirm how much money it made during the pandemic. But CSO and Founder Jarrod Morgan maintained that the company provides a valuable service to education. “If we don't or somebody doesn't … make sure that ... the students that are getting the credit are the ones that did the work,” he said, “then over time, it really waters down the value of the education credential.”
Andrea Kohlenberg, based in Chicago, felt ProctorU violated her right to privacy as a student. She is now suing the company in a class action lawsuit. (Photo: Eddie Quinones/YR Media)
Fear of Biometric Burglary
Andrea Kohlenberg felt ProctorU violated her rights — and she’s taking the company to court.
Before she took her LSAT — the required test for admission to law school — Kohlenberg said she was totally fine with the idea of being proctored. “It makes sense in theory,” she said. “These are major exams. Cheating would be really bad academically.” But as Kohlenberg learned some of the things ProctorU would require from her, she got increasingly worried.
“My initial concern was security on my laptop. If they can take control on my computer, I have sensitive data on my computer. That would be serious if someone else got that. I have medical information and documents. I didn’t know if those would be safe.” said Kohlenberg.
Kohlenberg lives in Illinois, where the Biometric Information Privacy Act (BIPA) was designed to regulate the storage and purchase of data.
According to BIPA, any company that collects any biometric information must do three things: protect that data using a “reasonable standard of care within the private entity’s industry” while they still need it, destroy any biometric data once it’s no longer useful to the company, and provide a timetable to the public for when the data will be destroyed.
Kohlenberg and two other students filed a class action complaint that demanded a jury trial, alleging that the company failed to do all three of those things.
“It’s one thing if you get your Social Security Number stolen, you can get your social number changed. But if someone steals your biometrics, those are things you can’t change about yourself,” Kohlenberg said.
YR Media reached out to ProctorU for its response to Kohlenberg’s allegations and other details in this story and did not hear back by deadline.
Talha Ahmad and Mohamed Bouftas passed a resolution demanding that Purdue University end all contracts with virtual proctoring services. (Kaiti Sullivan/YR Media)
College Governments Take On The Fight
Kohlenberg’s story is echoed by two other students we talked to. Talha Sarfraz Ahmad and Mohamed Amine Bouftas are trying to bring about change at Purdue University by using the power of student government to push back against virtual proctoring services.
Ahmad was spurred to action by an experience he had with Respondus. He said that when he first tried to use it, Respondus locked him out of his iPad for eight hours. Because all of his notes were on that device, he said he had to rush to the student library to print them out in time for his exam. YR found that out of 79 petitions on Change.org, 33 petitions made complaints against Respondus.
Ahmad was infuriated. With Bouftas’ help, they drafted a resolution against virtual proctoring services at Purdue.
Ahmad said, “I would argue that using these ... built a lot of suspicion between professors and students. And when so many people fall into problems like being falsely flagged for cheating or things like that, it makes me wonder, like, is this really achieving the objective of education?”
When asked to comment on Ahmad’s experience, Respondus said students can “restart the iPad” to leave “Assessment Mode” or the device will automatically do so after 8 hours. Ahmad told YR Media that a pop-up notification said the iPad would be locked for 8 hours.
Students aren’t the only ones taking a stand against virtual proctoring services. Some colleges discourage it. The Academic Senate at San Francisco State University (SFSU) passed a resolution that makes a case to restrict or ban the use of third-party proctoring. “Third-party remote proctoring is expressly designed to minimize cheating and rarely provides any student support,” the resolution states. “This casts faculty in the role of police catching cheaters, rather [than] faculty acting as mentors to facilitate student success.”
SFSU Professor Dr. Jackson Wilson, the vice chair of the academic senate at SFSU, helped author the SFSU resolution after hearing multiple concerns from students and faculty. He described virtual proctoring software as “a troublesome service … that we should be highly suspicious of.”
However, Andrew Roderick, assistant vice president of academic technology at SFSU, said the resolution is non-binding and is not policy. He knew of one program that used proctoring software on its culminating exam required for graduation.
Roderick said in an email, “Based on their experience using remote proctoring software, they are unlikely to use it in the future as it presented a number of logistical and technical challenges to the group of students participating in the exam.”
The resolutions passed by student governments and faculty are usually not obligatory, and universities make exceptions to use virtual third-party proctoring for certain programs. California State University Office of the Chancellor wrote a memorandum asking campus leaders to address concerns with virtual proctoring but stopped short of banning the software. UCSB Faculty Association made a similar gesture by writing a letter asking the university to curb the use of ProctorU (which the university appears to have ignored).
Filing a petition online is another popular tactic for making a statement against virtual proctoring software, and sometimes it works. The City University of New York’s petition, “Do Not Let CUNY Violate Student Privacy,” gained more than 28,000 signatures. In an update to the petition on Change.org, Aharon Grama, the chief of staff of CUNY’s undergraduate government, wrote that he is fairly certain that the university discontinued the use of virtual proctoring software after experiencing difficulties with ProctorTrack. CUNY did not return YR Media’s request for comment.
Talha Ahmad (left) and Mohamed Bouftas (right) organized against virtual proctoring at Purdue University after Ahmad was locked out of his iPad for 8 hours right before an exam. Photo: Kaiti Sullivan/YR Media
A Pandemic Trend That Bends
We’ve seen the impacts these virtual proctoring services are having on students now, during the pandemic. As the vaccine rollout continues and the country starts to reopen, what can we expect from proctoring apps in the future?
UCSC Vice Provost Herbie Lee said his institution rolled funding left over for ProctorU on its $200,000 purchase order into the 2020-21 academic year, authorizing continued funding for the software.
YR uncovered new patents for other virtual proctoring services, including one pending patent that works by positioning a camera directly behind students, watching over them as they take an exam. Another pending patent filed in June 2020 outlined plans for headgear that looks like Google Glass that students would wear so the device could monitor them during a test.
Screenshot from a 2020 pending patent filed by David S. Breed and Wendell C. Johnson for an invention designed to ensure that a test-taker isn’t getting help or otherwise cheating.
ProctorU Founder Jarrod Morgan said at the end of the day, it's up to schools to decide whether they want to discipline students when irregularities are flagged. ProctorU just provides the proctors and the recordings.
So what are some of the alternatives out there? What are students asking for?
Bouftas, who helped Ahmad draft a resolution against virtual proctoring at Purdue, suggested less invasive alternatives, such as virtual conferencing services already being used. “Another thing we tried to address is modifying the way assessments are given, so that cheating is harder to do,” Bouftas said.
Mohamad Bouftas and Talha Ahmad continue to discuss the bill they passed with Purdue’s senate, sparking campuswide awareness of proctoring services and the impact they can have on students. Sergine Beaubrun completed the bar exam and is now working towards becoming a criminal lawyer and prosecutor focused on fighting injustice. Meanwhile, Andrea Kohlenberg remains a class action plaintiff and representative for others suing ProctorU over their experience with the LSAT.
“We're all in this together,” Beaubrun said. “Let's fight for a better system.”
- Zoe Harwood
- Lead Producer:
- Jenn Cain
- Amber Ly
- Marjerrie Masicat, Pedro Vega Jr.
- Radamés Ajna, Devin Glover
- Lissa Soep
- Danielle Parenteau Decker
- Special Thanks:
- Dan Kopf, Matt Drange and the Electronic Frontier Foundation for its reporting on students’ petitions opposing proctoring apps.
YR Media has joined forces with the App Inventor team at M.I.T. to make stories, apps and learning resources about A.I. through an equity lens. Stay tuned for more. We are grateful for support in this work from the National Science Foundation. The opinions, findings, and conclusions or recommendations expressed are those of the makers of Surveillance U and do not necessarily reflect the views of the NSF.