Proctorio's facial recognition is racist.

Also, it kind of sucks in general.

It's much more likely to not recognize Black people, for example


Wait. That's it?

Failures in the real world of their facial recognition algorithm are widely known, even if Mike is in denial (EDIT: see page 21, paragraph 2-3 of the letter to the senate, “fewer than five” issues due to race). See @Procteario [1] [2] for more; or for generic failures, even this image a friend sent me.

Now I have the data to prove it.

I talked about how I got my hands on their facial recognition data in my previous post – with my hands on their 'prized algorithm', I decided to take it on a test drive.

I set out to look for a representative dataset, and I found FairFace – a facial recognition dataset balanced for race, gender and age. I found their GitHub, which included a link to the dataset. I downloaded their [Padding=1.25] set, and ran the facial recognition algorithm against the 10954 faces stored in val.

Proctorio includes all four OpenCV models – and runs all four of them at once too, despite that being... quite inefficient. But, I gave them the benefit of the doubt: I count an image as a “pass” in the above graph if any one of the models thought it found anything like a face at all; though often times it just doesn't work that way.

It could not find the Black woman's face. It could not find the face in the foreground. It thinks the ear is a face.

Several “failures” were also embarassingly obvious: It doesn't think there was a face at all. There is.

Note that all images featuring faces were taken from the FairFace dataset by Kimmo Kärkkäinen and Jungseock Joo. Original images licensed under the CC-BY 4.0 license.

EDIT: Added raw data.

Race Pass Fail Pass % Relative % from avg
Black 669 887 42.99 -28.72
Middle Eastern 718 491 59.39 -1.53
White 1258 827 60.34 0.05
East Asian 971 579 62.65 3.88
Southeast Asian 917 498 64.81 7.46
Indian 984 532 64.91 7.63
Latino / Hispanic 1089 534 67.1 11.26
Total 6606 4348 60.31

Audits, audits...

I earlier had the experience of looking through Proctorio's “zero knowledge encryption” claims, and their audit from a 'White Oak Security', which I found... less than promising.

Now, I have the privilege of looking through their facial recognition suite, the centerpiece of their “unbiased, unblinking” service (quote from home page), supposedly undergoing an audit by a 'BABL AI', which has been supposedly happening for months now.

As an outsider to their company, I managed to pull their facial recognition outside of their application and run it against an open-source dataset in a weekend.

With this track record of audits that take too long and lead to nowhere, I'm starting to question not just Proctorio, but the questionable people they contract with to rubber-stamp their broken products.