Amazons Rekognition system is an inexpensive way of facial recognition. But seems like it’s not reliable enough for police use.
In a recent test conducted by ACLU, The American Civil Liberties Union. They tested the Amazon’s open Rekognition API for the faces of all the 535 members of Congress against 25000 public mugshots. None of these mugshots was of the congressmen but shockingly. The system generated 28 false matches of the congressmen with the criminals.
Could Cost A Life
This has raised some serious concerns over Rekognition’s use by the Police forces. In their statement. The speaker for ACLU said, “An identification — whether accurate or not — could cost people their freedom or even their lives, Congress must take these threats seriously, hit the brakes, and enact a moratorium on law enforcement use of face recognition.”
Amazon Points Poor Calibration
When a source reached Amazon for a statement on these tests, the spokesman for the Company said that the results were due to poor calibration. The ACLU tests were performed at a confidence threshold of eighty per cent while Amazon recommends a threshold of 95 per cent for human face recognition, where a false recognition may have severe consequences.
“While 80% confidence is an acceptable threshold for photos of hot dogs, chairs, animals, or other social media use cases,it wouldn’t be appropriate for identifying individuals with a reasonable level of certainty.”, the spokesperson informed.
However, currently Amazon does not enforce these recommendations during the setup and law enforcement agencies could be using this threshold without any check.
It’s Not Hypothetical
The recent experiments were designed keeping an eye on Amazon’s partnership with the Washington County Sheriff’s Department in Oregon where the images were tested against a database of thirty thousand public mugshots.
Jacob Snow who organised ACLU’s test in North Carolina, said that this was not a hypothetical situation. These results are parallel to situations where Rekognition is being actually used.
The test result also pointed out a racial bias in Rekognition where 39% of the wrongly matched faces were Black Americans whereas only 20% were White Americans.
These findings also confirmed the disparities found by NIST’s Facial Recognition Vendor Test. Which showed that the error rates were higher in facial recognition of women and African Americans.
The test has raised some serious concerns on how unregulated can the use of this system be? It also sparked a strong reaction from Congress members. Three Congress members, Sen. Markey, Rep. Gutiérre, and Rep. DeSaulnier (D-CA) signed onto an open letter to Amazon’s CEO, asking for a full list of law enforcement agencies using the technology and inquiring about safeguards for using it on children younger than thirteen.