Diving Deep Into Facial Recognition

Facial Recognition: of all the biometric modalities it’s the one the world seems most ambivalent about. Seen from one angle, it’s the future of the biometric industry. While it might not be the most trusted or the most used, it’s cornered the market on excitement. In the mobile world, the iPhone X made waves late last year using facial recognition rather than capacitive fingerprint scans to unlock phones.

Last month Qualcomm and AMS announced a joint project to develop competing sensors for Android devices. Tokyo Stadium will employ facial recognition for the upcoming Rugby World Cup as well as the 2020 Olympic Games. Delta just announced an all-facial-recognition airport terminal in Atlanta. In total, the market for facial recognition generated USD 17.91 billion in 2017, according to a report by Allied Market Research, with an impressive predicted >26% growth through 2025.

At the same time, the technology seems increasingly mired in controversy. Surveillance cameras equipped with facial recognition have led to the forced internment and “re-education” of as many as one million Muslims from ethnic minority groups in western China. In the United States, reports that both Immigration and Customs Enforcement (ICE) and the Drug Enforcement Administration (DEA) have begun concealing facial recognition surveillance cameras in public street lights with no public or court oversight has fed concerns about a general lack of regulation for biometric technologies. And speaking at the Web Summit in Lisbon just last month, Microsoft President Brad Smith spoke about the need to place limits on facial recognition technology before “the year 2024 looks like the book 1984.”

Given such strong support for and simultaneous resistance to facial recognition, it seemed appropriate to do a slightly deeper dive into the use cases, the controversies, the arguments for and against widespread use, and the regulations surrounding the tech. What limitations exist, by design or convention? What if any should exist? What are individual companies doing to self-regulate, and what is the likelihood of greater government oversight?

Because so much has been happening with facial recognition that we haven’t already covered, we’ve decided to spend the entire month on just that modality. We’ll have a new post every week throughout December. Next year we’ll be back to our favorite subject: the joys of palm vein!

The year 2024 looks like the book 1984.

Some Common Use Cases

Before diving into the meaty controversy, let’s first take a basic look at some of the applications for facial recognition technology. We’ll start with the least-controversial and build from there.

#1 – Access

Identity verification for access control is certainly among the least-controversial uses for facial recognition, and it represents a significant share of the market. In a very simplified form, facial recognition access solutions ask the same question: does this face pattern match one on file in the system? If yes then the system opens the door, beeps, alerts the authorities, or whatever it’s programmed to do. Users are registered and presumably consenting parties.

They look directly into the sensor or camera for access. Housing complexes might employ facial recognition for convenient access so that residents carrying bags only need look up to access their building or door.

The system being deployed for the Rugby World Cup mentioned above falls squarely into this use case. Athletes and authorized members of the press will get access with a glance. Spectators using the same system will be restricted to a more limited set of permissions, maybe to VIP box seating but not the locker rooms. The reverse has been implemented in New Zealand to restrict casino access for “problem gamblers,” consenting individuals who feel they can’t trust their self-control.

#2 – Payments

There are also numerous facial recognition solutions designed to reduce friction in payments. Some seem to be aiming for entirely frictionless shopping. Tech giant Lenovo’s Beijing campus now sports an unstaffed convenience store where cameras around the store verify identity for payments. Walk in and take what you want; the cameras recognize your face and bill you. Sam’s Club has announced plans for a similar model store in Dallas that combines facial recognition with a mobile app, and Alipay allows the same at a pharmacy in Zhenzhou, dubbed the “Future Pharmacy.”

Some are as much marketing as payment solutions, like Yum China’s “Smile-to-Pay,” which gives select KFC customers the inexplicable ability to purchase chicken with a smile (plus, obviously, the price of said chicken). I can never decide whether I think Smile-to-Pay is primarily fun or creepy. It’s meant to be light-hearted and convenient, but also weird and patronizing. Sad people need chicken, too, after all.

#3 – Apps

Other fun or gimmicky applications of the technology include Snapchat’s kitty ear filter, which tracks a user’s face in real time, or the Google Art Selfie app, which compares users’ facial features with famous works of art (the latter was restricted under Illinois biometric legislation on the grounds that users did not give informed consent, but more on that later). These fun, gimmicky uses are often provided at no extra cost to the user but act as marketing or inexpensive data mining.

#4 – Surveillance

Finally, there is a wide range of surveillance applications of the technology. These solutions are geared towards governments, large retail chains, metro systems, and even public schools. Some systems incorporate other biometric identification systems or AI analysis. These solutions are where the bulk of the controversy surrounding facial recognition lies, though certainly not all of it.

According to a Buzzfeed report from August, retail giants Target, Walmart, and Lowe’s have all experimented with facial recognition for store surveillance. These stores already employed surveillance security cameras designed to reduce theft or respond quickly to vandalism or violence on site. Facial recognition is seen as an upgrade for those systems, with greater efficiency and lower costs. Of course, these chains don’t have access to government databases of faces, or to large private databases like Facebook’s or Google’s. They must register individual biometric signatures themselves. FaceFirst’s CEO Peter Trepp explained how many retailers offer shoplifters or other offenders a choice between criminal prosecution and “voluntarily” providing their biometrics. This is coercion, forcing a person to agree to something they wouldn’t otherwise by threatening them. Trepp says retail applications now constitute half of FaceFirst’s business.

Surveillance solutions have also been extensively marketed for schools, at least in one case as a direct response to school shootings in the United States. RealNetworks began offering its facial recognition security service SAFR to schools in the U.S. and Canada free of charge earlier this year, to combat school shootings. The company launched a version designed to operate in rural, low-bandwidth areas in India shortly thereafter. In a July interview, Real Networks founder Rob Glaser explicitly clarified that he was aware of the controversy surrounding the tech, and that he would welcome greater national regulation. It is not entirely clear however what the system could do to prevent the vast majority of school shootings, in which the perpetrators were students themselves and so authorized to be present.

Other school surveillance solutions tout the benefits of facial recognition in monitoring student activity or simplifying attendance taking for teachers (as a former teacher, this seems ludicrous). One pilot by Looplearn geared at 11th graders in Victoria, Australia met with significant backlash earlier this year from both students and local school board administrators, neither of which group was adequately informed of the pilot beforehand (read about it in this awesomely hard-hitting student reporting). In a March statement responding to facial recognition (and surveillance in general) in U.S. schools, the American Civil Liberties Union cautioned that, “Communities and schools need to think about what kind of message they are sending to our kids when they monitor them in school like they were prisoners in a detention facility.”

#5 – Government Surveillance

Finally, there are facial recognition solutions for government surveillance. These include China’s country-wide surveillance net, “Sharp Eyes” and the FBI’s “Next Generation Identification System,” which compares facial recognition data with the FBI database of mugshots. There’s a lot to say about these big government applications. They represent some of the largest capital investments in the technology, and they affect the greatest numbers of individuals, for better or worse. These will be discussed in greater detail in upcoming posts focused on controversial uses of facial recognition, as well as broad differences in technology between different solutions.