This week, we highlight a process undertaken by the New York State Education Department to evaluate the use of facial recognition technology in schools that could serve as a model of democratic accountability for policing agencies.
Last week, New York became the first state in the country to ban the use and purchase of facial recognition technology (FRT) in its public schools.
Privacy advocates and parents alike are rightfully celebrating this decision (even as some wish it had gone ever further). As we’ve testified in the law enforcement context, facial recognition technology raises serious risks to privacy, free expression, and racial justice. What’s more, the accuracy of facial recognition systems—and their potential for bias—are entirely unproven in real-world contexts. A decision that declines to use New York’s school children as test subjects for biometric technology is the right one.
But no matter what you think of the merits of New York’s decision, we all should be celebrating the process that led to it as a model of democratic accountability in action.
What is democratic accountability and why does it matter?
Democratic accountability means the public has a voice in setting transparent, ethical, and effective policies and practices before the government acts. It looks like lawmaking by public officials in a way we can all watch and comment on. This kind of lawmaking ensures our basic commitment to democracy, assures the legitimacy of the actions government takes, and leads to better policy.
It sounds straightforward—and sometimes, like with facial recognition in New York schools, it is.
Policing and the democratic accountability gap
But there is at least one area of government where this kind of accountability is especially rare: policing. Too often, policing agencies are allowed to police themselves, making crucial decisions with a range of serious consequences for the public without public input, participation, or deliberation.
Nowhere is this more true than with law enforcement’s use of emerging technologies, like facial recognition. From the Baltimore Police Department’s decision to fly aerial surveillance flights to the NYPD’s deployment of a robotic dog in a public housing building—the approach frequently taken by policing agencies is “deploy first, ask (public) permission later.”
This approach is undemocratic—and it’s a bad deal for communities and police alike. Effective policing requires trust and public buy-in. Making high-stakes decisions without public input—or even notifying the public about those decisions ahead of time—breeds distrust and can lead to backlash, like departments having to forfeit the use of the tools.
New York schools and facial recognition technology: democratic accountability in action
Which bring us back to the case of FRT in New York schools.
After community backlash to one school district’s decision to implement facial recognition technology to monitor school property before seeking legislative authorization, the New York State Legislature passed a bill suspending further use or purchase of the technology until it could better assess its benefits and risks in the school context.
To enable this evaluation, the statute charged the Office of Information Technology Services (ITS), in consultation with the Education Department, with analyzing the use of biometric technology in New York schools and issuing a public report. The law further required ITS to issue recommendations on 12 discrete topics, ranging from the privacy implications of facial recognition technology, to its effectiveness, to potential disclosure and auditing requirements.
ITS appears to have taken that responsibility seriously—its 49-page report followed 18 months of careful study and was based on a “comprehensive review of literature discussing the implications of FRT”; a widely distributed public survey of parents, students, school staff, law enforcement and civil rights groups; a virtual public hearing; and outreach to subject matter experts in data and student privacy, civil liberties and civil rights, and other key community stakeholders.
Based on this comprehensive review, ITS concluded that “there are discernible risks to the use of this technology in school settings” that “outweigh any documented benefits.”
Last week, New York State Education Commissioner Betty Rosa issued her final order banning FRT in New York schools. In reaching her decision, Commissioner Rosa “considered the recommendations of the biometrics report conducted by [ITS] . . . and concluded that serious concerns surrounding the use of facial recognition technology do not outweigh its claimed benefits.”
Trust the (democratic) process
New York’s approach to facial recognition in schools ensured the public had a voice in the process and relied on actual evidence. It was democratic and accountable. Perhaps no coincidence it has been roundly praised.
This process should serve as a model for how all public agencies—especially those granted the greatest power, like law enforcement—should evaluate whether or not to use emerging technologies like facial recognition: First, study the costs and benefits before the technology is legislatively authorized, making sure to meaningfully engage the public throughout this evaluation. Then, publicly report any findings. Finally, make a policy decision based on these findings.
This is democratic accountability in action. Our nation’s policing agencies should be taking notes.