|
One of the perennial ethical issues of the computer age is whether one’s personal welfare is being threatened by new technologies and new uses of existing technologies. Consider this succession of headlines over the past 25 years: London has blanket CCTV coverage; residents caught 70 times a day on camera Data brokers and marketing firms cull your data from hundreds of sources US cities ask residents to link their home cameras to citywide network US law enforcement tracks your movements by pinging your cellphone Doorbell cameras document your neighbors’ lives Alexa, Siri and other home systems listen in on your lives Websites put cookies on your computer to track all your activity online Your boss puts keystroke trackers on your office computer to track everything you do How Apple AirTags are being used by stalkers Your grocery store knows your nutrition and your medical problems Your data is being sold on the dark web Social media algorithms feed users more and more extreme content Individuals outed on social media for every kind of behavior and identity IOT devices are more hackable
This past week, growing concern about one new technology of surveillance—the license plate reader—erupted once again in the news. Cameras on major thoroughfares or at the entrance and exit of communities log every vehicle passing by. Undeniably a crime deterrent and occasionally essential to the solution of car theft, local burglary, and occasionally violent crimes, the technology has made many uneasy about its invasion of privacy and its potential for misuse. The technology was originally developed and deployed in the U.K. as a counter-terrorism tactic, later adopted in the late 1990s by the U.S. Customs and Border Patrol. Widespread use only started after 2017, when Garrett Langley and two Georgia Tech friends figured out how to redesign and downsize license plate readers so they were affordable to local communities. The result was a revolution. Today, more than 5,000 communities use 120,000 Flock Safety (the company they founded) devices. Most significantly, Flock created a national database that collects all the local data and allows nationwide searches for cars. There were few controls on how long the data stayed in the database nor on how local police officers used it. The latest concern regarding license plate readers started with Washington Post reporting in early August that more than 50 police and law enforcement officials had misused the system to stalk former wives, lovers, and romantic rivals (see here). The misuse, arguably, was foreseeable and is why privacy advocates get upset about large databases. Flock found itself in the crosshairs of a national backlash. Several jurisdictions had already recently dropped Flock as a provider because of its loose privacy controls. In my own neighborhood, several communities and Stanford University announced recently they were not renewing contracts with Flock over privacy concerns. Stanford, which values the license-plate readers as a deterrent to crime against vulnerable students on an open campus, has now switched to a competing company, Genetec Cloudrunner, whose system is designed to be used by a single institution, incorporating by design many privacy protections (see here). Both the New York Times and the Post published lengthy articles this week on how Flock has responded to the most recent controversy (see here and here). Before evaluating the steps they are taking, I want to consider a broader point about the ethics of introducing any new technology. Living in Silicon Valley and teaching at Stanford, the origin of so many technologies, it has been impossible to avoid addressing the ethical obligations of the innovators who brought us computers, the internet, the Internet of Things, social media, and most recently, artificial intelligence. I addressed the ethical obligations of startups and others who were pioneering new technologies in my 23 years teaching at the Stanford Business School. Here are the principles I taught then (slightly updated!). I think they are still valid. At the conceptual or business plan stage, developers of any new technology should identify how the technology might be deliberately misused or might have unintended harmful side effects. Solutions to these identified risks should be “baked in” during the design stage, for example, disabling problem capabilities or using them “by design.” Companies should monitor problems that arise “in use,” tracking problems unforeseen at the introduction but apparent when put to use in anticipated or unanticipated “use cases.” Companies should maintain the capability of responding rapidly to these unanticipated problems, providing fixes or recalling products or systems for software or hardware “fixes.” Companies should be transparent about unexpected problems, or problems they underestimated in the design process, notifying users and the public of current or past exposure. Companies should take accountability for the harms, even unintended harms, caused by the technology.
It is relatively easy to describe these broad principles, but in class we dealt with some of the hard cases. How much potential for misuse should motivate a company to alter an early design, to redesign the product already in use, to issue a software update, to withdraw the product entirely from the market? How transparent should a company be if it is trying to fix a security or misuse problem before revealing the problem publicly? Are there some technology products that do so much good that you have to accept the misuse and negative side effects of releasing it? And by the way, what does it mean to “take accountability?” Flock arguably fell woefully short on several of these ethical principles. Its standard product allowed any police official to track any car with little oversight. It was foreseeable that there are bad apples among even the police—who would misuse the technology to stalk others. There were many opportunities in the design process to address these risks. Flock did not act as evidence of significant misuse accumulated. Over the nine years since Flock’s introduction and widespread use, the number of cases of misuse grew, known primarily to Flock. Flock did little until the recent Washington Post story motivated it to take emergency action to protect its market. There is a question even now whether Flock has done enough to address the risk of misuse. Flock’s CEO this week said that it now requires police departments to make use of what had previously been only an option: officers must now log the criminal case number related to every search of the database, and every department must use a new capability that audits and flags “abnormal activity” by any user. Flock says only one-third of departments previously used those safeguards. To me, Flock CEO Garrett Langley’s claim that they did not make these changes earlier because they expected municipalities to require them or departments to voluntarily adopt them sounds disingenuous. Flock is also lowering the “default” time the data is retained in the national system from 30 to seven days, though a department can extend that time. Ethical responsibility for new technologies also falls on the users who adopt them! In choosing its new license plate reader system this month, Stanford announced that it had set strict limits on how it uses the system. Data will be visible in the cloud for only one hour. After that, the data will be transferred to a Stanford server where only Stanford has access. Stanford also released information on how the Flock cameras help investigate crimes in the past: a serial thief, a serial package thief, and stalking cases. Finally, I wonder whether we have heard the last regarding the misuse of other surveillance systems. How are our cities and towns handling all the data collected by their CCTV and public camera systems?
|
