In an increasingly data-driven world, the notion of employing artificial intelligence to deter crime is seductively compelling. The notion, variously referred to as predictive policing, is a science-fiction-like idea: sophisticated computer codes sift through large bodies of past data to predict where crime is most likely to occur or even name those at high risk of offending. The promise is a future of more intelligent, more effective law enforcement that is able to proactively deploy resources, pre-empt violence before it can happen, and break free from reactive, old-fashioned policing. But this promise of technology is accompanied by a deep and sobering question: can a system founded on the discriminatory bases of history ever deliver a just and equal future? The quest for algorithmic justice is riddled with the danger of automating and amplifying the same injustices it is trying to overcome.

The mechanics of such predictive systems differ, but they tend to fit into two categories. The first is place-based prediction. Such models consume years' worth of historical crime data, often arrest records and emergency calls. The algorithm searches for patterns in all this data, mapping out "hot spots"—streets, blocks, or neighborhoods where crime has often been reported in the past. The software then creates maps that mark these areas as having a high chance of future criminality. The second, and more contentious, category is person-based prediction. These systems scan information on individuals, such as their arrest records, their friends and associates, and occasionally even more amorphous information from social media or other online sources. They give individuals a risk score, a figure that allegedly measures their probability of being a participant in a violent crime, either as a victim or perpetrator. Police are then told to target these hot spots and hot individuals.

The basic fallacy of such a method can be briefly stated as "garbage in, garbage out." An AI is not imbued with a built-in understanding of crime, justice, or society. It is a pattern-discovery machine, and its universe is whatever data it is trained on. If what it is trained on is records of arrests, the algorithm is not learning about crime per se; it is learning about policing. It learns where police have been dispatched most frequently in the past and whom they have decided to arrest. This information is not an unadulterated, objective history of crime. It is a record of decades of human choice, policy, and, importantly, bias. If a police agency has previously over-policed poor and minority communities, the data will reflect a dense cluster of crime in those communities. The algorithm, noticing this trend, will then send even more police to those same neighborhoods. It creates a vicious cycle. Increased police presence results in increased arrests, which the algorithm takes as increased crime, and then calls for even increased police presence. The system does not find crime; it finds and perpetuates its own discriminatory training data.

This issue of discriminatory data is amplified by the problem of proxy variables. When an algorithm is prevented from using overtly racial information, it will often use proxies that serve as a surrogate for race. Postal code, income level, or even the prevalence of specific slang terms within social media updates can become very good surrogates for race and ethnicity. A pattern-finding algorithm may learn in short order that it's a good idea to target individuals from a particular zip code or with a particular income level, essentially re-implementing racial profiling under the guise of mathematical objectivity. The bias is in the code, not visible to the naked eye, but catastrophic in consequence. The outcome is that current social inequalities are not only reflected by the algorithm but are strengthened and validated by the imprimatur of "data-driven" science.

In addition, the notion of pre-predicting individual criminality is philosophically and practically problematic. Human behavior is not a straightforward, deterministic consequence of previous behaviors. It is shaped by an infinite variety of variables: chance, choice, circumstance, and support systems. To give an individual a risk score is to regard them not as a free human being, but as a statistical likelihood. This can have disastrous effects well before any crime has been committed. A high-risk score can result in increased police scrutiny, stops, and questions more often, and a greater feeling of being targeted within a community. This can undermine confidence between the public and the police, making it less likely for them to work together and actually making the public less safe. It can also create a self-fulfilling prophecy: if you observe a group of individuals enough, you are virtually certain to catch them doing something they shouldn't be doing, even if it's something very minor that would normally go unspoken against in some other situation.

The call for algorithmic justice also presents enormous due process and transparency issues. Most of the algorithms employed for predictive policing are "black boxes." Their internal mechanisms are treated as corporate trade secrets, so neither the public nor those being policed can know how a risk score was derived or a hot spot was created. How can you dispute a police deployment tactic or an individual risk assessment if you can't view the evidence or the rationale behind it? This is against a fundamental principle of justice: being allowed to confront your accuser. If the accuser is a clandestine algorithm, then that right is meaningless.

This doesn't mean that data isn't useful in enhancing public safety. Used in the right way, data can even reveal real patterns, like an increase in burglaries after a power failure or the particular mechanisms of gang violence that can best be solved with special social programs instead of police raids. The trick is to turn the attention away from forecasting who is going to offend and toward the conditions that contribute to crime. Data can be used to direct resources toward community centers, mental health services, job training programs, and infrastructure improvements—addressing the root causes of crime rather than just its symptoms. This is a form of prediction that leads to investment and support, not just surveillance and punishment.

The question of whether AI can predict crime fairly ultimately hinges on our definition of fairness. Is fairness a statistical idea, where the algorithm is "correct" based on its own erroneous data? Or is fairness an ethical and social idea, demanding that the system does not reinforce existing disadvantages and honors the dignity and worth of all people? The evidence to date indicates that the two definitions are opposed. A predictive-accuracy-optimizing algorithm that takes biased data into account will virtually certainly be socially unfair.

The quest for algorithmic justice is thus one of the most pressing challenges of our era. It compels us to look the demons of the past in the eye that haunt our data. Before we can rely on an algorithm to see into the future, we have to make sure it's not just repeating the biases of history. A fairly system would demand more than superior code, though – a complete rewriting of the data we draw upon and the objectives we aim at. It would take transparency, regular audits, and above all, the humility to accept that certain things about human existence, such as the possibility of redemption and change, cannot and must not be captured in a simple number. Safer communities are not found on the road to better prediction, but through greater investment in justice itself.

.    .    .