A More Impartial Machine

The past several years have revealed a long and incomplete list of the ways in which machine learning algorithms discriminate. They falsely rank black males as more likely to criminally offend, they find women ill suited to tech jobs, they classify lighter-skinned women as “more beautiful,” and so on. The existence of such bias is clear. Remaining is the question of what to do about it.

A new paper by Soheil Ghili at Yale SOM, with two colleagues from the Yale Institute for Network Science, offers a partial solution. Published in the proceedings of the 2019 conference for the Association for the Advancement of Artificial Intelligence, the article lays out a novel method drawn from statistics and econometrics that helps reduce the “latent discrimination” present in algorithms.

Consider the algorithms in courts that are used to assign a risk score to criminal defendants: they provide a simple number that predicts the likeliness somebody will reoffend after arrest, and therefore an indicator of whether somebody should be allowed to walk free on bail. Though an algorithm cannot, for obvious reasons, explicitly consider a defendant’s race when calculating this score, it may take into account variables like one’s prior record of arrests, or annual income, or zip code of residence. All of these secondary variables happen to be correlated with race, which means that they can introduce latent discrimination into the result, in this case systematically assigning higher risk scores to African Americans than whites.

To reduce this bias, Ghili and his colleagues introduce a method called “train then mask,” in which they train their algorithm on one set of data that includes sensitive variables (like race), and then mask and control for these variables when actually applying the algorithm. This process is designed to filter out the effects of correlated variables and dampen latent discrimination. (See the paper for a full description of the model.)

The researchers test this method in two different domains: they look first at financial information of German citizens and attempt to predict a person’s credit risk; the algorithm works well. They next look at the personal information of 5,000 convicted criminals to predict their likelihood of reoffending within two years. Again, the algorithm appears to produce accurate results with a promising reduction in bias.

“We evaluate the performance of our algorithm on several real-world datasets and show how fairness for these datasets can be improved with a very small loss in accuracy,” Ghili and his colleagues conclude. The results of their test, “suggest that train-then-mask as an algorithm can be easily employed to alleviate (explicit and latent) discrimination in various datasets”—a key finding for anybody invested in the application of machine learning.