AI News, Artificial Intelligence’s White Guy Problem

Artificial Intelligence’s White Guy Problem

Histories of discrimination can live on in digital platforms, and if they go unquestioned, they become part of the logic of everyday algorithmic systems.

Another scandal emerged recently when it was revealed that Amazon’s same-day delivery service was unavailable for ZIP codes in predominantly black neighborhoods.

The complexity of how search engines show ads to internet users makes it hard to say why this happened — whether the advertisers preferred showing the ads to men, or the outcome was an unintended consequence of the algorithms involved.

Otherwise, we risk constructing machine intelligence that mirrors a narrow and privileged vision of society, with its old, familiar biases and stereotypes.

Artificial Intelligence’s White Guy Problem

Histories of discrimination can live on in digital platforms, and if they go unquestioned, they become part of the logic of everyday algorithmic systems.

Another scandal emerged recently when it was revealed that Amazon’s same-day delivery service was unavailable for ZIP codes in predominantly black neighborhoods.

The complexity of how search engines show ads to internet users makes it hard to say why this happened — whether the advertisers preferred showing the ads to men, or the outcome was an unintended consequence of the algorithms involved.

Otherwise, we risk constructing machine intelligence that mirrors a narrow and privileged vision of society, with its old, familiar biases and stereotypes.

'I think my blackness is interfering': does facial recognition show racial bias?

There was the voice recognition software that struggled to understand women, the crime prediction algorithm that targeted black neighbourhoods and the online ad platform which was more likely to show men highly paid executive jobs.

Despite the need, a vetted methodology in machine learning for preventing this kind of discrimination based on sensitive attributes has been lacking.” The paper was one of several on detecting discrimination by algorithms to be presented at the Neural Information Processing Systems (NIPS) conference in Barcelona this month, indicating a growing recognition of the problem.

Nathan Srebro, a computer scientist at the Toyota Technological Institute at Chicago and co-author, said: “We are trying to enforce that you will not have inappropriate bias in the statistical prediction.” The test is aimed at machine learning programs, which learn to make predictions about the future by crunching through vast quantities of existing data.

“It just looks at the predictions it makes.” Their approach, called Equality of Opportunity in Supervised Learning, works on the basic principle that when an algorithm makes a decision about an individual - be it to show them an online ad or award them parole - the decision should not reveal anything about the individual’s race or gender beyond what might be gleaned from the data itself.

Ghosts in the Machine

In a brightly lit office, Joy Buolamwini sits down at her computer and slips on a Halloween mask to trick the machine into perceiving her as white.

That’s because facial detection algorithms made in the U.S. are frequently trained and evaluated using data sets that contain far more photos of white faces, and they’re generally tested and quality controlled by teams of engineers who aren’t likely to have dark skin.

As a result, some of these algorithms are better at identifying lighter skinned people, which can lead to problems ranging from passport systems that incorrectly read Asians as having their eyes closed, to HP webcams and Microsoft Kinect systems that have a harder time recognizing black faces, to Google Photos and Flickr auto-tagging African-Americans as apes.

A seminal 2012 study of three facial recognition algorithms used in law enforcement agencies found that the algorithms were 5–10% less accurate when reading black faces over white ones and showed similar discrepancies when analyzing faces of women and younger people.

“Just being goofy, I put the white mask on to see what would happen, and lo and behold, it detected the white mask.” Facial analysis bias remains a problem in part because industry benchmarks used to gauge performance often don’t include significant age, gender, or racial diversity.

LFW includes photos that represent a broad spectrum of lighting conditions, poses, background activity, and other metrics, but a 2014 analysis of the data set found that 83% of the photos are of white people and nearly 78% are of men.

There’s “limited evidence” of bias, racial or otherwise, in facial analysis algorithms, in part because there simply haven’t been many studies, says Patrick Grother, a computer scientist specializing in biometrics at the National Institute for Standards and Technology and lead author of the 2010 NIST study.

She is joined by a team of volunteers who support her nonprofit organization, the Algorithmic Justice League, which raises awareness of bias through public art and media projects, promotes transparency and accountability in algorithm design, and recruits volunteers to help test software and create inclusive data training sets.

“But these aren’t necessarily the people who are going to be most affected by the decisions of these automated systems…What we want to do is to be able to build tools for not just researchers, but also the general public to scrutinize AI.” Exposing AI’s biases starts by scrapping the notion that machines are inherently objective, says Cathy O’Neil, a data scientist whose book, Weapons of Math Destruction: How Big Data Increases Inequality and Threatens Democracy, examines how algorithms impacts everything from credit access to college admissions to job performance reviews.

Both are reasonable and seemingly objective parameters, but if the company has a history of hiring and promoting men over women or white candidates over people of color, an algorithm trained on that data will favor resumes that resemble those of white men.

Risk assessment tools used in commercial lending and insurance, for example, may not ask direct questions about race or class identity, but the proprietary algorithms frequently incorporate other variables like ZIP code that would count against those living in poor communities.

They analyzed more than 2 billion price quotes across approximately 700 companies and found that a person’s financial life dictated their car insurance rate far better than their driving record.

Higher insurance prices for low-income people can translate to higher debt and plummeting credit scores, which can mean reduced job prospects, which allows debt to pile up, credit scores to sink lower, and insurance rates to increase in a vicious cycle.

The algorithm was equally accurate at predicting recidivism rates for black and white defendants, but black defendants who didn’t re-offend were nearly twice as likely to be classified as high-risk compared with similarly reformed white defendants.

“The real issue is that we have, for a long time, been able to avoid being very clear as a society about what we mean by fairness and what we mean by discrimination.” There are laws that could provide some protection against algorithmic bias, but they aren’t comprehensive and have loopholes.

(It’s currently against the law to unintentionally discriminate on the basis of sex, age, disability, race, national origin, religion, pregnancy, or genetic information.) Proving disparate impact is notoriously difficult even when algorithms aren’t involved.

“If we want to have best practices, we should be testing a lot of versions of the software and not just relying on the first one that we’re presented with.” Since algorithms are proprietary and frequently protected under non-disclosure agreements, organizations that use them, including both private companies and government agencies, may not have the legal right to conduct independent testing, Selbst says.

The 12 research teams received contracts under the Explainable AI program aim to help military forces understand the decisions made by autonomous systems on the battlefield and whether that technology should be used in the next mission, says David Gunning, Explainable AI’s program manager.

Operating similarly to the way the National Transportation Safety Board investigates vehicular accidents, Shneiderman’s safety board would be an independent agency that could require designers to assess the impact of their algorithms before deployment, provide continuous monitoring to ensure safety and stability, and conduct retrospective analyses of accidents to inform future safety procedures.

Rise of the racist robots – how AI is learning all our worst impulses

In May last year, a stunning report claimed that a computer program used by a US court for risk assessment was biased against black prisoners.

The program, Correctional Offender Management Profiling for Alternative Sanctions (Compas), was much more prone to mistakenly label black defendants as likely to reoffend – wrongly flagging them at almost twice the rate as white people (45% to 24%), according to the investigative journalism organisation ProPublica.

The accusation gave frightening substance to a worry that has been brewing among activists and computer scientists for years and which the tech giants Google and Microsoft have recently taken steps to investigate: that as our computational tools have become more advanced, they have become more opaque.

(“It’s impossible to know how widely adopted AI is now, but I do know we can’t go back,” one computer scientist says.) It’s impossible to know how widely adopted AI is now, but I do know we can’t go back But, while some of the most prominent voices in the industry are concerned with the far-off future apocalyptic potential of AI, there is less attention paid to the more immediate problem of how we prevent these programs from amplifying the inequalities of our past and affecting the most vulnerable members of our society.

Last year, Lum and a co-author showed that PredPol, a program for police departments that predicts hotspots where future crime might occur, could potentially get stuck in a feedback loop of over-policing majority black and brown neighbourhoods.

For Samuel Sinyangwe, a justice activist and policy researcher, this kind of approach is “especially nefarious” because police can say: “We’re not being biased, we’re just doing what the math tells us.” And the public perception might be that the algorithms are impartial.

Programs developed by companies at the forefront of AI research have resulted in a string of errors that look uncannily like the darker biases of humanity: a Google image recognition program labelled the faces of several black people as gorillas;

This sort of approach has allowed computers to perform tasks – such as language translation, recognising faces or recommending films in your Netflix queue – that just a decade ago would have been considered too complex to automate.

In London, Hackney council has recently been working with a private company to apply AI to data, including government health and debt records, to help predict which families have children at risk of ending up in statutory care.

Lum and her co-author took PredPol – the program that suggests the likely location of future crimes based on recent crime and arrest statistics – and fed it historical drug-crime data from the city of Oakland’s police department.

The program was suggesting majority black neighbourhoods at about twice the rate of white ones, despite the fact that when the statisticians modelled the city’s likely overall drug use, based on national statistics, it was much more evenly distributed.

As if that wasn’t bad enough, the researchers also simulated what would happen if police had acted directly on PredPol’s hotspots every day and increased their arrests accordingly: the program entered a feedback loop, predicting more and more crime in the neighbourhoods that police visited most.

And while most of us don’t understand the complex code within programs such as PredPol, Hamid Khan, an organiser with Stop LAPD Spying Coalition, a community group addressing police surveillance in Los Angeles, says that people do recognise predictive policing as “another top-down approach where policing remains the same: pathologising whole communities”.

The scientific literature on the topic now reflects a debate on the nature of “fairness” itself, and researchers are working on everything from ways to strip “unfair” classifiers from decades of historical data, to modifying algorithms to skirt round any groups protected by existing anti-discrimination laws.

These things are going to eliminate bias from hiring decisions and everything else.’” Meanwhile, computer scientists face an unfamiliar challenge: their work necessarily looks to the future, but in embracing machines that learn, they find themselves tied to our age-old problems of the past.

Even artificial intelligence can acquire biases against race and gender

One of the great promises of artificial intelligence (AI) is a world free of petty human biases.

Hiring by algorithm would give men and women an equal chance at work, the thinking goes, and predicting criminal behavior with big data would sidestep racial prejudice in policing.

So “ice” and “steam” have similar embeddings, because both often appear within a few words of “water” and rarely with, say, “fashion.” But to a computer an embedding is represented as a string of numbers, not a definition that humans can intuitively understand.

Using it, Bryson’s team found that the embeddings for names like “Brett” and “Allison” were more similar to those for positive words including love and laughter, and those for names like “Alonzo” and “Shaniqua” were more similar to negative words like “cancer” and “failure.” To the computer, bias was baked into the words.

The program also inferred that flowers were more pleasant than insects and musical instruments were more pleasant than weapons, using the same technique to measure the similarity of their embeddings to those of positive and negative words.

For example, it looked at how closely related the embeddings for words like “hygienist” and “librarian” were to those of words like “female” and “woman.” For each profession, it then compared this computer-generated gender association measure to the actual percentage of women in that occupation.

People have long suggested that meaning could plausibly be extracted through word cooccurrences, “but it was a far from a foregone conclusion,” says Anthony Greenwald, a psychologist at the University of Washington in Seattle who developed the IAT in 1998 and wrote a commentary on the WEAT paper for this week’s issue of Science.

Algorithmic Bias: From Discrimination Discovery to Fairness-Aware Data Mining (Part 2)

Authors: Carlos Castillo, EURECAT, Technology Centre of Catalonia Francesco Bonchi, ISI Foundation Abstract: Algorithms and decision making based on Big ...

2018 Isaac Asimov Memorial Debate: Artificial Intelligence

Isaac Asimov's famous Three Laws of Robotics might be seen as early safeguards for our reliance on artificial intelligence, but as Alexa guides our homes and ...

AI in Public Sector: Tool for inclusion or exclusion?

March 1, 2018 Panel Event AI in the Public Sector: Tool for inclusion or exclusion? This panel was organized by the Taskar Center for Accessible Technology as ...

Machine Learning: Google's Vision - Google I/O 2016

Google has deployed practical A.I. throughout its products for the last decade -- from Translate, to the Google app, to Photos, to Inbox. The teams continue to ...

Jacob Silverman: "Terms of Service: Social Media and the Price of Constant Connection"

Jacob Silverman's recently-released book TERMS OF SERVICE is a call for social media users to take back ownership of their digital lives. Integrating politics ...

re:publica 2016 - Kate Crawford: Know your terrorist credit score!

Find out more at: How are our lives being changed by the rise of machine learning, big data and ..

Paradise or Oblivion

Paradise or Oblivion This video presentation advocates a new .

2nd Annual McGowan Forum on Ethics: The Challenge of Big Data

A panel of writers, corporate leaders and Government officials examine the ethical responsibility of those who compile and track citizen's personal data.

Humanities Innovators in a Tech World | Thursday May 17th

Sponsored by the Dorrance Scholarship Programs, the College of Humanities presents "Humanities Innovators in a Tech World," the first symposium in the ...

The War on Drugs Is a Failure

The War on Drugs is a campaign of prohibition and foreign military aid and military intervention being undertaken by the United States government, with the ...