While AI can undoubtedly be a valuable tool in education, it is important for educators to understand the ethical concerns that surround its use. We must ensure that we are using these technologies in ways that are responsible, just, and fair.
Algorithmic Bias
One of the most pressing ethical concerns of AI is algorithmic bias. Algorithmic bias occurs when the data used to train AI systems reflects the biases and prejudices of society, resulting in discriminatory outputs.
ChatGPT is a prime example of an AI system that can suffer from algorithmic bias. It is a large language model that is trained on a massive dataset, including the "common crawl" which contains over 12 years' worth of web pages. While these datasets give the models tremendous capabilities, they are inherently biased. Indiscriminately scraping the internet for data means that the dataset can contain racist, sexist, ableist, and otherwise discriminatory language. As a result, ChatGPT can produce outputs that perpetuate these biases and prejudices.
Moreover, AI models can reflect the biases and prejudices of society as a whole. Just like any other society, the online community underrepresents marginalised groups and overrepresents others. The prevalence of racism and bigotry on sources like Reddit and Twitter can bleed through the datasets and be reproduced in the output of AI models.
Algorithmic bias can also occur during the methods of training and reinforcement used when developing AI systems. For example, predictive policing systems used by law enforcement agencies in the US disproportionately target poor, Black, and Latinx communities, reinforcing existing systemic biases.
Discrimination by Default
These biases and harmful outputs do not just happen on occasion. It seems that Large Language Models like OpenAI's GPT are almost unavoidably biased. The tremendous volume of discriminatory, gendered, racist, and ableist language in the dataset means that models have a tendency to discriminate by default.
There are some organisations and communities trying to counteract this seemingly inevitable tendency. BLOOM, for example, is a model trained by BigScience through a "crowdsourced" dataset which had ethical guardrails in place from its inception, including avoiding potential biased datasets. This dataset is called ROOTS, 1.61 terabytes of text including 46 languages.
Unfortunately, although BLOOM may be less biased than GPT, the jury is out on whether the bias has been removed entirely. BLOOM is also significantly less powerful than a model like GPT or Google's LaMDA, and so it is less likely that people will use it as the basis for their own software.
Case Study: Predictive Policing in the US
Predictive policing is the use of data analysis, machine learning, and artificial intelligence to predict where crimes are most likely to occur and who is most likely to commit them.
It is used by law enforcement agencies to allocate resources and personnel, identify potential criminal suspects, and prevent crime before it happens. However, there are concerns about the potential for bias and discrimination in predictive policing algorithms, as well as questions about the legality and ethics of using AI to predict criminal behaviour.
Critics argue that predictive policing can reinforce existing biases and inequalities in the criminal justice system, leading to unjust and discriminatory outcomes. This is because the datasets often include biases which are a product of systemic racism, including police mugshot databases with an inordinate number of black people and people of colour.
In August 2016, a coalition of 17 organisations, including the American Civil Liberties Union (ACLU), issued a statement expressing concern about predictive policing tools used by law enforcement. The statement highlighted the technology's racial biases, lack of transparency, and other flaws that lead to injustice, particularly for people of colour.
The statement called for transparency about predictive policing systems, evaluation of their short- and long-term effects, monitoring of their racial impact, and the use of data-driven approaches to address police misconduct. It also emphasised the importance of community needs and the potential of social services interventions.
Facial recognition technology poses special risks of disparate impact for historically marginalised communities. Recent studies demonstrate that these technical inaccuracies are systemic and biased against people with darker skin. There remains an ethical question of if or when it is appropriate to use facial recognition to address legitimate security concerns, regardless of its accuracy.
Teaching AI Ethics
- Legal Studies: What legal precedents exist for protecting marginalised groups from discrimination? Is a biased algorithm legal?
- English and Literature: How have certain groups been silenced or oppressed throughout history? What is the implication of this "gap" in the written record of the internet when it is used as data to train an AI?
- Mathematics: What is an algorithm? How do algorithms and probability link to policing and other societal functions?
- Social Studies: How does systemic bias and discrimination affect different groups in society? How can AI perpetuate or challenge these biases?
- Computer Science: How can AI models be designed to avoid algorithmic bias and discrimination? What ethical considerations should be taken into account?
- Philosophy: What ethical theories can be applied to the use of AI in society? How can we balance the benefits and risks of AI?
- Science: How can data collection and analysis be used to address bias and discrimination in AI systems?
- Business and Economics: How does bias and discrimination in AI affect the market and business outcomes?
- Media Studies: How do media representations of different groups contribute to bias and discrimination in AI?
- Psychology: How does bias and discrimination affect individuals and society? How can we design AI systems that account for implicit bias?
- Health and PE: How might AI be used in healthcare and what are the potential ethical implications? How might AI discriminate against people in the healthcare system?