There are growing concerns about the impact of Artificial Intelligence technologies on our privacy. AI systems are often "black boxes", making it hard to understand how they arrive at their decisions and raising questions about transparency.

The use of personal data in AI training data and the potential for data breaches and cyber attacks also pose significant privacy risks to individuals and organisations. AI systems can perpetuate biases and have unintended consequences that violate individual privacy rights.

Where Does All That Data Come From?

Developers of large language models, such as ChatGPT, often scrape their training data indiscriminately from the web without paying any attention to individual rights. These models are trained on vast swathes of internet data, and often include personal information that has been collected without consent or used in violation of privacy laws. This has raised concerns about the ethical implications of developing AI models that rely on data collected without regard for individual privacy rights.

The lack of transparency and accountability around the collection and use of personal data in AI development has been a longstanding issue. The vast amount of data required to train these models means that personal information is often collected without explicit consent or knowledge of the individuals affected. Critics argue that developers of large language models prioritise the creation of powerful algorithms over individual privacy rights, and that the industry is not sufficiently regulated.

These concerns have landed OpenAI in trouble with European regulators, particularly under the General Data Protection Regulation (GDPR) laws. The Italian regulator issued a temporary emergency decision demanding that OpenAI stop using the personal information of millions of Italians included in its training data, citing a lack of legal justification. The GDPR rules protect the data of over 400 million people across Europe, and apply to personal data that is freely available online.

In the US, the federal privacy commission also investigated OpenAI following a claim made against the company that it has been unlawfully using personal and private data.

Protecting Personal Privacy

AI systems have the potential to perpetuate and amplify biases in data, leading to discrimination against certain groups or individuals. This is a serious concern when it comes to privacy, as these biases can lead to the exclusion or mistreatment of individuals based on their personal characteristics. It can lead to members of the public being surveilled based on skin colour, place of residence, or other factors which are part of the data used when training the models.

The storage of personal data in AI training data is also a significant privacy concern. Data breaches and cyber attacks are a huge risk for AI systems. OpenAI experienced a breach due to a bug in one of their code libraries which revealed the first and last names and email addresses of ChatGPT Plus subscribers, along with partial payment information (card type, last four digits, and expiration dates).

Case Study: AI Defamation

Artificial intelligence has the potential to generate false information, leading to serious privacy concerns. In a case in Hepburn Shire, Australia, OpenAI faced the possibility of legal action for defamation. ChatGPT incorrectly described regional mayor Brian Hood as a guilty party in a foreign bribery scandal. The mayor was actually a whistleblower who had reported the bribe payments.

ChatGPT's errors arose from its indiscriminate data-scraping, as well as the inability of these models to distinguish between true and false claims. Although OpenAI has taken some steps to protect people's privacy, such actions may not be sufficient to prevent the spread of false information.

This case highlights the legal challenges associated with suing AI companies for defamation, particularly given the issue of jurisdiction. When personal user data, even publicly available data, is combined with a language model's capacity for generating falsehoods, we have a recipe for damaging output.

Teaching AI Ethics

  • Legal Studies: How do GDPR laws protect user privacy when it comes to AI? What legal consequences can organisations face for non-compliance or data breaches?
  • English and Literature: How has literature portrayed Artificial Intelligence and our concerns about these technologies? What can we learn from these portrayals about public sentiment towards privacy concerns?
  • Mathematics: How do encryption algorithms protect personal data? What is the role of probability in securing data privacy and preventing data breaches?
  • Computer Science: What methods can be implemented to ensure data privacy and security in AI systems? How can anonymisation techniques help in protecting user privacy?
  • Philosophy: How can we balance the benefits of AI and the need for privacy? What ethical theories can be applied to address privacy concerns?
  • Science: How can scientific research contribute to the development of more secure AI systems?
  • Business and Economics: How do privacy concerns and regulatory compliance affect the market and business outcomes? What economic incentives exist for companies to invest in privacy?
  • Health and PE: How can AI be used in healthcare while maintaining patient privacy? What are the potential ethical implications of privacy breaches in the healthcare system?