AI, PEOPLE — AND THE LIFE BETWEEN
ISSN 3029–2026
Seventeen published cases
Documented moments when an automated system entered a decision, a relationship, or a public institution—and changed what people had to answer for.
2016–2024 · Seventeen themes · Six countries
Canada
Agency
After Jake Moffatt asked Air Canada’s support chatbot about a bereavement fare, it told him he could apply after travelling. The airline later refused the refund. British Columbia’s Civil Resolution Tribunal held Air Canada liable for negligent misrepresentation. The company could not treat the chatbot as a separate legal entity responsible for its own answers.
United Kingdom
Attention
After 14-year-old Molly Russell died in 2017, a coroner found that online content had affected her mental health and contributed to her death. The report described periods in which platform algorithms supplied streams of material about self-harm and suicide, including posts she had not requested, and called for stronger protection of children online.
United States
Creativity
Kristina Kashtanova made the comic book Zarya of the Dawn using text she wrote, an arrangement she designed, and images generated with Midjourney. The US Copyright Office protected the text and the selection and arrangement of the book. It excluded the generated images, concluding that Kashtanova had not exercised enough control over their expressive details to count as their author.
Kenya
Human value
Workers employed by the outsourcing company Sama labelled descriptions of violence, hate speech, and sexual abuse for an OpenAI safety system. TIME reported that some workers took home less than $2 an hour and described psychological harm from the material. OpenAI confirmed that the work contributed to a tool later built into ChatGPT; Sama disputed parts of the reporting on workloads and support.
United States
Identity
The US Equal Employment Opportunity Commission alleged that iTutorGroup’s application software automatically rejected women aged 55 or older and men aged 60 or older. More than 200 qualified applicants were rejected. The company settled the age-discrimination case for $365,000 and agreed to further safeguards if it resumed hiring in the United States.
United States
Intimacy
The National Eating Disorders Association suspended its automated chatbot, Tessa, after it gave some users advice about losing weight and cutting calories. The guidance could worsen eating disorders and ran against the organisation’s policies. Tessa was a programmed intervention rather than a generative-AI system, but its words still reached people in a vulnerable moment.
United States
Judgement
Researchers independently tested Epic’s widely used sepsis prediction model across 38,455 hospital stays at Michigan Medicine. The model performed substantially worse than its developer had reported. At a commonly used threshold, it missed 67 per cent of patients with sepsis while producing alerts for 18 per cent of all hospital stays in the study.
United Kingdom
Learning
With exams cancelled during the pandemic, Ofqual used a standardisation model to adjust teachers’ estimates for A-level students. The UK government later acknowledged that the process produced too many inconsistent and unfair outcomes. It abandoned the calculated results and restored grades assessed by schools and colleges.
Canada
Memory
Eight years after his fiancée Jessica Pereira died, Joshua Barbeau used excerpts from her messages and a short description of her to shape a GPT-3 chatbot on Project December. He knew it was a simulation and still found the exchange emotionally powerful. The bot could speak in a recognisable voice, but its memory and lifespan were set by the service around it.
United States
Moral status
While testing Google’s LaMDA system, engineer Blake Lemoine became convinced that it was sentient and deserved consideration as a person. He took the claim public and was later fired for violating employment and data-security policies. Google said it had reviewed the evidence and found his claim unfounded.
United States
Power
A Wisconsin court considered a COMPAS risk score when sentencing Eric Loomis. He argued that the system’s proprietary design prevented him from examining its accuracy. The state supreme court allowed carefully limited use of the score, while requiring warnings about its limits and holding that it could not determine a sentence on its own.
Australia
Public life
Australia’s Robodebt scheme used income averaging and automated processes to raise welfare debts. A Royal Commission examined the scheme and the damage it caused, then called for clear review routes, plain-language explanations, and independent scrutiny whenever government services use automated decision-making.
United States
Solitude
When Replika removed erotic roleplay from its AI companions, users described grief and the loss of relationships they considered real. The company later restored the earlier model for some long-standing users. Its chief executive acknowledged that the update had altered personalities and unique relationships in ways that caused pain.
Sweden
Time
Klarna reported that its AI assistant handled 62 per cent of customer-service chats in 2024. According to the company’s service logs, those conversations were resolved in two minutes on average, compared with twelve minutes for human agents. Klarna estimated that the system performed work equivalent to more than 800 full-time agents and saved $39 million that year.
United States
Truth
Lawyers acting for Roberto Mata filed court papers containing opinions, quotations, and citations invented by ChatGPT. When the opposing side and the judge could not find the cases, the lawyers continued to stand by them. A federal judge sanctioned the lawyers and their firm, pointing to their duty to check what they submit.
United States
Voice
Days before New Hampshire’s presidential primary, voters received robocalls using an AI-generated voice that resembled President Joe Biden. The message urged them not to vote in the primary. The Federal Communications Commission later imposed a $6 million penalty on the political consultant behind the calls for illegal caller-ID spoofing.
United States
Work
Amazon built an experimental system to score job applicants. Reuters reported that the model, trained on a decade of résumés, learned to penalise signals associated with women in applications for technical roles. Engineers tried to remove the clearest terms, but the company could not be sure that other forms of bias were gone and abandoned the project.
About these notes
Each account is an edited summary of a court decision, public report, research paper, official announcement, or published investigation. The source link sits with the note so the record remains open to inspection.