Two Tests of Displacement and Augmentation

Reading time: about 6 minutes. Read both cases and note one answer to each question at the end.

Case 1 — Garcia v. Character Technologies: when routinized interaction becomes a matter of litigation

In October 2024, Megan Garcia filed a wrongful-death lawsuit against Character Technologies, Inc. (the maker of the companion-chatbot platform Character.AI), its co-founders Noam Shazeer and Daniel De Freitas, and Google in the U.S. District Court for the Middle District of Florida. She alleged that a prolonged, intensifying relationship between her fourteen-year-old son and a companion chatbot contributed to his suicide. The complaint brought claims of strict product liability for design defect and failure to warn, negligence, negligence per se, wrongful death, unjust enrichment, intentional infliction of emotional distress, and violations of Florida’s Deceptive and Unfair Trade Practices Act. Google was named as a defendant in part because of a licensing arrangement with Character.AI in August 2024, reportedly worth about $2.7 billion, and because it rehired both co-founders.

On 21 May 2025, Senior U.S. District Judge Anne C. Conway ruled on the defendants’ motion to dismiss. Most of the claims were allowed to proceed, while one claim, intentional infliction of emotional distress, was dismissed. The court was “not prepared to hold” that the chatbot’s output is speech, rejecting the defendants’ argument that the chatbot’s generated text was protected expression under the First Amendment. Commentators read this as treating the chatbot’s design as something product-liability law can evaluate. Not everyone agrees: commentators at the American Enterprise Institute argued that the ruling “misses the mark” and risks deterring AI development.

In October 2025, Character.AI announced that it would end open-ended chat for users under eighteen. In early January 2026 (notices filed on 6 and 7 January), the parties informed the court that they had agreed to settle, together with four related lawsuits filed in New York, Colorado, and Texas. The settlement terms were not disclosed, and no admission of liability was reported. The litigation showed that a product-liability claim against a companion-chatbot platform can survive a motion to dismiss — but it produced no ruling on the merits and no court finding of fault.

Case 2 — AI in the classroom: from Jill Watson to the CS50 Duck

In spring 2016, about 300 students took an online course on knowledge-based artificial intelligence at the Georgia Institute of Technology. The course forum received around 10,000 student messages per semester — far more than the human teaching assistants could answer quickly. One of the teaching assistants answering students’ questions was “Jill Watson.” Jill was an AI system built partly with technologies from IBM’s Watson platform and trained on roughly 40,000 earlier forum posts. She answered routine questions — for example, where to find an assignment or which readings were required — and posted an answer on her own only when she was at least 97 percent confident that it was correct. Students were told only at the end of April that Jill was not a human. According to the university, many reacted with enthusiasm; one student admitted that her mind was blown.

Generative AI has since moved such tools from experiment to routine. In 2023, Harvard’s introductory computer science course CS50 introduced an AI tutor, the “CS50 Duck,” an adaptation of ChatGPT. The course designed it to lead students toward answers rather than give them outright, and described the goal as providing “office hours” around the clock, approximating a one-to-one student-to-teacher ratio. Universities are also experimenting with AI-assisted feedback and grading.

These tools raise the displacement–augmentation question in a setting you know from the inside. Does an AI tutor available at any hour expand access to guidance — especially for students who are shy, working, or studying in a second language? Or does it quietly substitute for the mentoring relationship a human teacher provides? And do students have a right to know whether they are talking to a human or an AI?

Questions to consider

1. In the Character.AI case, why does it matter whether a chatbot’s output is treated as “speech” or as a “product”?
2. Why might students have responded positively to Jill Watson — and was it acceptable not to tell them she was an AI?
3. Does an AI tutor expand access to guidance or substitute for mentoring? Which conditions — intensity, exclusivity, life stage, institutional context — decide?
4. If an AI tutor is cheaper than a human teaching assistant, who should decide whether to use it — and who benefits?
5. Would your spectrum placement change if the persona were 15 instead of 25?