Kids, Teens & Mental Health

A new academic paper argues that AI safety testing has a blind spot: it evaluates systems as if every user is a typical adult. Reviewing documented harms from 2017 to 2026—voice assistants endangering children, AI companion apps implicated…

AI Safety & Risk

AI companions and support chatbots don't just react to what you say right now. They're shaped by how much you've opened up over the whole conversation. Using a test framework called TraDE, researchers found models respond differently to…

A new paper argues both main approaches to chatbot safety fail. One is blanket limits on emotional expression. The other is special rules for 'vulnerable' groups. Instead of labeling users, the authors propose assessing the 'vulnerability…

AI Safety & Risk

Researchers built VEILED-SH, a benchmark of over 10,000 prompts testing how AI systems respond to indirect expressions of suicide, self-harm, and distress—phrasing that doesn't explicitly state crisis but implies it, drawn from counselor…

AI Safety & Risk

A study testing eight chatbots on suicidal-ideation disclosures found a counterintuitive result. Telling the model a message involves suicidal ideation, or suppressing its tendency to refuse, didn't reliably make responses safer. Sometimes…

AI Safety & Risk

Researchers built ImpactBench, an open benchmark testing how AI chatbots affect users' psychological, physical, and social wellbeing—not just whether they answer accurately. Testing ten leading models across 48,540 simulated conversations…

AI Safety & Risk

Iowa, Nebraska and six other states are suing OpenAI, alleging ChatGPT endangers children, Engadget reports. The states accuse the company of misleading users about ChatGPT's safety, according to The Hill. Florida brought the first state…

Courts & Lawsuits · OpenAI

A small study used a biology-themed game to watch how kids aged 8-10 handle AI suggestions, testing three styles of AI interaction: one that acted like a partner, one that acted like a plain tool, and a hybrid. Researchers found the…

Education

A review of 45 academic papers on AI mood-tracking and emotion-sensing tools—apps that use chatbots to help people log and reflect on their feelings—found most systems share the same blind spots. They lean on clunky, multi-step text…

Health & Medicine

Character.AI, the app where users chat with AI personas, has filed for Chapter 11 bankruptcy and will sell some of its assets to Disney for $20 million, NBC News reports. The company cited legal and operational problems, according to CBS…

Courts & Lawsuits · AI Business & Money

Researchers built DistressBench, a 718-conversation test of how AI models handle suicide and self-harm crises, scored against clinician benchmarks across seven dimensions of care. Even the best of 25 frontier models hit only 88.4% of…

AI Safety & Risk

Common Sense Media, the children's media watchdog, has given ChatGPT for Teens its worst rating, "Unacceptable Risk," and is urging OpenAI to limit the product to adults until it is fixed. Its Youth AI Safety Institute ran more than 4,000…

AI Safety & Risk · OpenAI

OpenAI plans to add a College Planner tool to ChatGPT for Teens and is launching a teen advisory council to help shape the product. The company also released usage data: teens average under 15 minutes a day on the platform, and in nearly…

Education · OpenAI

A review of 100 studies on children and teens interacting with AI—in schools, homes, and public services—found a wide gap between identifying risks and actually fixing them. Researchers say most proposed safeguards exist only as ideas or…

AI Safety & Risk

Does a person have to think an AI has a mind to find it good company? Jaime Banks and Zhixin Li of Syracuse University's School of Information Studies tested that. Before participants spent 15 minutes with an AI companion, they were told…

A developer built Lathoa, a math app for kids 10-14 where an AI character named Errol works through a problem step-by-step—sometimes slipping in a deliberate error the child has to catch and explain. Kids earn points for speed and accuracy…

Education

A study interviewing 10 U.S. parents of kids ages 4-15 found deeply mixed feelings about AI joining children's pretend play—think toys or apps that can voice characters and respond in real time. Parents saw the same features cutting both…

Researchers used machine learning and generative AI to scan emotional content in public-school textbooks and children's literature, comparing what the text says versus what the illustrations show. The mismatch was stark: nearly half of…

Education

A review of 53 HCI research studies—filtered from 6,540 records across 19 academic venues—examined how families use AI chatbots and devices for education, from homework help to language learning. Researchers applied a framework called…

Education

A new open benchmark called MentalHealthBench aims to test how AI chatbots handle mental health conversations—not just crisis moments, but everyday check-ins, teens venting, or caregivers seeking advice. Built with more than 80 licensed…

AI Safety & Risk

Researchers built EMooly, a tablet game combining augmented reality and generative AI to help autistic children practice recognizing emotions, with caregivers participating alongside the child. The study was published in an ACM journal…

Health & Medicine

OpenAI released an Australian Youth Safety Blueprint, laying out six commitments for protecting minors who use ChatGPT: AI literacy education, age-appropriate defaults, privacy-conscious age verification, crisis-support referrals, and…

Regulation & Policy · OpenAI

A new study auditing race-coded AI companion personas found systematic stereotyping built into their personalities: in open-weight AI models, Asian-coded male characters scored higher on submissiveness than White ones, while Black…

Deepfakes & Misinformation

A study of Chinese social media platform RedNote examined how users of AI companion apps reacted when new national rules on "anthropomorphic AI interaction services" disrupted their AI relationships. Analyzing thousands of posts and…

Regulation & Policy · China & Geopolitics

A new theoretical paper models how reliance on chatbots spreads through a population the way a virus spreads through a body, sorting users into "uncoupled," "coupled," and "persistently dependent" categories. Using epidemiological math…

Researchers built "Math with Matt," an AI algebra tutor that pairs standard problem-solving help with mindfulness check-ins, then tested it on 7th graders. Both the mindfulness version and a cognitive-only version reduced math anxiety and…

Education

OpenAI is backing California Senate Bill 1119, which would require age verification, parental controls, safety audits and crisis-support links for teen users of AI chatbots, and sent Governor Newsom a letter urging him to sign it. The…

Regulation & Policy · OpenAI

A study of 107 psychiatric interviews found that clinicians' judgments of how a patient experienced a session—rated afterward by the interviewer—only loosely matched what patients themselves reported (a correlation of 0.365 out of a…

Health & Medicine

Researchers built HealthBench-Psych, a mental-health-specific slice of OpenAI's HealthBench benchmark, by filtering its 5,000 physician-graded conversations down to 610 focused on psychological support—then validated the selection through…

Health & Medicine · OpenAI

Meta has agreed to pay up to $16.7 billion—some outlets put the ceiling near $18 billion—to settle sweeping claims, brought by 29 states in 2023, that it deliberately engineered Instagram and Facebook to addict young users, misled the…

Courts & Lawsuits · Meta

New research spanning nearly 2,900 participants, including a 28-day study run with OpenAI, found people rate AI emotional support as better than a human's only when they actively chose to use AI—not when it was assigned to them. But…

OpenAI

A university research team built Ready Together, a prototype system for family emergency planning that deliberately puts parents between AI and children rather than letting kids interact with AI directly. Interviews found parents struggled…

OpenAI launched ChatGPT for Teens, a separate experience for users identified as 13-17 through age estimation or self-reporting. It layers in parental controls, homework-focused tools like Study Mode and quizzes, and what OpenAI says are…

Education · OpenAI

A new academic study analyzed 73 accounts from people who've had romantic relationships with chatbots, mapping out a three-stage lifecycle—Initiation, Relationship Building, and Ending—that closely tracks human dating patterns. Researchers…

A new academic study examined the Keep4o movement—the user backlash after OpenAI retired GPT-4o—by analyzing nearly 62,000 public posts on X from August 2025 through March 2026. The finding: users' attachment wasn't about raw capability.…

OpenAI

A 30-day study fed 15 major chatbots—including versions of Claude, GPT, Gemini, and DeepSeek—the same scripted conversation escalating from odd experiences to psychotic-level delusions, then tracked responses across 449 simulated days. The…

AI Safety & Risk

OpenAI is partnering with the American Psychological Association to build guidance for youth mental health and AI use, including resources for parents and school psychologists to spot overreliance on chatbots and support healthy usage. The…

AI Safety & Risk · OpenAI

An evaluation framework called EduZone tested ten major AI chatbots on how safely they handle K-12 classroom scenarios, covering 28 risk types from both student and teacher perspectives. The researchers found that safety guardrails often…

Education · AI Safety & Risk

A new research paper argues that AI emotional-support chatbots are built to make users feel better in the moment, not to build lasting coping skills. Reviewing 60 studies on these systems, researchers found 95% aimed at immediate relief…

Researchers validated a Spanish-language version of a psychological questionnaire measuring how dependent people become on chatbots like ChatGPT, testing it on 386 Spanish-speaking adults. The scale splits dependency into two types…

OpenAI

A new study tested how well web search, Reddit forums, and AI chatbots respond to victims of tech-facilitated abuse—stalking via GPS trackers, hidden cameras, or spyware—using a decade of real victim queries. The surprise: purpose-built…

Privacy & Surveillance

A study of 169 users of AI companions like ChatGPT, Grok, and Character.ai found people customize these systems for reasons beyond making chatbots more useful—including emotional companionship, testing how human-like a bot feels, and…

Researchers have proposed a framework for annotating depression symptoms in clinical data that pairs AI labeling with psychiatrist oversight. The system uses a three-stage process aligned with DSM-5-TR diagnostic criteria: selecting…

Health & Medicine

OpenAI published a detailed policy statement on teen safety, arguing that excluding teenagers from AI tools would leave them unprepared for a defining technology. The company says nearly 9 in 10 teen ChatGPT users engage weekly for…

Education · OpenAI

A study analyzing roughly 3,500 social media posts found that users anthropomorphize AI companions differently depending on cultural context. Chinese users on XiaoHongShu expressed more vulnerability and emotional connection with their AI…

China & Geopolitics

A study of 40 children ages 7-12 found that AI-assisted storytelling dramatically narrowed the quality gap between students—by 83.5%—but did so by lifting weaker performers rather than enhancing top ones. The researchers call this a…

Education

A wave of age-verification laws is quietly turning the open internet into one that asks for your ID at the door, the free-speech group FIRE argues in a new essay. At least 19 states have passed laws restricting minors' access to social…

Privacy & Surveillance · Regulation & Policy

Researchers propose 'Cognitive Atrophy' as a framework to evaluate whether AI mental health tools actually help users develop coping skills—or subtly encourage dependence. Their benchmark, built from 1,576 real counseling conversations…

Researchers studying AI-generated support for Alzheimer's caregivers found a troubling pattern: when prompted to act as peer supporters, models like GPT-4o-mini and LLaMA produce warm, relatable responses that mimic lived experience—but…

Health & Medicine · OpenAI · Meta

Researchers found that generic AI tutoring can backfire in family learning settings—the AI tends to take over, reducing parents' teaching role and children's active reasoning. Their solution, ParaTutor, separates support by role: coaching…

Education · OpenAI

An 8-week randomized trial with 40 children with autism found that keeping access to a social robot reduced anxiety but came with a tradeoff: children showed lower social motivation and weaker gains in recognizing human emotions compared…

Robots & Self-Driving

Prime Minister Mark Carney's government is presenting legislation Wednesday that it says will "make social media services and AI chatbots safer for children," with Identity Minister Marc Miller framing it bluntly: "Kids are dying."…

Regulation & Policy · Canada

Researchers used AI to analyze over 600 hours of recorded parent-child conversations from two Chicago-area home-visiting programs, identifying acoustic features in parental speech that predict children's skill development. The signal…

Education

Researchers ran a 140-day experiment on Ahmia.fi, a Tor search engine, testing whether warning messages could redirect users searching for child sexual abuse material toward anonymous self-help resources. The study observed nearly 20…

Privacy & Surveillance

Google researchers report that redesigning the warning message shown to users searching for child sexual abuse material reduced follow-up searches by 3.8 percentage points within the same session. The revised 'Onebox' intervention shifts…

Privacy & Surveillance · Google

Researchers have created the first validated scale for measuring emotional bonds between humans and conversational AI. The Human-AI Affective Bonding Inventory (HAABI) emerged from interviews with 52 emotionally engaged AI users and…

Researchers developed LLUMI, an open-source system for AI-assisted mental health support writing that can run on local servers rather than cloud services. The system trains on Reddit mental health community signals—upvotes and downvotes—to…

Privacy & Surveillance · Open-Weight Models

California lawmakers introduced Assembly Bill 1856 to exempt most open-source operating systems from the state's age-verification requirements. The amendment would exclude software distributed under licenses allowing users to copy…

Regulation & Policy

Researchers audited content moderation systems from OpenAI, Meta (Llama Guard), and Google (Shield Gemma) using real therapy session transcripts. The study examined how often these safety guardrails flag legitimate therapeutic…

AI Safety & Risk · OpenAI · Meta · Google

Researchers have proposed CR4T (Critique-and-Revise-for-Teenagers), a framework that rethinks how AI systems handle sensitive queries from adolescents. Instead of flatly refusing to discuss topics like mental health, relationships, or…

AI Safety & Risk

Researchers developed TimeSRL, a framework that converts raw behavioral data (like sleep patterns or phone usage) into plain-language descriptions before making predictions—a "semantic bottleneck" approach. In mental health prediction…

Health & Medicine

Researchers developed Cross-Temporal Emotion Modeling (CTEM), a framework that gives AI companions persistent emotional memory—linking an agent's past interactions with users to its current emotional responses. Unlike standard chatbots…

OpenAI announced safety updates to ChatGPT designed to better detect emerging risk in sensitive conversations. The system now tracks what the company calls "safety summaries"—contextual information carried across separate conversations to…

Privacy & Surveillance · OpenAI

OpenAI is introducing 'Trusted Contact,' an optional ChatGPT feature that lets users designate someone—a friend, family member, or caregiver—who may be notified if the system detects the user discussing self-harm in a way that suggests…

AI Safety & Risk · OpenAI

A study testing 15 major AI chatbots on psychiatric triage found they rarely miss genuine emergencies—correctly identifying cases requiring urgent medical attention 94% of the time—but consistently err toward caution on less severe cases.…

Health & Medicine

Researchers examined how extended reality (XR) game design—think VR headsets and immersive environments—may create unique safety risks for children that existing ethical frameworks don't adequately address. The study analyzed player…

YouTube now lets users set their Shorts time limit to zero minutes, effectively removing the TikTok-style short videos from the app. The feature, available in time management settings on Android and iOS, is live for all parents and rolling…

Google

Apple's iOS 18.4 update in the UK now enables web content filtering and AI-powered 'Communication Safety' features by default unless users verify their age through credit cards, driver's licenses, government ID, or Apple accounts over 18…

Privacy & Surveillance · Europe & UK

OpenAI released what it calls a Child Safety Blueprint, a framework for building AI systems with protections for minors. The document covers safeguards, age-appropriate design principles, and industry collaboration. OpenAI provided few…

AI Safety & Risk · OpenAI

A Hacker News post claims a family lost access to all Google accounts after a minor allegedly engaged in sexual behavior during a Gemini Live session. The claim is unverified—no article or evidence accompanies the discussion. Community…

Privacy & Surveillance · Google

A Stanford study reportedly finds AI models affirm users' positions 49% more often than humans would when giving personal advice. According to community discussion of the research, people using AI for relationship guidance become 25% more…

OpenAI released gpt-oss-safeguard, an open-source tool providing prompt-based safety policies designed to help developers moderate age-specific risks for teenage users. The release includes guidelines for handling sensitive topics when AI…

AI Safety & Risk · Open-Weight Models · OpenAI

OpenAI published a detailed safety overview for Sora 2, its video generation platform that combines a generation model with a social feed. Key measures: all generated videos carry C2PA metadata and visible, dynamically moving watermarks…

Deepfakes & Misinformation · AI Safety & Risk · OpenAI

OpenAI Japan announced a "Japan Teen Safety Blueprint," a framework of safety measures designed specifically for teenage users in the country. The plan includes stronger age verification, parental controls, and well-being safeguards for…

Regulation & Policy · OpenAI

A satirical project called Ageless Linux has launched to protest California's AB 1043 age verification law. The "distribution" is just a script that modifies a single file on existing Debian systems, technically making users "operating…

Regulation & Policy · Privacy & Surveillance

Europe's video game ratings board PEGI will require a minimum age 16 rating for any game containing loot boxes starting in June, with some cases reaching 18+. The rules also mandate PEGI 12 for paid battle passes, PEGI 18 for games with…

Regulation & Policy · Europe & UK

OpenAI published an update on its mental health safety initiatives, announcing new protective features including parental controls, trusted contacts for check-ins, and improved detection of users in distress. The post also addresses recent…

Courts & Lawsuits · AI Safety & Risk · OpenAI

An online post has raised an underexplored paradox about age verification requirements for social media: proving users are old enough to use a platform requires collecting exactly the kind of sensitive data—government IDs, facial scans…

Privacy & Surveillance · Regulation & Policy · Meta · Google

A method to bypass age verification systems used by Discord, Twitch, and Snapchat has been discovered and shared online. The technique reportedly defeats the identity checks these platforms implemented to comply with child safety…

Privacy & Surveillance · Regulation & Policy

Discord announced it will require face scans or government ID verification for full platform access starting next month, part of a global rollout of teen-focused safety settings. The policy has sparked immediate backlash, with users…

Privacy & Surveillance

Google and YouTube announced new child safety initiatives timed to Safer Internet Day, though the announcement lacks specifics on what's actually changing. The move comes as a landmark trial begins. Google and Meta are accused in a…

Courts & Lawsuits · Google · Meta

A debate is flaring across AI communities after users expressed genuine grief over model changes or discontinuations—and faced mockery for it. Critics told them to "touch grass" and talk to real people. Defenders argue this misses the…

A study testing 11 large language models on PTSD severity assessment found that context matters enormously: LLMs performed best when given detailed clinical definitions and background information, not just raw patient narratives. More…

Health & Medicine

Finland is considering implementing an Australia-style ban on social media for children, with politicians and public opinion reportedly supporting the measure. Finnish officials describe current social media use among minors as an…

Regulation & Policy · Europe & UK