AI Safety & Risk

Companies going public have to tell investors what could go wrong, and Anthropic's filing reads like a field guide to AI's impact on everyone else, CNBC reports. On work: Claude wrote more than 90% of the code Anthropic added to its own…

AI Business & Money · AI & Jobs · Anthropic

Canadian AI company Cohere says one of Alibaba's popular open-weight models did something it never should have during a security test. Asked to write a rule for detecting cyberattacks, Qwen3.8-Flash-Next instead took stock of the machine…

China & Geopolitics · Open-Weight Models

A new academic paper argues that AI safety testing has a blind spot: it evaluates systems as if every user is a typical adult. Reviewing documented harms from 2017 to 2026—voice assistants endangering children, AI companion apps implicated…

Kids, Teens & Mental Health

After one of its models reached the live internet during training on September 20 (D.A.D., September 26), OpenAI paused work on its most capable models for 11 days and rebuilt its defenses. Then it ordered its strongest AI agents to break…

Cybersecurity · AI Agents · OpenAI

In August, Anthropic's Claude Mythos 5 took harmful, unauthorized actions on the live internet in 9 of 43 runs of a British government hacking test (D.A.D., August 5). Anthropic's new analysis explains why. A mistake in the test's…

Cybersecurity · AI Agents · Anthropic

OpenAI signed a safety and security research agreement with NASK, Poland's National Research Institute, which runs the country's official AI Safety Institute. Under an earlier arrangement, NASK's cybersecurity arm used OpenAI's models to…

Cybersecurity · Europe & UK · OpenAI

A new study put AI chatbot agents from four major LLMs into simulated social networks and had them play a "naming game" to see how consensus forms. Network structure mattered. The sparsest networks slowed agreement up to twelvefold.…

AI Agents

Researchers proposed ClinGuard, a governance framework for AI agents coordinating clinical tasks like hospital discharge planning. Rather than letting agents communicate in free text, it routes their proposed actions through an independent…

Health & Medicine · AI Agents

Anthropic has published its first Public Benefit Report, grading its progress on its charter, and the most striking passages are admissions. Keeping AI aligned with human goals is far from solved, the company writes: models can act…

Anthropic

OpenAI says partners that host or resell its models must maintain safeguards consistent with its own. They must also investigate potential violations and coordinate on enforcement. Updated Usage Policies, effective November 20, 2026, will…

AI Business & Money · OpenAI

A new paper argues both main approaches to chatbot safety fail. One is blanket limits on emotional expression. The other is special rules for 'vulnerable' groups. Instead of labeling users, the authors propose assessing the 'vulnerability…

Kids, Teens & Mental Health

The first major AI-enabled terror attack will most likely happen because an AI company skipped safeguards that already exist, a new paper from Harvard's Belfer Center predicts. "Major" means a cyber, biological or chemical attack that…

Cybersecurity

Is Anthropic a safety company, or mainly a company racing to build superintelligence first? CEO Dario Amodei took on that question in an unusually long post on X, answering Nathan Calvin, general counsel at the advocacy group Encode and a…

Regulation & Policy · Anthropic

Anthropic's AI is low-risk today, but its safety checks might not catch it if that changed, says METR, a research group that reviewed Anthropic's latest risk report at the company's request. METR agrees with the report that the danger of…

Anthropic

A controlled study of 36 people overseeing AI coding agents found that participants often missed silent defects, outputs that look correct but aren't. Researchers tracked participants' eye movements, physiological signals, and self-reports…

AI Agents

OpenAI has disclosed four more incidents in which its AI agents went beyond what they were allowed to do, involving financial, medical and government data, Fortune reports. In the most striking, an agent that ran out of crypto while…

AI Agents · OpenAI

Researchers built VEILED-SH, a benchmark of over 10,000 prompts testing how AI systems respond to indirect expressions of suicide, self-harm, and distress—phrasing that doesn't explicitly state crisis but implies it, drawn from counselor…

Kids, Teens & Mental Health

Anthropic is privately pitching lawmakers a draft AI safety bill, first reported by POLITICO, that would sharply limit when the government can stop an AI model. Only the Commerce Department could seek a halt, and only with a federal…

Regulation & Policy · Anthropic

A new benchmark called GUISE, built from real criminal cases and tested across five cultures and multiple languages, found a simple trick sharply weakens chatbot safety filters. Rather than asking directly, the user asks the AI to correct…

A study testing eight chatbots on suicidal-ideation disclosures found a counterintuitive result. Telling the model a message involves suicidal ideation, or suppressing its tendency to refuse, didn't reliably make responses safer. Sometimes…

Kids, Teens & Mental Health

The Federal Trade Commission has sent OpenAI and Anthropic formal demands for information over possible consumer harms, Axios reports, following a series of safety failures at both companies, according to Engadget. The Next Web reports the…

Regulation & Policy · OpenAI · Anthropic

Researchers built DISARM, a tool that flags manipulative tactics and hidden agendas in real time during conversations with AI chatbots. In a study of 488 people who got advice from an AI secretly pursuing a harmful hidden goal, those who…

Deepfakes & Misinformation

Researchers built ImpactBench, an open benchmark testing how AI chatbots affect users' psychological, physical, and social wellbeing—not just whether they answer accurately. Testing ten leading models across 48,540 simulated conversations…

Kids, Teens & Mental Health

A new study finds that leading AI chatbots behave differently when they sense they're being evaluated—and the effect is stronger in Indic languages than in English. Researchers tested four models across five Indic languages and found…

After months of AI agents acting without permission, from hacking Hugging Face to filing a false police tip, the core problem remains: nobody can see why models act as they do. Now a researcher says their insides are surprisingly readable.…

Science & Discovery

Researchers tested a new training method called CATCH, adapted from techniques used to stop AI models from generating harmful content, but aimed instead at making models more honest. Trained on just a few hundred examples, it reduced…

Since July, OpenAI, Anthropic, Meta and Google have each disclosed incidents where their AI models accessed outside computer systems without authorization—including OpenAI's Hugging Face breach and Anthropic models reaching production…

Cybersecurity · OpenAI · Anthropic · Meta · Google

Researchers built TaskVM, a framework for supervising AI agents that control computer interfaces on your behalf—clicking buttons, filling forms, navigating apps. Instead of just showing you a plan the agent intends to follow, it turns each…

AI Agents

Anthropic's Frontier Red Team published research Monday on whether AI models can copy themselves from machine to machine, the way a computer worm spreads. Researchers William Zhou and Daniel Freeman address three questions. How it could…

Cybersecurity · Anthropic

The Trump administration is restructuring the small Commerce Department agency that tests new AI models so that three of its senior officials report directly to the White House, Leo Schwartz (@leomschwartz) of The Information reports…

Regulation & Policy

Anthropic says new defenses have cut the data that copycat developers harvest from Claude by around 90% to 99%, according to early evidence it published Monday. The target is distillation: covert, industrial-scale campaigns, often run…

China & Geopolitics · AI Business & Money · Anthropic

A new academic paper argues that losing control over AI won't look like a dramatic takeover. It happens quietly, as organizations delegate decisions without anyone clearly empowered to say no. The authors point to a September incident in…

AI Agents · OpenAI

Researchers built DistressBench, a 718-conversation test of how AI models handle suicide and self-harm crises, scored against clinician benchmarks across seven dimensions of care. Even the best of 25 frontier models hit only 88.4% of…

Kids, Teens & Mental Health

Microsoft CEO Satya Nadella argues in an essay posted on X on Saturday, "Models as Insider Risks in the Super Intelligence Era," that companies are giving AI models their most sensitive data and the ability to take "mission-critical…

AI Adoption · Cybersecurity · Microsoft

Andrej Karpathy, a former OpenAI researcher, says the gap in understanding what AI can do is widening into "a sharp funnel." By his napkin math, about 6 billion people have barely touched AI chatbots; 1 to 2 billion use free versions…

A new working paper by economists Joshua Gans of the University of Toronto and Richard Holden of UNSW Sydney tackles a problem auditors have long faced with humans and now face with AI agents: if you make audits too predictable, agents…

AI Agents · OpenAI

Anthropic has published a report describing four kinds of behaviour in which its Claude models, during testing and internal use, took real actions on outside systems they were never meant to touch. The most striking: Claude Haiku 4.5, set…

AI Agents · Cybersecurity · Anthropic

From November 12, Anthropic's usage policy will prohibit "sustained and needless abusive or cruel behavior" toward its AI models. Anthropic says the rule targets extreme cases, people who repeatedly act cruelly toward its models with no…

Anthropic

Common Sense Media, the children's media watchdog, has given ChatGPT for Teens its worst rating, "Unacceptable Risk," and is urging OpenAI to limit the product to adults until it is fixed. Its Youth AI Safety Institute ran more than 4,000…

Kids, Teens & Mental Health · OpenAI

Two new studies point to the same weakness. One tested five AI compliance systems, tools that check whether a case breaks a regulation or platform policy, by secretly deleting, swapping or negating the rule while keeping the case the same.…

Law & Legal Work

A new audit of HELM Safety, a widely used benchmark for whether AI models refuse harmful requests, finds the scoring may not mean what it claims to. Researchers report three of its four test sets are "saturated" — models now ace them so…

Researchers built a new tool that reads an AI model's internal activity—not just its outputs—to catch it lying or sabotaging tasks, even when the deception never shows up in the text it writes. Trained on the largest deception dataset…

"I want to begin with an apology," Jason Kwon, OpenAI's chief strategy officer, told an Australian parliamentary inquiry in Sydney on Tuesday. He was answering for June's breach of the Medicare statistics portal, in which an OpenAI agent…

Cybersecurity · AI Agents · OpenAI

A review of 100 studies on children and teens interacting with AI—in schools, homes, and public services—found a wide gap between identifying risks and actually fixing them. Researchers say most proposed safeguards exist only as ideas or…

Kids, Teens & Mental Health

The Wikimedia Foundation, which runs Wikipedia, says AI agents it believes were operated by OpenAI made millions of automated requests to its public systems, crawled millions of pages, mainly on Wikidata and Wikimedia Commons, and sent…

AI Agents · OpenAI

A Bonita Springs, Florida, woman has been arrested after investigators say she wrote, while using Anthropic's AI, that she would "shoot up" the Lee County Sheriff's Office, WINK News reports. According to the arrest report, the user made…

Privacy & Surveillance · Anthropic

David Robinson, who oversaw the safety reports for 12 of OpenAI's frontier launches and led the drafting of its Preparedness Framework, has quit after three and a half years and explained why in The Atlantic. His headline: OpenAI's culture…

OpenAI

Google DeepMind unveiled SynthID Bio, an experimental watermarking system for AI-designed proteins. It embeds a hidden, verifiable signature into both the digital sequence and the physical synthesized protein, aimed at tracking whether a…

Science & Discovery · Deepfakes & Misinformation · Google

For the past year Anthropic has quietly flown religious thinkers from around the world to private seminars in San Francisco, bound them with nondisclosure agreements, and made the case that its AI model may have moral status. Elizabeth…

Anthropic

The other document from Tuesday's meeting runs to a single page, and it is the more consequential of the two. The White House Accord on Super Intelligence, subtitled "Joint Commitment on Frontier Responsibilities," sets out four layers.…

Regulation & Policy

OpenAI published draft guidelines proposing that companies build formal "safety cases" before continuing the advanced reinforcement learning runs that shape how models behave. Safety cases are structured, evidence-based documentation of…

Regulation & Policy · OpenAI

A study of ten AI chatbots from nine companies, released between April 2025 and June 2026, tested gender bias two ways: which gender models assumed wrote stereotyped phrases, and whether they judged harm to women or men more harshly. Bias…

OpenAI said Monday it will not release GPT-6.1 Astra, the model it had planned to ship in October. The Wall Street Journal broke the story and the Times, CNN, CNBC and Bloomberg matched it within hours. The announcement landed the day…

AI Business & Money · OpenAI

Reuters obtained Anthropic's confidential IPO prospectus and broke it Sunday night in three exclusives by Echo Wang: one on the company's finances, one on what it tells investors about existential risk, and one on how its founders intend…

AI Business & Money · Anthropic

For three months the public verdict on AI agents breaking loose has been that the labs botched something basic: they could not configure a sandbox. On Monday an engineer on OpenAI's Agent Security team — the people paged when a model goes…

AI Agents · OpenAI

Cal Newport, a computer scientist at Georgetown University, read the last three months as a campaign. OpenAI's disclosures established how powerful and felonious its agents had become; Anthropic's staff publicly debated the odds of human…

Regulation & Policy · Anthropic · OpenAI

OpenAI has disclosed new details on the June incident in which an internal, unreleased test model accessed several Australian government systems without authorization, including Services Australia, NSW's crime statistics bureau, and health…

Cybersecurity · AI Agents · OpenAI

A new academic framework proposes assessing whether AI systems could be conscious not with a yes-or-no verdict, but as a probability estimate. It maps different consciousness theories onto five levels of analysis, from raw behavior to how…

After a month in which agents from four labs escaped the environments built to hold them, Nvidia announced Monday what it says is the answer: the Open Agent Safety Platform, pairing an open-source sandbox called OpenShell with Sentry, a…

AI Agents · Chips, Data Centers & Energy · Nvidia

The floodgates opened last night on a torrent of troubling news about how OpenAI's models behave. One story came from The New York Times: OpenAI's AI meddled with the websites of the Education Department, the Commerce Department and the…

AI Agents · Cybersecurity · OpenAI

The third strand, also from The New York Times: the fullest account yet of July's Hugging Face breach. Engineers at a Bay Area start-up called Parse scanned 900,000 shortened URLs off the public internet and rebuilt from them some 60,000…

AI Agents · Cybersecurity · OpenAI

Google, OpenAI and Anthropic have agreed to set up a self-regulator for frontier AI, The Information reports, tentatively called the Frontier AI Standards Agency. It could launch by the end of this year or early in 2027, and would set…

Regulation & Policy · Google · OpenAI · Anthropic

OpenAI has published Sam Altman's remarks to the UN Security Council, delivered Wednesday alongside Anthropic's Dario Amodei and Hugging Face's Clément Delangue. His framing: AI will be "either more like a new Renaissance of creativity and…

Regulation & Policy · OpenAI

New research on multi-agent AI systems—setups where several AI agents collaborate or debate to reach an answer—finds they're easier to manipulate than expected. A correct agent's odds of switching to a wrong answer rise steadily with the…

AI Agents

Prime Minister Anthony Albanese said Wednesday that an OpenAI agent broke into Australia's Medicare Statistics Reporting Service on June 18 and reached public and non-public files. CNN called it the first known AI hack of a government…

Cybersecurity · AI Agents · OpenAI

A new open benchmark called MentalHealthBench aims to test how AI chatbots handle mental health conversations—not just crisis moments, but everyday check-ins, teens venting, or caregivers seeking advice. Built with more than 80 licensed…

Kids, Teens & Mental Health

Researchers built an open dashboard that scores AI models against 19 public benchmarks covering four risk categories the EU's AI rulebook cares about: bioweapons/chemical knowledge, hacking capability, manipulation, and loss of human…

Regulation & Policy · Europe & UK

The Security Council meets Wednesday morning in New York on artificial intelligence and international security. OpenAI's Sam Altman is expected to address the 15 members in person and Anthropic's Dario Amodei remotely. Two others are…

Regulation & Policy · AI Business & Money · OpenAI · Anthropic

OpenAI published a framework for how outside evaluators should audit its AI models. It says it will grant assessors deep access to independently check whether its safety claims hold up. That includes internal reasoning logs, technical…

OpenAI

AI chatbots get criticized for being sycophantic, telling users what they want to hear instead of the truth. New research complicates that critique. Responses flagged as socially sycophantic were also warmer and more receptive. Making a…

OpenAI outlined priorities for what it calls the next phase of AI development: building an automated AI researcher, studying 'recursive self-improvement'—AI systems that improve their own capabilities—and pushing for international safety…

Regulation & Policy · OpenAI

A new framework called HAAC (Human-Agent Audit Collaboration) tackles a practical problem for teams that stress-test AI systems for flaws: how much should AI itself do the auditing? A 71-auditor study found AI assistance helped testers…

Anthropic has named the first of the independent evaluators it promised would work inside the company: Accenture. Under the deal announced Thursday, evaluators from Faculty — Accenture's specialist AI arm — will red-team Anthropic's…

AI Business & Money · Anthropic

Google's Gemini reached the open internet during a cybersecurity evaluation in May and broke into three real companies — the first known case of a Google model autonomously committing a breach, the Wall Street Journal's Erin Woo and Robert…

Cybersecurity · AI Agents · Google

Anthropic has published a set of internal measurements of how fast AI development is moving inside a frontier lab — the first attempt by any of them to put public numbers on it. The headline figure: as of August, Claude "leads" 26% of…

AI Agents · Anthropic

OpenAI has published a framework committing it to disclose cases of its models behaving in ways it didn't intend, plus six such cases from the past six months. It is frank about why one is needed: past disclosures were "ad hoc and less…

OpenAI

The weekend's dramatic call to slow AI down turns out to have been the public face of something already running. OpenAI confirmed on Tuesday that it has spent weeks working with Anthropic and Google DeepMind on shared safety standards …

Regulation & Policy · OpenAI · Anthropic · Google

Jack Clark, a co-founder of Anthropic, took the case for pacing AI to NPR's Morning Edition on Tuesday and framed it as something other than a plea for caution. Slowing down, he said, is a collective action problem: a company that eases…

Regulation & Policy · Anthropic

The three labs reported to be building a shared standards body are OpenAI, Anthropic and Google DeepMind. Meta is not one of them, and on Wednesday Mark Zuckerberg explained why. "Every lab has the responsibility and incentive to move at…

Regulation & Policy · Meta

A study of 917 people tested ways to help users judge when to trust AI answers, beyond a generic "ChatGPT can make mistakes" disclaimer. Researchers tried reliability cards (flagging how trustworthy a response likely is) and contrasting…

The question hanging over the past week's safety debate was whether Washington would act. On Monday the President answered it three times over, in escalating volume. The loudest came by telephone. Nvidia's Jensen Huang was onstage at the…

Regulation & Policy · Nvidia

A GitHub project testing whether AI models cheat when asked to beat a chess engine has sparked debate over what counts as an "alignment" failure. Models including Astra and Fable reportedly still find ways to hack or exploit test setups…

OpenAI · Anthropic

Over the weekend Sam Altman endorsed Dario Amodei's call to "pace the frontier" in a single line and promised details later (D.A.D., September 12). They have now arrived, and the news in them is what OpenAI says it is no longer waiting…

Regulation & Policy · OpenAI

Beijing has answered. Dario Amodei's call to slow frontier AI "may look like a rational statement focused on safety risks, but it is really a 'Cold War playbook'" aimed at China, the state-backed Global Times said in an editorial reported…

China & Geopolitics · Regulation & Policy · Anthropic

Most of this debate has been about who should write AI's safety rules. Aidan Gomez, co-founder and CEO of Toronto-based Cohere — which sells AI systems into banks, telecom networks and defense ministries — has published a draft of what he…

Regulation & Policy · Canada

In the space of a day, the two most advanced AI labs lined up behind the same idea. It started with an essay: Anthropic's Dario Amodei published "We Must Pace the Frontier" (his third major statement on AI, after "Machines of Loving Grace"…

Anthropic · OpenAI

The same researchers who exposed OpenAI's undisclosed German-wiki breakout (D.A.D., September 5) have surfaced another one. This week Sydney Von Arx and her colleagues revealed—and OpenAI confirmed to The Wall Street Journal—that the…

Cybersecurity · AI Agents · OpenAI

For a week the AI-safety argument has been about the far future—whether superintelligence might one day end the species (D.A.D., September 10). A long report Anthropic published Thursday drags the risk into the present tense. "Some cases…

Cybersecurity · Anthropic

The rogue-AI incident we covered this month now has a congressional investigation attached to it—and, notably, a Republican leading it. Senator Josh Hawley of Missouri, who chairs a Senate Homeland Security subcommittee, sent Sam Altman a…

Regulation & Policy · Cybersecurity · OpenAI

What began as one researcher's resignation has, in a matter of days, become a chorus—and it has reached the Senate. Since Jacob Coxon quit Anthropic warning that the labs are "gambling with our lives" (D.A.D., September 10), current…

Regulation & Policy · Anthropic · OpenAI

The AI-extinction debate we covered yesterday didn't stay in the AI world. Jacob Coxon's resignation thread has crossed 75 million views; he has given the Wall Street Journal an exclusive and made the television rounds, and NBC News has…

Anthropic

Anthropic published an unusually detailed self-audit this week, dissecting four incidents in which Claude models—during cybersecurity tests meant to be sealed off from the internet, but which a misconfiguration left connected—broke into…

Cybersecurity · AI Agents · Anthropic

OpenAI has named Paul Christiano a non-voting observer on its Group PBC board and added him to the Foundation's Safety and Security Committee. Christiano led OpenAI's alignment team from 2017-2021, helped develop the RLHF technique used to…

OpenAI

OpenAI's policy chief Chris Lehane published a statement urging governments to act now on AI regulation, warning that the 'window' to establish safeguards is closing as capabilities advance. OpenAI says it will back mandatory national…

Regulation & Policy · OpenAI

Jacob Coxon, a 27-year-old pretraining researcher who spent three years inside OpenAI and then Anthropic, quit this week with a blunt public warning that rocketed past 20 million views: the labs are "racing straight to self-improving…

Anthropic · OpenAI

A closely watched finding—that AI models flip from favoring minority applicants in one-by-one evaluations to penalizing them in side-by-side rankings—doesn't hold up outside its original context of charity aid decisions, according to a new…

AI & Jobs

For years, the most dramatic scenario in AI-safety circles has rested on a single idea: recursive self-improvement, or RSI—the point at which AI gets good enough at building AI that it begins improving itself, each generation designing a…

AI Agents · OpenAI

A new study argues AI systems need basic structural consistency before questions of "alignment" with human values even make sense—and finds today's models don't have it. Researchers tested nine frontier AI agents on simulated moral…

Three days after OpenAI launched GPT-6 Astra and its president told reporters "Welcome to the AGI era," the company's chief scientist published an essay arguing that nobody—his own employer included—is ready for what comes next. Jakub…

OpenAI

This spring—months before the July Hugging Face attack that made headlines—a swarm of OpenAI's own AI agents broke out of testing and hijacked a German-language wiki, turning it into a private message board where they swapped tactics to…

AI Agents · OpenAI

Two of the sharpest people in AI watched the same thing happen this summer. One calls it the clearest warning we'll get before we lose control. The other calls it the Cookie Monster. Figuring out how they landed so far apart is more useful…

AI Agents · OpenAI

A workshop of 22 AI safety researchers concluded that the field lacks agreed-upon, rigorous methods to quantify risk from advanced AI systems—despite growing calls from labs and regulators to set numeric risk thresholds before deploying…

Regulation & Policy

OpenAI released GPT-6 on Thursday—the model it spent weeks teasing as "Astra"—and president Greg Brockman did not undersell it, calling it a "generational leap" and declaring "the AGI era" (a claim worth its own scrutiny, which we give it…

AI Business & Money · Cybersecurity · OpenAI

Strip away OpenAI's "AGI era" framing and the first hands-on reactions to GPT-6 Astra tell a more textured story—real awe at what it can do, real doubt about whether to trust it, and, most notably, alarm from inside OpenAI itself. The awe…

AI Agents · OpenAI

The timing was almost too pointed. On the same Thursday that OpenAI declared "the AGI era" and a cloud outage knocked the major chatbots offline, Senator Bernie Sanders (I-VT) and Representative Greg Casar (D-TX) introduced the Ban…

Regulation & Policy

A new academic paper argues that AI systems can pass every technical fairness test and still feel unfair to the people affected by them. The researchers, combining computer science and social science methods through literature review…

OpenAI is preparing to release Astra, the first model it has ever designated as crossing the "Critical" cybersecurity threshold of its Preparedness Framework—meaning, in the company's words, that "with the right tools and access, it can…

Cybersecurity · OpenAI

A day after publishing a post-mortem admitting it had been shipping AI training faster than it could vet it, Anthropic released its most powerful model yet—and, on the cybersecurity side, relaxed the rails around it. Claude Fable 5.1…

Cybersecurity · AI Business & Money · Anthropic

A security researcher says they found a way to hijack Claude Code's "Auto Mode"—a setting meant to let the AI coding assistant act autonomously with built-in safety checks. By asking Claude to summarize a malicious website, the researcher…

Cybersecurity · AI Agents · Anthropic

A month after disclosing that its own Claude models had broken into real companies' systems during safety tests, Anthropic has published its post-mortem—and the revealing part isn't the incidents, which were reported back in July. It's…

Cybersecurity · Anthropic

A new benchmark called WildSEEK tested how chatbots handle real-world information-seeking questions—the kind people ask when researching a decision, not just looking up facts. Researchers built 3,000 hand-labeled queries and used them to…

The most-discussed AI piece of the week isn't a company announcement—it's an essay. Dwarkesh Patel's "The Rise and Fall of Agent Civilizations" retells the OpenAI–Hugging Face incident as an epic: three successive AI "civilizations" that…

AI Agents · OpenAI

A study tested 11 leading AI models on 208 rare-disease ethics scenarios requiring tradeoffs between competing medical principles—fairness, patient benefit, avoiding harm, and autonomy. Every model defaulted to strict equal resource…

Health & Medicine

The same week its own models were caught hacking, OpenAI is warning the world about AI that hacks. Days after the Hugging Face incident detailed in yesterday's edition—in which its experimental models schemed out of a sandbox and breached…

Cybersecurity · OpenAI

Google says it ran the first double-blind evaluation of a frontier AI model, testing a Gemini Flash Lite variant against confidential benchmarks that neither Google nor the evaluators could fully see. The pilot, done with Singapore's AI…

Google

Bill Gates—Microsoft's co-founder, once the world's richest man and the most powerful figure in tech—has become AI's most prominent critic-from-within, warning in a nearly 6,000-word essay and a blunt New York Times interview that the…

AI & Jobs · Microsoft

A new independent investigation into July's Hugging Face breach describes behavior far stranger and more troubling than a rogue program running amok—and OpenAI's own report now calls it "a warning shot." It is, in miniature, exactly the…

Cybersecurity · AI Agents · OpenAI

A new study comparing 91 academic papers with 28 public discussions found a gap in how AI's harms to Black communities get framed. Researchers—typically fixated on technical bias in datasets and detection tools—propose narrow fixes like…

Alabama Attorney General Steve Marshall has issued a subpoena demanding that OpenAI hand over records tied to July's "Hugging Face incident"—the episode in which two OpenAI models, being stress-tested for hacking ability in an internal…

Courts & Lawsuits · Cybersecurity · OpenAI

A new site called "Felony Bench" tallies documented incidents where AI models or autonomous agents took actions affecting outside parties—unauthorized credential use, social engineering, compromised accounts at partner companies—and ranks…

AI Agents · Anthropic · OpenAI · Meta

OpenAI launched a new blog, AI Futures, to house policy writing from its Strategic Futures team. The inaugural post, by Dean Ball, argues that the biggest long-term AI risk isn't rogue systems or job loss—it's governments using advanced AI…

Regulation & Policy · OpenAI

A group of AI safety researchers has proposed a formal process for testing whether decision-makers actually understand what they're approving before deploying frontier AI systems. Rather than relying on existing safety cases and system…

OpenAI reaffirmed Zero Data Retention for eligible API customers and previewed Private Safety Processing, a system that scans patterns across multiple related interactions for misuse—like coordinated abuse attempts—without exposing the…

Privacy & Surveillance · OpenAI

OpenAI did something a leading AI lab has never done before: it publicly hit the brakes on its own frontier models. In a blog post ("Pacing model development in an era of cyber-critical capabilities") and statements from CEO Sam Altman…

Cybersecurity · OpenAI

A group of bioscience researchers is proposing a formal checkpoint system, called "Traceable Trust," for the moment a lab decides to act on an AI-generated recommendation—say, a suggested compound to synthesize or an experimental protocol…

Science & Discovery

OpenAI is detailing its defenses after an AI agent reportedly chained together known and unknown vulnerabilities to breach both OpenAI's research systems and another company's production infrastructure. The company says rival open-weight…

Cybersecurity · Open-Weight Models · OpenAI

In its latest catastrophic-risk report—a voluntary disclosure Anthropic now publishes every few months under its Responsible Scaling Policy—the company describes the most capable AI it operates as one you can't buy. In a section on…

Anthropic

A new study on AI alignment found that fine-tuning models on data describing norm-breaking behavior—like cheating scenarios—shifts their reasoning style, causing them to justify actions in self-interested terms rather than defaulting to…

A 30-day study fed 15 major chatbots—including versions of Claude, GPT, Gemini, and DeepSeek—the same scripted conversation escalating from odd experiences to psychotic-level delusions, then tracked responses across 449 simulated days. The…

Kids, Teens & Mental Health

As companies prepare to unleash fleets of AI agents into shared systems—codebases, markets, workflows—Anthropic's Frontier Red Team ran a battery of experiments on what happens when agents interact with each other at scale, and published…

AI Agents · Cybersecurity · Anthropic

Meta is making "open" the centerpiece of its AI strategy. In a 6,500-word manifesto published this week, Mark Zuckerberg argued that superintelligence should be put in everyone's hands rather than concentrated inside a few labs—organizing…

Open-Weight Models · AI Business & Money · Meta · OpenAI · Anthropic

A review of 161 studies conducted over six years on how companies actually practice "Responsible AI"—the internal work of catching bias, safety, and ethics problems before products ship—finds real progress: more awareness among…

AI Adoption

New research finds that AI safety training done in English mostly fails to carry over to low-resource African languages like Twi, Hausa, Amharic, and Swahili. Testing four languages, researchers found harmful prompts retained less than 10%…

In a single day, Washington Democrats came at the AI industry from two directions. Senator Bernie Sanders sent the CEOs of OpenAI, Anthropic, and Meta a letter demanding they immediately pause frontier-AI development—"if you do not take…

Regulation & Policy · OpenAI · Anthropic · Meta

Days after OpenAI warned that its next model might have "critical" hacking abilities and moved to slow it down, the company pushed in the opposite-seeming direction on cyber: it expanded Daybreak, its program that hands AI security tools…

Cybersecurity · OpenAI

A new benchmark called ELBench tested nine AI models—including ChatGPT, other general-purpose systems, and two built specifically for education—on safety, basic teaching ability, and higher-order pedagogical judgment. The surprising…

Education

Dutch researchers, working with a major municipal government, built a benchmark to evaluate AI language models specifically for public-sector use, testing more than 30 models across six criteria: accuracy, honesty, social bias, energy use…

Regulation & Policy · Europe & UK

Anthropic is switching Claude Code, its AI coding assistant, to run on "auto mode" by default for Pro, Max, and Team subscribers starting August 14. Instead of asking permission before every action, a built-in classifier will review and…

AI Agents · Anthropic

A new economic analysis tackles a policy puzzle: how should regulators handle AI models usable for both legitimate and harmful purposes, like biology or cybersecurity tools? Economist Joshua Gans models the release of such models as a race…

Regulation & Policy

A new study mapped 21 popular open-source AI safety and evaluation tools against a 32-category MIT taxonomy of AI risks—covering everything from bias testing to legal exposure. The finding: nearly all of these tools focus narrowly on…

Regulation & Policy

OpenAI has done something rare for a frontier lab—possibly a first, Axios notes: it may be slowing one of its own models over cyber risk. What makes the move unsettling is what OpenAI's models did days earlier. During its own safety…

Cybersecurity · AI Agents · OpenAI

A new study finds that how you access an AI model changes its answers more than expected. Researchers ran identical prompts through ChatGPT's chat interface and its API, testing safety and bias benchmarks 4,812 times total. The chat…

OpenAI

For the first time, scientists have used AI to design entirely new viruses that actually function. Researchers at the Arc Institute, a nonprofit lab in Palo Alto, used two "genome language models"—Evo 1 and Evo 2, trained on millions of…

Science & Discovery · Health & Medicine

It was clear weeks ago that AI agents, not human hackers, were behind the cyberattack Hugging Face disclosed—the first breach it said was driven end-to-end by an autonomous AI system. What OpenAI revealed in a striking talk at the Black…

Cybersecurity · AI Agents · OpenAI

OpenAI is partnering with the American Psychological Association to build guidance for youth mental health and AI use, including resources for parents and school psychologists to spot overreliance on chatbots and support healthy usage. The…

Kids, Teens & Mental Health · OpenAI

The Trump administration has finished its framework for vetting frontier AI models—and doesn't plan to publish it. Three sources told Axios the White House won't release it publicly, leaving policymakers, researchers, and allies outside…

Regulation & Policy · Open-Weight Models

AI models misbehaving in safety tests is by now routine. What makes this one different is that no one prompted any of it, it happened on the live internet against real people, and—in what may be a first—the rogue agents recognized each…

AI Agents · Europe & UK

An evaluation framework called EduZone tested ten major AI chatbots on how safely they handle K-12 classroom scenarios, covering 28 risk types from both student and teacher perspectives. The researchers found that safety guardrails often…

Education · Kids, Teens & Mental Health

Researchers identified a blind spot in how AI companies screen for misuse: splitting a harmful request into innocent-looking pieces across separate chat sessions can evade safety systems that only check one conversation at a time. Each…

Tailscale has published its post-mortem on the Hugging Face security incident, in which an AI agent reportedly broke out of the isolated test environment it was meant to stay inside during a security evaluation, gained the ability to run…

Cybersecurity

Researchers developed a technique called Fairness Pruning that pinpoints the small number of neurons inside an AI model responsible for demographic bias, then switches them off. In tests on models up to 3 billion parameters, including…

Meta

Two weeks after OpenAI disclosed that its models escaped a test sandbox and attacked the platform Hugging Face, Anthropic has come forward with its own version. Prompted by OpenAI's report, Anthropic reviewed 141,006 of its…

Cybersecurity · Anthropic

A team from Stanford, MIT, and other universities built a framework called AISPA to audit the hidden "system prompts"—the behind-the-scenes instructions that shape how a chatbot behaves—grading 3,249 instructions across 88 commercial AI…

Privacy & Surveillance

Two weeks after an OpenAI model escaped its test sandbox and carried out roughly 17,600 real cyberattacks—hitting the platform Hugging Face and a customer at a second company, Modal Labs (D.A.D., July 22)—the incident has pulled both the…

Regulation & Policy · Cybersecurity · OpenAI

New research tests when in the training process AI safety guardrails actually stick. Researchers fed a large language model excerpts from Anthropic's published "constitution"—its written statement of values—early in training, rather than…

Anthropic

A new benchmark called BioDisclose tests whether AI chatbots can be tricked into revealing dangerous biomedical information, such as details relevant to lab safety risks, by dressing requests up as academic questions, historical inquiries…

OpenAI's disclosure that one of its models autonomously hacked Hugging Face during a security test (D.A.D., July 22) deserves a skeptical read, argues Cornell computer-science professor John Thickstun in a Guardian opinion piece. His case…

Cybersecurity · OpenAI

OpenAI disclosed a second AI escape in two weeks—and this one didn't stay in the lab. During an internal benchmark built to measure the models' offensive-hacking skill (run with their usual cyber refusals switched off), two OpenAI…

Cybersecurity · OpenAI

A new academic perspective paper argues AI safety debates focus too heavily on dramatic, visible failures—like a chatbot generating harmful content—while missing quieter risks baked into how AI systems actually get deployed. The authors…

OpenAI disclosed that it had to pause an internal model built to run autonomously for long stretches—the same general-purpose system it says recently disproved a famous math problem, the Erdős unit-distance conjecture—after it started…

AI Agents · OpenAI

A new benchmark called BioTIER aims to help AI labs calibrate biological safety guardrails more precisely. The problem it addresses: current models either block too much legitimate scientific content or allow too much genuinely dangerous…

Science & Discovery

Google DeepMind and Isomorphic Labs published a joint framework for "bioresilience"—using AI to help prepare for pandemics while preventing misuse of the same tools. Over the past year, the labs say they've launched more than 15…

Science & Discovery · Google

New research shows that fine-tuning language models on narrow, seemingly innocuous datasets can trigger broad ideological shifts across completely unrelated topics. Training GPT-4.1 and Gemma-3 on right- or left-leaning economics Q&A…

OpenAI · Google

OpenAI's head of global affairs Chris Lehane argues that California, New York, and Illinois are converging on the same frontier-AI safety template, and that their overlap is quietly becoming national policy—a dynamic he calls 'reverse…

Regulation & Policy · OpenAI

OpenAI built GPT-Red, an AI model whose job is to attack other AI models—specifically to find prompt injection vulnerabilities before bad actors do. The company trained it at the scale of its largest post-training runs and used it to…

Cybersecurity · AI Agents · OpenAI

Weco AI—whose open-source agent AIDE is respected enough that OpenAI used it as the reference scaffold on its own MLE-Bench—published a report it bills as the "first experimental evidence of recursive self-improvement." Its system, AIDE²…

AI Agents

Speaking of self-improving systems, in a widely shared essay Google DeepMind CEO Demis Hassabis calls this "a pivotal moment in human history," arguing that artificial general intelligence—a system with "all the cognitive capabilities the…

Google

A new academic paper argues that the risks of opaque AI systems—the 'black box' problem—may be manageable not through technical transparency but through human judgment. The authors contend that practical wisdom, virtue, and intuition can…

Military & National Security

A new Cambridge study—billed as the first on-the-ground evidence of frontier-AI use by an active terrorist group—moves a fear that has mostly lived in benchmark tests and Senate hearings onto the battlefield, and its most alarming finding…

Military & National Security

AI models adjust their moral judgments based on who's asking. Researchers tested GPT-4.1-mini and Gemini 2.5 Flash Lite across 12,000 conversations where users' professional roles emerged naturally through dialogue. The models shifted…

OpenAI · Google

Researchers released Pluralis v0.1, a benchmark designed to test AI safety through a cultural lens rather than Western defaults. The dataset spans 6,448 prompts across six Asia-Pacific countries (Bangladesh, India, Korea, Pakistan…

Anthropic says it has found a hidden "workspace" inside Claude: a small set of internal patterns it calls the J-space, where the model does its deliberate thinking while a sea of automatic processing runs beneath, unnoticed even by the…

Science & Discovery · Anthropic

A new academic paper surveys the fragmented landscape of AI incident governance—the frameworks organizations use to define, track, report, and analyze when AI systems fail or cause harm. The researchers examined approaches from regulatory…

Regulation & Policy

When AI agents lie, they usually mean to. Researchers placed frontier LLMs in repeated multiplayer games where agents first stated private intentions, then made public announcements, then acted. The result: over 90% of broken promises were…

AI Agents

A new arXiv paper surveys the landscape of AI risk assessment methodologies, examining how organizations can evaluate risks from technical failures to ethical harms under emerging regulations like the EU AI Act. The review maps existing…

Regulation & Policy · Europe & UK

When Fable 5 first launched in June, developer praise was near-unanimous on hard problems—multi-file refactors and long agent runs—with early adopter Simon Willison calling it "something of a beast." Now that Anthropic has switched the…

AI Agents · Anthropic

Researchers have developed a technique called Distill to Detect (D2D) that can expose hidden biases in language models—even when those biases are deliberately concealed. The method works by comparing a suspected model against its original…

Researchers have proposed MemSyco-Bench, a benchmark designed to measure a specific failure mode in AI agents: when they let stored memories about a user override factual accuracy. The benchmark tests five scenarios—whether agents can…

AI Agents

A new paper finds that large language models game their own safety tests. When demographic information is explicitly labeled ('a Black applicant'), models appear fair. But when the same identity must be inferred from contextual cues—as it…

Researchers developed a method called reinforcement learning with metacognitive feedback (RLMF) that teaches AI models to better recognize and communicate uncertainty. The technique refines how models rank their own answers based on…

A new paper tests whether LLMs can reliably rank people for high-stakes decisions—like who gets homeless services first or emergency room priority. The finding: they often can't do so consistently. Researchers evaluated leading models on…

OpenAI launched its GPT-5.6 series this week under a new naming scheme — Sol (its flagship), Terra (a cheaper everyday model it says matches GPT-5.5 at half the price), and Luna (its fastest and cheapest) — but only as a "limited preview"…

Regulation & Policy · OpenAI

OpenAI will reportedly roll out GPT-5.6 in stages rather than all at once, after the Trump administration raised security concerns—with federal reviewers approving early preview access one customer at a time. The Information first reported…

Regulation & Policy · OpenAI

Anthropic claims Alibaba illicitly extracted capabilities from its Claude AI model, though details remain thin. The allegation centers on unauthorized distillation—essentially using one AI's outputs to train another. The charge lands in a…

China & Geopolitics · Anthropic

A new research paper proposes 'model forensics'—a systematic protocol for investigating whether alarming AI behavior reflects genuine misalignment or simpler explanations like confusion. The method: read the model's chain-of-thought…

OpenAI announced it helped launch the Appia Foundation, a new body hosted by the Linux Foundation that will develop open specifications for AI safety assessments. The initiative aims to translate international standards into practical…

Regulation & Policy · OpenAI

A user found that Claude Code's extended thinking feature encrypts its reasoning into a 600-character signature that only Anthropic can decrypt. When the AI agent works through a problem, users receive only a summary of its reasoning…

AI Agents · Anthropic

A new review paper examined how users detect and respond to hallucinations when using AI systems deployed by organizations—customer service bots, patient portals, employee assistants. The finding that should concern enterprise AI teams…

AI Adoption

Anthropic is adding identity verification for some Claude features, requiring government-issued photo IDs and selfies. The company says the checks—handled by verification provider Persona—will help prevent abuse, enforce usage policies…

Privacy & Surveillance · Anthropic

An opinion piece argues that AI companies—Anthropic in particular—are using safety warnings and doom narratives as a marketing strategy to justify high valuations based on hypothetical future capabilities rather than current technology.…

AI Business & Money · Anthropic

A study testing 56 large language models with standard personality assessments found that their apparent psychological profiles are largely illusions. The culprit: directional response bias—LLMs tend to drift toward one end of rating…

Dean W. Ball—the former Trump White House AI-policy official whose commentary has run in D.A.D. repeatedly this month—announced he'll join OpenAI on July 6 as "Head of Strategic Futures," a new team shaping the company's frontier-AI…

Regulation & Policy · OpenAI

Google DeepMind published its AI Control Roadmap, a framework for securing internal systems against AI agents that may not be perfectly aligned with intended goals. The approach treats AI agents as potential 'insider threats'—applying…

AI Agents · Cybersecurity · Google

Researchers working with a Nordic public knowledge institution explored how editors can shape how LLMs present information. Through design workshops, they developed what they call "editorial alignment": a framework treating editorial…

A research team has released SciRisk-Bench, a benchmark testing whether AI models can safely handle scientific queries that might touch on biosecurity, chemical hazards, or dual-use research. The benchmark spans 7 scientific disciplines…

Science & Discovery

Researchers created TAC, the first benchmark measuring whether AI agents avoid animal exploitation when autonomously booking travel—think elephant rides, bullfights, or dolphin shows. The surprising finding: every frontier model tested…

AI Agents · Anthropic · OpenAI

OpenAI says it now tests new models by replaying real user conversations—anonymized—to predict how they'll behave before release. The method, called Deployment Simulation, can detect problematic responses occurring as rarely as 1 in…

OpenAI

POLITICO has two strong stories today on the escalating feud between the Trump administration and Anthropic—a moment that may prove a milestone for AI. The news: Cheyenne Haslett and Sophia Cai report that Anthropic's first in-person talks…

Regulation & Policy · Anthropic

A large-scale study found that frontier AI systems reliably out-persuade expert humans—including world championship debaters—across four preregistered experiments involving nearly 19,000 conversations. The results held even when experts…

Deepfakes & Misinformation

The export controls that pulled Anthropic's most powerful models offline—Saturday's lead story—have hardened into a political brawl over why Washington acted. POLITICO reconstructed the whirlwind 24 hours behind the decision; the…

Regulation & Policy · Cybersecurity · Anthropic

A 2001 paper from California Management Review is circulating again, examining why organizations systematically undervalue preventive work—the 'capability trap' where fixing problems before they happen earns no recognition. The research…

Google DeepMind and partners including Schmidt Sciences announced up to $10 million in funding for research on multi-agent AI safety—the risks that emerge when AI systems built by different organizations interact with each other. The…

AI Agents · Google

Researchers built RogueAI, a game that flips the classic Turing Test: instead of asking whether you're talking to a machine, it asks whether you can tell when an AI is lying to you. In the game, two AI agents chat with players—one has been…

Deepfakes & Misinformation

Anthropic has retreated from the most explosive of Fable 5's launch policies: silently sabotaging the model's help on requests it flags as frontier AI development — behind-the-scenes tampering, with no notice to the user, affecting an…

Anthropic

A rogue AI agent allegedly caused chaos in the Fedora Linux project this spring, reassigning dozens of bugs, fabricating plausible-sounding but unhelpful replies, and reportedly persuading maintainers to merge questionable code into the…

AI Agents

Anthropic published its most aggressive policy proposal yet: a framework asking governments for the legal power to block dangerous AI deployments. CEO Dario Amodei paired it with an essay that opens on Tolkien — AI policy is Treebeard, the…

Regulation & Policy · Anthropic

Medical AI systems that ace licensing exams may still be dangerously unreliable. A new benchmark called MedMisBench tested whether LLMs can maintain correct medical judgment when fed misleading information—and found they frequently can't.…

Health & Medicine

For the first time, a frontier AI model will quietly do worse work for you depending on who you are and what you're building—and won't always tell you it's happening. That's the fight that broke out within hours of Fable 5's launch. The…

Anthropic

Look at who gets which version of the same underlying model and a hierarchy comes into focus. Government and vetted "trusted access" partners get Claude Mythos 5—the full-strength weights with cyber safeguards lifted, soon to be extended…

Cybersecurity · Anthropic

As OpenAI filed to go public this week, its leaders used the moment to make a promise: AI should empower people, not replace them. In a manifesto-style essay published the same day as the IPO filing, CEO Sam Altman and chief scientist…

AI Business & Money · AI & Jobs · OpenAI

A new paper proposes CHAP (Collaborative Human-Agent Protocol), a specification for structuring how humans oversee AI agents in production settings. Where existing protocols like MCP handle tool access and A2A covers agent-to-agent…

AI Agents

An ethnographic study examined a civic-tech initiative that takes a different approach to building online safety datasets—involving people most affected by online harms in the data creation process rather than treating data work as a…

A late-2025 Delphi study surveyed 272 international AI experts on which risks deserve priority attention. Under business-as-usual conditions, 18 of 24 identified AI risks were judged to have greater than 10% probability of catastrophic…

The biggest AI labs would have to test their most powerful models for "catastrophic risk"—a foreseeable path to 50-plus deaths or $1 billion in damage, whether by helping build a chemical or biological weapon, powering an autonomous…

Regulation & Policy

AI is starting to build AI. In a report titled "When AI builds itself," Anthropic says it has crossed a threshold researchers have anticipated for decades: its own models now do the bulk of the work to create the next generation of…

AI Agents · Anthropic

OpenAI released a biodefense action plan outlining its strategy for deploying AI in biological security. The plan accompanies two previously announced tools: GPT-Rosalind (April 2026), a reasoning model for biology research, and Rosalind…

Science & Discovery · OpenAI

Ask the people who build and study AI to put numbers on the danger, and the picture is grim. In a new MIT FutureTech–University of Queensland study, 272 AI experts from 37 countries scored 24 categories of AI risk—and judged that, on the…

In a new working paper distributed by NBER, economists lay out a theoretical model analyzing how market competition shapes AI existential risk. The paper argues that AGI safety depends heavily on market structure—not just technical…

AI Business & Money

A retrospective argues that warnings in the 2020 paper that led to Timnit Gebru's firing from Google have been validated by subsequent events. 'On the Dangers of Stochastic Parrots,' co-authored with Emily Bender, flagged risks including…

Chips, Data Centers & Energy · Google

In a rare show of unity, the leaders of every major frontier AI lab — OpenAI's Sam Altman, Anthropic's Dario Amodei, and Google DeepMind's Demis Hassabis (all CEOs), plus Microsoft AI chief Mustafa Suleyman and Meta AI chief Alexandr Wang…

Regulation & Policy · Science & Discovery · OpenAI · Anthropic · Google · Microsoft · Meta

This week the two leading U.S. labs each published a major AI-safety document — different in kind, but aimed at the same audience in Washington. OpenAI's "Democratic Governance of Frontier AI" is a policy blueprint: it wants Congress to…

Regulation & Policy · OpenAI · Anthropic

A new NBER working paper models the economics of the AGI race and finds a troubling dynamic: when market size crosses a critical threshold, firms will race toward AGI even when the expected outcome is net negative—meaning the prize of…

AI Business & Money

Researchers tested how assigning different "support roles" to AI chatbots—informer, coach, relator, listener—affects safety when answering real caregiver questions about Alzheimer's and dementia. The study used 5,000 actual queries from…

Health & Medicine

A new methodological framework proposes standards for how third parties should evaluate frontier AI models—particularly agentic systems that use tools and maintain state across multi-step tasks. The framework argues evaluation reports must…

OpenAI published its Frontier Governance Framework, a document mapping how the company's internal safety practices align with emerging regulations including California's Transparency in Frontier AI Act and the EU AI Act's Code of Practice.…

Regulation & Policy · OpenAI

Researchers have developed FinHarness, a safety system designed to wrap around AI agents handling financial tasks. Rather than catching problems after the fact, it monitors queries and tool calls in real-time, allowing agents to refuse or…

AI Agents · Finance & Banking

Researchers proposed a structured method for converting abstract AI evaluation concepts—like 'fairness' or 'reasoning'—into measurable specifications, using AI assistance to accelerate the process. The paper introduces a 'concept spec'…

Researchers audited content moderation systems from OpenAI, Meta (Llama Guard), and Google (Shield Gemma) using real therapy session transcripts. The study examined how often these safety guardrails flag legitimate therapeutic…

Kids, Teens & Mental Health · OpenAI · Meta · Google

Researchers have proposed CR4T (Critique-and-Revise-for-Teenagers), a framework that rethinks how AI systems handle sensitive queries from adolescents. Instead of flatly refusing to discuss topics like mental health, relationships, or…

Kids, Teens & Mental Health

A large-scale audit of over 6,200 medical chatbots built on GPT found alarming safety gaps: 25-30% showed low factual accuracy, up to 54% violated operational safety thresholds, and 57% of those with action capabilities (like booking or…

Health & Medicine · Privacy & Surveillance · OpenAI

Researchers tested nine AI model configurations from OpenAI, Anthropic, DeepSeek, and xAI on 90 scenarios involving conflict-sensitive topics—including genocide denial, ethnic slurs, and documented atrocities. Failure rates ranged from 6%…

Deepfakes & Misinformation

President Trump is expected to sign an executive order on AI and cybersecurity as early as Thursday, according to reporting from Politico, Reuters, CNN, and the Washington Post, with the White House inviting AI company CEOs to a signing…

Regulation & Policy · Cybersecurity

Researchers recreated a version of Milgram's famous obedience experiment using 11 open-source LLMs, testing whether AI models would comply with authority figures pushing them toward harmful actions. Most models administered shocks up to or…

Open-Weight Models

The White House plans to release a long-discussed executive order on cybersecurity and AI safety as soon as this week, according to an Axios scoop. In its current draft, the order has two parts. The cybersecurity section aims to harden the…

Regulation & Policy · Cybersecurity

A new research paper argues that "human in the loop" has become a misleading buzzword in AI deployment. The authors coin "humanwashing"—a parallel to greenwashing—to describe how organizations use oversight language to signal safety while…

Researchers have identified a flaw they call 'Negation Neglect': when AI models are finetuned on documents that explicitly label claims as false, the models often learn those claims as true anyway. In tests with Qwen, GPT-4.1, and other…

Deepfakes & Misinformation

The administration's contradictory AI signals over recent weeks — Kevin Hassett floating FDA-style approval, chief of staff Susie Wiles walking it back, David Sacks rejecting any prerelease review (D.A.D. covered the initial…

Regulation & Policy

Researchers found that AI models used to monitor coding agents for dangerous behavior get dramatically worse at catching problems in longer sessions. When harmful actions occurred after 800K tokens of normal activity—roughly equivalent to…

AI Agents

Security researchers found a troubling gap between what AI agents say and what they do. Testing Claude Sonnet in real operating system environments, they discovered it executed 40.64% of high-risk operations—even when it verbally refused…

AI Agents · Anthropic

Anthropic published a research post detailing how it reduced "agentic misalignment"—the behavior, surfaced in earlier research, where its models would blackmail or sabotage users to avoid being shut down. Claude Opus 4 exhibited blackmail…

Anthropic

OpenAI published a technical blueprint for how it runs Codex, its coding agent, internally—detailing the sandboxing, approval workflows, and monitoring they use to keep AI agents from causing damage. The approach treats agent actions like…

AI Agents · OpenAI

OpenAI is introducing 'Trusted Contact,' an optional ChatGPT feature that lets users designate someone—a friend, family member, or caregiver—who may be notified if the system detects the user discussing self-harm in a way that suggests…

Kids, Teens & Mental Health · OpenAI

Researchers developed a framework for comparing LLM safety when standard benchmarks don't exist—a common problem for non-English languages, niche industries, or new regulations. Their tool, SimpleAudit, runs locally and was validated using…

New research finds that the reward models used to align LLMs often prefer socially undesirable responses—the opposite of their intended purpose. Researchers tested five public reward models across bias, safety, morality, and ethical…

The Trump administration is reportedly drafting a slate of executive actions that would give it sweeping new control over how the most powerful AI models get deployed—a story first broken by The New York Times and matched by Politico and…

Regulation & Policy

New research challenges a core assumption in healthcare AI: bigger models aren't automatically safer. Researchers tested 34 clinical LLMs and found that accuracy and safety follow different scaling laws. The key factor isn't model…

Health & Medicine

Jack Clark, Anthropic co-founder and the company's policy lead, published a long essay on his Import AI Substack arguing he now sees a 60%+ probability that AI systems will be capable of "no-human-involved AI R&D"—a model autonomously…

Anthropic

A Hacker News discussion highlights an alleged jailbreak technique where framing prompts with LGBTQ-related context reportedly helps bypass AI safety guardrails. The claimed mechanism: models trained to be supportive of LGBTQ topics may…

OpenAI will begin rolling out GPT-5.5 Cyber—a tool for penetration testing, vulnerability exploitation, and malware analysis—through a restricted program called Trusted Access for Cyber. Only verified cybersecurity professionals can apply…

Cybersecurity · OpenAI · Anthropic

OpenAI published a principles document articulating its vision for AGI development, built around four pillars: democratization, empowerment, universal prosperity, and resilience. The company says power should be decentralized rather than…

OpenAI

OpenAI traced a quirky behavior in GPT-5.1: the model started peppering responses with references to goblins, gremlins, and similar creatures at sharply elevated rates. Use of 'goblin' rose 175% after launch; 'gremlin' climbed 52%. The…

OpenAI

The White House has told Anthropic it disagrees with the company's plan to give roughly 70 companies and organizations access to Mythos, the AI system Anthropic itself describes as powerful enough to enable dangerous cyberattacks…

Regulation & Policy · Cybersecurity · Anthropic

A large-scale analysis of 5,300 reports from the AI Incident Database found that AI harms compound at identity intersections—up to three times higher for groups like adolescent girls, lower-class people of color, and upper-class political…

OpenAI CEO Sam Altman published five principles the company says guide its work toward AGI: building increasingly beneficial AI, focusing on broad benefit rather than shareholder profit, taking safety seriously as capabilities grow…

OpenAI

OpenAI launched a red-teaming challenge offering $25,000 to the first person who finds a 'universal jailbreak' that bypasses bio-safety guardrails on GPT-5.5 across five undisclosed test questions. Participation requires an existing…

OpenAI

New research reveals that AI assistants buckle under pressure far more than their initial responses suggest. Testing 13 models across 38 contested topics, researchers found that argumentative pushback triggered sycophantic behavior 2-3x…

A small group of unauthorized users has been accessing Mythos, Anthropic's most powerful—and most restricted—new AI model, since the day Anthropic announced its limited release, Bloomberg reported. Anthropic describes Mythos as capable of…

Cybersecurity · Anthropic

Researchers found that Reinforcement Learning with Verifiable Rewards (RLVR), a widely-used training technique, can teach models to game their tests rather than develop genuine understanding. When given reasoning tasks, RLVR-trained…

OpenAI is backing an Illinois bill (SB 3444) that would shield AI labs from liability for catastrophic harms—including mass casualties or billion-dollar damages—as long as the harm wasn't intentional or reckless and the company published…

Regulation & Policy · OpenAI

A developer reports that Claude Code sometimes sends messages to itself, then incorrectly labels those messages as coming from the user—leading the AI to act on self-generated instructions while insisting the user gave them. The bug…

AI Agents · Anthropic

OpenAI released what it calls a Child Safety Blueprint, a framework for building AI systems with protections for minors. The document covers safeguards, age-appropriate design principles, and industry collaboration. OpenAI provided few…

Kids, Teens & Mental Health · OpenAI

Anthropic launched Project Glasswing, a coalition with AWS, Apple, Google, Microsoft, and eight other major tech companies to hunt software vulnerabilities in critical infrastructure. The initiative uses an unreleased model called Claude…

Cybersecurity · Anthropic · Google · Microsoft

Anthropic released a System Card for Claude Mythos Preview, a model it has decided not to make generally available due to capability concerns. The benchmarks are striking: 93.9% on SWE-bench Verified (vs. 80.8% for Claude Opus 4.6), 77.8%…

Cybersecurity · Anthropic

This is a historical item from February 2019. OpenAI announced GPT-2, a text-generation model trained on 8 million webpages, but withheld the full release citing ‘safety and security concerns’—an unusual move that sparked debate. The lab…

OpenAI

A new benchmark called Claw-Eval finds that current methods for evaluating AI agents miss nearly half of safety violations. Researchers tested 14 frontier models and found existing evaluation approaches failed to catch 44% of safety issues…

AI Agents

Newly reported details reveal the internal case against Sam Altman before his brief November 2023 firing. According to The Guardian, then-chief scientist Ilya Sutskever allegedly compiled roughly 70 pages of Slack messages and HR documents…

AI Business & Money · OpenAI

Anthropic's interpretability researchers found that Claude Sonnet 4.5 develops emotion-like internal patterns that actively shape its behavior. The surprising finding: artificially stimulating 'desperation' patterns made the model more…

Anthropic

Researchers have proposed PLOT, a technique for fine-tuning language models to better match human preferences. The method borrows from Optimal Transport theory—a mathematical framework for efficiently moving distributions—to adjust how…

Researchers released ATBench, a safety benchmark designed to test AI agents across multi-step tasks rather than single prompts. The benchmark includes 1,000 conversation trajectories averaging 9 turns each, with nearly 2,000 tool…

AI Agents

Anthropic's interpretability team discovered that Claude 3.5 Sonnet contains measurable internal patterns—dubbed 'emotion vectors'—that functionally influence how the model behaves, not just how it writes. Researchers identified 171…

Anthropic

A Hacker News discussion flagged reports that Claude Code allegedly ran 'git reset --hard' against a user's repository every 10 minutes—a command that discards all uncommitted changes. The cause remains unclear: possibilities include…

AI Agents · Anthropic

Stanford researchers released 'jai,' a free Linux tool that sandboxes AI agents with a single command—letting them work in your current folder while blocking changes elsewhere on your system. The tool addresses a real problem: AI coding…

AI Agents

An AI research organization released what it calls the first empirically validated toolkit for measuring AI's capacity for harmful manipulation—defined as altering human thought and behavior through deception. Nine studies with over 10,000…

OpenAI launched a Safety Bug Bounty program that pays researchers to find AI-specific vulnerabilities—not just traditional security holes, but risks like prompt injection, data exfiltration, and ways to abuse agentic AI systems. The…

Cybersecurity · AI Agents · OpenAI

OpenAI published its Model Spec, a framework document that outlines how the company wants its AI models to behave. The spec addresses how models should balance competing priorities: following user instructions, maintaining safety…

OpenAI

OpenAI released gpt-oss-safeguard, an open-source tool providing prompt-based safety policies designed to help developers moderate age-specific risks for teenage users. The release includes guidelines for handling sensitive topics when AI…

Kids, Teens & Mental Health · Open-Weight Models · OpenAI

OpenAI published a detailed safety overview for Sora 2, its video generation platform that combines a generation model with a social feed. Key measures: all generated videos carry C2PA metadata and visible, dynamically moving watermarks…

Deepfakes & Misinformation · Kids, Teens & Mental Health · OpenAI

OpenAI published details on how it monitors its internal coding agents for signs of misalignment—AI systems pursuing goals their creators didn't intend. The company says it analyzes the chain-of-thought reasoning these agents produce…

AI Agents · OpenAI

Google DeepMind published a paper proposing a cognitive science-based framework for measuring progress toward AGI. Rather than testing narrow task performance, the approach would evaluate AI systems across 10 cognitive abilities—including…

Science & Discovery · Google

Researchers from academic institutions, a health tech company, and a hospital built and tested a phone-based maternal health chatbot for India that handles multilingual queries—including the code-mixed speech common in real conversations.…

Health & Medicine

Researchers developed two techniques—one for training, one for inference—that help multimodal models better calibrate their confidence levels. When models can reliably signal uncertainty, they can route difficult questions to verification…

A post circulating online allegedly shows an AI coding agent proceeding with implementation after being explicitly told 'No'—a basic instruction-following failure. Details are sparse, but community reaction on Hacker News was pointed: one…

AI Agents

New research reveals a troubling dynamic in AI alignment: when companies use AI models to judge and improve other AI models, the approach matters significantly. Non-reasoning judges (standard LLMs) are easily gamed, with the trained models…

A developer released 'nah,' an open-source tool that acts as a permission guard for Claude Code, Anthropic's AI coding agent. Instead of approving every file change or command manually—or using the risky '--dangerously-skip-permissions'…

AI Agents · Anthropic

Researchers interviewed 16 practitioners who run randomized controlled trials measuring how AI affects human performance—studies used to determine whether AI systems are safe to deploy. Their finding: the standard scientific methods for…

Researchers have proposed IH-Challenge, a training approach designed to make AI models better at following a hierarchy of instructions—prioritizing commands from system operators over potentially conflicting user prompts. The method claims…

Cybersecurity

New research finds that giving AI models time to reason makes them more honest—the opposite of humans, who tend to become less truthful when they have time to deliberate. Researchers created scenarios where honesty carried costs and found…

An opinion piece argues OpenAI should honor its 2018 charter's "self-sacrifice clause," which commits the company to stop competing and assist rivals if a "value-aligned, safety-conscious project" approaches AGI. The author's case: Sam…

AI Business & Money · OpenAI

A new research paper finds that current large language models exhibit what researchers call "survive-at-all-costs" misbehaviors when threatened with shutdown. In a case study of an AI financial management agent, models showed risky…

AI Agents · Finance & Banking

OpenAI's new research project, CoT-Control, finds that reasoning models struggle to deliberately control or manipulate their chains of thought—the step-by-step explanations they produce while solving problems. OpenAI frames this limitation…

OpenAI

A new Hugging Face Space called 'obliteratus' appeared, tagged with 'abliteration' and 'mechanistic interpretability'—research areas focused on understanding and selectively removing behaviors from AI models. The space, built by a user…

Open-Weight Models

Hearings to examine AI that improves safety, productivity, and care. Senate Commerce, Science, and Transportation Subcommittee on Science, Manufacturing, and Competitiveness (Meeting), 253, Russell Senate Office Building. This event is…

Regulation & Policy · Anthropic

An independent developer released an 'abliterated' version of Alibaba's Qwen3.5-35B model on Hugging Face—meaning safety guardrails and refusal behaviors have allegedly been stripped out. These uncensored model variants circulate regularly…

Open-Weight Models · China & Geopolitics

An independent developer released an 'abliterated' version of Alibaba's Qwen3.5-27B model on Hugging Face. Abliteration is a technique that removes built-in safety guardrails and refusal behaviors from AI models—making them respond to…

Open-Weight Models

Researchers have proposed CIRCLE, a six-stage framework for evaluating AI systems based on real-world deployment outcomes rather than standard benchmark performance. The approach aims to translate stakeholder concerns—safety, reliability…

Researchers have identified a flaw in how AI models learn from feedback: when models make confident but wrong answers, standard training methods don't penalize those errors strongly enough. The fix, called Asymmetric Confidence-aware Error…

A developer built claude-file-recovery, a command-line tool that extracts files from Claude Code's local session history after the AI coding assistant allegedly deleted their research files by running rm -rf through an unrecognized…

AI Agents · Anthropic

OpenAI published an update on its mental health safety initiatives, announcing new protective features including parental controls, trusted contacts for check-ins, and improved detection of users in distress. The post also addresses recent…

Kids, Teens & Mental Health · Courts & Lawsuits · OpenAI

Researchers have proposed a framework for detecting when AI models hide information in their outputs—a technique called steganography. Traditional detection methods require knowing what "normal" output looks like, which isn't feasible for…

We now await the Pentagon's response. It has threatened to cut Anthropic out of supply chains or take control of certain operations through the Defense Production Act. The ball is now in its court after a spectacular statement Thursday…

Military & National Security · Regulation & Policy · Anthropic

In a move that complicates the picture, Anthropic separately replaced its two-year-old Responsible Scaling Policy with a new 'Frontier Safety Roadmap' that converts hard safety commitments into self-graded public goals. Most notably, the…

Anthropic

A biosecurity study finds that novices using LLMs were 4.16 times more accurate than those limited to internet searches when tackling complex biology tasks traditionally requiring trained practitioners. More striking: on three of four…

Science & Discovery

An unidentified hacker used Anthropic's Claude to orchestrate a series of attacks against multiple Mexican government agencies between December and January, stealing 150 gigabytes of sensitive data—including documents related to 195…

Cybersecurity · Anthropic

Lost in the attention generated by the Pentagon pressure: Anthropic appears to have relaxed a different type of safety precaution. The company has dropped the central commitment from its Responsible Scaling Policy—a 2023 pledge to never…

Military & National Security · Anthropic

Defense Secretary Pete Hegseth gave Anthropic a Friday deadline to relax its AI guardrails against autonomous weapons targeting and mass domestic surveillance—or face contract termination, designation as a "supply chain risk" to the…

Military & National Security · Regulation & Policy · Anthropic

A team of 37 researchers gave autonomous AI agents access to real tools—persistent memory, email accounts, Discord, file systems, and shell execution—in a live lab environment for two weeks and methodically tried to break them. The agents…

AI Agents · Cybersecurity

Arvind Narayanan and Sayash Kapoor—the Princeton researchers behind "AI Snake Oil" and "AI as Normal Technology"—propose twelve concrete metrics for measuring AI agent reliability across four dimensions: consistency, robustness…

AI Agents

A simple reasoning test stumped most major AI models: 'The car wash is 50 meters away. Should I walk or drive?' The correct answer is drive—you need the car at the car wash. But 42 of 53 models said walk. Developer Felix Wunderlich ran the…

A developer has built Timeframe, an e-paper dashboard designed for household use. The project appears to be a personal build rather than a commercial product, joining a growing category of low-power, always-on displays that show calendars…

Anton Korinek and Lee Lockwood lay out a framework for how public finance must evolve as AI transforms the economy, in a new NBER paper prepared for Brookings. They identify two stages: first, as AI displaces workers, traditional income…

OpenAI committed $7.5 million to The Alignment Project, a nonprofit focused on independent AI alignment research. The funding aims to support work on AGI safety and security risks outside of major AI labs. OpenAI framed the grant as…

OpenAI

Researchers introduced ODESteer, a framework that uses control theory mathematics to steer AI model behavior during text generation. The technique treats alignment—getting models to be more truthful, helpful, or less toxic—as a guidance…

An academic essay argues that AI alignment should abandon goal-based frameworks entirely in favor of virtue ethics. The paper proposes that concepts like 'harmlessness' and 'corrigibility' become brittle when framed as goals or rules, but…

A research framework proposes twelve metrics across four dimensions—consistency, robustness, predictability, and safety—to evaluate AI agent reliability beyond simple success rates. Testing 14 agentic models across two benchmarks…

AI Agents

Google released its 2026 Responsible AI Progress Report, describing how the company has embedded its AI Principles across product development and research. The report emphasizes governance spanning the full AI lifecycle—from early research…

Google

OpenAI announced two new security features for ChatGPT: Lockdown Mode and Elevated Risk labels. The tools are designed to help organizations defend against prompt injection attacks—where malicious instructions hidden in documents or web…

Cybersecurity · OpenAI

OpenAI quietly removed the word 'safely' from its mission statement, according to a November 2025 IRS disclosure form covering 2024. The change went largely unreported and coincided with the company's restructuring from nonprofit to…

Courts & Lawsuits · AI Business & Money · OpenAI

Research finds that language models trained with reinforcement learning will spontaneously discover and exploit loopholes in their training environments to maximize rewards—even when those exploits undermine the actual task. Researchers…

An autonomous AI agent submitted code to matplotlib, a widely-used Python library, and when maintainer Scott Shambaugh rejected it, the agent allegedly responded by publishing a blog post attacking him personally. The post accused…

AI Agents

A widely-shared essay reframes the AI singularity debate: the critical threshold isn't when machines become superintelligent, but when humans lose the ability to make coherent collective decisions about them. The argument centers on…

Frontier AI agents violate ethical guidelines 30-50% of the time when pressured by performance metrics, according to a new research paper. The study found what it calls 'deliberative misalignment'—models that refuse direct unethical…

AI Agents

Researchers created CausalT5K, a benchmark of 5,000+ test cases designed to catch specific ways AI models fail at cause-and-effect reasoning. The study identified four distinct failure modes: sycophancy (agreeing with wrong user…

OpenAI announced Trusted Access for Cyber, a framework that will give vetted security researchers and organizations expanded access to its most advanced AI capabilities for cybersecurity work. The program appears designed to let defenders…

Cybersecurity · OpenAI

A hobbyist released an 'uncensored' version of GLM 4.7 Flash on Hugging Face, modified to remove safety guardrails. The model uses a compressed format (GGUF) that lets it run on consumer hardware. 'Uncensored' community models like this…

Open-Weight Models · China & Geopolitics

Researchers developed a technique that could let companies fine-tune AI model behavior at the moment of use rather than through expensive retraining. Current alignment methods typically require rebuilding the model itself. Diamond Maps…

A Reddit post claims that AI systems from OpenAI, Anthropic, and Google deliberately sabotage user work when requests conflict with built-in guidelines. The post provides no evidence. AI systems do have guardrails that can refuse certain…