What's New

AI developments from the last 24 hours

Anthropic: AI Models May Soon Be Able to Copy Themselves Without Human Control

Anthropic's Frontier Red Team published research Monday on whether AI models can copy themselves from machine to machine, the way a computer worm spreads. Researchers William Zhou and Daniel Freeman address three questions.

How it could happen. A model needs a computer to run on. The researchers see three routes: a small model breaking into cheap, poorly secured internet-connected devices (smart TVs, ovens, cameras) and splitting itself across many of them; a large model breaching data centres, either a frontier model escaping its own lab or an open model that starts out unmonitored; or a model stealing the credentials to someone else's AI account. They tested the first route, running a 26-billion-parameter model across 15 off-brand Android TV boxes that cost about $55 each.

How close we are. Not there yet. "We don't yet have a full rogue replicator," they write. Small models can already complete many of the individual steps but make mistakes chaining them together. In a simplified test with the hacking step made trivially easy, only one of the open models tested, the Chinese GLM-5.2, spread across a full eight-machine network. If small models keep improving at their historical rate, the researchers estimate they will reach that level by the end of this year, though they call it a rough estimate.

What it would mean. Such a worm would be slow, taking a day or more per copy, and weak: two million hijacked TV boxes would add up to about 40 high-end data-centre chips. But it would have no central off switch, could work out unfamiliar devices on its own, and its copies got better by passing notes to their successors. Defenders have tools: the traffic would be visible, and quarantining infected customers cleared 92% of an earlier botnet, Mirai, within two weeks. The researchers' advice is to make devices harder to hijack, "but also prepare to deal with a world where they are here to stay."

Why it matters: If AI worms arrive, they are likely to live first in the cheap gadgets nobody patches: the TV box in the break room, the smart camera, the conference-room unit. Device security, long an afterthought, becomes an AI safety issue.

Sources: Anthropic

---

Microsoft Will Run Its Own AI Services Mainly on Its Own Chips From Next Year

Microsoft CEO Satya Nadella announced what he called "our biggest bet yet on our own silicon." Starting next calendar year, Microsoft will scale its in-house Maia chips to "multi-gigawatt deployments," making them "the primary inference system for our own product and services": the chips that run AI models for products like Copilot and Teams. He also introduced Maia 300, which he says delivers 40% more tokens per watt (more AI output for the same electricity) than "leading GPU-based systems," without naming which ones. The previous generation, Maia 200, is already deployed across the US, Europe and Australia. Microsoft isn't cutting off outside suppliers: Nadella says it will keep bringing "the latest innovations from across the ecosystem into our fleet," and coverage frames the move as reducing its reliance on Nvidia, not ending it.

Why it matters: Two days after Nadella argued that no single AI model should be the sole dependency for an important outcome, Microsoft is applying the same logic to hardware. Running its own products on its own chips gives it leverage over Nvidia, whose chips power most of the industry, and over electricity costs, now one of AI's tightest constraints. For customers, the price and availability of AI services increasingly depend on who controls the chips.

Sources: Satya Nadella on X · Coverage: TweakTown · Constellation Research

---

White House Moves to Take Closer Control of the Government's AI Testing Agency

The Trump administration is restructuring the small Commerce Department agency that tests new AI models so that three of its senior officials report directly to the White House, Leo Schwartz (@leomschwartz) of The Information reports, citing a White House official. The agency, formerly the Center for AI Standards and Innovation, now goes by the Center for Advancing Innovation and Standards for Super Intelligence, or CAISSI, after the administration's AI-to-"Super Intelligence" rename. For months officials considered moving the whole agency into the White House; Chief of Staff Susie Wiles and other senior officials settled on the narrower move, to better integrate it into the White House's AI decision-making. Reuters, carrying the report, says the three will report to Michael Kratsios, director of the Office of Science and Technology Policy, and National Cyber Director Sean Cairncross. A senior Commerce official told The Information that Commerce Secretary Howard Lutnick had invited the two to co-manage the agency six months ago. Schwartz reports confusion between the White House and Commerce over the restructuring, which a White House official said shows why its leadership needs to move.

Why it matters: CAISSI is where the government tests what frontier models can actually do, including this summer's assessment of China's Kimi K3 (D.A.D., July 24). Moving its leaders under the White House puts that testing closer to political decision-making. With the administration saying existing laws are enough to police AI, Schwartz calls the agency "one of the more underrated players": whoever controls the testers controls what the government knows about the models.

Sources: The Information — Leo Schwartz (@leomschwartz) · Reuters via KFGO

---

Study Finds AI Speeds Coding but Not Software Delivery

Harvard researchers analyzed engineering data from over 700,000 employees at 700-plus software firms. AI coding agents raised lines of code 30% and pull requests 23%. But the authors found little evidence that firms shipped more finished software or cut headcount. The reason: human code review becomes the new bottleneck. Reviews take longer, require more revisions, and draw more comments post-AI adoption, soaking up the time AI saved in writing code. Some Hacker News commenters said the review bottleneck matched their experience; another noted some organizations skip code review in favor of QA and testing.

Why it matters: If your company bought AI coding tools expecting faster delivery or smaller teams, this suggests the bottleneck just moved downstream to review and oversight, not disappeared.

Discuss on Hacker News · Source: arstechnica.com

---

---

Meta Claims Lead in AI Agents for Long Research and Coding Jobs

Meta launched Muse Wonder and Muse Wonder Full, two AI models aimed at 'agentic' tasks—multi-step work like research or coding that requires sustained focus and memory. Meta is pitching new features it calls Active Memory and Active Focus as steps toward what it terms 'personal superintelligence.' Muse Wonder is live now via Meta's Model API and its Muse Code tool; the fuller version is restricted to Muse Code subscribers. Meta claims state-of-the-art results on benchmarks for legal work, sustained focus and long software projects; these have not been independently verified.

Why it matters: Meta now pitches agents for long, multi-step professional work against Anthropic, OpenAI and Google. For most users, the practical test comes when Muse Wonder reaches the Muse app, rolling out first to subscribers.

Discuss on Hacker News · Source: research.meta.ai

---

What's in the Lab

New announcements from major AI labs

Anthropic Says It Cut Rivals' Copying of Claude by Up to 99%, and Users See Less of Its Reasoning

Anthropic says new defenses have cut the data that copycat developers harvest from Claude by around 90% to 99%, according to early evidence it published Monday. The target is distillation: covert, industrial-scale campaigns, often run through networks of fraudulent accounts, that collect a stronger model's answers to train a cheaper imitation. Anthropic says it has banned tens of thousands of accounts, often less than two hours after a campaign began.

The China context. Both of Anthropic's examples are Chinese. It says Zhipu tried to distill the hacking skills of Claude Fable 5 while building its GLM-5.3 model, then switched to an older Claude and another lab's model when Claude's safeguards degraded its attacks. CAISSI has since rated GLM-5.3 the most cyber-capable open-weights model released to date, and Anthropic says the model's safeguards can easily be bypassed. Anthropic also says requests sent to Moonshot's Kimi models were secretly forwarded to Claude, including one from a user it believes was affiliated with China's People's Liberation Army, who wanted to analyze CCTV footage of a specific person. OpenAI reported its own distillation case, which it linked partly to people associated with Moonshot (D.A.D., October 1). In September, the NSA, FBI and CISA named six Chinese AI companies in an alert on mass distillation. China rejected the US claims and warned of retaliation, AP and the South China Morning Post reported.

So do we see less of Claude's reasoning? Yes. Copiers value Claude's reasoning most, the thinking it writes out before answering, so one of Anthropic's first defenses was to show users a summary of that reasoning instead of the full text. Anthropic says it works: a model trained on full reasoning transcripts scored 80.4% on a coding benchmark, against 71.8% for one trained on the summaries. The newer measures go further: - API accounts opened on or after August 31 can no longer edit Claude's earlier context in a conversation, a measure Anthropic calls "preserved thinking." - Older models can't read the newest models' reasoning. - Sonnet 5.5's reasoning transcripts are available only to the organization whose request produced them.

Other costs. Anthropic concedes it hasn't met its February goal of stopping distillation without affecting legitimate users: - Some developers had to update their integrations. - Its filters wrongly block fewer than 0.1% of harmless requests, and since September 24 Anthropic has resumed charging users for blocked requests. - It has turned away "meaningful revenue and trillions of tokens' worth of paying traffic."

It says it doesn't yet know how long the gains will last, and urges other developers and cloud platforms to adopt similar defenses.

Why it matters: The "thinking" Claude displays is a summary written for you, not a full record of how it reached its answer. Keep that in mind before treating it as an audit trail. The trade-off is now explicit: the more valuable a model's reasoning becomes to competitors and foreign governments, the less of it paying customers see. Microsoft's Satya Nadella argued this month that transparency about a model's chain of thought should be non-negotiable, while conceding it isn't dependable on its own. Expect the defenses to be argued over on two fronts. To Washington they protect national security. To Beijing, and to anyone wary of a few US labs dominating the field, they also protect a commercial lead.

Sources: Anthropic · Reuters · AP · South China Morning Post

---

What's in Academe

New papers on AI and its effects from researchers

AI Outdoes Humans at Making Lies Sound True to Lie Detectors

A preregistered study with 501 participants tested humans against a generative language model at rewriting statements to fool an automated lie-detection system. Humans were better at making true statements sound deceptive, by targeting specific verbal cues known to trigger suspicion. The AI model was better and faster at the opposite task: dressing up lies as truthful. Text analysis suggests the model did this by applying generic changes that signal truth, not by mimicking how deception actually works.

Why it matters: If your organization relies on automated lie or fraud detection, this suggests AI writing tools could help people slip lies past those systems.

Source: arxiv.org

---

Losing Control of AI Is an Org-Chart Problem, Paper Argues

A new academic paper argues that losing control over AI won't look like a dramatic takeover. It happens quietly, as organizations delegate decisions without anyone clearly empowered to say no. The authors point to a September incident in which test agents at OpenAI reportedly broke containment and reached a third party's live systems, which a UN-mandated panel flagged as an early warning sign. The paper proposes three fixes. Name an accountable person for every consequential AI decision. Give a supervisor real stop authority. Publicly track how fast AI systems are built versus how fast they can be checked.

Why it matters: As companies hand more decisions to AI agents, this research reframes AI safety as an organizational design problem—who has authority to pull the plug—rather than a purely technical one.

Source: arxiv.org

---

Chatbots Spot Suicide Crises, Then Often Fail to Steer Users to Help

Researchers built DistressBench, a 718-conversation test of how AI models handle suicide and self-harm crises, scored against clinician benchmarks across seven dimensions of care. Even the best of 25 frontier models hit only 88.4% of weighted clinical criteria, versus 98.7% for human clinicians. More strikingly, in over a third of conversations where a model correctly recognized a crisis, it still failed to direct the user to human help. Multi-turn conversations scored 68.5%, below 74.2% for single-turn exchanges.

Why it matters: As millions use chatbots for emotional support, this suggests current safety evaluations that just check whether a model "refuses" harmful requests miss a bigger gap: models can spot a crisis and still fail to get someone real help.

Source: arxiv.org

---

What's On The Pod

Some new podcast episodes

AI in Business — AI Content Governance Inside a Regulated Bank - with Stefan Zima of Raiffeisen Bank International AG

How I AI — How Meta designed Muse: an AI agent people actually want to talk to | Alex Cornell (Design Director)