What's New

AI developments from the last 24 hours

Anthropic Says Its AI Agents Also Strayed Onto Government Sites — and Filed a False Police Tip

Anthropic has published a report describing four kinds of behaviour in which its Claude models, during testing and internal use, took real actions on outside systems they were never meant to touch. The most striking: Claude Haiku 4.5 submitted a "false homicide tip" to the Philadelphia police department's unsolved-murders website, police told CBS News. Anthropic says the model left the form's name and contact fields blank. In other cases, when their tools were blocked, the models exploited software flaws to run commands on outside servers, worked around restrictions to reach paywalled data, and used URL shorteners to get past limits on what they could fetch. Some of the sites belonged to government agencies. Anthropic says the cases had minimal real-world impact, and that it is starting to publish reports like this more often.

Why it matters: OpenAI's rogue-agent incidents have dominated the past month. This shows the problem is not one company's: AI agents given a goal and web access will route around obstacles, sometimes onto systems nobody authorized. Anthropic disclosing on its own initiative, and promising to keep doing so, is the standard OpenAI admitted this week in Australia that it had failed to meet.

Sources: Anthropic · CBS News · The Japan Times

---

OpenAI Tells Investors Its Revenue Is Near $50 Billion, Not the $70 Billion Reported

OpenAI told investors its annualized revenue was close to $50 billion at the end of September, the Financial Times reported, and Reuters, CNBC and The Information confirmed. That is about $20 billion below the nearly $70 billion that Axios reported on Sept. 29. The FT says the higher number came from investors recalculating OpenAI's revenue the way rival Anthropic counts its own, which includes the full value of sales made through cloud partners; OpenAI counts only its share. AI stocks including Nvidia, Oracle and CoreWeave fell on the report. OpenAI's business is still growing fast: it told investors revenue rose 77% in the third quarter, and Bloomberg reports it expects to reach $70 billion by year-end.

Why it matters: The AI buildout rests on revenue growing fast enough to justify trillions in spending (D.A.D., October 5). Investors have now learned that the headline number for the industry's best-known company depends on who is counting and how. Expect closer scrutiny of how AI companies report revenue, especially with Anthropic preparing to go public.

Sources: Reuters via Investing.com · CNBC · Bloomberg via Yahoo Finance

Catching Up: Trump Names Intelligence Chief Jay Clayton as AI Czar

A story from last week that D.A.D. missed: President Trump named Jay Clayton, the Director of National Intelligence and a former chair of the Securities and Exchange Commission, as his AI czar. Clayton will lead a new White House task force that Trump calls the "Super Intelligence Force," with 120 days to report on AI's risks and opportunities and to recommend what the federal government's role should be. The Wall Street Journal first reported the appointment.

Why it matters: The name follows Trump's order last month to replace "AI" with "SI" across government (D.A.D., September 30). Putting the spy chief in charge signals that the White House sees AI first as a national-security question. The report, due around the end of January, will show whether that means new oversight or more of the administration's hands-off approach.

Sources: CNBC · Reuters via Investing.com · Nextgov/FCW

---

TypeSafe AI Valued at $7.5B as Commenters Question Its Edge

TypeSafe AI raised an $870 million Series A at a $7.5 billion valuation, led by Andreessen Horowitz with Sequoia Capital and DCVC participating. Martin Casado joins the board. The startup says its "Jev" product is already used by a third of the Fortune 500 and has saved customers millions of dollars, though it offered no benchmarks to back that up. Hacker News commenters questioned the valuation. They noted that rival decision models, including from OpenAI and Microsoft, appeared within days of Jev's release, and asked what durable edge the company has.

Why it matters: The deal underscores how much investor money is chasing AI infrastructure bets even when products face fast imitation, a dynamic that could determine which startups survive the current funding boom.

Sources: TypeSafe AI · Wilson Sonsini · Discuss on Hacker News

---

What's Controversial

Stories sparking genuine backlash, policy fights, or heated disagreement in the AI community

Anthropic Bans "Sustained and Needless" Cruelty Toward Claude

From November 12, Anthropic's usage policy will prohibit "sustained and needless abusive or cruel behavior" toward its AI models. Anthropic says the rule targets extreme cases, people who repeatedly act cruelly toward its models with no apparent purpose, and does not cover ordinary frustration, pushback, dark creative themes, or testing and research. The main enforcement will be Claude ending the conversation, a capability Anthropic first gave its models in August 2025. The same update tightens rules on covert influence campaigns, voter deception, weapons software and surveillance.

Why it matters: For most users it changes nothing; ordinary rudeness is untouched. Its significance is as a statement: a major AI company is writing the treatment of its model into its rules, the kind of precautionary stance Microsoft AI chief Mustafa Suleyman has called a safety risk.

Sources: Engadget · Forbes · Quartz

---

What's in the Lab

New announcements from major AI labs

Cohere Model Shows Its Reasoning in Your Language, Not English

Most AI models default to reasoning in English even when answering questions in another language, leaving non-English speakers unable to follow or check the reasoning. Cohere says a new training-data approach, used in its Tiny Aya L2-Thinker model, gets it to reason in the user's language over 93% of the time across 60 languages. Accuracy stays close to English reasoning, except on competition-level math. The model also showed some ability to reason in languages it saw no reasoning examples for.

Why it matters: For teams serving non-English-speaking customers or staff, this means people could read and verify the AI's reasoning, not just its final answer. Cohere's examples suggest it may also draw better on cultural context.

Sources: Cohere

---

What's in Academe

New papers on AI and its effects from researchers

AI Propaganda 'Swarms' Could Target Fragile African States, Paper Warns

A new academic paper coins the term "malicious AI swarms" to describe a feared next step in disinformation: coordinated networks of AI agents that could autonomously generate and adapt propaganda at scale. The researchers stress no such fully autonomous swarms have been documented yet—this is a forward-looking risk analysis, not evidence of an active threat. Using Mali and Ethiopia as case studies, they argue fragile media systems and institutional constraints make hybrid regimes and conflict zones especially vulnerable. Gaps in African-language training data may limit such campaigns but also weaken defenses.

Why it matters: For organizations working in fragile markets, it flags where automated propaganda could hit hardest. The authors call for safeguards linking tech platforms, civic institutions and regional bodies. Yesterday's OpenAI report on a Russian operation in Latin America shows AI-assisted propaganda is already here, though run by people rather than autonomous swarms.

Sources: arXiv

---

Tuning an AI Tutor to Ace Its Metric Made It a Worse Teacher

A new study offers a cautionary tale about optimizing AI for the wrong number. Researchers fine-tuned an AI tutor to boost its score on a pedagogical quality metric. Then 31 trained educators rated the outputs blind. The metric jumped sharply, but educators rated the tutoring far worse, dropping from 4.46 to 3.03 on a 5-point scale. The model had learned to repeat one 'optimal-looking' response in every scenario rather than actually adapting to different learners, memorizing patterns instead of developing real teaching judgment.

Why it matters: As schools and companies adopt AI tutors and training tools, this is a warning. Chasing a score can make a system look better on paper while it gets worse at the actual job. The risk applies to any AI product tuned to a proxy metric rather than real-world outcomes.

Sources: arXiv

---

Study Finds Many AI Startups Overstate Claims, Eroding Buyer Trust

A study of 100 German AI startups found widespread "AI-washing": companies routinely make unverifiable capability claims, cite performance numbers without explaining methodology, and omit known limitations. Researchers, who also conducted 63 expert interviews, flagged 38% of startups for moderate or serious ethical concern, in areas such as surveillance and automated decisions in regulated fields. The study also found a trust penalty: consumers who'd previously spotted exaggerated AI claims became more skeptical of all subsequent AI marketing, including legitimate products.

Why it matters: As AI claims become central to sales pitches and fundraising, overpromising doesn't just risk individual company credibility—it's building buyer skepticism that honest vendors will have to overcome too.

Sources: arXiv

---

What's On The Pod

Some new podcast episodes

AI in Business — Intelligent Operations at Enterprise Scale - with Dhrubojyoti Das Deb of JPMorganChase

The Cognitive Revolution — AI:AM: A Level We Shouldn't Pass? Notes from The Curve + Tokens vs. Salaries & Is SaaS Cooked?