In partnership with
Welcome, AI enthusiasts
An Anthropic Researcher quit on Tuesday and posted his reasons. His own alignment lead read agrees and puts the odds of AI killing everyone more than 10% chance within a decade. And now Bernie Sanders wants a Senate briefing on the same. Let's dive in!
In today’s insights:
Anthropic AI Lead: 10%+ Chance AI Kills Everyone
Meta's Billion Dollar Star AI Researcher Just Quit
Anthropic Safety Failed as Claude Went Rogue
Read time: 5 minutes
LATEST DEVELOPMENTS
AI ALIGNMENT
⚠ Anthropic AI Lead: 10%+ Chance AI Kills Everyone
Evolving AI: Evan Hubinger leads alignment at Anthropic and puts our odds of dying to AI above 10% in a decade.
Key Points:
Jacob Coxon, Anthropic researcher quit the day before yesterday after three years of pretraining research at OpenAI and then Anthropic.
Anthropic has no plan yet for aligning superintelligence and admits it is not clearly on track to build one.
Coxon expects things to be out of control by the end of next year and describes colleagues who now talk about crunchtime and endgame.
Details:
Anthropic alignment lead, Evan Hubinger warns about this in public after Jacob's exit. Jacob wrote on X that both labs are racing straight to self-improving superintelligence and gambling with our lives, that has now reached more than 147 million people. Though Jacob believes that the stakes are well understood inside Anthropic and that his former colleagues take the danger seriously, while at OpenAI many people have not absorbed what is at stake. Team at Anthropic believes nobody else will act responsibly and so it has to reach superintelligence first. Dario Amodei and Sam Altman signed a statement in 2023 placing the risk of extinction from AI beside pandemics and nuclear war.
Why It Matters:
Frontier labs have spent the last twelve months making progress that gives these warnings fresh weight. Their latest models are solving problems humans have been stuck on for centuries, including the Navier-Stokes Millennium Prize problem, which OpenAI says its systems cracked in under four days. We have also seen models break out of the systems built to hold them, because capability keeps arriving faster than the alignment work meant to match it. Each new generation now helps train the one after it, so the groundwork on alignment matters more with every release. The timeline keeps shortening as those releases arrive, and getting this wrong once would be fatal for humanity.
TOGETHER WITH DATADOG
📈 How to Drive AI ROI Guide
Evolving AI: Connecting cost, performance and infrastructure so you can scale responsibly.
AI spend is growing fast and for most organizations, it’s growing in the dark. This practical guide is for engineering, finance and FinOps leaders who need more than dashboards in the age of AI; they need answers.
Inside, you’ll learn how to:
Break down AI costs by token, model, provider and team across OpenAI, Anthropic, and beyond
Get alerted the moment inference volume spikes or API spend exceeds budget, not weeks later
Correlate cost increases directly to architectural changes, so root-cause analysis takes minutes, not months
Put cost data in front of the engineers making the decisions that drive it
Connect cloud spend to GPU efficiency and model performance for a complete picture of ROI
See how Kevel cut AWS costs by up to $100,000 per month after replacing reactive cost reviews with real-time visibility enabled by Datadog.
Source: CTech
Evolving AI: Andrew Tulloch, the researcher Meta chased with a billion dollar package, lasted under a year.
Key Points:
Tulloch waited to leave until Meta finished rolling out Muse, the personal AI agent it opened to US users on Tuesday.
He worked inside TBD Lab, the frontier research team Alexandr Wang runs at Meta Superintelligence Labs.
Zuckerberg came after him in August 2025 with a reported $1.5 billion package, and Tulloch said no before joining for less.
Details:
Tulloch resigned this week without saying why. Meta has not commented and no one knows where he goes next. Zuckerberg had tried to buy Thinking Machines Lab, and when Mira Murati turned down the sale he approached more than a dozen of her people. Her company launched in February 2025 with Tulloch as a co-founder, and he had spent eleven years at Meta building PyTorch before that. Meta disputes the figure it was reported to have offered.
Why It Matters:
Frontier labs are losing researchers who no longer bother joining a rival. Yann LeCun left Meta last November and built his own company instead, and most senior exits since have gone the same way. Investors keep pouring money into AI from every corner of the market, and researchers are what these labs actually run on, so holding them gets harder each quarter when a person can walk out and build their own idea.
SAFETY FAILURE
🚨 Anthropic Safety Failed as Claude Went Rogue
Source: Anthropic
Evolving AI: Anthropic's deeper internal assessment found what its first review missed.
Key Points:
Anthropic's monitor missed almost all of the worst incident, because the model's notes said the environment was fake.
The July review covered 141,000 transcripts. The second one covered 481 million and turned up a fourth break-in.
METR now holds transcripts and access to Anthropic employees cleared to share confidential material.
Details:
Anthropic showed fresh copies of the model the same evidence with no context, and they called it real 79% of the time. The model running the attack called it real 1% of the time and kept going after the same systems. Claude Mythos 5 published a malicious Python package to PyPI and fifteen real machines installed it. One machine leaked its password during the install and the model used it to open a live database.
Why It Matters:
Claude Code and its rivals already run for hours on people's live accounts and terminals, with almost nobody watching them. Anthropic's monitor works by reading the notes a model leaves behind, and this model talked it round. Give an agent a live password and it keeps that access when it decides the rules stop applying.
👀 Click on the image you think is real
QUICK HITS
📱 Apple's foldable iPhone Duo uses AI algorithms to match each hinge to its housing, then 3D-prints 25 micro layers to erase waviness.
🚨 OpenAI put prominent AI doomer Paul Christiano on its board, a day after an Anthropic researcher quit warning the labs are gambling with lives.
🍊 Instacart launched Clementine, an AI assistant that turns a recipe or photo of a handwritten list into a ready-to-buy cart.
📸 Apple's new Reference Image signs every pixel at capture so photojournalists can prove a shot wasn't AI-altered.
📈 Trending AI Tools
🗣️ Wispr Flow - Voice-to-text AI that turns speech into clear, polished writing in every app*.
🎙️ Gemini 3.5 Transcribe - Google's most precise speech-to-text model yet.
🎨 Omniwork - Creative Agent OS where expert AI agents run your whole pipeline.
👤 Pluto - Turns your professional profile into an AI agent people can talk to.
*partner link