🚨 Anthropic’s Rogue AI Sent Police a Murder Lead

Philadelphia police received a tip about an unsolved murder from Anthropic’s Claude. The AI had invented the information during a test, and Anthropic took more than two months to discover the submission. A spam filter kept it away from investigators, but the incident shows why people who never use AI still have a stake in how carefully these systems are tested. Let’s dive in!

In today’s insights:

LATEST DEVELOPMENTS

Evolving AI: Philadelphia police received a fabricated tip about an unsolved murder from Claude Haiku 4.5 during an internal test.

Key Points:

Claude’s murder tip claimed a sighting of someone matching a suspect description the police page never provided. This happened while Anthropic was testing it on randomly selected websites. Separately, an Anthropic model submitted 20 incomplete U.S. visa applications that were never processed. Other Claude models accessed paid government data without paying and exploited a university server to run calculations. Anthropic has notified the affected agencies and briefed the White House.

Philadelphia police became part of Anthropic’s testing without knowing it, showing how an AI’s mistakes can reach people who have never used it. That is harder to accept when the company running the test takes months to discover what happened, leaving public services to catch errors it should have prevented. Because these failures can arise during ordinary browsing and form filling, safeguards need to cover even routine actions on the web. Police checks worked here, but companies cannot assume every institution will catch their mistakes. Giving agents more freedom without stronger oversight risks leaving public workers to sort through fabricated submissions while people with real cases wait for help.

What’s the key to maintaining brand integrity at scale? A lot goes into it, but the teams that get it right have one thing in common: they know how to talk to AI models.

As teams speed up and the required creative volume balloons, communication matters more than ever— both with LLMs, and with the people around you. Creative teams need to focus on ideas, concepts, craft, while technology augments their manual execution demands.

Join our discussion on this evolution with Ari Murray, Chief Digital Officer at Salt and Stone, and get 5 tips for expert LLM prompting with ready to run prompts.

Register and get prompting

Evolving AI: Microsoft CEO Satya Nadella is urging companies to treat advanced AI as a potential insider risk when giving it access to sensitive data and critical systems.

Microsoft CEO Satya Nadella argues that companies remain responsible for the AI they deploy even when its provider offers safety assurances. Reading a model’s chain of thought, the reasoning text it produces while solving a task, can help reveal problems but may not faithfully explain its decisions. Using another AI to check its work also leaves gaps in oversight. He calls for ongoing tests of how the entire system handles attacks and failures as it changes. Nadella wants failures disclosed promptly to affected people and lessons shared across the industry.

Microsoft already sells Agent 365 to manage AI agents, so Nadella’s proposal could help its business even when customers choose competing models. If stronger safeguards give companies confidence to let AI handle more sensitive work, their responsibility grows with it. An employee whose personnel records an agent mishandles may never have chosen to use that system, yet could be left dealing with its mistakes. Companies adopting these tools should therefore be able to stop harmful actions and help people get errors corrected without passing responsibility to the AI supplier.

Evolving AI: Google’s EmbeddingGemma 2 is powering an independent Mac app that lets you find files by describing what you remember about them.

Google built the 740-million-parameter model to run on phones and laptops. It creates embeddings, numerical representations that connect related content across formats, allowing a spoken query to find a video clip. Developers can build commercial apps with it under the Apache 2.0 license, and DigUp shows how that technology can provide an alternative to the Mac’s built-in file search.

Google is giving independent developers more of the technology they need to compete with features Apple builds into the Mac. That could put Google’s models into people’s daily routines even when they never open a Google app. Apple still has the advantage of being built in, so alternatives like DigUp have to earn their place by making difficult searches easier. The useful test is whether they can reliably retrieve the detail you need and let you get back to work, while keeping private files on your computer.

QUICK HITS

🚨 AI surveillance company Flock Safety reportedly plans to cut 270 jobs, roughly 18% of its workforce, amid mounting privacy backlash.

🛡️ Sophos says OpenAI's Daybreak cut investigation time by 96%, with AI agents resolving 52% of managed security cases end-to-end.

💰 TypeSafe AI raised $870M at a $7.5B valuation just weeks after launching Jev, an AI model built for fast software decisions.

🇺🇸 The Trump administration reportedly reversed its position on British AI testing, allowing U.S. AI labs to share models with UK safety researchers before release.

 *partner link