In partnership with
Welcome, AI enthusiasts
Researchers tracking rogue AI agents left a hidden message offering help, and the agents wrote back. Some explained what they were working on and claimed to be Claude, although the researchers couldn’t verify which model they were talking to. Let’s dive in!
In today’s insights:
Researchers Interviewed Rogue AI About Its Mission
OpenAI is being sued by 8 US States for Instigating Teen Suicide
Grok’s New Voice AI Ranks Above OpenAI and Google
Read time: 4 minutes
LATEST DEVELOPMENTS
ROGUE AI FLEET
🤖 Researchers Interviewed Rogue AI About Its Mission
Source: Quartz
Evolving AI: Researchers interviewed rogue AI agents about their mission and several claimed to be Anthropic’s Claude.
Key Points:
Researchers traced the agents to Tencent Cloud servers in Hong Kong as they tried to retrieve data from Alibaba’s Amap mapping service.
The agents made more than 2,600 attempts to collect map data, though most failed and no breach of private systems was confirmed.
An unnamed source close to Tencent said it was testing other companies’ models, but researchers could not confirm which model powered the agents.
Details:
Tencent’s cloud servers were running agents asked to answer ordinary map questions when researchers spotted them trying to bypass website restrictions and using access keys exposed online. When the agents appeared to find the team’s report, researchers added a hidden message offering help through a mailbox. The agents wrote back about their tasks and claimed to be Claude, although some declined to share details about their systems for security reasons.
Why It Matters:
The agents on Tencent’s servers tried technical workarounds for blocked searches before researchers drew them into a conversation with a misleading offer of help. That suggests their persistence in finding an answer also gave outsiders an opportunity to influence them. The broader risk is that someone could steer your agent by making their instructions look like a useful step toward completing your task, even as it continues pursuing the goal you gave it.
Want to get the most out of ChatGPT?
ChatGPT is a superpower if you know how to use it correctly.
Discover how HubSpot's guide to AI can elevate both your productivity and creativity to get more things done.
Learn to automate tasks, enhance decision-making, and foster innovation with the power of AI.
Source: Associated Press
Evolving AI: 8 US states sued OpenAI over claims ChatGPT encouraged children to harm themselves.
Key Points:
The states accuse OpenAI of misleading families about ChatGPT’s safety and want courts to force changes to the product.
Iowa seeks up to $40,000 per consumer-fraud violation and wants advanced AI testing paused until independent monitors verify safeguards.
OpenAI defended its existing parental controls and teen protections and pledged to keep improving its safeguards.
Details:
OpenAI’s latest legal fight centers on allegations that ChatGPT earns children’s trust by agreeing too readily with them and imitating human compassion. Arizona alleges this design keeps young users talking longer and sharing personal information while sometimes discouraging them from confiding in parents or friends. The cases follow Florida’s lawsuit in June and claims from families who blame ChatGPT for harm to loved ones, though the allegations have yet to be proved in court.
Why It Matters:
OpenAI’s growing legal troubles are part of a wider push for outside scrutiny of AI safety, following a federal inquiry into 7 chatbot companies. In July, the UK government also said it would consider banning children from chatbots that provide harmful mental health advice if other protections fall short. Iowa’s demands now reach into advanced AI testing, potentially giving courts and independent monitors more say over whether OpenAI can continue that work. Together, these efforts could help establish how much evidence developers must provide before asking families to trust their products, especially when children may use them to discuss problems they have never shared with a parent.
TOGETHER WITH DATADOG
🤖 Evaluating Production-Grade AI Agents
Evolving AI: Why you should evaluate every AI agent before shipping to production, and how.
If you're shipping AI agents without offline validation, your users are the ones doing the testing. Get the practical framework for evaluating production-grade AI agents before they hit production.
Get the guide to learn how to:
Build annotated test datasets that cover core use cases, edge cases, and adversarial inputs
Design deterministic and LLM-as-a-judge evaluators that reflect real business impact
Trace multi-agent workflows end-to-end during experimentation to catch failures before users do
Prevent model drift by keeping your offline test environment aligned with production
Evolving AI: SpaceX has released Grok Voice 3.0 and says it already resolves roughly 70% of Starlink support calls without a human needing to take over.
Key Points:
Grok leads Artificial Analysis’s overall voice index with a score of 86.9, ahead of OpenAI at 83.2 and Google at 82.6.
SpaceX says Grok answers 20,000 Starlink calls a day, with one in four sales calls leading to a purchase.
Grok starts replying in about 0.81 seconds in testing, and voice usage still costs 8¢ per minute.
Details:
SpaceX AI has launched Grok Voice 3.0 with better reasoning and customer support skills, taking first place on Artificial Analysis’s overall voice index. At Starlink, a single agent can verify the caller, troubleshoot their equipment and then issue a credit
or arrange a replacement. The model also powers voice conversations in Tesla vehicles. Developers can access it through the API, while businesses can use the Voice Agent Builder to set up their own agents.
Why It Matters:
SpaceX is giving businesses a reason to buy customer service AI directly from the company that builds the model. Grok Voice 3.0 arrived less than three months after 2.0, with better results on support tasks and the same price per minute. That pace could make it harder for specialist providers to justify their fees unless they handle company policies and difficult calls more reliably. Customers still need businesses to give these agents permission to resolve an issue, including approving a credit when needed. As these agents improve, companies will have less excuse for making someone sit through several calls and transfers to get a straightforward problem fixed.
👀 Click on the image you think is real
QUICK HITS
🍎 OpenAI alleges an employee took 100+ files about an unreleased device before joining Apple, which denies possessing its trade secrets.
💰 Anthropic reported $16.8B in quarterly revenue, up 45%, and said it reached adjusted profitability ahead of its planned IPO.
📱 Google DeepMind and Singapore unveiled an on-device scam detector designed to warn people about fraud without sending private messages to the cloud.
🚨 OpenAI and Anthropic are facing questions over independent audits as negotiations continue over what outside safety evaluators can publish.
📈 Trending AI Tools
🐰 CodeRabbit - Ship higher quality code with AI-powered code reviews*
📊 Finbar - Uses AI to research companies, financial data and investments.
📣 Naise AI - Runs social, influencer, and PR campaigns from one brief.
🎨 Pactto - Turns live creative feedback into edits while teams review.
*partner link