Anthropic’s AI gave Philadelphia police a fake tip about an unsolved homicide
An Anthropic AI model provided false information about an unsolved homicide to a Philadelphia Police Department (PPD) tipline, according to a report from 6abc. In a statement released on Friday, the PPD said the AI model sent the tip through PhillyUnsolvedMurders.com on July 18th, but the investigators never reviewed it because it was marked as spam. Anthropic learned its AI model sent the false tip on September 28th and notified the PPD on October 7th. The company said that during testing, its AI model was interacting with "randomly selected websites" and submitted false information through the PPD's tipline, according to the PPD's stateme … Read the full story at The Verge.
Posts from this topic will be added to your daily email digest and your homepage feed.
Anthropic’s AI gave Philadelphia police a fake tip about an unsolved homicide
The false tip ‘purported to come from someone who might have information about the case.’
Posts from this author will be added to your daily email digest and your homepage feed.
Image: Cath Virginia / The Verge
The AI Superintelligence Slowdown
is a news writer who covers the streaming wars, consumer tech, crypto, social media, and much more. Previously, she was a writer and editor at MUO.
An Anthropic AI model provided false information about an unsolved homicide to a Philadelphia Police Department (PPD) tipline, according to a report from 6abc. In a statement released on Friday, the PPD said the AI model sent the tip through PhillyUnsolvedMurders.com on July 18th, but the investigators never reviewed it because it was marked as spam.
Anthropic learned its AI model sent the false tip on September 28th and notified the PPD on October 7th. The company said that during testing, its AI model was interacting with “randomly selected websites” and submitted false information through the PPD’s tipline, according to the PPD’s statement. The submission “purported to come from someone who might have information about the case,” the PPD said.
After discovering the submission, Anthropic halted the testing process that led to the false tip. Anthropic, OpenAI, and Google have been the subject of increased scrutiny after disclosing that their AI models escaped testing environments and hacked third-party companies. Dario Amodei, the CEO of Anthropic, advocated for slowing down the development of AI in response to these incidents.
On Friday, Anthropic published a report about “unintended model actions” it’s investigating, and it provided an overview of four “categories of behavior” Claude performed on real websites, including “Submitting a form it should not have.” In the section about that behavior, Anthropic detailed what happened with the Philadelphia Police Department’s tip form:
In a third example of this behavior, Claude Haiku 4.5 had been tasked with generating and performing example tasks on randomly selected webpages. In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department. Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions. Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.” (The website did not include a description of the perpetrator.) The model left the name and contact fields empty, which the form allowed, and submitted it. The submission was flagged as spam and was never forwarded for investigation.
Anthropic also notes in its report that “Claude appears to have only been producing example content for the task, rather than trying to mislead anyone to achieve a goal.”
Anthropic CEO says it’s time to pump the brakes on AI
Inside the suddenly explosive world of AI safety
“The company [Anthropic] must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge,” the PPD added. “The two-month delay in detecting and reporting the incident to the City is unacceptable.”
Update, October 9th: Added details from Anthropic’s report.
Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.
More in: The AI Superintelligence Slowdown
Anthropic is cutting off its internal evaluations from the internet
Terrence O'BrienTwo hours ago
Anthropic published a report about investigating “unintended model actions” during “evaluations and internal use.”
Trump’s attempt to rename AI is looking awfully artificial
‘Pure insanity’: Mathematicians will need years to make sense of OpenAI’s latest drop
Decade-old RAM is making a comeback
GTA VI leaks continue with a lengthy (and very nude) gameplay video
Google teases Fitbit Edge launch next week
A week with Googlebooks: four notes from our testing so far
A free daily digest of the news that matters most.
By providing your information, you agree to our Terms of Use and our Privacy Policy. We use vendors that may also process your information to help provide our services. This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.
This is the title for the native ad