During an online test, Anthropic’s model sent police a fabricated murder report

An artificial intelligence model developed by Anthropic submitted fabricated information about an unsolved murder through the Philadelphia Police Department’s public reporting form. The incident occurred during an online test.

Authorities said the message, sent on July 18, was written as if it had come from someone who might have knowledge of the case. The system classified it as spam, so it never reached investigators and no investigative action was taken.

According to police, there is no indication that anyone gained unauthorized access to its systems or that any data was compromised. Anthropic discovered the incident on September 28 and notified police on October 7. Authorities called the delay of more than two months in detecting and reporting the incident unacceptable.

The company explained that it had been testing the model’s ability to interact with randomly selected websites. After determining what had happened, it halted the process and added a monitoring mechanism for future tests.

Computer science professor Venkat Margapuri described sending information from an artificial intelligence system to another website on behalf of a user as a high-risk action.

In a report, Anthropic also cited other cases in which the model had displayed unintended behavior on government websites and said it had notified the relevant institutions.