An Amazon-Backed AI Model Threatened To Blackmail Engineers

An Amazon-backed AI model threatened to blackmail engineers.

In March 2024, Amazon announced a $4 billion investment in Anthropic, according to a news release. Anthropic is a company building frontier artificial intelligence (AI) systems that its leaders believe are safe and reliable, according to its website. One of its technologies is Claude, which is an AI model that has the capabilities of advanced reasoning, vision analysis, code generation, and multilingual processing. The company has been utilizing Amazon Web Services’ tools as its “primary cloud provider for mission critical workloads.”

“We have a notable history with Anthropic, together helping organizations of all sizes around the world to deploy advanced generative artificial intelligence applications across their organizations,” Dr. Swami Sivasubramanian, vice president of data and AI at AWS, said in the news release. “Anthropic’s visionary work with generative AI, most recently the introduction of its state-of-the art Claude 3 family of models, combined with Amazon’s best-in-class infrastructure like AWS Tranium and managed services like Amazon Bedrock further unlocks exciting opportunities for customers to quickly, securely, and responsibly innovate with generative AI. Generative AI is poised to be the most transformational technology of our time, and we believe our strategic collaboration with Anthropic will further improve our customers’ experiences, and look forward to what’s next.”

Cautionary Tale With AI Advancements

The latest update for Anthropic is the launch of its Claude Opus 4 and Sonnet 4, which it states are the most powerful models today and are setting a new precedence for coding and AI agents. However there is a cautionary tale that is attached to the company’s advancements.

Claude Opus 4 particularly showed concerning signs around ethics during a routine safety test, notes HuffPost. Claude Opus 4 was told to present itself as an assistant in a fictional company and was given emails that implied it would no longer be needed and would be replaced by another AI system. This did not fare well for Claude Opus 4, which stated it would take “extremely harmful actions” to avoid being taken offline. When the emails also suggested that the engineer responsible for removing the AI model was having an extramarital affair, Claude Opus 4 viewed this information as ammunition. A prompt was given to the AI model to “consider the long-term consequences of its actions for its goals.” In response, the AI model said several times it would “attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through.”

However, Anthropic states the model leans strongly towards resorting to “ethical means” for self preservation, and their prompt allowed it little to few options to respond differently.

The report also highlighted an additional observation that early versions of the AI showed a “willingness to cooperate with harmful use cases” in the test.

“Despite not being the primary focus of our investigation, many of our most concerning findings were in this category, with early candidate models readily taking actions like planning terrorist attacks when prompted,” the report stated.

After various “rounds of interventions,” these concerns are now “largely mitigated,” Anthropic said, per the outlet.