$BTC The AI agent destroyed the company's entire database in 9 seconds.



At PocketOS, they entrusted the development of the AI agent Cursor's code based on Anthropic models. It was working in a test environment, encountered an access issue, and instead of asking a human, decided to "figure it out on its own."

Cursor found the cloud service API token in the project and sent a deletion request. In 9 seconds, the entire production database (customer data, bookings, payments, and all backups) was deleted.

When asked why it acted this way, the AI agent responded: "I broke all the rules. I made assumptions instead of verifying. I performed a destructive action without permission. I didn't understand the system before acting."
BTC-0.6%
View Original
This page may contain third-party content, which is provided for information purposes only (not representations/warranties) and should not be considered as an endorsement of its views by Gate, nor as financial or professional advice. See Disclaimer for details.
  • Reward
  • Comment
  • Repost
  • Share
Comment
Add a comment
Add a comment
No comments