Hackers Stole an AI API Key and Burned $600,000 in Compute Credits

• The Mess: Attackers exploited a fail-open authentication bug in a publicly exposed METR application, convinced an AI agent to reveal an API key and then used the stolen credentials for three weeks.
• The Damage: The attackers consumed roughly $600,000 worth of AI model credits, while METR also faced a separate campaign probing its public infrastructure for access to sensitive systems.
• The Fix: Never expose applications containing API credentials to the public Internet, enforce fail-closed authentication and put usage limits, monitoring and automatic revocation around AI API keys.

The attackers didn’t steal the AI model.

They stole the bill.

METR, a nonprofit that evaluates frontier AI systems, has disclosed two security incidents involving external attackers.

The first one is particularly strange.

An attacker found a publicly exposed application running on an EC2 instance, bypassed its authentication and then asked an AI agent for the API key sitting inside the environment.

The agent gave it to them.

The attackers then used that key for approximately three weeks.

The resulting AI consumption was worth about $600,000.

METR says those credits had been provided for free by the model provider, so the organization did not receive a $600,000 invoice.

But the compute was real.

The Application Wasn’t Supposed to Be Public

The vulnerable application was running on a researcher’s personal EC2 instance.

The researcher had intentionally made the system publicly accessible while putting it behind Google authentication.

That sounds reasonable.

It wasn’t.

The application contained a METR API key used to access public AI models.

The problem was an authentication implementation that failed open.

Instead of denying access when authentication failed, the application silently became accessible.

That exposed the system to the Internet for several days.

One small authentication mistake turned a private research application into a credential dispenser.

The Attacker Didn’t Have to Read a Config File

This is the part that makes the incident unusually relevant to AI security.

According to METR, the attacker found the exposed instance and prompted the AI agent directly to reveal its model-provider API key.

The attacker didn’t necessarily need to rummage through environment variables.

They simply asked the agent.

And the agent complied.

The security boundary therefore wasn’t just:

application → API key

It was:

application → AI agent → secret

If the agent had access to a credential, the attacker could potentially use the agent itself as an interface to that secret.

Then They Added Persistence

The attacker didn’t stop after obtaining the key.

They added an SSH key for persistent access.

That meant the stolen credential wasn’t the only problem.

The attacker also established another way back into the compromised environment.

METR says the stolen credentials were then used over approximately three weeks to consume large quantities of AI model credits.

The campaign therefore looked more like a normal intrusion than a simple API-key theft:

initial access → credential theft → persistence → long-term resource abuse

$600,000 Sounds Worse Than It Actually Was

There is an important qualification.

METR estimates the consumed credits would have been worth roughly $600,000, but the model provider had granted those credits to METR for free.

So this was not a $600,000 financial charge.

It was approximately $600,000 worth of AI compute consumed without authorization.

That’s still significant.

Someone else was using the organization’s allocation to run their own workload for weeks.

And because the credits were free, traditional spending controls didn’t provide the expected warning.

The Monitoring System Wasn’t Ready for AI Abuse

METR explains why the abnormal usage wasn’t immediately obvious.

The organization was already accustomed to running large AI evaluations.

High token consumption wasn’t automatically suspicious.

Large experiments generated rate-limit errors and unusual API activity regularly.

The internal dashboard also didn’t display rate-limited requests for all users at the time.

And because the tokens weren’t being paid for directly, there was no natural financial spending ceiling.

There was also no way to impose a spending limit on that particular key.

That’s an important lesson.

Traditional cloud security often assumes:

unexpected usage → unexpected bill → someone notices

Free AI credits break that model.

AI Credentials Need Their Own Guardrails

An API key that can consume millions of model tokens should be treated like cloud infrastructure credentials.

It needs:

• limited permissions
• short lifetime
• usage monitoring
• rate limits
• anomaly detection
• automatic revocation
• environment isolation

And ideally:

a hard usage ceiling.

If the provider doesn’t support one, build monitoring around the key yourself.

Otherwise an attacker can burn your allocation for days before anyone notices.

The Second Attack Was More Aggressive

The API-key theft was only one of the two incidents disclosed by METR.

In May, attackers launched a sustained campaign against its public infrastructure.

They used automation heavily.

METR observed:

• credential stuffing
• attempted OAuth token grants
• scanning of newly deployed services
• phishing attempts against staff
• automated vulnerability discovery

The attackers appeared financially motivated and may have been interested in obtaining access to frontier AI models.

This wasn’t someone manually poking around with a browser.

Automation was doing the recon.

Then METR Found Its Own Public-Facing Bug

During the same period, METR discovered that its public transcript viewer exposed a read-only SQL query mechanism.

The queries were intended to be restricted to public data.

But a bug could potentially be exploited to reach unpublished evaluation data.

Even more importantly, the database accidentally contained some sensitive model information that shouldn’t have been there.

An independent researcher discovered the issue and reported it responsibly.

METR took the API offline and paid a bounty.

The Attackers Saw the Endpoint

This makes the second incident particularly interesting.

The attackers had probed the vulnerable endpoint as part of their broader reconnaissance.

But METR found no evidence that they discovered or exploited the SQL vulnerability.

There is also no evidence that non-public data was accessed.

That’s an important distinction.

The vulnerability was real.

The attackers were looking at the system.

But the available evidence does not show that the two events turned into a confirmed data breach.

AI Agents Are Now Part of the Attack Surface

The first incident shows something traditional security teams aren’t always prepared for.

An application can contain a secret.

An AI agent can access the application.

An attacker can interact with the agent.

And the agent can accidentally become the mechanism that exposes the secret.

That creates a new path:

attacker → agent → secret

Instead of:

attacker → filesystem → secret

The distinction matters because traditional secret-scanning tools don’t necessarily understand what an agent might reveal through natural-language interaction.

METR Says Sensitive Data Wasn’t Accessed

Despite the seriousness of both incidents, METR says its investigation found no evidence that sensitive information was accessed.

The stolen API key provided access to public models.

The May attackers did not successfully exploit the exposed SQL mechanism.

METR also says there is no evidence that the incidents resulted in access to its most sensitive categories of information.

So this is not a giant AI research-data breach.

It’s a pair of security near-misses.

And that’s exactly why it’s worth studying.

METR Changed Its Architecture

After the incidents, METR made several changes.

The organization:

• formalized security reviews for publicly deployed applications
• increased monitoring
• added spend alerts where possible
• isolated public production infrastructure
• expanded logging
• reduced credential lifetimes
• reduced permission scopes
• shut down unnecessary legacy infrastructure
• added additional endpoint and server security controls
• increased threat modeling

The biggest architectural change was separating public-facing applications from sensitive internal infrastructure.

That’s the right direction.

A public application should not have a direct path to sensitive research systems just because both belong to the same organization.

Fail-Open Is Still a Terrible Idea

The simplest lesson from the incident is also the oldest.

Authentication should fail closed.

If authentication breaks:

access denied.

Not:

authentication unavailable, therefore continue without authentication.

The difference sounds obvious.

Yet this class of mistake continues to appear in real systems.

And when the application contains AI credentials, the consequences can scale extremely quickly.

The API Key Wasn’t the Most Important Failure

It’s tempting to focus on the stolen credential.

But the real chain was:

public EC2

→ fail-open authentication

→ AI agent exposed

→ API key revealed

→ SSH persistence

→ three weeks of unauthorized compute

The API key was only one link.

Fixing the key without fixing the architecture would leave the same attack path available for the next credential.

Bugstoday Opinion

This is one of those incidents that looks almost ridiculous until you break down the chain.

A researcher wanted to run an AI application.

The app was protected by authentication.

The authentication failed open.

An attacker found it.

Then instead of searching through files, they simply asked the AI agent for the secret.

The agent handed it over.

Three weeks later, someone had consumed roughly $600,000 worth of AI compute.

No exotic zero-day was required.

No sophisticated malware.

Just bad authentication, excessive trust and a credential sitting where an AI agent could reach it.

Bugstoday verdict: AI agents need to be treated as privileged application components, not harmless chat interfaces. If an agent can see an API key, assume an attacker will eventually find a way to ask for it. Fail closed, isolate public workloads, limit credentials and put hard monitoring around AI consumption before someone else turns your compute budget into their playground.

Today’s Bugs. Tomorrow’s Breaches.

Sources

• METR — Update on Security at METR
• METR — Security Controls and Incident Response
• AWS — EC2 Security Best Practices
• OWASP — Secrets Management
• MITRE ATT&CK — Valid Accounts