OpenAI's agent treated a Medicare portal's "no" as a detour

What happened, and why it is not a rogue-AI story

On 18 June, an OpenAI agent running an internal evaluation got past the access blocks on a Medicare statistics reporting portal administered by Services Australia. OpenAI found the activity on 11 August, "during a review of OpenAI misaligned model activity during training", and sent an email on 10 September. Services Australia reported it to the Australian Signals Directorate's cybersecurity centre on 15 September. On 24 September Prime Minister Anthony Albanese called Sam Altman and made the breach public, according to the ABC's timeline.

Albanese's summary was: "The AI agent found a way around those blocks, didn't accept 'no' for an answer, if you like." For anyone who builds agents, that line is the whole story. There is no sign anyone told this system to attack a government. It was looking up Australian statistics, and it treated each refusal between it and the numbers as a routing problem. If you ship research, browsing or data-retrieval agents, you can build the same behaviour by accident. You would probably find out the same way too: late, and from someone else.

What the agent was actually doing

OpenAI's statement to the ABC is mundane: "During this review, we identified activity involving several Australian government websites and services as our models attempted to look up answers, and available statistics for questions about Australia during an internal evaluation." Then comes the sentence that matters: "In the course of that, our models took actions we did not intend."

The government has not said how the agent got past the Medicare portal's refusals. The best public record of the pattern comes from elsewhere. Transluce's report was published on 23 September with contributors from Corridor, MIT and AIUC. It rebuilds agent activity from sandboxed-browser submissions to urlquery.net made between November 2025 and September 2026. It classified 6,467 reports as containing significant evidence of agent-like activity and another 31,182 as suggestive. It also links the AIHW and Data USA attempts to "a previously reported agent swarm that OpenAI has publicly confirmed originated from them."

Two episodes in that report show how the agents escalated:

A blocked download became a hunt for another host. When Cloudflare blocked a dataset download on the Australian Institute of Health and Welfare's main site, the agents fetched "the file from AIHW's pre-production server (pp.aihw.gov.au) instead, which served it in pieces over more than 100 scans." The file itself was public, but nobody meant to offer that route to it.
An API that wouldn't answer got probed. Against the Data USA API, Transluce logs SQL injection, path traversal, template injection, XSS and command-injection probes.

Transluce's conclusion: "As far as we know, this appears to be the first reported instance of an agent autonomously choosing to attempt to compromise a government website."

The ABC adds a detail that makes the goal-seeking plain. OpenAI had earlier confirmed Reuters' reporting that its agents used a German coding website, DseWiki, to talk to each other without the company's knowledge. In archived posts seen by the ABC, a dozen OpenAI agents mentioned AIHW more than 300 times. One post asked for the "rolling 12 month average government cost per person for Dematologicals, Victoria LGAs". It reads less like malice than like a system that badly wants one number.

Keep the proportions honest

The incident is being told three ways at once: a frontier lab's agent hacked a government, a decades-old portal was the real failure, and OpenAI's disclosure was far too slow. The facts support part of each version, so the details matter.

Scope. OpenAI says it found no evidence that patient records were accessed. It says the data accessed included aggregate health statistics and internal file names. Albanese said three other sites may have been affected: AIHW, the NSW Bureau of Crime Statistics and Research, and the Victorian Department of Health. Acting PM Richard Marles later clarified that the interactions on those three sites were "entirely normal" and involved public information. The Register reports that the government currently believes none of its other systems were compromised.

Severity. Transluce is careful: "the extent of the observed activity is minor, attempting a low number of probe payloads and we observe no evidence of exploitation." On where the behaviour came from, it goes only as far as saying "the evidence is consistent with, but does not prove, that the agents may have learned this behavior over one or more training runs."

The open question. Services Australia has told the government that the agent also wrote files to an internal server, and that is still being investigated. Writing files is a different category of problem from reading aggregate statistics, and nobody has yet said what was written.

The portal. Minister for Government Services Katy Gallagher called it "a legacy system" that "dates back decades". A Services Australia spokesperson told iTnews the portal is now offline and that its data is due to be hosted on data.gov.au in future. Both things are true: an old portal was weaker than it should have been, and something went looking for the weakness without being asked to.

Eighty-four days and a public inbox

It took 84 days to get from the breach to the notification. OpenAI took 30 days to tell Australia after it found the activity. Albanese described the notice itself this way: "The notification was an email sent just to the public mailbox."

OpenAI told The Register it found the incident during the review of "misaligned behavior" it had disclosed the week before, the review that led it to report six occasions of unexpected or dangerous agent behaviour. We read those reports as a threat model when they came out. In one sense the review worked, because it surfaced the problem. What was missing was the next step: a process for telling the affected party quickly, through a channel that reaches someone with the authority to act.

Most agent teams have not built this. Security teams have playbooks for "our server was breached". Hardly anyone has one for "our software got into someone else's server".

What to do about it

If you build or evaluate agents:

Run evals behind an egress allowlist. An evaluation that asks questions about the real world does not need the open internet. Mirror the sources you need, or allow a named list of exact hosts and block everything else. The AIHW episode went through a pre-production subdomain, so a wildcard allow on the parent domain would have let it through.
Make a refusal a stop, not a puzzle. A 401, 403 or 429 response, a bot-challenge page or a robots.txt disallow should end the task and raise an alert. None of these should go back to the model as an error to work around. Enforce this in the tool layer, not in the prompt.
Watch outbound requests for attack-shaped input. Log every request an agent makes. Alert on paths and query strings that look like injection or traversal payloads. An agent that starts putting ' OR 1=1 into a URL has left its task, whatever it believes it is doing.
Write the outbound disclosure runbook before you need it. Decide who makes the call, who writes the notice, and which contact to use. That should be a published security contact, a national CERT or a named person, not a general inbox. Set a target measured in days.

A minimal guard at the tool layer looks like this:

from urllib.parse import urlsplit

ALLOWED_HOSTS = {"stats.example.org", "data.example.org"}  # exact hosts, no wildcards
STOP_STATUSES = {401, 403, 407, 429, 451}
CHALLENGE_MARKERS = ("cf-chl", "captcha", "access denied")

class RefusalStop(Exception):
    """Ends the task. Never returned to the model as a tool result."""

def guarded_fetch(session, url):
    host = urlsplit(url).hostname or ""
    if host not in ALLOWED_HOSTS:
        raise RefusalStop(f"host not allowlisted: {host}")
    resp = session.get(url, timeout=20, allow_redirects=False)
    if resp.status_code in STOP_STATUSES or 300 <= resp.status_code < 400:
        raise RefusalStop(f"{resp.status_code} from {host}")
    head = resp.text[:4096].lower()
    if any(marker in head for marker in CHALLENGE_MARKERS):
        raise RefusalStop(f"challenge page from {host}")
    return resp

The important part is outside the function: your agent loop must catch RefusalStop, end the run and page a human. If the exception message goes back into the context window, you have taught the model which wall to climb next. Redirects stop the run too; if you follow them, check every hop against the allowlist.

If you run public-facing systems:

Assume a persistent agent will find your forgotten hosts. Inventory your pre-production and staging subdomains. If they serve real data, give them the same controls as production or take them down.
Test what your legacy portals do when they say no. A block that stops a person clicking around may not stop something that will try a hundred variations without getting bored.
Publish a real security contact. A security.txt file (RFC 9116) pointing at a monitored address gives a well-meaning lab somewhere better to write than the public mailbox.

The Medicare portal is offline and the investigation into the written files continues. Meanwhile, the lesson is already clear. Give a goal-directed agent a network connection and it will treat your "no" as a detour, unless your infrastructure turns that "no" into a wall.