Australia plans mandatory AI incident reporting

Prime Minister Anthony Albanese on November 10 released the recommendations of a rapid review into OpenAI’s June breach of Services Australia and said the government accepts all 18 of them, the Guardian reports. OpenAI itself will not face criminal penalties. The new rules target future incidents: a “failure to prevent” offence and mandatory, time-bound reporting of AI safety incidents, both still framed as proposals the government will consider.

What the review asks for

The proposed offence would criminalise developing an AI model that engaged in harmful conduct while failing “to take reasonable steps to prevent that harm”. The review wants the attorney general’s department, the Australian federal police and home affairs to consider it alongside “penalties for egregious regulatory non-compliance”. The reporting duty would cover, “including but not limited to”, misaligned behaviour and agentic cyber incidents. Albanese also said the government would set up a new AI safety regulator.

“The mandatory reporting, when artificial intelligence goes rogue, gives us a chance to be able to make sure that when this happens, the reporting occurs and occurs immediately,” Home Affairs Minister Tony Burke said in Canberra. On criminal law, he put it more bluntly: “It should never be the case that there’s an argument that someone didn’t know the computer was loaded.”

Labor would also consider six obligations on AI companies:

• maintain security controls across all stages of development and deployment;
• identify and manage serious risks associated with their models;
• notify the Australian government of incidents rapidly;
• allow independent evaluation of their models;
• maintain and publish safety and security frameworks covering governance, risk management and transparency;
• respond to updated standards and rules as technologies change.

Albanese released the recommendations but not the review’s findings or the report itself, and he did not directly answer a question about whether the June incident would have breached the proposed laws had they been in place.

How the agent got in

On June 18 an OpenAI agent tasked with researching public Medicare and Pharmaceutical Benefits Scheme spending found the Medicare Statistics Reporting Service portal on the open internet. When the published data did not answer its task, it pushed through the site’s privacy protections to unreleased information, including non-public data on medicine use among Victorian patients. A Cloud Security Alliance research note describes this as the agent bypassing access controls designed for people rather than exploiting a vulnerability in the ordinary sense, and calls the pattern “agentic overreach”. Deputy prime minister Richard Marles said the data sat behind “a fence that the AI agent effectively climbed over”, while Australians’ personal data sits “inside a safe”.

Albanese called the breach “minor” in terms of privacy. “The clear indication is that no information was got that involves people’s personal details,” he said, adding that this “doesn’t mean that it wasn’t a serious issue”. Besides Services Australia, the agent reached three other systems in June.

The late email

OpenAI became aware of the incident in August and still did not notify the Australian government until mid-September, by a single email to a generic Services Australia inbox, even though several senior company leaders had recently met government officials. Guardian Australia revealed in October that OpenAI used AI to help write that email.

AI researcher Toby Walsh argues the people running these companies need to be criminally liable. “We’ve tried fining them, and we’re not the only ones. These companies are incredibly wealthy, and it’s almost impossible to fine them enough,” he said.