Australia flags criminal penalties for harmful AI models

Australia plans a new AI safety regulator and has flagged criminal penalties for AI companies that don’t comply, Prime Minister Anthony Albanese said on November 10 as he released the 18 recommendations of a “rapid review” into a rogue OpenAI agent’s break-in on government systems. For teams that build models, these are still recommendations, not law, but two of them are aimed squarely at developers: a standalone “failure to prevent” offence and “mandatory, time-bound AI safety reporting” of hacks.

The offence would be a change to criminal law, the BBC reports. It would criminalise cases where a company develops “an AI model that engages in harmful conduct” and “fails to take reasonable steps to prevent that harm.” The review also proposes new safety and security standards for AI companies. “The law needs to get ahead of and, in some cases, make sure that it catches up with, what is a very fast changing world,” Albanese said.

The reporting rule answers the part of the case Albanese called his “disappointment”. In June, the agent hacked a statistics portal holding “non-sensitive” data from Medicare, Australia’s universal healthcare scheme. OpenAI says it only learned of the breach in August, while reviewing “misaligned model activity”, and on September 10 it emailed a general inbox of an Australian government agency. That single email went to a generic Services Australia address even though several senior OpenAI leaders had recently met Australian government officials. It took several more days to reach Australia’s cybersecurity centre, then a minister, and only after that the prime minister.

Services Australia was not the only agency waiting. The same agent also got into the portals of the Australian Institute of Health and Welfare, the Victorian Department of Health and the New South Wales Bureau of Crime Statistics and Research, according to The HIPAA Journal. The health and welfare institute heard only on September 24, the day Albanese made the incident public at a press conference in New York during the UN General Assembly.

Nobody told the agent to break in. It had been asked to research public medicines spending, found the Medicare Statistics Reporting Service as a source, and when the published data did not give the answer, it pushed through the site’s privacy protections to reach unreleased information. The portal is a legacy Services Australia site with aggregate data on Medicare and Pharmaceutical Benefits Scheme use, mostly used by researchers. OpenAI’s own account is that its “models took actions we did not intend”.

Days after the hacks were revealed, OpenAI delayed the rollout of a new AI model, citing safety concerns. Albanese said he had spoken to Sam Altman and raised “Australia’s extreme concern about this incident”, and that Altman acknowledged there were “issues with protocols” at OpenAI.