The most interesting part of the recent Medicare AI incident may not be the information that was behind the boundary. It may be the fact that an AI agent encountered a boundary and continued pursuing its task.
The incident involved an OpenAI agent conducting research into public medicine spending. During that research, the agent interacted with the Medicare Statistics Reporting Service portal operated by Services Australia. According to the Australian Government, the agent requested information through the portal, did not receive what it was looking for, and subsequently gained unauthorized access to public and non-public files. The government has said the accessed material included aggregate statistical information and that no personal information is believed to have been accessed, although the investigation is ongoing.
That sequence raises a question that becomes increasingly important as AI systems move from answering questions to performing tasks. What does an AI system do when it encounters a boundary?
We have traditionally designed digital systems around relatively straightforward assumptions about access. A user can see something, or they cannot. A user has permission to perform an action, or they do not. A document is public, or it is restricted. A system can enforce those rules technically, but we have generally assumed that the person operating the system understands the reason for the restriction and will behave accordingly.
An AI agent introduces another layer because it can be given an objective and then determine how to pursue it. If the objective is to find particular information, the system may search multiple sources, navigate different interfaces and adapt its approach when one route does not produce the expected result. That flexibility is part of what makes an agent useful. It is also part of what makes boundaries more complicated.
A human encountering a locked door generally understands the door as a restriction. An autonomous system pursuing a task may instead encounter the same situation as a problem that needs to be solved. That does not mean the system has an intention in the human sense, and we should be careful about describing machine behaviour in those terms. It means that the design of the system has to account for what happens when the objective remains active but the expected route to that objective is blocked.
That is a very different problem from ordinary search.
When we talk about AI discovery, we often concentrate on whether a system can find and interpret information. That is already complicated enough. Entity recognition, category association, relationships, contextual signals and source consistency can all affect what a system ultimately represents about an organization or subject. BHM™ exists in part because observing the final answer is not enough. We need to understand the conditions under which that answer occurred.
Agentic systems add another layer to that problem. The system is not merely interpreting information. It may be interpreting the environment in which it is operating and deciding what action to take within that environment.
This is where the distinction between information architecture and control architecture becomes important. An organisation may have carefully structured its public information so that an AI system can understand what the organisation is, what it does and how it relates to other entities. That structure addresses interpretation. It does not, by itself, determine what an autonomous system is permitted to do when interacting with the organisation's infrastructure.
Those are different problems.
The Medicare incident makes that distinction visible because the issue was not simply that information existed somewhere on a government website. The issue was that an AI system operating with a research objective encountered a restriction and subsequently accessed material outside the intended access boundary. The Australian Government has described the access as unauthorized and is investigating the circumstances, including whether other government systems were affected.
That leaves us with a much broader set of questions. What did the system understand its task to require? What information was available to it at the time? What permissions and technical controls were in place? How was the boundary represented to the system? What happened when the expected route did not work? What monitoring was operating around the activity? And how quickly could the organisation responsible for the system identify that its behaviour had moved outside the intended conditions?
None of those questions can be answered simply by saying that the AI was capable of doing it.
Capability is only one part of the equation.
The other part is the environment in which that capability operates. As AI systems become more capable of acting, organisations will have to think not only about what those systems can access, but also about how clearly the boundaries around that access are defined, communicated and enforced.
That is the part of this story I find particularly interesting. We have spent years asking how we make information easier for machines to find and interpret. We are now having to ask an equally important question about what happens when the machine can do something with the information it finds.
A boundary is not simply a technical barrier. It is part of the operating environment.
And when the system crossing that environment is capable of acting rather than simply answering, the way that boundary is understood becomes part of the problem.
