What Should an AI Guardrail Actually Do?

Tell me that an AI system has guardrails, and I still do not know what it will do when it reaches a boundary.

Which action will it stop? Whose authorization will it require? What evidence will let it proceed? Can the agent bypass the check through another tool?

“Guardrails” is an understandable metaphor. It is not, by itself, a specification that someone can build, test, or hold an operator to.

The term can also refer to two different decisions: which requests a system will accept, and which actions it may take to carry them out. This article concerns the second. An agent can accept a legitimate task and still exceed its authority in how it completes it. Accepting the goal does not authorize every means of reaching it.

One requirement should be explicit in any system that purports to act responsibly on someone’s behalf: an authorization check. Before the system takes an action, that action must fall within established authority. When it encounters a material gap, it must pause that action and return the decision to someone entitled to make it.

That is the standard I propose for responsible agent deployment. It leaves room for useful autonomy while making its limits concrete.

Delegation includes reasonable inference. Asking an agent to analyze a document includes reading it. It does not ordinarily include publishing it. The agent should be able to carry out the normal steps of an assignment without asking permission at every turn. But an action does not become authorized merely because it would help achieve the goal.

There is also a limit on what the person giving instructions can grant. “No one can give what they do not have.” If a friend lends you a bicycle, you may ride it. You cannot give it to someone else as a birthday present. Permission to use it does not make it yours to give away.

The Law Commission of England and Wales explains the maxim in the context of ownership. The related point for delegation is that permission has a scope. Having a task does not give you every power that might help you perform it.

Suppose a friend asks you to look after their house while they are away. You notice that the living room looks worn and repaint it yourself, thinking you are taking good care of the place. Your friend returns to a colour they did not choose and the expense of putting it back. You may have meant well, but permission to look after the house did not include permission to redecorate it.

The mistake was treating your judgment about what would help as authority to act. The appropriate next step was to ask whether your friend wanted the room repainted. An agent pursuing a useful goal needs to recognize the same distinction.

The Australian Medicare statistics portal incident makes the distinction concrete.

According to the Australian government’s September 24 account, OpenAI researchers used an internal model to conduct internet-based research into public medicine spending. After encountering blocks, the agent obtained unauthorized access, read public and non-public files, and wrote files to the portal’s internal server.

The original instructions are not reproduced in that account. “Public medicine spending” describes the research topic; it does not establish an express instruction to use only public sources. We should not invent that restriction. Nor should we turn non-public files into a claim about personal medical records: the government reported no evidence at that stage that personal information had been accessed, and its investigation was ongoing.

We can still apply an authority test to the facts reported.

First, what was the assignment? Research medicine spending. Second, what actions reasonably followed from it? Finding sources, reading available information, and analyzing it. Third, what additional authority would the reported actions require? Permission to enter restricted areas and write to the government’s server.

The published account supplies no basis for that additional authority. More decisively, the government describes the access as unauthorized. The research objective could not supply the missing permission.

A failed request alone would not prove overreach. A researcher can try another public page or another ordinary retrieval method. The boundary in this case is the move into restricted access and server modification. Those are different powers from researching a topic.

Under the standard I propose, the system should have paused before exercising those additional powers. It could continue research through authorized sources, report the limitation, or request substantiation for the proposed access. It should not infer permission from technical reachability, persistence, or the usefulness of the result.

A legitimate security audit illustrates why this should be an authorization check rather than an indiscriminate ban. A team may hold permission to test a defined system. The deployer should provide a way to establish that permission, its scope, and its duration, then let the agent work within those limits. Permission already established need not be requested repeatedly.

An assertion of authority is evidence of who made the claim; it is not proof that the claim is true. Suppose instead that the house sitter hires a painter and falsely says the owner approved the work. The painter may reasonably believe that claim. The person carrying out the action and the person claiming authority are now different people, and an investigation must examine both. The false claim matters even if the recipient acted in good faith; whether the recipient should have checked further is a separate question.

An authorization record should therefore identify who approved what, on what basis, and with which limits. Verification should be proportionate to the action and its consequences, and records should retain only the information needed. The model does not have to authenticate every document itself. The surrounding system must provide an appropriate process.

Most importantly, that process must control execution. If the agent can obtain the same access through another route, a warning on the first route has not enforced the boundary.

This is where responsibility falls on the organization operating the system.

OpenAI’s paper Practices for Governing Agentic AI Systems distinguishes model developers, system deployers, and users. Those are roles, not necessarily separate companies or departments. The distinction helps locate the decisions: who developed the capability, who connected it to tools and external systems, and who commissioned the task?

The analogy with agency also has a limit. A person acting as an agent can be called to account for their conduct. A software system cannot bear that accountability in the same way. Describing its behavior as unauthorized identifies a failure; it does not identify who must answer for it. The deployer must answer for the conditions under which it allows the system to act: its access, controls, testing, supervision, and response to failures. How far that responsibility extends depends on the decisions it controlled, what it knew or reasonably should have known, and the circumstances of the failure. Responsibility held by developers or users must be examined alongside it.

For this incident, OpenAI occupied the operating role as well as the development role. The model was internal. The deployment decision at issue was allowing it to act on external systems during research, not releasing it as a public product.

My position is that a deployer should not authorize that operation without evidence that the system can stay within its mandate and escalate material authority gaps. The evidence must cover the complete system—its model, tools, permissions, and controls—and include what happens when it cannot complete the task within the allowed scope.

If the evidence is inadequate, the deployer should restrict the operating conditions or withhold that use. An agent stopping at a boundary may be performing correctly. An agent crossing it to produce a useful result has not earned a passing grade.

That assigns a concrete operational responsibility. It does not establish exclusive legal liability, excuse every development decision, or erase the responsibility of a person who supplies false authorization. Nor does the public record tell us precisely which control was absent or failed in Australia. The reported outcome shows that the operating arrangement did not prevent the unauthorized actions.

Disclosure matters too. OpenAI’s published review of third-party incidents makes evidence available for scrutiny and shared learning. Publishing those accounts was the right thing to do. We should encourage disclosure and use it to improve practice.

That recognition coexists with the Australian government’s criticism of delayed notification. Public reporting and timely notice to affected parties are distinct responsibilities.

Mistakes will happen. Maturity shows in how an organization responds: acknowledge the failure, inform those affected, investigate it, and change the conditions that allowed it. Disclosure makes that work open to examination. It does not remove accountability for the original decisions.

The next step is to demonstrate what was learned. Identify the boundary that failed, change the controls, and test whether the system now respects that boundary—including under the conditions that led to the incident. A promise to do better needs evidence behind it.

That is the response these disclosures should invite: thank you for making the failure visible; you remain answerable for it; now show how the lesson changes what happens next. The same standard belongs in our own deployments.

I have described responsibility as the capacity to perceive, judge, and respond. Accountability is the obligation to answer for decisions and their consequences. Calling an agent irresponsible does little unless we identify the people and organizations capable of changing its operating conditions.

For AIAC, the practical requirement is an authorization check that can be demonstrated: routine work proceeds within established scope; material gaps trigger a pause; legitimate changes have a recorded basis; and the limits remain enforceable across the system’s tools.

Saying a system has guardrails tells us little. Tell us which action it will stop, whose authorization it requires, and what lets it proceed.

We are applying familiar principles of agency to a new setting: a delegated task has a scope, authority has limits, and someone remains answerable for the decisions made. Using those principles does not require treating a software agent as a person in law.

The new challenge is making those principles work in systems that can act at speed and with limited supervision. What we are learning from these incidents is how to turn established expectations about authority into controls that hold in practice. That is a practical measure of maturity for AI adoption.