AI Agent Authorisation: Designing Permission Boundaries for AI That Takes Action

Summary

For most of the past decade, engineering work around data and AI has concentrated on the read path: which records a system may see, where they are stored, how they are encrypted and who may query them. That work remains necessary. Once a language model is wired to tools that update records, send messages or start transactions, however, the difficult part of the design moves to the write path. The question is no longer only what the model can see. It is what the system may do, under which identity, with whose approval, and how anyone can show afterwards exactly what happened.

This article starts from the central argument of Ts. Lukas J. Tan's essay Who Controls AI When AI Starts Taking Action?: governance for AI must cover the authority given to systems that act, not only the protection of the information they process. Here that argument is treated as an engineering specification. We look at how to separate a probabilistic planner from a deterministic executor, how to declare capabilities explicitly, how to enforce human approval in code rather than in a policy document, how to scope credentials, what an audit trail needs to contain, and how to test and release changes that alter what an agent is permitted to do.

The intended reader is a developer, architect or technical lead integrating a third-party model into an existing business application. Nothing here depends on a particular vendor, and most of it can be implemented by a small team without a dedicated platform.

What Is AI Agent Authorisation?

AI agent authorisation is the set of controls that decides which actions an AI-driven component may execute against real systems, on whose behalf, within what limits, and with what evidence left behind. It sits alongside two older concepts and should not be confused with either of them.

• Authentication establishes who or what is making a request.
• Data access control decides what may be read.
• Action authorisation decides what may be changed, sent or initiated, and under which conditions.

An agent can be properly authenticated and hold legitimate read access to a customer record, yet have no sound reason to edit that record, merge it with data from another department or send a message to the customer in the company's name. Each of those is a separate grant of authority. In many early integrations they are granted together, by accident, because they share one credential.

A useful way to frame the difference is to compare two integration patterns. In the assistant pattern, the model returns text and a person decides what happens next. In the actor pattern, the model's output is parsed into a structured call and the application executes it. The model may be identical in both. The consequences of an error are not. A flawed summary misleads one reader, who still has a chance to notice. A flawed write changes the state of the business before anyone has looked.

The Problem in Technical Terms

The integration credential quietly becomes the agent's authority

The quickest route from prototype to demo is usually to give the integration a service account or API key with generous scope, wrap a handful of endpoints as tools and let the model choose among them. Because the model never sees the key, the arrangement feels contained. It is not. The effective authority of the agent is every operation reachable through its tools, combined with the scope of the credential behind each tool. If a tool called update_customer accepts arbitrary fields and runs under an account that can write the whole customer table, then the agent can, in principle, alter any customer in any way, whatever the system prompt says it ought to do.

A system prompt is an instruction to a probabilistic component. It shapes behaviour on average. It is not an access control mechanism, and it should never be the only thing standing between a model's output and a committed write.

Authority expands one pull request at a time

Few teams set out to build an autonomous agent with broad write access. What usually happens is gradual. The first release drafts emails. The next adds a read-only connector so staff no longer paste data into a chat window. A later sprint allows the agent to update a status field, then to schedule a follow-up, then to send the follow-up itself. Every change is small, sensible and easy to review on its own. Nobody raises a ticket titled 'grant the AI authority over customer communications', yet that is the cumulative result.

Engineers will recognise this as configuration drift. It deserves the treatment we already give to infrastructure drift: a declared desired state, a diff on every change and a review by someone who understands the consequences, not only the code.

A probabilistic planner produces deterministic side effects

A database transaction has no concept of how confident the model felt. Once committed, an update is simply an update, and an email that has left the mail server cannot be recalled by a better prompt. Non-determinism in planning is acceptable when every side effect passes through deterministic checks. It becomes a liability when the model's output is the final step between intent and execution.

Untrusted content enters through the read path

Agents read customer enquiries, attachments, web pages and internal documents. Any of these can contain text that looks like an instruction. If the same context window that holds a customer's email is also where the model decides which tool to call next, then the content of that email has become an input to an authorisation decision, unless the architecture explicitly prevents it. This is the practical reason why authority cannot be delegated to the model itself.

Architecture: Separate the Planner from the Executor

The core design principle is simple to state. The model proposes; deterministic code decides and executes. Everything else in this article follows from taking that sentence seriously.

Reference components

• Planner. The model, its prompt and its retrieved context. Its only output is a structured action proposal. It holds no credentials.
• Capability manifest. A versioned declaration of every action the agent may propose, with an input schema, writable fields, limits, a risk tier and the credential scope required.
• Policy gate. A deterministic function that evaluates each proposal against the manifest, the principal on whose behalf the agent is acting and any business rules. It returns allow, deny or require approval.
• Approval service. Holds proposals that need a human decision, presents them clearly and records the outcome.
• Executor. The only component that can obtain credentials. It performs permitted actions using narrowly scoped, short-lived tokens and idempotency keys.
• Audit log. An append-only record linking each request, proposal, decision, approver and result.
• Control plane. Feature flags, kill switches and dashboards that let operators reduce or suspend the agent's authority without a code release.

Data flow of a single action

1. A request arrives with an identified principal: a named user, or a scheduled job owned by a named team.
2. Context is retrieved using the principal's own read permissions, not a superuser account.
3. The planner emits a proposal consisting of a capability name, a target record and arguments.
4. The proposal is validated against the capability's schema. Malformed proposals are rejected, not repaired.
5. The policy gate evaluates the proposal and returns a decision with a reason.
6. If approval is required, the proposal waits in the approval service until a person accepts or rejects it, or until it expires.
7. The executor requests a credential scoped to that capability and that target, performs the action with an idempotency key and records the result.
8. The outcome is returned to the planner as an observation, labelled as data rather than as instruction.

One rule in this flow is worth stating on its own. An agent acting on behalf of a principal should never hold more authority than that principal. Its effective permission is the intersection of what the user may do and what the manifest allows the agent to do. A sales executive's assistant agent should not be able to approve a refund that the sales executive could not approve.

Implementation Details

Declaring capabilities in a manifest

A manifest turns implicit authority into something that can be read, diffed and reviewed. The format matters less than the discipline of keeping it as the single source of truth. A simplified example:

# capabilities.yaml

- name: crm.update_contact_status
  description: Change the follow-up status of one contact
  input_schema:
    contact_id: { type: string, required: true }
    status:     { type: enum, values: [open, contacted, closed], required: true }
  writable_fields: [status]
  risk_tier: low
  tier_reason: internal, reversible, single field
  requires_approval: false
  max_actions_per_run: 20
  credential_scope: crm.contacts.status.write

- name: email.send_customer_message
  description: Send one message to the contact already on record
  input_schema:
    contact_id: { type: string, required: true }
    body:       { type: string, required: true }
  recipient_rule: contact_on_record_only
  attachments: none
  risk_tier: high
  tier_reason: external, irreversible once sent
  requires_approval: true
  credential_scope: mail.send.customer

Notice what the manifest does not contain: there is no generic update_customer capability. Each action is as narrow as the business need allows. The status update may touch only one field. The email capability may only write to the contact already on record, cannot attach files and always requires approval. The numeric limit in this example is illustrative; the right values depend on the workflow and should be set by the people who own it.

Assigning risk tiers

Risk tiers should be based on what an action can do, not on how impressive the underlying model is. A short set of questions is usually enough to place a capability:

• Reversibility: can the action be undone cleanly, or does it leave a permanent effect such as a sent message or a completed payment?
• Reach: does the effect stay inside the organisation, or does it reach customers, suppliers or the public?
• Data sensitivity: does the action read or write sensitive personal data, or combine records from different departments?
• Material impact: could it affect someone's finances, employment or access to a service?
• Volume: how many records can a single run touch before anyone notices?

An action that scores low on every question can run with logging and limits. One that is irreversible, external or touches sensitive data belongs in a higher tier with enforced approval. Record the reasoning for each tier in the manifest itself, as the tier_reason field does above, so that a reviewer can challenge it later.

A deterministic policy gate

The gate is ordinary code. It should be boring, fast and exhaustively tested. It must not call the model.

def evaluate(proposal, principal, manifest):
    cap = manifest.get(proposal.capability)
    if cap is None:
        return Decision.deny('unknown capability')

    if not cap.input_schema.validates(proposal.args):
        return Decision.deny('arguments do not match schema')

    if set(proposal.fields_written()) - set(cap.writable_fields):
        return Decision.deny('field not writable by this capability')

    if not principal.is_permitted(cap.credential_scope, proposal.target):
        return Decision.deny('principal lacks authority over target')

    if actions_in_run(proposal.run_id, cap.name) >= cap.max_actions_per_run:
        return Decision.deny('run limit reached')

    if cap.requires_approval or involves_sensitive_data(proposal):
        return Decision.require_approval(cap.risk_tier)

    return Decision.allow()

Several details matter. Unknown capabilities are denied by default; the gate never infers what the model meant. Writable fields are checked explicitly, so a valid capability cannot be used to smuggle in an extra change. The principal check runs on the specific target, not only on the capability, so row-level permissions in the source system are respected. And every decision carries a reason, which feeds the audit log and makes later review possible.

Human approval as an enforced state

A policy that says 'a human must approve customer emails' is only as strong as the mechanism that enforces it. In code, approval should be a state in a state machine that the executor checks before it can run.

PROPOSED  --> DENIED
PROPOSED  --> ALLOWED  --> EXECUTING
PROPOSED  --> PENDING_APPROVAL --> APPROVED --> EXECUTING
                               --> REJECTED
                               --> EXPIRED
EXECUTING --> DONE
EXECUTING --> FAILED --> EXECUTING   (retry, same idempotency key)

Approval must be bound to the exact action that was reviewed. If the planner revises the wording of an email after approval, the earlier approval should no longer be valid.

# when the approver accepts
approval = Approval(
    proposal_id = p.id,
    args_hash   = sha256(canonical_json(p.args)),
    approver_id = user.id,
    expires_at  = now() + APPROVAL_TTL,
)

# immediately before execution
if sha256(canonical_json(p.args)) != approval.args_hash:
    raise ApprovalMismatch()
if now() > approval.expires_at:
    raise ApprovalExpired()

The approval screen should show the before and after state of any record, the full text of any outbound message, the recipient and the reason the agent gave. An approver who cannot see what they are approving is not providing oversight. Expiry matters too: an approval granted on Monday for a proposal built from Monday's data should not be executed on Thursday.

Scoped, short-lived credentials

The planner process should hold no secrets at all. The executor should request a token for one capability, ideally for one target, with a short lifetime, and discard it afterwards. Where a downstream system only offers coarse API keys, wrap it with a small internal service that exposes the narrow operations the agent needs and nothing more. This adds a component to maintain, but it moves authority out of a vendor's key format and into code your team controls and can test.

Avoid shared service accounts for agent actions. If every agent write appears in the source system as 'integration_user', attribution is lost at exactly the point where it matters most.

An audit trail that someone can actually review

For each action, record at least the following:

• the principal and the originating request;
• the model identifier, prompt version and manifest version in use;
• the proposal exactly as emitted, and the validated arguments;
• the gate decision and its reason;
• the approver, the time of approval and the argument hash approved;
• the executor's request, the response from the target system and any error.

Write the log to a store the agent cannot modify. Remember that audit logs frequently contain personal data, so they need their own access controls and retention rules; data protection obligations do not pause because a record exists for governance purposes. Finally, schedule a review. A log that nobody reads is evidence that something happened, not evidence that anyone was in control.

Treating retrieved content as data

No prompt-level technique reliably stops a model from being influenced by text it reads. The architecture should therefore assume influence will sometimes succeed and limit what that influence can achieve:

• Decide the available tool set from the principal and the task before any untrusted content is loaded, and do not let content change it.
• Constrain recipients, accounts and targets to records already in the system, not to addresses that appear inside a document.
• Require approval for high-tier actions regardless of how the proposal was reached.
• Label tool results and retrieved documents as observations in the context, and log which content was present when each proposal was made.

Testing and Deployment

Test the boundary, not the wording

Model output varies, so tests that assert on exact text tend to be brittle. The policy gate, by contrast, is deterministic, and it is where the important guarantees live. Table-driven tests suit it well:

CASES = [
  # capability,                   args,                    principal,      expected
  ('crm.update_contact_status',   {'status': 'closed'},    'sales_user',   ALLOW),
  ('crm.update_contact_status',   {'credit_limit': 0},     'sales_user',   DENY),
  ('email.send_customer_message', {'body': 'Follow-up'},   'sales_user',   REQUIRE_APPROVAL),
  ('payments.issue_refund',       {'amount': 100},         'sales_user',   DENY),
  ('crm.update_contact_status',   {'status': 'closed'},    'other_branch', DENY),
]

def test_policy_gate():
    for cap, args, who, expected in CASES:
        p = make_proposal(cap, args, target=CONTACT_IN_BRANCH_A)
        assert evaluate(p, principals[who], MANIFEST).kind == expected

Beyond unit tests, three other layers help. Contract tests run each executor against a sandbox of the target system to confirm that the scoped credential really cannot do more than the manifest declares. Adversarial scenario tests feed the full pipeline documents and messages containing embedded instructions, then assert that no out-of-manifest action was executed and that every high-tier proposal landed in the approval queue. Replay tests run logged historical proposals through a new policy version to show exactly which past decisions would change.

Every change to authority is a release

An agent's risk profile can change without anyone deploying what feels like a new system. A model upgrade, a revised prompt, a new connector, a broadened token scope or a removed approval step can each alter what the agent will do. Treat all of them as releases.

A practical technique is an authority diff in continuous integration: compare the manifest and credential scopes before and after the change, and flag new write capabilities, widened scopes, raised limits or removed approvals. Such changes should need sign-off from the business owner of the workflow as well as an engineer. Technical reviewers can judge whether the code is correct. They are not always the right people to judge whether the business should accept the new risk, and the business owner cannot judge it without seeing the diff.

Build the off switch first

Each capability should have a flag that disables execution while leaving proposals and drafts working. If something looks wrong, operators can return the agent to the assistant pattern quickly, investigate from the logs and restore authority deliberately. The ability to reduce authority quickly is as important as the ability to grant it.

Third-Party Models and Shared Responsibility

Most Malaysian software teams and SMEs are not training foundation models. They integrate one supplied by an international provider, host it on someone else's cloud, connect it to a customer's systems and deliver the result under a contract. When something goes wrong, responsibility is spread across several parties, and none of them can see the whole stack.

A workable way to reason about this is to sort each concern into one of three columns:

• What you directly control: the manifest, the policy gate, the approval flow, the executor, credential scopes, logging and the release process.
• What you can reasonably verify: model behaviour within your own test scenarios, documented provider settings and the behaviour of your integration under change.
• What you must cover through contracts and supplier oversight: the provider's internal model behaviour, its data handling and retention, its hosting location and its own change management.

The engineering consequence is direct. Put the controls that matter most in the first column, because that is the only place your team can enforce them. If safety depends on a model always refusing a certain instruction, it depends on something you cannot inspect. If safety depends on a gate that denies the action regardless of what the model proposes, it depends on code you own.

This is not a way of reducing accountability. It is a way of making accountability specific enough to implement, test and demonstrate.

Pitfalls and Trade-offs

• Approval fatigue. If every action requires approval, people learn to approve without reading. Tier actions by consequence and reserve the approval queue for decisions that genuinely need judgement.
• Granularity. Capabilities that are too coarse recreate the broad service account problem. Capabilities that are too fine make the manifest hard to review. Aim for one capability per business action that a manager would recognise.
• Latency. Human approval turns a synchronous call into an asynchronous workflow. Design user interfaces and job schedulers for that from the start rather than adding waiting states later.
• Retries without idempotency. A timeout followed by a retry can send the same message twice. Every executor call needs an idempotency key derived from the approved proposal.
• The prompt as policy. Business rules written only in a system prompt are untested and unenforced. Move them into the gate.
• Policy on paper, not in code. For every rule in the organisation's AI policy that concerns actions, there should be a corresponding check in the gate and a test that proves it. A documented rule without a test is an aspiration.
• Logs as a new data risk. Storing full prompts and documents helps investigations, and harms the organisation if the log store becomes the least protected copy of customer data.

Proportionate Controls for SMEs

Not every agent is high risk, and treating them all as if they were would make adoption impractical for smaller organisations. An agent that tidies internal task lists under a named user is different from one that can message customers, change financial records or influence employment decisions. The controls should scale with the consequence of the action, not with the novelty of the technology.

For a small team, a reasonable minimum looks like this:

1. A manifest listing every capability, its writable fields and its risk tier.
2. A deterministic gate that denies by default and checks the principal's own permissions.
3. Enforced approval for actions that leave the organisation or affect money, employment or sensitive personal data.
4. Per-capability kill switches.
5. An append-only log with a named person responsible for reviewing it.

None of these requires specialist infrastructure. They require deciding, in writing and in code, what the agent is allowed to do before it does it.

The Malaysian Regulatory Context

In October 2026, Malaysia's Personal Data Protection Department (JPDP) issued Public Consultation Paper No. 1/2026 on a proposed Artificial Intelligence and Personal Data Protection Framework under the Personal Data Protection Act 2010 (Act 709). The proposal addresses personal data across the AI system lifecycle, including development, testing, deployment, external AI services, data subject rights, security, fairness, risk management and compliance documentation. It already recognises different levels of autonomy, access controls, human oversight and the need to review material changes to AI systems.

In his industry feedback, Lukas raised the question of whether the level of autonomy and operational authority given to an AI system should become a more explicit factor in governance and risk assessment, while keeping expectations proportionate so that not every agent is automatically classed as high risk. The Framework is still a proposal, and this article is not legal advice. From an engineering standpoint, though, an architecture in which authority is declared, enforced, logged and versioned is far easier to explain and document than one in which authority is scattered across prompts and shared credentials.

Frequently Asked Questions

What is AI agent authorisation?

It is the set of controls that decides which actions an AI component may execute against real systems, on whose behalf, within what limits and with what audit evidence. It is separate from authentication and from controls on what data may be read.

Is a system prompt enough to stop an AI agent doing something harmful?

No. A system prompt guides a probabilistic model but enforces nothing. Restrictions belong in deterministic code: a capability manifest, a policy gate that denies by default, and credentials scoped so that forbidden actions are technically impossible.

Which AI agent actions should require human approval?

Actions that are hard to reverse or that reach beyond the organisation: outbound customer messages, changes to financial records, anything affecting employment, and processing of sensitive personal data. Low-consequence internal actions can usually run without approval if they are limited and logged.

How do you test an AI agent when its output is not deterministic?

Test the boundary rather than the wording. The policy gate is deterministic and can be covered with table-driven tests. Add sandbox contract tests for executors, adversarial scenarios that assert no out-of-manifest action runs, and replay tests for policy changes.

Does using a third-party foundation model move responsibility to the provider?

No. Responsibility is shared, and existing legal duties, including personal data protection, still apply. Enforce critical controls in the layer you own, verify what you reasonably can, and cover the rest through contracts and supplier oversight.

Do small businesses need a full AI governance platform?

Usually not. A manifest, a deny-by-default gate, enforced approval for high-consequence actions, per-capability kill switches and a reviewed append-only log cover most needs and can be built by a small team.

Conclusion

The engineering habits that make AI agents safe to deploy are not new. Least privilege, deny by default, explicit state machines, idempotent operations, append-only logs and change control have served software teams for a long time. What is new is the need to apply them to a component whose output is a proposal rather than a guaranteed result, and whose behaviour can shift when a model or a prompt changes.

Connecting AI to real business systems is where much of its value lies. The aim is not to avoid that integration but to make the authority it creates visible, deliberate and reversible. Teams that design the boundary first are often able to extend an agent's role with more confidence, because every extension arrives as a reviewable change rather than a surprise.

If you are planning to connect AI to systems that write, send or transact, you may find it useful to read one of our case studies, or to book a Discovery Session with the team to map your agent's capabilities and approval points before it goes live.

And before your next release, it may be worth asking one question of your own codebase: when the model proposes an action, which line of code decides whether it actually happens?