Which Business Decisions Should AI Be Allowed to Make?

A business may describe a proposed system as an “AI sales assistant” or an “AI operations agent”. Neither name explains what it should be allowed to decide.

Within sales, drafting a follow-up message differs from changing a quoted price. Within operations, suggesting a schedule differs from cancelling a customer’s booking. The same system may perform several tasks that deserve different levels of authority.

List the specific decisions and actions involved. For each, ask who currently has authority, what information they need and what happens if they are wrong. This reveals where AI can assist without taking over the underlying decision.

NIST’s AI Risk Management Framework calls for documented human and AI roles. In practical terms, “the team supervises the AI” is too vague. Name the role that can approve an action and the person responsible for keeping the policy current.

The policy should be understandable to the people using the system. If they cannot explain where its authority ends, an exception is likely to become an improvised decision at the point when clarity matters most.

The following model provides a working starting point. It should be applied to a particular task, not used as a blanket rating for a whole product.

Consider a customer refund. AI might summarise the order history, recommend eligibility, prepare the refund for approval or issue it within a tightly defined policy. These are different delegations, even if the interface looks similar.

A business may permit execution for a narrow, reversible administrative action while requiring approval for a modest financial action that affects another person. The amount involved is one factor, but it is not the only one. Reputation, privacy, fairness, contractual commitments and the difficulty of correction may matter more.

Make the transition between levels explicit. If evidence is incomplete, the system should move to a lower level of authority or stop for review. It should not fill the gap with an unsupported assumption.

This also makes supplier discussions clearer. Ask vendors to demonstrate where the boundaries are enforced. A policy described in a sales presentation needs to appear in the actual permissions, workflow and approval behaviour.

A useful decision review looks at three questions.

First, what could the action change? A draft summary changes little until someone relies on it. An update to a delivery address may redirect goods. An access change may expose information. Trace the effect beyond the immediate screen.

Second, how reversible is the action? Editing an internal note may be straightforward. Recalling a message already sent to a customer is not. Reversing a payment may require external cooperation. Restoring a deleted record may not restore the work that depended on it.

Third, how strong is the evidence? A request supported by consistent, current records differs from one based on conflicting documents or ambiguous language. A system can be confident in its wording while the underlying evidence remains weak.

Do not compress these questions into an unexplained score. Record the important reasons for the chosen boundary. For example: “Approval required because the action changes customer access and identity has not been verified.” That statement can guide implementation and review.

New Zealand’s NCSC and partner agencies warn that agentic AI risks extend across integrations and downstream actions. Review the whole chain of consequences when setting a limit. A harmless-looking update in one tool may trigger a significant action in another.

Where decisions affect employment, credit, healthcare or other regulated activities, establish the applicable local requirements and specialist review before deployment. A general delegation policy cannot settle those obligations across countries.

A human approval step can fail even when it is technically present. Reviewers may lack context, face too many requests or assume that the system has already checked the important facts.

Give the reviewer a concise decision package: the proposed action, the evidence used, the relevant policy, unresolved uncertainty and the consequence of approval. Keep the original material accessible so that a summary can be checked.

The action should remain pending until the required approval is recorded. Avoid ambiguous buttons that combine reviewing a recommendation with executing it. Where the consequence warrants it, separate the person preparing the action from the person authorising it.

AWS guidance on human review for critical AI decisions emphasises risk-based escalation and meaningful reviewer context. Translate that into the workload your business can support. If there are only two qualified reviewers, the design must account for their availability.

Test rejection as carefully as approval. Can the reviewer change the proposed action? Can they ask for more information? Does rejection produce a useful next step, or simply send the same request around again?

A review step is effective only when someone can understand the decision and reasonably choose a different outcome.

Also decide what happens when the approval expires. A time limit can trigger an escalation or cancellation. It should not quietly grant permission unless that behaviour has been deliberately assessed and authorised.

Keep a sample of completed decisions, escalations and rejected proposals for review, subject to appropriate privacy and retention controls. Look for recurring reasons why people disagree with the system. These may reveal weak instructions, missing information or a task that has been delegated too broadly.

Inspect near misses as well as completed errors. A reviewer who repeatedly catches the same wrong recommendation is maintaining the service through effort that should be visible. Treat that pattern as a reason to improve or restrict the system.

Test boundary conditions whenever policies, tools or models change. Include requests just inside and outside a limit, conflicting records and attempts to obtain an unauthorised action. Confirm that the system stops when it should, using the permissions of the real production account.

Provide a clear way to suspend execution while retaining useful read-only assistance. A business may need to pause automatic refunds without losing access to order summaries. That distinction helps people respond proportionately when a problem appears.

Finally, review whether the delegation still serves the business. A task that initially needed approval may become suitable for limited execution after sufficient evidence. Another may need stronger review as its consequences grow. Authority should change through a deliberate decision, not accumulate as new features are connected.

When commissioning AI and automation work, include these boundaries in the requirements and acceptance criteria. They define what the service is allowed to become.