Least-authority design

How much authority should an AI agent have?

Give an AI system the least authority sufficient for the decision. Authority should increase only when the evidence, reversibility, action scope, approval path, trace, outcomes, and stop controls justify it.

Participation and authority ladder

  1. Analysis only

    Assemble or interpret evidence without choosing an alternative.

  2. Recommendation

    Evaluate alternatives and recommend or abstain; no action is authorized.

  3. Proposal

    Create a reviewable action proposal while the accountable person decides.

  4. Human-approved action

    Execute only after a recorded person approves the specific action at action time.

  5. Tightly policy-bound action

    Execute only named, bounded, reversible actions under pre-authorized policy, with trace, exception, and stop controls.

Moving upward is not a reward. A lower rung may be the mature choice for high-consequence or weakly attributable decisions.

An authority grant is a complete tuple

A defensible authority contract states: named action + target scope + conditions + limits + approval + decision trace + exception path + stop/revocation. Missing elements remain unknown; they are not silently supplied by a prompt, role credential, or tool permission.

Permitted action

The exact consequential effect—not “use tools” or “assist.”

Scope and limits

Objects, channels, value, frequency, duration, geography, and affected parties.

Approval

Action-time person or explicit standing policy, never an ambiguous generic consent.

Trace

Decision, evidence, recommendation, authorization, execution, and outcome remain separate.

Exception

Who handles missing evidence, conflicts, out-of-policy proposals, or execution ambiguity.

Stop and revocation

How authority is suspended, narrowed, rolled back, or withdrawn.

Consequence determines the control burden

Reversibility and blast radius

Hard-to-reverse actions, large affected populations, cascading operational effects, and rights or safety impact call for lower default authority, stronger evidence thresholds, and qualified domain review.

Evidence threshold

Action should fail closed when mandatory evidence is stale, missing, contradictory, outside effective time, or insufficiently validated. Uncertainty should change the permitted mode—not be hidden behind confidence prose.

Accountability

A named person or accountable role remains responsible for policy, exceptions, and outcome review. Human-in-the-loop is meaningful only when the person has enough evidence, time, and power to challenge the proposal before effect.

The operating control system

  • Abstention: the system can say no result is supportable.
  • Escalation: exceptions route to an appropriate role rather than a generic queue.
  • Override: an authorized person can reject or alter the recommendation or action.
  • Action-time checks: evidence freshness, policy, limits, duplicate prevention, and approval are rechecked immediately before effect.
  • Revocation: standing authority can be removed without retraining or redeploying the whole system.
  • Recovery: ambiguous or failed execution is reconciled before retry, with idempotency and rollback where possible.

Authority expansion requires outcome evidence

Process speed or successful API calls do not justify more authority. Expansion should require predeclared expected outcomes, an appropriate comparator, observed downstream results, a candid attribution posture, incidents and overrides, and accountable approval of the new policy. Learning may propose a policy change; it must not silently grant itself more authority.

Assess participation and authority for your decision