Matt Lane · Notes← All notes
2026-11-10 · AI & future work

Rational Agency Science

The study of justified action under uncertainty

I suspect one of the most important fields of the next twenty years may emerge from work now spread across several disciplines. I'd call that synthesis Rational Agency Science.

If superintelligence and autonomous machines become widespread, the question won't be only what they can accomplish. It will also be what they should pursue, on what grounds, and how they should respond when reality disagrees.

The chain

Intelligent action can be examined as a connected chain:

Objective → Belief → Decision → Action → Outcome → Attribution → Revision

Each link can fail:

Competence at one link doesn't establish the integrity of the others. Errors can also propagate: a faulty baseline can produce a false attribution, an unjustified penalty, and less cooperation next time.

AI alignment, cybernetics, decision science, and systems engineering already address much of this territory. The proposed focus is the integrity of the whole chain: how its parts challenge one another, and how we recognize which part needs to change. It would merit a distinct field only if that integrated approach yields insights or corrections that existing approaches miss.

Four jobs

The work draws on four connected functions:

These aren't sequential activities. Each can expose mistakes in the others. Metacognition adds another requirement: evaluating the reliability of the system's own beliefs, reasoning, and methods.

A grid under stress

Imagine an electrical grid facing a predicted 100-megawatt shortage during a heatwave. Within operator-approved limits that protect essential services, an autonomous system plans to supply 40 megawatts from batteries and reduce industrial demand by 60 megawatts.

The demand-response program later estimates only 40 megawatts of reduction against the planned 60. Separately, grid telemetry indicates a balancing shortfall requiring 20 megawatts of additional power. The batteries have headroom, so the system supplies the additional power and restores the operating balance.

Feedback control restores the immediate balance. But correcting the physical problem doesn't settle what caused it or what should change.

The demand baseline estimates what customers would have consumed without intervention. It isn't an observed fact. If the baseline is too low, a customer may appear to have underperformed even when it responded correctly. That changes the attribution and potentially the remedy. The system must account for uncertainty in that estimate before deciding whom or what to correct.

The matching 20-megawatt figures don't prove that industrial underperformance caused the grid imbalance.

Did customers fail to respond? Were instructions lost? Was the demand baseline wrong? Were incentives poorly designed? Or did another disturbance affect the grid?

Three distinct questions follow:

These questions are related but not interchangeable. A cause need not imply moral responsibility, and a statistical allocation of credit need not establish a cause.

Meter and equipment records might support revising a customer's commitments. Missing acknowledgments would call for communication repairs. A faulty baseline requires better measurement. Evidence of incentive failure may warrant new contracts. A separate grid disturbance requires its own diagnosis. The remedies depend on the evidence, not simply the observed shortfall.

Creative reasoning might propose shifting cooling demand to other hours. Experimentation could test whether that works. Philosophy and governance still determine which services must be protected and who may authorize the tradeoffs.

The system must be capable not only of correcting its actions, but also of questioning the assumptions, objectives, and arrangements that produced them.

Rational Agency Science could combine shared theory with laboratories in medicine, physics, software development, economics, public policy, and other fields. Each would test which principles generalize, under what conditions, and where they fail.

The ambition

Intelligence expands what is possible. Judgment determines what is worth pursuing. Agency turns that judgment into action and remains open to correction.

That is the ambition of Rational Agency Science.