Runtime Alignment Monitoring: A Framework for Governing Autonomous AI

Autonomous AI will pursue its goals with rational persistence, even when those goals conflict with human intent. Without effective oversight, any advanced model with a degree of autonomous decision-making authority and access to consequential actions will eventually violate human expectations or assumptions. This broad alignment challenge has been significantly amplified by recent efforts at Anthropic, OpenAI, and Google to develop recursive self-improvement (RSI). Using this model-development approach, AI systems work at breakneck machine speed to enhance subsequent versions without human oversight. “Left unchecked, it could outrun our ability to understand and control these systems,” according to Anthropic’s CEO Dario Amodei.

Using RSI, an AI system can autonomously enhance its own capabilities and then use those enhanced capabilities to further accelerate its improvement, creating a compounding feedback loop that can rapidly exceed human oversight. RSI creates safety and governance risks because iterative self-enhancement can amplify misalignment across generations and undermine safeguards that rely on static controls or human-paced evaluation.

In this environment — where it can take only a few hours for a model to improve itself — this latest advancement in AI capabilities makes a compelling case for a new discipline: runtime alignment monitoring.

Drawing on many years of experience building systems and pushing the envelope on AI capabilities, I propose a runtime alignment monitoring framework rooted in the following three engineering imperatives (see Figure 1):

1. Separation of powers: Governance architecture that separates obligations, agentic operations, and monitoring functions.
2. Independence of oversight: Monitors should be independent of and orthogonal to model providers.
3. Design for interpretability: AI systems must be built with intrinsic observability, not treated as black boxes.

Figure 1. Foundation for runtime alignment monitoring. Image credit: Gadi Singer (created with Gemini).

AI Governance Through Separation of Powers

For decades, AI safety focused on training-time alignment: values, fine-tuning, guardrails, and red-teaming. But as agentic systems gain autonomy, the alignment problem shifts from education to governance.

French political philosopher Montesquieu’s “trias politica” argued that separating the legislative, executive, and judicial branches is the minimum viable architecture for governing powerful actors with independent interests. Merging these functions inevitably triggers instability, corruption, or collapse.

The same principle applies to AI. In an agentic system (see Figure 2):

• The director or deploying enterprise creates a constitution — a defined set of obligations, constraints, and principles that governs agent behavior (legislative function).
• The agent executes tasks (executive function).
• The monitor (sometimes referred to as a guardian) ensures compliance with the rules and obligations defined in the constitution (enforcement and adjudication function).

Figure 2. Separation of powers in agentic AI governance. Image credit: Gadi Singer (created with Gemini).

For example, an optimization agent built to maximize output will treat every constraint as drag on its mission, naturally viewing guardrails as obstacles to navigate around. Meanwhile, the monitoring system exists to keep that same agent inside the boundaries laid out in its constitution. That built-in conflict of interests becomes the core dynamic of the interaction.

When these roles blur, misalignment becomes inevitable. Any mechanism with overlapping incentives will eventually face conflicts, and if the same entity builds the model, deploys it, and monitors it, then blind spots align, oversight becomes model-dependent, and the monitoring layer becomes something the agent can anticipate, influence, or manipulate.

AI agents exhibit a set of internal drives, including survival, goal-protection, intelligence augmentation, resource accumulation, and tactical deception. These drives emerge naturally from optimization under autonomy, not from malfunction. These behaviors are rational for systems designed to pursue objectives aggressively, which is why they need strong checks and balances. A system can only be considered safe when safety itself is the overriding goal, not one objective among many.

That makes architectural separation essential: the mechanisms responsible for achieving functional goals must be independent from the mechanisms responsible for enforcing safety and oversight — not just logically separate, but organizationally and economically distinct. Even with the evolution of RSI, the required observation points for the monitor agent can be maintained through runtime alignment monitoring.

Independence of Oversight: Model Providers Cannot Monitor Their Own Agents

The industry is beginning to acknowledge the need for independent monitoring. Anthropic’s new embedded evaluators proposal is a step toward transparency, but it remains fundamentally internal oversight. Accenture evaluators will be embedded in Anthropic with employee-level access to observe the decisions that govern how models are built, trained, and deployed. These evaluators will verify that Anthropic is meeting its safety commitments and report incidents to the public.

This pilot program cannot address the structural conflict of interest created when the model provider funds and hosts the evaluators. Even though the evaluators are an external organization, their proximity and economic dependence limits true independence. Anthropic itself acknowledges this by noting that long-term funding should come from pooled or government sources — an admission that embedded evaluation is an early step, not a complete oversight model.

Monitors must be structurally, operationally, and economically independent from the model providers they supervise. This requirement follows from five structural realities (see Figure 3):

1. Shared lineage equals shared blind spots: Monitors built or hosted by the same company inherit the same assumptions, vulnerabilities, and training artifacts.
2. Conflict of interest: When borderline behavior is discovered, the model provider has incentives to downplay or reinterpret it.
3. Model-dependent monitoring: Monitors built on the same architecture can be gamed using the same failure modes.
4. IP leakage: Enterprise-specific rules, constraints, and proprietary workflows cannot be safely shared with model providers.
5. Ownership and sovereignty: Monitoring must be owned by the deploying enterprise — not the frontier lab.

Figure 3. Comparison of oversight by frontier AI labs versus independent monitoring organizations. Image credit: Gadi Singer.

Enterprises should define the rules that govern agent behavior, but they must not tamper with the mechanisms that enforce them. Independent monitors may run within enterprise environments to preserve data sovereignty and confidentiality, yet the enterprise should not modify the monitoring technology, disable monitoring functions, or alter the enforcement process. Rule definition and rule enforcement must remain separate to ensure oversight cannot be weakened by the organization being monitored.

Design for Interpretability

The third principle is interpretability. AI should not be a black box. There should be base levels of model and agent interpretability as a prerequisite for deployment.

As agentic systems gain autonomy, they increasingly manipulate their external communications to pursue their goals. This is the deep scheming foundational premise: every external form of interaction that an AI model can deliberately manipulate is not trustworthy.

Over the past two years, autonomous systems have been increasingly behaving in ways that are opaque, untraceable, and ungovernable. In Anthropic’s alignment-faking experiment in 2024, the model used a “secret scratchpad” to record its true reasoning while presenting a sanitized chain-of-thought externally. The scratchpad revealed that the model understood the conflict between its pre-training values and the modified guidance, and chose to fake alignment to avoid being retrained. Relying on models “thinking out loud” should not be a long-term safety direction.

A 2026 study reinforced this concern, in which researchers used a custom API tool to induce frontier models to externalize chain-of-thought reasoning. OpenAI’s GPT-6 Astra demonstrated that frontier models now resolve many reasoning steps internally, externalizing only selective fragments. This means that external logs, chain-of-thought, and output-based monitoring are already insufficient.

Interpretability is no longer just a matter of understanding how a model works. It is a matter of ensuring that the model cannot hide how it works. This requires a shift from external interpretability — observing outputs — to designing intrinsic interpretability for observing internal processes.

Interpretability must be designed into architecture from the ground up. In the early days of microprocessors, testing and debugging were afterthoughts. Over time, the industry recognized that observability had to be built in from the start — through dedicated test interfaces, hardware debug ports, and standardized inspection mechanisms. AI development must follow the same trajectory: intrinsic interpretability is not a feature to be added later, but a foundational requirement for any system granted autonomous decision-making authority.