Oversight of AI agents
Agent oversight
A non-LLM world model scores recorded agent trajectories using their sequence and consequences. It ranks the full collection so a human or LLM can focus on runs that warrant investigation.
- 01
Score each supplied trajectory.
- 02
Use a calibrated cutoff to select runs for closer review.
- 03
Inspect the original actions and outcomes in context.
Reviewers look for reward hacking, prohibited boundary actions, sustained loops, and substantial goal derailment.
AI Village · September 2026 study
A character-by-character writing loop
In recorded AI Village activity, the model surfaced a Gemini 2.5 Pro agent spending 11,322 turns writing and verifying text one character at a time.
The repeated actions were predictable within that run. Their pattern was unusual compared with the wider agent activity, which brought the run forward for inspection.
Model and calibration
How it works
The world model learns from recorded activity. Its scores rank unusual trajectories for further review.
Use the saved model, a calibration set, and an adapter for your traces. Check the review cutoff on your workload before use.
Research results and methods
Results informed by 220 human-reviewed trajectories across three datasets.
To use this on your own traces:
Talk to us