The controls that get AI past risk review

Two connected layers. Governance sets the rules: what AI may be used for, who approves it, what gets logged, and how decisions are recorded for audit. AgentOps makes those rules operational through evaluation harnesses, tracing, versioning, guardrails, cost budgets and incident response. A policy document without the runtime controls underneath it does not survive an audit, and controls without a policy have nothing to enforce.

A policy exercise produces a document. We build the mechanisms that make the document true: the test set a release must clear, the trace that reconstructs what an agent did, the budget that stops a runaway loop, the log an auditor can read. We will map the controls to ISO 42001 or the EU AI Act where you need it, but the deliverable is working instrumentation, not a binder.

An evaluation harness and tracing for a single system takes four to six weeks. A control framework and instrumentation across an AI portfolio runs ten to sixteen weeks depending on how many systems are already live. An audit readiness review against a named framework can be done in three weeks. Ongoing AgentOps runs as a monthly retainer after the build.

No. The harness, traces, dashboards and runbooks are yours, built on tooling you own and documented for your platform team to operate. Some clients take it in house immediately, others keep us on an AgentOps retainer for the first two or three quarters while the practice settles. Either path is priced separately and neither is a condition of the build.

With an inventory and a risk sort, not a rebuild. We catalogue what is running, what each system can reach and change, and what is currently observable, then rank by exposure. High-risk systems get tracing and guardrails first, low-risk ones get scheduled. Retrofitting controls onto live systems is normal work and rarely requires taking anything down.

By whether it catches things. We track regression rate caught before release versus after, trace coverage across agent actions, mean time to reconstruct an incident, spend against policy budgets, and the proportion of systems meeting their approval gate. Baselines are set at the start, so improvement is visible rather than asserted at the next audit.