Filtered by: AI & Agents × clear
A managed evaluation service can take over the judge, but not the contract tests, the drift canary, or anything needing provenance for the judge itself. Testing one produced four faults that each returned a plausible number instead of an error.
Per-message filters like Model Armor correctly pass most of a multi-turn probe. Adding the conversation tier that catches it, and the alerting fallback that turned out to matter more.
It takes ten minutes to generate a research report, a deck, or a recommendation list now, and the ability to explain any of it is what is quietly slipping. The fix I landed on is to have the AI turn around and interview me.
Seven or eight layers all answering some version of "is this allowed". Two questions sort almost all of it: where does each one bind, and who can actually change it.
FinChat already had five layers of policy, and every one of them evaluates an action that is already happening. None could see a change that does not exist yet. What OPA is, and how I filled that gap.
Model Armor blocks a prompt and tells the user, and on most builds that is where it stops. Here is the full path from a guardrail violation to a correlated ServiceNow incident on Google Cloud, with screenshots from the running system.
SR 26-2 punted generative AI out of model risk scope and told you to govern it anyway. The FINOS AI Governance Framework is a free catalogue of 46 risks and controls, with the regulatory mapping already done.
I was challenged to work out what it would take to think of agents as colleagues. Taken seriously it becomes an operating model: identity, scoped access, governed onboarding context, supervision, evaluation, deprovisioning. Here's what worked when I built that, and the three places it broke.
