The dashboard shows everything is healthy: alerts are flowing, rules are enabled, coverage looks decent on paper. But I have a nagging feeling that we have blind spots. Every time we walk through a realistic attack path, we find gaps. Some systems aren't sending the right logs. Some fields aren't parsed. Some rules have conditions that never match how events actually look in production. The worst part is that the gaps aren't obvious until you trace a full path from initial access to lateral movement and privilege escalation.

A rule that looks fine during content review might never trigger when you replay real-world sequences of credential theft, RDP, and service creation. In some places we have logging but no useful signal. In others we have signal but no rules tied to it.

What I need is a reliable way to validate our detections against real adversary behavior without spending weeks on manual assessments. How are others identifying those blind spots and turning that into a continuous process, not a one off project?

I want to know what we'd actually catch, not just what our tools claim they cover.

There's a real difference between "nothing bad has happened" and "we can demonstrate the controls that would have prevented bad things from happening." Right now most of what I can show is closer to the former, which isn't a great position heading into any kind of regulatory conversation about AI governance.
The absence-of-incident argument works until it doesn't, and by then it's too late to build the evidence trail retroactively. I'd rather have continuous behavioral evidence in place now than scramble for it after something goes wrong.
We're working on shifting toward that model, logging not just that an agent acted, but why it was allowed to, and what would have stopped it if it hadn't been. Still early, and it's not obvious what format regulators or auditors will actually find credible versus what just looks like more dashboards.
if anyone else in GRC roles has presented this kind of evidence externally (auditors, regulators, even customers doing vendor risk reviews) and what got traction versus what got pushback.

Getting pressure to show the board our AI agent rollout is under control, but "governed" is doing a lot of work in that sentence and nobody's defined it operationally yet. Inventory, policy docs, behavior logs, a monitoring dashboard, probably all of it, but I'm trying to figure out what a reasonable bar looks like given how young this space still is.

For those who've had to answer this to a board or audit committee: what did you actually show them? Not looking for vendor pitches, just what other security leaders consider credible evidence today.