COMMENTARY: AI definitely beats any headcount increase. But the bigger question to ask is how it is making decisions. AI is starting to classify data, flag records for retention, support legal hold decisions, and surface compliance issues across client environments. But a log showing the final action is not enough anymore. Customers and auditors will want to know what signals were used, what policy was applied, who reviewed the decision, and where human judgment stayed in control. That is where AI governance becomes a real managed service issue, not just another policy conversation.
More than two decades have passed since I started working in this industry, but "automation" has meant a slightly different thing at every stage of its evolution. For quite some time, it was pretty simple: fire off a rule, open a ticket, let a person decide. I've spent years running large-scale security operations, including Optiv's managed security services operation, where hundreds of analysts had access to the tooling capable of collecting all sorts of data, and humans did the analysis. It was a clear division of responsibility and accountability: when the phone rang, a person made a decision.
Now, the boundary has shifted, and very few organizations in the channel have noticed it. In large part, it's due to the nature of the change: invisible, subtle, and not directly related to the core business. While it's understandable, the problem is that such a fundamental change won't be addressed by an auditor or by a compliance incident until it's too late. New systems make decisions, and it's not just that; the current systems can decide whether a piece of data falls under legal hold, whether it needs to be quarantined, and how to properly manage its retention period. As for an MSP, this is truly great news; it's impossible for a team of analysts to manage all these nuances manually in forty clients' environments. From a productivity perspective, AI beats any headcount increase I can recall approving in the past.
Here's the question I ask everyone I advise now, and nine times out of ten, it leads to a long pause - when your tool made this decision last Tuesday, can you show me how it made it?
Reasoning is missing in your logs
Most of the organizations focus only on the final action that has been performed. While it's possible to track the reclassification of a document or place a mailbox on legal hold, there is almost never any logging for the reasoning behind these actions. Most of the tools simply don't log what factors were considered in a decision, whether it could have turned out differently if another parameter had been changed, or even whether there might be a possibility that an algorithm got it wrong. The action leaves a footprint, while the decision-making process typically doesn't.
This is precisely what regulators want to audit for now. According to NIST AI RMF, the ability to trace and understand decisions your algorithms make is not a bonus, it's a requirement. If your organization uses AI in its compliance work, traceability and accountability will be among the things it will be assessed against. And here's the problem I've faced myself: when an algorithm starts making decisions that are consistently good, people stop questioning them, and that's when trouble begins. A decision made with confidence is the most dangerous thing a system can make, as confidence is something most of us instinctively trust. I have personally seen highly skilled analysts agreeing to decisions made by an algorithm simply because they sounded convincing. My advice for everyone I consult is always the same: AI can make decisions with confidence, but only people can assume responsibility for them.
Not everything should be automated
There are plenty of tasks where automation makes perfect sense. Managing classification and retention of records is definitely the one where an algorithm wins hands down every time. It's high-volume, rule-based, and repetitive. This is something these algorithms do best.
It requires discipline, though, in distinguishing between routine work and human judgment. Reclassification of documents is work, an automated process. Deciding that a certain category of records doesn't need to be retained anymore is a judgment – it's potentially risky, but a human has to make it. The same logic applies to monitoring tasks: while embedding AI into your continuous governance processes gives you much more visibility, it's still monitoring. Automated and real-time visibility is great, but it's still visibility; it's not actionable unless everything is recorded for review.
The channel faces a risk, too. Modern algorithms become capable of taking actions that can be devastating, including data destruction. Autonomous agents have already caused significant damage to organizations simply by deleting sensitive data. Controls here are pretty trivial, though – make sure to classify any action that can cause destruction, limit its scope, and ensure a person has to approve it.
Keeping a human in the loop is far from being nostalgic. When it comes to highly risky tasks with potential compliance implications, it's a vital step to prevent disaster.
A framework isn’t evidence
If your organization decides to align its use cases with NIST AI RMF, go for it. Any effort to bring a bit of structure to an otherwise implicit process can only help. Yet, I have been sitting through countless meetings when a beautiful control matrix fell apart after an auditor started asking who actually reviews all those exceptions, and how often. These days, auditors know how the game works: telling them you are compliant with NIST is meaningless unless you can prove it.
A framework helps you shape your thinking and decisions, nothing more.
So, this is what I would like to tell the channel. The potential of modern algorithms is great, and MSPs using both AI and true governance discipline will beat other MSPs trying to rely purely on their teams of analysts. Yet, it's time to recognize that the bar has been raised: your algorithm doesn't just have to be able to make the right decision. It must be capable of explaining how it arrived at that decision – and what decisions must be made by people, despite being based on a huge amount of data and algorithms' impressive capability to analyze it.
And this scrutiny is coming soon.
MSSP Alert Perspectives columns are written by trusted members of the managed security services, value-added reseller and solution provider channels or MSSP Alert's staff. Do you have a unique perspective you want to share? Check out our guidelines here and send a pitch to [email protected].
