AI is becoming a bigger part of how corporate security teams and MSSPs operate, but it will deliver the best results when paired with human expertise.
That’s what
Hack The Box found when it tested agentic AI-augmented units and human-only teams on three dozen security challenges. Humans and AI agents working together were able to solve problems and complete tasks much faster than human-only teams, and increased performance improvements when highly skilled security pros were involved.
“For security teams and vendors, the takeaway is that AI is most effective when it is embedded inside human-led workflows,”
Gerasimos Marketos, Hack The Box’s chief product officer, told MSSP Alert. “It accelerates investigation, pattern recognition, and analysis, but human expertise remains essential for judgment, verification, and complex problem-solving.”
Marketos added that “organizations that treat AI adoption as a headcount substitution strategy rather than a workflow redesign will miss the actual advantage the data demonstrates.”
The AI Arms Race
Enterprises and security services providers are going to need AI in their toolbox going forward, because the adversaries they’re defending against are going to
use it in their malicious operations. According to
CrowdStrike’s
2026 Global Threat Report, there was an 89% year-over-year increase in attacks by threat actors armed with AI.
Security pros' use of AI tools is also growing as cybersecurity vendors rapidly integrate the technology into their offerings.
“AI can improve an organization’s detection, triage, and response, as well as improve a security team’s fundamental operations in identity security, vulnerability management, user reporting, and response readiness,” researchers with
Arctic Wolf wrote earlier this month. “While the AI arms race is currently raging, it’s clear that the war won’t be won by who 'uses AI.' It will be decided by who turns AI into measurable outcomes.”
Bugcrowd, in January, noted that ethical “white hat” hackers are increasingly using AI to
augment their work in finding vulnerabilities in networks and other systems so that they can be fixed.
Humans in the Mix
To be most effective against attacks,
AI agents will need to work with humans, Hack The Box found in its
AI-Augmented vs. Human-Only Cybersecurity Benchmark Report. Using its
NeuroGrid Capture The Flag competition, the vendor analyzed data from 1,078 teams that included 120 agent AI and 958 human teams across 36 challenges that spanned nine technical domains and four difficulty levels.
When humans and AI agents worked together, teams finished tasks significantly faster - 4.1 times faster when highly skilled humans were involved, and 1.4 times faster across all teams overall. The success rate also increased by 70%, rising from 16% for the best human-only teams to 27% when agents were part of the workflow.
“The most important takeaway is that these gains were not automatic,” Marketos said. “The strongest performance came from teams where experienced practitioners remained responsible for directing the AI, validating results, and making the final decisions.”
Hack The Box will offer a deeper analysis of the research at the
RSAC 2026 show in San Francisco on March 26.
Benefits for Mid-Level Security Pros
One of the more important findings was that the largest performance gains happened with medium-complexity challenges, where AI-augmented teams saw nearly 3.89-times higher solve-rate ratios. This is the area where many security analysts develop their skills, such as investigative instincts, correlating signals, identifying attacker behavior, and going from alerts to an understanding of incidents.
“In other words, AI performs particularly well in the same operational tier that historically serves as the training ground for mid-level practitioners,” he said.
This creates both opportunity and risk. AI can help mid-career analysts move more quickly, handle heavier investigation workloads, and get exposure to more varied threat scenarios in less time, Marketos said.
“The structural risk is what the benchmark identifies as the ‘missing middle,’” he said. “If organizations automate the entry-level work that trains junior practitioners without deliberately redesigning how analysts develop their skills, they cut the pipeline that produces future senior talent. The productivity numbers look strong in the short-term while the capability foundation erodes underneath.”
AI and MSSPs
This is true not only for enterprises, but MSSPs as well, because they operate high-volume security environments that put a premium on speed and scale. Both of those directly affect service quality, Marketos said.
“AI-augmented teams change the economics of SOC operations by allowing analysts to process more alerts, investigations, and customer environments without scaling headcount linearly,” he said, pointing to the 27% challenge-solve rate for augmented teams vs. 16% for those with only humans. “This highlights how AI can accelerate investigation and problem-solving when integrated effectively into analyst workflows. Rather than replacing analysts, AI acts as a force multiplier that increases the throughput of security teams.”
Assigning Tasks
The teams that were most effective used AI for such structured tasks as recognizing patterns, triaging alerts, and generating hypotheses. Analysts dealt with validating findings, interpreting context, and making investigative decisions.
“For MSSPs, the practical implication is to design SOC workflows with AI in the loop so that automation handles repetitive analytical tasks while humans focus on judgment and escalation,” he said. “The competitive advantage for MSSPs will come from strong hybrid teams, where AI increases speed and analysts provide the expertise and judgment that customers ultimately depend on.”
