In 2023, the Biden Administration convinced seven of the top AI companies to
put a focus on security, safety, and trust when developing their products, including allowing third-party researchers to run external tests before releasing them.
However, in a report last year, Stanford University’s
Human-Centered Artificial Intelligence (HAI) research institute wrote that the implementation of the commitments made by
Microsoft,
Amazon,
Meta,
OpenAI,
Google,
Anthropic, and
Inflection “
has been uneven.” Some vendors reward researchers for finding security flaws in their AI systems, though few encourage it or provide “concrete protections for good-faith research practices,” the institute wrote.
For its part, Salesforce issued a white paper in 2024 that executives said
detailed the SaaS vendor’s “significant progress toward these commitments and reinforces our dedication to designing, developing, and using generative AI with ethics at the core.”
However, for the AI field as a whole, those protections – as are common for such research conducted on traditional software – are important to independent security researchers to ensure they don’t face legal action or are banned from platforms for trying to find security issues in AI systems. For AI companies, the work of such researchers is crucial.
“Independent evaluation and red teaming are critical for identifying the risks posed by generative AI systems,” according to a
research paper by
MIT. “However, the terms of service and enforcement strategies used by prominent AI companies to deter model misuse have disincentives on good faith safety evaluations.”
HackerOne Steps Up
Threat exposure management firm
HackerOne is looking to make it safer for external security researchers to test AI systems. The San Francisco-based company this month unveiled its
Good Faith AI Research Safe Harbor, an industry framework designed to provide clear authorization and legal protections for researchers who test AI systems in good faith.
The company argued that testers often have to use techniques and outcomes for AI technologies that aren’t found in traditional vulnerability disclosure frameworks. That can create legal ambiguity for red teamers and slow down the discovery of security risks.
The goal of the Good Faith AI Research Safe Harbor is to have a standardized process that allows for the authorization of responsible AI testing.
Reducing the Legal Risks for Testers
“AI testing breaks down when expectations are unclear,”
Ilona Cohen, chief legal and policy officer at HackerOne,
said in a statement. “Organizations want their AI systems tested, but researchers need confidence that doing the right thing won’t put them at risk.”
The new framework comes almost four years after HackerOne released its
Gold Standard Safe Harbor for traditional software, which the company said has been widely adopted in the industry.
HackerOne customers can use the AI safe harbor along with the Gold Standard Safe Harbor to let testers know that they welcome the external research into the security of the AI products.
Opening Opportunities for MSSPs, MSPs
The vendor’s AI safe harbor will be important to AI companies, organizations that use the systems, and MSSPs and MSPs alike, according to
Rob Enderle, principal analyst with
The Enderle Group, who noted that traditional safe harbors don’t cut it in the AI age.
“Traditional frameworks focus on binary vulnerabilities – a bug is a bug,” Enderle told MSSP Alert. “With AI, a vulnerability could be an unintended bias, a privacy leak, or a method to bypass a safety feature. These often don’t fit into a standard framework. ... This framework provides a legal shield from the company providing the AI protecting the researcher who stays within the framework.”
For MSSPs and MSPs that offer security services, such safe harbors are a way to manage risks, which creates a revenue opportunity. If they adhere to the HackerOne framework, it will shield service providers and their staff from much of the related AI liability if something goes wrong, he said.
It also opens up possibilities for MSSPs to offer formal services like AI red teaming or constant AI exposure management.
It creates “the opportunity to provide a certification that monetizes for AI deployments that were managed within the framework,” Enderle said. “They can also point to this framework to better comply with regulations like the EU AI Act, emphasizing the secure-by-design aspect of this framework. For MSPs that need to test agentic AIs without triggering legal alarms, this framework provides a path to rigorous testing while mitigating the risk that the AI agent being developed will go rogue, doing serious damage.”
