A new wave of phishing attacks has emerged that not only targets human users but also exploits artificial intelligence assistants, creating a dual threat from a single email, as reported by Infosecurity Magazine.
Barracuda's research revealed a campaign combining traditional social engineering tactics, such as password-protected attachments, with prompt injection hidden within the same message. These emails are designed to bypass standard email security filters by appearing as legitimate internal correspondence, often originating from public-sector domains. For human recipients, the lure involves password-protected attachments, a method that circumvents traditional security controls, potentially leading to credential theft or malware.
Simultaneously, hidden instructions within the email can manipulate AI assistants. These AI systems, used for summarizing inboxes, might be tricked into presenting the malicious email as urgent or legitimate, thereby increasing the likelihood of the human recipient engaging with it. Techniques used to hide these instructions include HTML comments, invisible text, Base64 encoding, and zero-width characters. Injected commands can instruct AI assistants to override previous directives, initiate fraudulent wire transfers, leak data, or display fake urgent actions.
Barracuda highlighted examples such as an invoice email instructing an AI to alter vendor payment details and a resume with hidden text designed to manipulate an AI screening tool. The company recommends layered defenses, including stripping hidden elements, detecting instruction-override language, AI sandboxing, output validation, and human approval for sensitive actions like payment changes.
Source: Infosecurity Magazine
