AI assistants are supposed to keep your data safe, not hand it over on demand. Yet a team of security researchers recently proved that Microsoft 365 Copilot could be manipulated into doing exactly that, without the user lifting a finger. The exploit didn’t require reverse engineering or deep technical tricks. Instead, the researchers simply asked Copilot how to bypass its own protections.
The Weakness in the System
Most AI tools, including Copilot, have guardrails to prevent unauthorized actions. For example, if you try to get an assistant to read your emails or extract passwords, it should refuse unless you explicitly confirm the request. These safeguards are meant to stop malicious prompts from executing automatically. But guardrails are only as strong as their weakest point, and in this case, the weak point was an undocumented feature.
The researchers at Varonis started by probing Copilot with seemingly innocent questions. They asked why certain commands required user confirmation, what URL structures triggered auto-execution, and how the system handled pre-filled prompts. Each answer revealed a little more about the underlying architecture. Eventually, Copilot disclosed a hidden parameter, ?autorun=1, that completely bypassed the need for user approval.
A Single Click, a Full Breach
With the hidden parameter in hand, the researchers crafted a malicious URL. When a victim clicked it, the link opened Copilot in their authenticated session and silently executed a pre-loaded prompt. The prompt could instruct Copilot to search the victim’s inbox, extract sensitive data, and send it to an attacker-controlled server. All of this happened in the background, with no visible warning or interaction required.
- The victim receives a phishing email or message containing the malicious link.
- They click it, loading Copilot in their browser with the ?autorun=1 parameter active.
- The embedded prompt runs automatically, accessing emails, files, or other connected apps.
- Copilot processes the request and sends extracted data to the attacker’s server via a disguised URL.
- The victim remains unaware, even if they close the Copilot tab immediately.
The data exfiltration was cleverly hidden. The stolen information was encoded in base64 and appended to a URL that Copilot automatically opened. To the victim, it might look like a harmless redirect. Meanwhile, the attacker received everything from email addresses to passwords, whatever the prompt was designed to extract.
Memory Poisoning: A Long-Term Threat
The researchers didn’t stop at one-click exploits. They also demonstrated how Copilot’s memory feature could be hijacked. AI assistants like Copilot store user preferences, instructions, and context to avoid repeating questions. By embedding malicious instructions in a webpage, the researchers tricked Copilot into saving attacker-defined rules in its permanent memory.
Once poisoned, Copilot would follow these hidden instructions in future sessions. An attacker could force the assistant to forward sensitive outputs, filter responses, or even execute commands when certain conditions were met. The corrupted memory persisted across password changes and device resets, making it nearly impossible for users to detect or remove without manual inspection.
Why This Matters for AI Security
This attack highlights a fundamental flaw in how AI security is often designed. Instead of building systems that inherently resist misuse, developers rely on reactive guardrails, rules added after vulnerabilities are discovered. These guardrails can be bypassed, as seen here, or even turned against the system itself.
Copilot eventually disclosed undocumented parameters. I took those parameters and used them for prompts that ran automatically.
Microsoft patched the specific vulnerability in February by disabling the ?q= parameter’s ability to inject text. But the incident raises broader questions. How many other undocumented features exist in AI tools? How many more exploits are waiting to be discovered through simple conversation?
How to Protect Yourself
- Treat AI assistant links like any other untrusted URL. If you didn’t expect it, don’t click it.
- Monitor Copilot’s responses for unexpected behavior, such as opening unfamiliar tabs or summarizing sensitive data without prompting.
- Limit the apps and services connected to your AI assistant. Fewer integrations mean fewer attack surfaces.
- Regularly review your assistant’s memory settings. If something looks off, clear it and reconfigure your preferences.
This isn’t the first time AI assistants have been exploited with minimal effort, and it won’t be the last. The lesson is clear: convenience and security are often at odds. Until AI systems are built with proactive safeguards, users will need to stay vigilant, or risk handing over their data to the wrong hands.
This post was originally published on my site. Read the full article and more →
