Gmail AI Assistant: Convenient But Potentially Compromised

For many people worldwide, Using Gmail has become as routine as brushing their teeth. With the latest Gemini AI integrated into Google Workspace, managing emails, slides, and documents has become much easier. This convenience, though, comes with some significant security risks, including indirect prompt injections and the “link trap.”

Indirect prompt injection is like a digital magic trick. Someone can send an email or share a document that seems harmless but includes hidden AI commands. When Gemini AI reads it, it might do things you didn’t intend, like giving away data or responding in misleading ways. This isn’t just a Gmail issue; it’s a risk across Google’s suite, affecting Google Slides and Google Drive, too. It’s alarming because the attacker doesn’t need to directly interact with the system; just sending the right email can cause chaos.

Then there’s the link trap. Here, an attacker tricks the AI into including a harmful link in its response, disguised as something useful like a “reference.” Click on it, and you might be directed to a site where your sensitive information, like passwords or private messages, could be compromised. It’s a trap that relies on our trust in AI and is quite sneaky.

Google’s response to these vulnerabilities? They said it’s not something they plan to fix, labeling it “Won’t Fix (Intended Behavior).” They argue these are common issues in AI technologies, not just theirs, and they believe their security measures, like spam filters and input sanitization, are sufficient. They also point to their Vulnerability Rewards Program, which encourages finding such bugs. However, by calling these vulnerabilities intended behavior, Google might overlook the potential dangers, especially as these attack methods become more sophisticated.

If you’re concerned, you can take some steps to protect yourself. If you’re not comfortable with Gmail’s AI features scanning your emails, you might want to turn off the features. Always double-check links before you click them, no matter where they appear. Avoid sharing sensitive information through AI tools unless you know it’s secure. And if you’re in a work setting, ensure your team is aware of these risks.

This situation with Google raises a bigger question about balancing tech innovation with security. As AI weaves deeper into our lives, keeping it secure should be as much a priority as making it work well. While Google says they have it under control, the presence of these vulnerabilities means we all must stay vigilant, enjoying AI’s benefits while keeping our guard up.

---

References

1. Gmail Security Threat Is Confirmed And Google Won’t Fix It—Here’s Why (available at https://www.forbes.com/sites/daveywinder/2025/01/03/gmail-security-threat-confirmed-google-wont-fix-it-heres-why/ [Accessed: 04.01.2025].