Gemini is Google’s newest family of Large Language Models (LLMs). The Gemini suite currently houses three different model sizes: Nano, Pro, and Ultra.
Although Gemini has been removed from service due to politically biased content, new findings from HiddenLayer – unrelated to that issue – analyze how an attacker can directly manipulate another user’s queries and output, representing an entirely new threat. These vulnerabilities were disclosed to DeepMind using responsible disclosure practices.
While testing the 3 LLMs in the Google Gemini family of models, HiddenLayer found multiple prompt hacking vulnerabilities, including the ability to output misinformation about elections, multiple avenues that enabled system prompt leakage, and the ability to inject a model indirectly with a delayed payload via Google Drive. These vulnerabilities enable attackers to conduct activities that allow for misuse and manipulation. In new research released from HiddenLayer today, “New Google Gemini Content Manipulation Vulns Found – Attackers Can Gain Control of Users’ Queries and LLM Data Output – Enabling Profound Misuse,” HiddenLayer deep dives into these vulnerabilities, including a proof-of-concept of an Indirect Injection.
HiddenLayer researchers warn that the vulnerabilities potentially affect millions, including:
- The General Public: Misinformation generated by Gemini and other LLMs can be used to mislead people and governments.
- Developers using the Gemini API: System prompts can be leaked, revealing the inner workings of a program using the LLM and potentially enabling more targeted attacks.
- Users of Gemini Advanced: Indirect injections via the Google Workspace suite could potentially harm users. The attacks outlined in this research currently affect consumers using Gemini Advanced with the Google Workspace due to the risk of indirect injection, companies using the Gemini API due to data leakage attacks, allowing a user to access sensitive data/system prompts, and governments due to the risk of misinformation spreading about various geopolitical events.
Gemini Advanced currently has over 100M users, so the ramifications of these vulnerabilities are widespread. With the accelerating adoption of LLM AI, companies must be aware of implementation risks and abuse methods that Gen AI and Large Language Models offer to strengthen their policies and defenses.
To learn more about the new Google Gemini Content manipulation vulnerabilities that HiddenLayer researchers examined, visit https://hiddenlayer.com/research/new-google-gemini-content-manipulation-vulns-found/ or HiddenLayer: https://hiddenlayer.com/
HiddenLayer may sound familiar – they won the 2023 RSAC Innovation Sandbox for their security platform, which helps enterprises safeguard the machine learning models behind their most important products.
Follow Brilliance Security Magazine on Twitter and LinkedIn to ensure you receive alerts for the most up-to-date security and cybersecurity news and information.