Building secure AI agents
By Steven Van ·
Vercel outlines defenses against prompt injection in AI agents and ships two sanitization packages, harden-react-markdown and markdown-to-markdown-sanitizer.
Vercel published guidance on securing AI agents against prompt injection, arguing it should be treated as a normal part of working with language models rather than a rare edge case. Its core advice: assume the attacker controls the entire prompt, including user input, retrieved documents, and any intermediate content passed to the model, and design tools accordingly.
- Scope tools to the authority of the caller rather than letting the model set parameters like a tenant ID, which can otherwise expose data across tenants.
- Treat any data the agent reads, such as scraped content or search results, as potentially containing injected instructions, since the model cannot distinguish user intent from injected text.
- Sanitize model output before rendering it as markdown or HTML, since an attacker can embed something like an image URL that exfiltrates data when it loads in a browser. Vercel cites a real exploit against GitLab Duo that worked this way.
- Never include secrets or tokens in prompts.
Alongside the post, Vercel released two packages: harden-react-markdown, a companion to react-markdown with secure defaults for links and images and support for allow-listing images, and markdown-to-markdown-sanitizer, for sanitizing markdown before publishing it to external parties such as GitHub or GitLab.
