We removed 80% of our agent’s tools
By Steven Van ·
Vercel cut its internal text-to-SQL agent down to one bash tool and saw success rates rise from 80% to 100%.
Vercel's internal data agent, d0, translates natural language questions into SQL queries against its analytics infrastructure so anyone on the team can ask for numbers in Slack without writing code. The team rebuilt it by removing most of its tools: instead of separate tools for schema lookup, query validation, and error recovery, the agent now has a single tool, execute bash commands, and reads the underlying Cube semantic layer files directly with grep, cat, and ls.
Vercel benchmarked the new "file system agent" against the old architecture on five representative queries:
- Success rate: 100% (5/5), up from 80% (4/5)
- Average execution time: 77.4 seconds, down from 274.8 seconds
- Average token usage: about 61k tokens, down from about 102k, 37% fewer
- Average steps: about 7, down from about 12, 42% fewer
Vercel says the approach works because its Cube semantic layer is already well-structured YAML, Markdown, and JSON, and that giving a model raw file access over a messier, undocumented data layer would just produce faster bad queries.
