# Testing if "bash is all you need"

By Steven Van · 2026-01-22

Braintrust's Ankur Goyal found a SQL agent beat bash and filesystem agents on accuracy, cost, and speed querying GitHub issue data.

Ankur Goyal from Braintrust used [Vercel](<https://creatorstoolbox.com/tools/vercel>)'s blog to test a claim from Vercel's own earlier post: that filesystems and bash are the best abstraction for AI agents, even for tasks that aren't about writing code. Goyal built an eval harness to check whether that holds when the task is querying structured data rather than exploring loose files.

The [test](<https://vercel.com/blog/testing-if-bash-is-all-you-need>) ran three agents against a dataset of GitHub issues and pull requests, with questions ranging from simple ("How many open issues mention 'security'?") to multi-step ("Find issues where someone reported a bug and later someone submitted a pull request claiming to fix it"):

- A **SQL agent** querying a SQLite database directly: 100% accuracy, $0.51 per run, 45 seconds.

- A **bash agent** navigating JSON files with just-bash: 52.7% accuracy, $3.34, 401 seconds.

- A **filesystem agent** limited to basic search and read tools: 63.0% accuracy, $3.89, 126 seconds.

SQL won outright. The bash agent used about seven times more tokens than SQL, cost six and a half times as much, and took nine times as long, while even the filesystem agent, without full shell access, beat it on accuracy. The results point to filesystems and bash working better for exploring unstructured context than for querying structured data.

[![](<https://assets.vercel.com/image/upload/contentful/image/e5382hct74si/htPohxP7MYbZKG3LP6iK7/65e0dfe1c2ed26aa4a28bb9e5db1cc86/slack-imgs.png>)](<https://vercel.com/blog/how-to-build-agents-with-filesystems-and-bash>)

![Vercel](<https://www.google.com/s2/favicons?domain=vercel.com&sz=128>)

Vercel

The platform for frontend developers — deploy, preview, and scale web apps and AI agents with zero config.

[View Vercel →](<https://creatorstoolbox.com/tools/vercel>)

[Original source](<https://vercel.com/blog/how-to-build-agents-with-filesystems-and-bash>)

[Read Testing if "bash is all you need" on Creators Toolbox](<https://creatorstoolbox.com/blog/vercel-testing-if-bash-is-all-you-need>)

---
Canonical source: https://creatorstoolbox.com/blog/vercel-testing-if-bash-is-all-you-need