Run Scale AI’s AgentEnv evals on Vercel Sandbox
By Steven Van ·
AgentEnv 0.9.1334 and later include a Vercel provider that runs the agent and its task environment in separate Sandboxes.
Vercel Sandbox can now run evaluations from AgentEnv, Scale AI's framework for testing agents against tasks and scoring their results. The Vercel provider ships in AgentEnv 0.9.1334 and later, as listed in the Sandbox ecosystem documentation.
Separate Sandboxes for agent and task
The provider runs the agent and its task environment in separate Sandboxes. The ecosystem page lists AgentEnv as an eval harness integration, for evaluating agents in separate environment and agent sandboxes.
Running the example
AgentEnv has an example that runs an evaluation on Vercel Sandbox, so you can inspect the task's score and the agent's recorded actions. After completing the setup guide, run this command from the cloned repository. It uses your existing Vercel CLI login:
./vercel-eval --scope YOUR_TEAM --project YOUR_PROJECT
Replace YOUR_TEAM with your Vercel team slug and YOUR_PROJECT with the name of an existing project.