Agent responsibly
By Steven Van ·
Vercel outlines guardrails like self-driving canary rollouts and continuous chaos testing to keep agent-generated code from degrading production.
Vercel has published an internal talk on why passing CI no longer proves a change is safe to ship. The company argues that coding agents produce pull requests that look flawless, complete with polished descriptions and reasonable test coverage, while remaining blind to production realities like traffic patterns, region-locked databases, or a Redis instance near capacity.
The post, Agent responsibly, draws a line between relying on agents and leveraging them: relying means shipping whatever the agent produces once tests pass, while leveraging means using agents to move fast while the engineer still understands exactly how the code behaves under load. Vercel's stated bar for a pull request is whether its author would be comfortable owning a production incident tied to it.
To back that up at the infrastructure level, Vercel says its core platform team is building:
- Stronger guardrails around shared infrastructure, with runtime validation at every stage of the deployment pipeline
- Stricter static checks at PR time, especially around feature flags
- Production-mirroring end-to-end testing in staging
- Read-only agents that continuously verify system invariants in production, auditing the assumptions made by generative agents
- Metrics such as defect-commit versus defect-escape ratios to surface rising risk across the platform
Vercel also points to self-driving deployments, where a degrading canary halts and rolls back automatically, and to continuous validation such as load tests and chaos experiments run outside of deploys, citing a rehearsed database failover last summer that it says kept a real Azure outage this year from affecting customers.
