Skip to main content
Post

Improving Developer Experience in the Age of Agents

Weaviate shared what it learned from testing how coding agents read its docs. In its write-up on developer experience in the age of agents, the team says some agents were getting truncated pages, missing code examples for certain languages, or missing sections. Those gaps led them to search elsewhere or write wrong code, and agents rarely report the problem.

To find these issues, the team ran several agents on the same task and read their logs to see which tools they used, what they searched, and which pages they read. Fetch tools in different coding environments returned different parts of the same page. Before the fixes, one fetch tool returned the right language examples in only 7 of 15 TypeScript runs and 3 of 15 Go runs. After the changes, it returned the right snippets in every run Weaviate checked. A corrected, outdated sentence on a concepts page also stopped agents from using an older client library.

Weaviate also shipped Markdown versions of its documentation pages, which it says cut token use on page loads by around 20%. In one example, an agent needed 23 turns to solve a task when the code was hidden, and 6 turns with the Markdown page. Re-running the tests also caught a regression where a fix for hidden code tabs made one fetch tool drop most of the Quickstart. A small layout change fixed it.

Weaviate says it's now looking at how to steer agents toward the supported Go v6 client instead of calling the gRPC API directly. It's also measuring how well agents build Weaviate apps from scratch, and exploring an agent-oriented Quickstart and agent help with cluster maintenance.

Bring AI-native applications to life with less hallucination, data leakage, and vendor lock-in

Profile
Loading replies