A smarter approach to AI cost controls: allocating spend by efficiency
By Steven Van ·
Blanket usage caps hold back your most efficient engineers.
Enterprises can now allocate Devin spend by how efficiently each engineer uses it, instead of applying a uniform usage cap. Cognition shipped three enterprise features for this: Efficiency Score, Devin Coach and efficiency-based approvals. The announcement argues that blanket caps hold back the most efficient engineers and give less efficient ones no feedback.
Efficiency Score
The score is a 0 to 100 rating of each engineer's recent Devin usage. For each session, including investigations and planning that never produce a PR, Devin estimates the productive engineering hours, meaning how long a human engineer would have needed for the same work. A session with a PR counts only if the PR was merged. For a session without a PR, a classifier judges whether Devin made real progress on the task. Cognition says it validated the hour estimates against engineers' own assessments of their sessions.
The score combines two measures, both converted to a rating between 0 and 1:
- Efficiency (E): productive hours per Agent Compute Unit (ACU), which combines LLM and cloud infrastructure cost.
- Volume (V): productive hours per day, regardless of cost.
The formula is Score = 100 × (E + λV(1 − E)), where λ = 0.2. Efficiency sets the starting score, and volume can add up to a fifth of the points left between that and 100. In Cognition's examples, based on the most recent 200 ACUs, a typical engineer (E 0.50, V 0.50) scores 55, and a heavy but wasteful one (E 0.27, V 0.95) scores 41.
The score reflects recent usage rather than a lifetime average. Engineers see it in My Analytics, under Session efficiency, as Healthy, Satisfactory, Needs Improvement, or Unknown when there is not enough data. Supporting metrics are merged PRs per day, ACUs per merged PR, sessions with delivered outcomes, and consumption by session size.
Devin Coach
Devin Coach gives feedback on a prompt before it runs, flags common mistakes and suggests a fix. The announcement lists these suggestion types:
- Asking for a plan before implementing, when a prompt leaves major design decisions open.
- Switching from Ultra, the most expensive mode, to the default mode for a simple task such as changing a button label.
- Starting a new session for an unrelated follow-up.
- Splitting unrelated tasks into child sessions.
- Flagging clearly non-work prompts, such as planning a vacation.
Efficiency-based approvals
When an engineer with a Healthy or Satisfactory grade nears their limit, Devin raises it automatically, up to a ceiling the admin sets. Requests from other engineers go to an admin for review.
