A Custom DSL for Token-Efficient Agents
Replacing JSON queries with a token-efficient DSL to cut latency in half
Designing a Custom DSL to Cut Agent Token Usage and Latency by 50%
Identified token inefficiencies in the platform’s agent workflows and designed a custom domain-specific language to replace JSON-based query generation, cutting per-step latency by roughly 50%.
My role: I diagnosed where the agent loop was spending tokens, designed the DSL and its execution model, and built it into the production agent workflow on a developer platform serving ~100k requests/day over a ~6M-document corpus.
Results
I made the agent cheaper and faster at every step by changing the language it used to express queries, not just the model behind it.
- Replaced verbose JSON query generation with a purpose-built DSL, cutting output tokens by ~50%.
- Reduced minimum per-step latency from 4s to 2s, compounding across multi-step workflows.
- Grounded the design in a concrete understanding of how token overhead accumulates inside an agent loop, rather than treating the LLM as a black box.
- Delivered the change in production on an agentic system operating at ~100k requests/day.