// THE SIX CAUSES
What actually drives the waste.
Each cause is structurally addressable — none requires "agents getting smarter." Each has a measurable lever in the SandLogic stack.
01
Hallucinated tokens
Fabricated content costs the same as useful tokens — and downstream agents burn their own tokens responding to nonsense.
When a model fabricates content, every fabricated token costs the same as a useful one. Worse, in agentic workflows, downstream agents waste their own tokens reasoning over the hallucination. A 5% hallucination rate at the LLM layer can become a 20%+ token-waste rate at the agentic-workflow layer because the error compounds. Detection happens at the output layer (multi-metric scoring identifies low-confidence outputs); prevention happens by routing uncertain outputs to confidence-weighted retry or human review before the next chain step.
02
Non-deterministic retries
Stochastic outputs force retries. Same prompt, different answer, retry until something passes validation. Each retry is a full token spend.
LLMs are non-deterministic by design — temperature and sampling drive output variance. In production agentic workflows, this variance manifests as retries: validation logic rejects an output that doesn't meet the schema or business rules, and the system re-runs the same prompt. Each retry is a full token cost. Stabilization happens at two layers: model layer (sampling discipline, deterministic mode where appropriate) and runtime layer (caching identical inputs to deduplicate retry-driven token spend). Combined, these reduce retry-driven waste 5-8% on agentic workloads.
03
Context overload
Prompts stuff ever-growing context windows — full chat history, full retrieval results, full tool schemas — to 'be safe.' Each unused token is paid for on every call.
The defensive habit of stuffing context windows is the single largest source of input-token waste in production agentic AI. Teams add full chat history (turns ago, irrelevant), full retrieval results (3-5 chunks when 1 would suffice), full tool schemas (50 tools listed when only 2 are relevant), and structural metadata 'in case the model needs it.' On every call, the entire context is paid for. Adaptive RAG with intelligent memory control inverts this — context is injected based on semantic relevance to the active step. Result: 40-60% input-token reduction on multi-turn workflows without quality loss.
04
Wrong-size models
Reaching for GPT-4-class on tasks a 2.5B model would solve costs 50-100× more per token — and is often slower.
Model selection in enterprise AI is dominated by the 'reach for the biggest, safest model' bias. The cost difference between a frontier-API call (per-token premium pricing) and a small-language-model call on the same workload is 50-100× per token — and the SLM is frequently faster end-to-end because the token count is lower per query. Right-sizing means matching the model to the workload: classification tasks don't need frontier-scale reasoning; FAQ answering doesn't need a 70B parameter model; voice-agent intent detection doesn't need GPT-4. The Shakti family (100M-30B) is engineered to outperform peers 2-3× its size — so the right deployment is almost always smaller than instinct suggests.
05
Inefficient runtime
Even the right model on the wrong runtime burns tokens-per-second on idle hardware.
Generic inference frameworks (vLLM, TensorRT-LLM, SGLang) deliver baseline throughput but leave significant efficiency on the table — particularly on shared workloads with multiple users, varied prompts, and RAG-augmented context. EdgeFlow's hybrid KV-cache reuse (prefix-level for shared prompts + entity-level for retrieved chunks) skips the model entirely on cache hits. Cache-aware scheduling routes requests based on cache locality rather than round-robin. The result: +73% throughput vs vLLM on NVIDIA L40s, +29% on A100 — more tokens-per-dollar without changing the model.
06
Unoptimized tool chains
Multi-step agentic workflows invoke tools redundantly — re-fetching data, re-reasoning, looping when a single deterministic call would suffice.
Agentic workflows by their nature involve multiple steps — search, retrieve, reason, validate, respond. Without orchestration discipline, agents re-invoke tools redundantly: the same database query runs three times because the agent forgot the result; the same RAG retrieval happens twice because the chain doesn't deduplicate; chains continue looping when a terminating condition has already been met. LingoForge orchestration deduplicates tool invocations, caches intermediate results, and short-circuits provably-terminated chains. The long tail of multi-step token leakage gets eliminated structurally rather than through agent-prompt engineering.