Gisting: Compressing LLM Agent context to increase throughput and cut cost
Shopify Engineering published Gisting on August 19, demonstrating LLM context compression that cuts agent token costs.
Token costs constrain the frequency and depth of LLM agent interactions in production commerce systems. Gisting applies compression to the context window an agent carries across a multi-turn session — the running history of user requests, tool outputs, and prior responses that the model reads at each step. Shopify Engineering's paper demonstrates that compressing this context reduces per-call token consumption and raises throughput without accuracy loss on tested benchmarks. For agentic commerce applications executing multi-step workflows — product lookup, inventory check, cart management, and payment confirmation — context window efficiency directly affects per-transaction compute cost. Shopify operates over 150 million merchant storefronts; a compression technique from its engineering team carries implementation weight for the commerce platform at the centre of the agentic retail stack. The technique applies to any agentic checkout system maintaining session context across multiple tool calls.