AT&T cuts AI costs 90% by rearchitecting orchestration with small language models

AT&T rebuilt its AI orchestration layer to handle 8 billion tokens per day, replacing monolithic large model calls with a multi-agent stack where LLM "super agents" direct smaller "worker" agents. The approach delivered up to 90% cost savings while maintaining accuracy. Chief data officer Andy Markus said small language models match LLM accuracy on domain-specific tasks. The team used the rearchitected stack with Microsoft Azure to build Ask AT&T Workflows, a drag-and-drop agent builder for employees. All agent actions are logged with role-based access and human oversight.

View full digest for February 26, 2026