A Priority Queue Became the Backbone of Meta's AI Infrastructure

Almost every asynchronous action at Meta passes through a single system most people have never heard of. The Facebook Ordered Queueing Service (FOQS), a fully managed, horizontally scalable priority queue, moves close to a trillion items per day for 300+ engineering teams, and it has become critical infrastructure for AI at Meta's scale. Async LLM inference, Llama serving, GenAI image generation, and AI compute demand control all ride on it.

his talk traces how a queue originally built to absorb massive backlogs and prioritize work across highly heterogeneous producers and consumers grew into the reliability layer beneath Meta's AI stack. It is an impact story rather than a feature tour: how one system came to serve hundreds of teams without them stepping on each other, how it keeps the most important work moving under enormous load, and how it evolved from isolated regional deployments into a globally distributed service that delivers region-level disaster recovery in seconds with zero client-visible downtime. The finale looks at how that same queue now acts as a control plane for shaping AI compute demand.

Speaker: Jasmit Saluja, Meta