AI Architectures at Scale: Chiplets, Interconnects, and Co-design for Energy Efficiency

Recent advancements in deep neural networks (DNNs), especially transformer-based large language models (LLMs), have driven significant progress in artificial intelligence (AI). As demand grows, models expand to trillions of parameters, potentially requiring dedicated nuclear power plants for data centers. While GPUs are commonly used, they are outperformed in energy efficiency by domain-specific accelerators (DSAs). Modern system-on-chip (SoC) designs utilize these DSAs to enable parallel workload execution, known as accelerator-level parallelism (ALP). SoCs need to scale to meet the growing demand but encounter challenges like reticle limits, yield issues, and thermal management. Chipletization - combining multiple chips in one package - offers a solution for improved scalability and composability, leading to what I call chiplet-level parallelism (CLP). Future systems will incorporate various domain-specific chiplets, enhancing parallel execution. Additionally, technologies like silicon photonics will be vital for scaling these architectures to bridge the gap to warehouse-scale computing. This talk will cover the challenges and optimizations for ALP, CLP, and beyond Moore’s architectures.

Speaker: Vikram Jain, UC Berkeley