Trends in Deep Learning Hardware

The current resurgence of artificial intelligence, including generative AI like ChatGPT, is due to advances in deep learning. Systems based on deep learning are transforming all aspects of life. Deep learning has been enabled by powerful, efficient computing hardware. The algorithms used have been around since the 1980s, but it has only been in the last decade - when powerful GPUs became available to train networks - that the technology has become practical. Advances in DL are now gated by hardware performance. Demand for training operations has increased 10Mx in the last decade and is currently growing by 16x per year. The autoregressive nature of the decode phase of LLM inference places particularly heavy demands on latency and memory bandwidth. The trend toward agentic systems places additional demands on latency. This talk will review the history of hardware for deep learning and how this hardware is adapting to the latest challenges.

Speaker: Bill Dally, NVIDIA, Corporation