Why are GPUs so hard to program - or are they?

When we designed Blue Waters, the most powerful petascale supercomputer for the NSF community, the single most pressing concern about its success was that "GPUs are too hard to program." The rise of GPU computing has significantly boosted the pace of progress in numeric methods, algorithm design and programming techniques for scalability at the processor architecture level. There is now a wealth of publications, teaching material as well as publicly available software libraries. For example, the CUDA counter part of the Math Kernel Library (MKL) now contains hundreds of functions covering nearly every domain of computational science. We also have the experience of teaching more than 10,000 students to write scalable parallel programs using CUDA in a Coursera MOOC. However, there has been a lack of practical languages and compilers that relieve programmers from heavy lifting. I will review the community's progress in developing scalable, portable, and numerically stable software, with some deeper discussions of contributions from the IMPACT group. I will also present some recent advances and current efforts in languages and compilers for developing scalable numerical applications.

Speaker: Wen-mei Hwu, Univ. of Illinois, Urbana-Champaign