Large scale data processing with MapReduce

Large scale data analytics frameworks (e.g., Apache Spark, Apache Hadoop, Map Reduce, FlumeJava, and Dryad) are now widely used to derive value from massive amounts of raw data. In this talk, I’ll describe how these frameworks work. I’ll introduce the Map Reduce paradigm, and describe why it made it much easier for users to write programs to analyze data using hundreds or thousands of machines. I’ll also discuss problems solved by successors like Apache Spark (invented at UC Berkeley!).

Speaker: Kay Ousterhout, Principal Software Engineer, ServiceNow