The AI Alignment Problem: Why it's hard, and where to start

If we can build sufficiently advanced machine intelligences, what goals should we point them at? The frontier open problems on this subject are less, "A robot may not injure a human, nor through inaction allow a human to come to harm," and more, "If you could formally specify the preferences of an arbitrarily smart and powerful agent, could you get it to safely move one strawberry onto a plate?" This talk will discuss some of the open technical problems in AI alignment, the probable difficulties that make those problems hard, and the bigger picture into which they fit; as well as what it's like to work in this relatively new field.

Speaker: Eliezer Yudkowsky, Machine Intelligence Research Institute