Learning with limited supervision

Many of the recent successes of machine learning have been characterized by the availability of large quantities of labeled data. Nonetheless, we observe that humans are often able to learn with very few labeled examples or with only high level instructions for how a task should be performed. In this talk, I will present some new approaches for learning useful models in contexts where labeled training data is scarce or not available at all. I will first discuss and formally prove some limitations of existing training criteria used for learning hierarchical generative models. I will then introduce novel architectures and methods to overcome these limitations, allowing us to learn a hierarchy of interpretable features from unalebeld data. Finally, I will discuss ways to use prior knowledge (such as physics laws or simulators) to provide weak forms of supervision, showing how we can learn to solve useful tasks, including object tracking, without any labeled data.

Speaker: Stefano Ermon, Stanford