Benchmark NLP Datasets - Livestream

Like many areas of AI, present-day NLP is data-driven. As a result, the available benchmark datasets are the primary factor in shaping the field itself. This has wide-ranging consequences for research, technology, and increasingly for society. How do we conceptualize different tasks, and which tasks receive the most attention from researchers? Which languages are adequately represented in our literature? Which groups benefit most from language technologies? Where will our systems deliver results that are embarrassing or worse? The answers to all these questions lie largely in the data we have for training and assessment. It is therefore in our best interests to deeply understand the datasets on which we are so dependent, and to seek out innovative new ways of collecting and validating relevant data. In this talk, I will report on a number of recent efforts to create more meaningful benchmarks for the field, and I will seek to identify persistent challenges and open questions in this area.

Speaker: Chris Potts, Stanford University

See weblink to register