End-User Auditing and Model Sketching: Tools for Centering Human Values in Machine Learning Model Authoring and Evaluation

As AI-based technology has rapidly expanded into high-impact, user-facing domains, we have observed myriad ways in which it can perpetuate harmful biases, embed problematic values, or entirely fail. In the face of these errors, we are starting to see that algorithmic fairness and explainability methods may not be sufficient fixes for technological tools that were designed incorrectly or incompletely from the start. To improve these systems, we need a shift in voice and power in technology design. In this talk, I'll share two lines of work in this vein. In the first project, End-User Auditing, we built a tool to empower individual, non-technical users to leverage their distinctive expertise and lead their own system-scale audits of machine learning models. In the second project, Model Sketching, we sought to address the problem of technical design fixation in ML by taking inspiration from sketching practices in design, which purposefully distill ideas into rough, minimal representations to explore high-level design directions. We developed a tool that enables users to interactively author functional, sketch-like versions of ML models, focusing their attention on higher-level, human-understandable concepts rather than lower-level, technical implementation details in the early stages of model development. Every individual has unique expertise built up from their interests and skills in combination with the communities and environments they inhabit - my research aims to allow users to share this expertise to raise unanticipated issues and suggest novel solutions as co-designers of algorithmic systems.

Speaker: Michelle Lam, Stanford

Room 126