Skip to main content
Zero-shot Learning for Model Diagnosis and Distributional Control in Generative Models

Zero-shot Learning for Model Diagnosis and Distributional Control in Generative Models

Abstract: In practice, metric analysis on a specific train and test dataset does not guarantee reliable or fair ML models. This is partially due to the fact that obtaining a balanced (i.e., uniformly sampled over all the important attributes), diverse, and perfectly labeled test dataset is typically expensive, time-consuming, and error-prone. To address this issue, first, I will describe zero-shot model diagnosis, a technique to assess deep learning model failures in visual data. In particular, the method will evaluate the sensitivity of deep learning models to arbitrary visual attributes without the need of a test set. Second, I will show how to add controllability to pre-existing generative models (i.e., how to sample specific regions of the generative model) without the need of additional training data or labels, and its applications to generate fair generative models, and efficiently sample specific sub-populations. Finally, I will also give an overview of a few other lab projects that involve sensing people for AR and VR applications.

Bio: Fernando De la Torre obtained his Ph. D. in Electronic Engineering from Ramon Llull University's La Salle School of Engineering in Barcelona in 2002. He has been a research faculty member in the Robotics Institute at Carnegie Mellon University since 2005. His areas of interest in research are machine learning and computer vision. In particular, applications to human health, augmented reality, virtual reality, generative models, and methods that focus on the data (not the model). He is the director of the Human Sensing Laboratory (www.humansensing.cs.cmu.edu). He has published over 225 peer-review conference papers/journals in computer vision and machine learning. In 2014, he founded FacioMetrics LLC to license technology for mobile human sensing (acquired by Facebook/Meta in 2016).


Host:
Prof. Arun Ross (rossarun@cse.msu.edu), Department of Computer Science and Eng.

(Date Posted: 2023-11-06)