One of the key shifts in statistical machine learning over the past decade has been the transition from handcrafted features to automated, data-driven representation learning, typically via deep neural networks. As these methods are being used in high stakes settings such as medicine, health care, law, and finance where accountability and transparency are not just desirable but often legally required, it has become necessary to place representation learning on a rigourous scientific footing. In this talk we will re-visit the statistical foundations of representation learning from the lens of nonparametric latent variable models, and discuss how even basic statistical properties such as identifiability and consistency are surprisingly subtle. We will also discuss new results characterizing the optimal sample complexity for learning simple nonparametric mixtures, which turns out to have a nonstandard super-polynomial bound. Time permitting, we will end with applications to deep generative models that are widely used in practice.

This talk is based on joint work with Wai Ming Tai and Ruiyi Yang.