On the Statistical Capacity of Deep Generative Models

1097 

Contributed Abstract 

Poster 

Edric Tam (1), David Dunson (2)

(1) Stanford University, N/A, (2) Duke University, N/A

David Dunson  
Duke University

Edric Tam  
Stanford University

Edric Tam  
Stanford University

Deep generative models are routinely used in generating samples from complex, highdimensional distributions. Despite their apparent successes, their statistical properties are not
well understood. A common assumption is that with enough training data and sufficiently large
neural networks, deep generative model samples will have arbitrarily small errors in sampling
from any continuous target distribution. We set up a unifying framework that debunks this belief.
We demonstrate that broad classes of deep generative models, including variational autoencoders
and generative adversarial networks, are not universal generators. Under the predominant case of
Gaussian latent variables, these models can only generate concentrated samples that exhibit light
tails. Using tools from concentration of measure and convex geometry, we give analogous results
for more general log-concave and strongly log-concave latent variable distributions. We extend
our results to diffusion models via a reduction argument. We use the Gromov–Levy inequality to
give similar guarantees when the latent variables lie on manifolds with positive Ricci curvature.
These results shed light on the limited capacity

Deep Generative Models|Diffusion models|Generative Adversarial Networks|Variational Autoencoders|Concentration of Measure|

Section on Bayesian Statistical Science

Bayesian Computation