Variational Autoencoders Pursue PCA Directions (by Accident)

CVPR 2019

Authors: Michal Rolinek*, Dominik Zietlow*, Georg Martius

Why variational autoencoders disentangle? Turns out it is due to innocent looking choice of diagonal posterior. Theoretical paper about an accidental connection between VAEs and (nonlinear) principle components (PCA).

Links: Arxiv

We looked into visual disentanglement mathematically. Turns out the canonical (beta-)VAE (by accident) does what one would want from deep PCA! Accepted to #cvpr2019. #deepPCAexists https://t.co/aQ3QEAl3xN pic.twitter.com/Gk1PvtcFlS

— Michal Rolínek (@MichalRolinek) March 5, 2019