Mutual Information and Representation Learning
February 01, 2020
Preliminaries
Data Processing Inequality
Mutual Information Maximization
sdfdsf asddasf asdfasdf 2323232 Often this summation is not tractable because of the cardinality of and .
Deriving the Contrastive Predictive Coding loss
We want to model
and we set it to
Mutual Information Minimization
On mutual information maximization for representation learning
Mutual information neural estimation