Self-Supervised Representation Learning
Broadly speaking, all the generative models can be considered as self-supervised, but with different goals:
Generative models focus on creating diverse and realistic images While self-supervised representation learning care about producing good features generally helpful for many tasks Image based
Exemplar-CNN (Dosovitskiy et al., 2015) Rotation of an entire image (Gidaris et al.
Kullback-Leibler divergence Information theory Quantify information of intuition1 Likely events should have low information content Less likely events should have higher information content Independent events should have additive information. For example, finding out that a tossed coin has come up as heads twice should convey twice as much information as finding out that a tossed coin has come…