PCAandAutoencoder
Can think of an autoencoder as a nonlinear extension of Principal Component Analysis (PCA). An autoencoder with a single hidden layer, a linear activation function, and MSE loss reduces to an PCA. The number of neurons in the single hidden layer determines the number of principal components.
Follow-ups
- PCA article reccomended by Saxon: https://www.mit.edu/~gari/teaching/6.555/LECTURE_NOTES/ch15_bss.pdf
- Autoencoders, Unsupervised Learning, and Deep Architectures
- Neural networks and principal component analysis: Learning from examples without local minima https://www.sciencedirect.com/science/article/pii/0893608089900142