In PCA, we construct a variance-covariance matrix and found the eigenvalues; eigenvectors are then interpreted as an axis of the ellipsoid fitted to the data.
My question is: is there any problem in finding axes of the ellipsoid when eigenvalues have multiplicity greater than 1? For example, if the largest eigenvalue occurs in double (the vector space associated has dimension=2), what will happen with the first principal axis? And will the second principal axis be constructed using also the largest eigen value?
Imagine you have a circle gaussian in two dimensions, PCA will capture the axis that explain the most variance, but in this case you can actually find infinite ammount of axes that have the same variance. This happens when you have eigenvalues with multiplicity greater than 1. Now you do not have an unique solution.