I am on the proof of the polar decomposition w/o using SVD:
Let $A$ be a real $n \times n$ matrix. Show that there exists $Q$: orthogonal, $P$: positive semi-definite symmetric matrix such that $A = QP$.
Consider the product $A^*A$, it is a self-adjoint and positive semi-definite (as we are in $\mathbb{R}$) symmetric matrix, so it has a decomposition $EDE^{-1}$ where $D$ is diagonal and $E$ is orthonormal and contains eigenvectors of $A^*A$. Now let $$ P = \sqrt{A^*A} = E\sqrt{D}E^{-1} $$ where $\sqrt{D}$ has diagonal elements being the squareroot of $D$. Notice that $||Px|| = ||Ax||, $ so we can expect the product $AP^{-1}$ to be orthogonal. Provided that $P^{-1}$ exists (i.e. $P$ is positive definite), we have $||AP^{-1}x||=||x||$, so we do find our desired decomposition by letting $Q = AP^{-1}$.
The problem is that if $A$ is not invertible, then $P$ will also not invertible, regarding the whole proof useless. How should I fix it in this case?
A standard workaround for singularities is to use analytic techniques. The below is an existence argument, not a uniqueness argument. Note Polar Decomposition is not unique when $\det\big(A\big)=0$.
$A_k := A + \frac{\delta}{k}I$ for some small enough $\delta \gt 0$. Then for all $k=1,2,3,...$
$A_kP_k^{-1}=Q_k \in O_n\big(\mathbb R\big)$
by compactness of $O_n\big(\mathbb R\big)$, the $Q_k$ form a bounded sequence and hence there is (at least one) convergent subsequence $k_1\lt k_2\lt...$ whose limit is given by $\lim_{i\to\infty}Q_{k_i}=Q \in O_n\big(\mathbb R\big)$ .
Similarly
$P_{k_i} = Q_{k_i}^TA_{k_i}$
whose limit exists because each term on the RHS does. And for avoidance of doubt:
1.) $P_{k_i}$ is symmetric for all $i$ hence the limit is too.
2.) by topological continuity of eigenvalues, since $P_{k_i}$ has all positive eigenvalues for all $i$, its limit has all real non-negative eigenvalues, i.e. it is PSD.