I notice an interesting phenomenon when I try to generate a random s.p.d. matrix. One way to generate an s.p.d. matrix is to first generate an arbitrary matrix $M$, and take $$A=MM^T$$ $A$ is guaranteed to be s.p.d. (maybe semi p.d., in the worst case).
When $M$ is a full random matrix, (say generated by the MATLAB command 'rand(n)+1'),A is usually not so ill-conditioned. But if I force $M$ to be a lower triangular matrix (say using the command 'tril(rand(n)+1)'), which can be seen as a Cholesky factor, $A$ turns out to be far more ill-conditioned.
To illustrate the case, I generate 50 $A$s with full $M$ and another 50 $A$s with lower triangular $M$, and plot the condition number as
I cannot explain why. Can anyone help with this ?
As I state in my comment, it suffices to show that $M$ is generally ill-conditioned when we force it to be lower-triangular.
Suppose that $M$ is lower-triangular. Let $e_1,\dots,e_n$ denote the standard basis vectors. Note that $M e_i$ is the $i$th column of $M$. The first column $Me_1$ has $n$ random non-zero entries, whereas the last column $Me_n$ has $1$ random non-zero entry. Note that $$ \kappa(M) = \max_{\|x\| = \|y\| = 1} \frac{\|Mx\|}{\|My\|} \geq \frac{\|Me_1\|}{\|Me_n\|} $$ if the random entries have average absolute value $\mu>0$, then we should expect $\|Me_n\|$ to average at $\mu$, whereas $\|Me_1\|$ averages at $\sqrt{n}\,\mu$, so that $\frac{\|Me_1\|}{\|Me_n\|}$ should average at $\sqrt{n}$ (or at least, the the log of the fraction should average at $\log(\sqrt{n})$).
In this way, we see that the shape of the matrix leads to a higher expected condition number.