For this question, please assume that I am talking about the condition number with respect to the spectral norm. That is, $\kappa_2(A) = \|A\|_2\|A^{-1}\|_2 = \frac{\sigma_{max}(A)}{\sigma_{min}(A)}$.
Many textbooks about linear regression claim that a large condition number implies the presence of multicollinearity. However, consider the following example:
Assume U is an orthonormal matrix, and define $\Sigma$ as follows: \begin{equation*} \Sigma = \pmatrix{10^{6}&0\\0&1} \end{equation*}
Now let A = U$\Sigma$I. We have that $\sigma_{max}(A) = 10^6$ and $\sigma_{min}(A) = 1$, so $\kappa(A) = 10^6$. However, the columns of $A$ are not correlated (in fact, they're orthogonal). Hence, it seems that multicollinearity is not a necessary condition for a large condition number. In this case, we have orthogonal predictors on vastly different scales. What are some other reasons for a large condition number?
(Side question: can anyone prove that a higher correlation among the columns of $A$ implies a larger condition number?).