Transpose of $(X'X)^{-1}$

Question

I am taking a Phd class in econometrics, and the following is used constantly, for $X$ a $n\times k$ matrix, $n \neq k$: $$(X'X)^{-1} = ((X'X)^{-1})'$$ with "$'$" standing for transpose. Having a rather weak background in linear algebra, I cannot understand why this is true.

For example: ${\underset{k\times1}{\underbrace{\left(\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}\underset{k\times1}{\underbrace{X'u}}\right)}}\underset{1\times k}{\underbrace{\left(\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}\underset{k\times1}{\underbrace{X'u}}\right)'}}}={\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}\underset{k\times n}{\underbrace{X'}}\underset{n\times n}{\underbrace{uu'}}\underset{n\times k}{\underbrace{X}}\underset{k\times k}{\underbrace{\left(X'X\right)^{-1}}}}$

To my understanding the far right expression on the RHS should be different.

Answer 1

Basically, what this says is that $(X^TX)^{-1}$ is symmetric. There are two results about matrices at work here.

First, the inverse of a symmetric matrix (if it exists) is itself symmetric, i.e., $(A^{-1})^T=A^{-1}$. See this question for a proof.

Second, for any matrix $X$, $X^TX$ is symmetric. To see this, consider a typical element in the product: $(X^TX)_{ij}=\sum_k(X^T)_{ik}X_{kj}=\sum_kX_{ki}X_{kj}$, but this is just the dot product of the $i$th and $j$th columns of $X$. Since the dot product is symmetric, $(X^TX)_{ij}=(X^TX)_{ji}$. Combining these two results gives you $((X^TX)^{-1})^T=(X^TX)^{-1}$.

Answer 2

First, note that $(X'X)'=X'(X'')=X'X$. Now suppose $A$ is an invertible matrix such that $A'=A$, with inverse $B$. I claim that $B=B'$; taking $A=X'X$ will then give your equation. Indeed, note that $AB=BA=I$, so taking transposes, $B'A'=A'B'=I'=I$, so $B'$ is an inverse of $A'$. Since $A'=A$, this means $B'=A^{-1}=B$.

Answer 3

First: $(X'X)'=X'(X')'=X'X$. Then:

$(X'X)^{-1}(X'X) = 1 = 1' = \big((X'X)^{-1}(X'X)\big)'=(X'X)'\big((X'X)^{-1}\big)'=(X'X)\big((X'X)^{-1}\big)'$

So $1=(X'X)\big((X'X)^{-1}\big)'$ which means $\big((X'X)^{-1}\big)' = (X'X)^{-1}$.