In OLS formula $(X'X)^{-1}X'Y$, is the second paranthesis omited?

Question

OLS formula is: $(X^TX)^{-1}X^TY$

Is correct that fully spelled-out formula is: $(X^TX)^{-1}(X^TY)$
(i.e. because $((X^TX)^{-1}X^T)Y$ is impossible.)

---

My calculation

$((X^TX)^{-1}X^T)Y$ seems impossible, if we add dimensions $$((X^T)_{m\times{n}}X_{n\times{m}})^{-1}(X^T)_{m\times{n}}Y_{n\times1} => A_{m\times{m}}(X^T)_{m\times{n}}Y_{n\times1}$$

$A_{m\times{m}}(X^T)_{m\times{n}}$ doesn't work, so there is implied parenthesis, to first do $(X^T)_{m\times{n}}Y_{n\times1}=B_{m\times1}$

---

• Is this correct?
• Is such implied order of matrix multiplication the norm?

Answer 1

The product of an $m\times m$ matrix with an $m\times n$ matrix is simply a $m\times n$ matrix - there's no problem here.

Furthermore, matrix multiplication is in general associative, so the parentheses are not necessary.

Answer 2

Let's only the exact case where $Y = X A$,

$$\begin{eqnarray} (X^T X)^{-1} X^T Y &=& (X^T X)^{-1} X^T (X A) \\ &=& \left((X^T X)^{-1} (X^T X)\right) A \\ &=& I A \\ &=& A \end{eqnarray}$$

If the solution is not exact this formula will still give you the best approximation. If $(X^T X)^{-1}$ is singular, you can solve use the pseudo inverse, and it gives the optimal solution as well.