Identities with quadratic forms

Question

Let $y_1,\dots, y_n$ be vectors in $\mathbb{R}^p$ and let $\mu$ be an additional vector in $\mathbb{R}^p$. Let $M\in\mathbb{R}^{p\times p}$ be a symmetric positive definite matrix. I have some difficulties into prove the following identity \begin{equation}\sum_{i=1}^n (y_i-\mu)'M^{-1}(y_i-\mu)=n(\bar{y}-\mu)'M^{-1}(\bar{y}-\mu)+\sum_{i=1}^n (y_i-\bar{y})'M^{-1}(y_i-\bar{y})\end{equation} where \begin{equation}\bar{y}\triangleq \frac{1}{n}\sum_{i=1}^n y_i\end{equation} I've tried to expand the generic product $(y_i-\mu)'M^{-1}(y_i-\mu)$ but this seems to me useless to prove the previous identity.

Answer 1

Hint You could replace $\mu$ by $t\mu$. It is clear that the coefficients of $t$ and $t^2$ are the same on both sides of the equality. It remains to prove the equality of the constant terms (w.r.t $t$), that is to say the same equality without $\mu$.

The equality that is left resembles the equation $E(X^2) = E(X)^2 + \text{Var}(X)$ in probability. The proof is similar by using $\frac{1}{n}\sum_{i=1}^n$ instead of $E$.

Answer 2

Using linearity of the transpose and matrix multiplication we get

\begin{align} (y_i-\mu)^TM^{-1}(y_i-\mu) &= (y_i-\overline{y})^TM^{-1}(y_i-\mu) + (\overline{y} - \mu)^TM^{-1}(y_i-\mu)\\ &= (y_i-\overline{y})^TM^{-1}(y_i-\overline{y}) + (y_i-\overline{y})^TM^{-1}(\overline{y}-\mu) + (\overline{y} - \mu)^TM^{-1}(y_i-\mu). \end{align}

Now since $(\overline{y} - \mu)^TM^{-1}(y_i-\mu)$ is a scalar and $M^{-1}$ is symmetric, we have $$(\overline{y} - \mu)^TM^{-1}(y_i-\mu) = \left((\overline{y} - \mu)^TM^{-1}(y_i-\mu)\right)^T = (y_i-\mu)^TM^{-1}(\overline{y} - \mu)$$ so $$(y_i-\mu)^TM^{-1}(y_i-\mu) = (2y_i - \overline{y}-\mu)^TM^{-1}(\overline{y}-\mu).$$ Summing over $i=1, \ldots, n$ and again using linearity gives \begin{align} \sum_{i=1}^n (y_i-\mu)^TM^{-1}(y_i-\mu) &= \sum_{i=1}^n (2y_i - \overline{y}-\mu)^TM^{-1}(\overline{y}-\mu) +\sum_{i=1}^n(y_i-\overline{y})^TM^{-1}(y_i-\overline{y}) \\ &= \left(2\sum_{i=1}^n y_i - n\overline{y}-n\mu \right)^T M^{-1}(\overline{y}-\mu)+\sum_{i=1}^n(y_i-\overline{y})^TM^{-1}(y_i-\overline{y}) \\ &= (2n\overline{y} - n\overline{y}-n\mu)^TM^{-1}(\overline{y}-\mu) +\sum_{i=1}^n(y_i-\overline{y})^TM^{-1}(y_i-\overline{y}) \\ &= n(\overline{y}-\mu)^TM^{-1}(\overline{y}-\mu) +\sum_{i=1}^n(y_i-\overline{y})^TM^{-1}(y_i-\overline{y}). \end{align}

Answer 3

\begin{align} & (y_i-\mu)'M^{-1}(y_i-\mu) \\[10pt] = {} & \big((y_i-\overline y) +(\overline y - \mu)\big)' M^{-1} \big((y_i-\overline y) +(\overline y - \mu)\big) \\[10pt] = {} & (y_i-\overline y)'M^{-1} (y_i-\overline y) + (y_i-\overline y)'M^{-1}(\overline y - \mu) \tag 1 \\[4pt] & {} + (\overline y - \mu)' M^{-1}(y_i - \overline y) + (\overline y - \mu)'M^{-1} (\overline y - \mu) \tag 2 \end{align}

Consider the second term on line $(1)$: $\quad (y_i-\overline y)'M^{-1} (\overline y-\mu).$ As $i$ goes from $1$ to $n,$ the factor $M^{-1}(\overline y-\mu)$ does not change, so $$ \sum_{i=1}^n \Big( (y_i-\overline y)'M^{-1} (\overline y-\mu) \Big) = \left( \sum_{i=1}^n (y_i-\overline y)' \right) M^{-1}(\overline y-\mu). $$ This last sum from $i=1$ to $n$ evaluates to $0.$ The same thing applies to the first of the two terms on line $(2).$

Now look at $(\overline y-\mu)'M^{-1}(\overline y-\mu).$ As $i$ goes from $1$ to $n,$ this term does not change, so the sum is $n(\overline y-\mu)'M^{-1}(\overline y-\mu).$