Finding the matrix of a quadratic form

Question

I want to find the matrix of quadratic form $Q= \sum^p_{i=1} (y_i - \bar y)^2$.

Please help me finding it.

---

For example I have found the quadratic form matrix for $Q= p\bar y^2$ as follows:

$$Q= p(\frac{y_1+y_2+...+y_p}{p})^2$$

$$\frac{\partial Q}{\partial y_1}=2 (\frac{y_1+y_2+...+y_p}{p})$$

$$....$$

$$\frac{\partial Q}{\partial y_p}=2 (\frac{y_1+y_2+...+y_p}{p})$$

$$\Rightarrow \frac{1}{2}\frac{\partial Q}{\partial y} =\left(\begin{matrix}1/p & 1/p & \cdots & 1/p \\ 1/ p &\cdots & \cdots \cdots & 1/p \\ \vdots & \vdots & \vdots \ddots & \vdots \\ 1/p & \cdots &\cdots&1/p \end{matrix}\right)\left(\begin{matrix} y_1 \\ ... \\ y_p \end{matrix}\right)= A\left(\begin{matrix} y_1 \\ ... \\ y_p \end{matrix}\right)$$ where $A$ is quadratic form matrix.

---

But I cannot find this matrix A for $Q= \sum^p_{i=1} (y_i - \bar y)^2$.

Thank you for helping.

Note: my trail solution for the question I asked as following ;

Is this true? If not, please show the solution. Thanks.

Answer 1

This is another solution which I believe is simpler than the other one. I suggest this one as it is more compact. Consider the following

$$\begin{array}{l} Q = \sum\limits_{i = 1}^P {{{\left( {{y_i} - \bar y} \right)}^2}} = \sum\limits_{i = 1}^P {\left( {y_i^2 - 2{y_i}\bar y + {{\bar y}^2}} \right)} = \sum\limits_{i = 1}^P {y_i^2} - 2\bar y\sum\limits_{i = 1}^P {{y_i}} + {{\bar y}^2}\sum\limits_{i = 1}^P 1 \\ \,\,\,\,\, = \sum\limits_{i = 1}^P {y_i^2} - 2\bar y\left( {P\bar y} \right) + P{{\bar y}^2} = \sum\limits_{i = 1}^P {y_i^2} - 2P{{\bar y}^2} + P{{\bar y}^2}\\ \,\,\,\,\, = \sum\limits_{i = 1}^P {y_i^2} - P{{\bar y}^2} \end{array}\tag{1}$$

and hence

$$\eqalign{ & {{\partial Q} \over {\partial {y_k}}} = {\partial \over {\partial {y_k}}}\left( {\sum\limits_{i = 1}^P {y_i^2} - P{{\bar y}^2}} \right) = \sum\limits_{i = 1}^P {2{y_i}{{\partial {y_i}} \over {\partial {y_k}}}} - 2P\bar y{{\partial \bar y} \over {\partial {y_k}}} \cr & \,\,\,\,\,\,\,\,\,\,\, = 2\sum\limits_{i = 1}^P {{\delta _{ik}}{y_i}} - 2P\bar y{1 \over P} = 2\left( {\sum\limits_{i = 1}^P {{\delta _{ik}}{y_i}} - \bar y} \right) \cr & \,\,\,\,\,\,\,\,\,\,\, = 2\left( {\sum\limits_{i = 1}^P {{\delta _{ik}}{y_i}} - {1 \over P}\sum\limits_{i = 1}^P {{y_i}} } \right) \cr & \,\,\,\,\,\,\,\,\,\, = 2\sum\limits_{i = 1}^P {\left( {{\delta _{ik}} - {1 \over P}} \right){y_i}} = 2\sum\limits_{i = 1}^P {{A_{ik}}{y_i}} \cr}\tag{2} $$

and finally we can conclude that

$${A_{ik}} = {\delta _{ik}} - {1 \over P}\tag{3}$$

or equivalently

$${A_{ik}} = \left\{ \matrix{ 1 - {1 \over P}\,\,\,\,\,\,\,\,\,\,\,\,i = k\, \hfill \cr - {1 \over P}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,i \ne k\, \hfill \cr} \right.\tag{4}$$

Answer 2

If you expand the sums for the first few values of $p$, a clear pattern emerges: $$\begin{align} \sum_{i-1}^2(y_i-\bar y)^2 &= \frac12(y_1^2+y_2^2-2y_1y_2) \\ \sum_{i=1}^3(y_i-\bar y)^2 &= \frac13(2y_1^2+2y_2^2+2y_3^2-2y_1y_2-2y_1y_3-2y_2y_3) \\ \sum_{i=1}^4(y_i-\bar y)^2&=\frac14(3y_1^2+3y_2^2+3y_3^2+3y_4^2-2y_1y_2-2y_1y_3-2y_1y_4-2y_2y_3-2y_2y_4-2y_3y_4) \end{align}$$ I would guess that, in general, $$ \sum_{i=1}^p(y_i-\bar y)^2 = \frac1p\left((p-1)\sum_i y_i^2-\sum_{i\neq j}y_iy_j\right) $$ You can now build the matrix for this quadratic form by inspection: $$ a_{ij}=\cases{ \frac{p-1}p & \text{if } i=j \\ -\frac1p & \text{if } i\neq j } $$ The proof follows the derivation in H.R.’s answer—expand the terms in the sum, pull $p$ out of the $\bar y$’s and rearrange.

Answer 3

Here is an abstract derivation. I prefer to use equations instead of words. So pay attention to what each equation is telling you. We have

$$\eqalign{ & Q = \sum\limits_{i = 1}^P {{{\left( {{y_i} - \bar y} \right)}^2}} \cr & \bar y = {1 \over P}\sum\limits_{j = 1}^P {{y_j}} \cr}\tag{1}$$

Now notice the following

$$\eqalign{ & {{\partial Q} \over {\partial {y_k}}} = {\partial \over {\partial {y_k}}}\sum\limits_{i = 1}^P {{{\left( {{y_i} - \bar y} \right)}^2}} = \sum\limits_{i = 1}^P {2\left( {{y_i} - \bar y} \right){\partial \over {\partial {y_k}}}\left( {{y_i} - \bar y} \right)} \cr & \,\,\,\,\,\,\,\,\,\,\, = 2\sum\limits_{i = 1}^P {\left( {{y_i} - \bar y} \right)\left( {{\delta _{ik}} - {1 \over P}} \right)} \cr & \,\,\,\,\,\,\,\,\,\,\, = 2\sum\limits_{i = 1}^P {\left( {\sum\limits_{j = 1}^P {\left( {{\delta _{ij}} - {1 \over P}} \right){y_j}} } \right)\left( {{\delta _{ik}} - {1 \over P}} \right)} \cr & \,\,\,\,\,\,\,\,\,\,\, = 2\sum\limits_{j = 1}^P {\left( {\sum\limits_{i = 1}^P {\left( {{\delta _{ij}} - {1 \over P}} \right)\left( {{\delta _{ik}} - {1 \over P}} \right)} } \right)} {y_j} \cr} \tag{2}$$

where ${{\delta _{ij}}}$ is the Kronecker-delta defined by

$${\delta _{ij}} = \left\{ \matrix{ 1\,\,\,\,\,\,\,\,\,\,\,\,\,\,i = j \hfill \cr 0\,\,\,\,\,\,\,\,\,\,\,\,\,i \ne j \hfill \cr} \right.\tag{3}$$

and we may conclude that

$$\eqalign{ & {A_{kj}} = \sum\limits_{i = 1}^P {\left( {{\delta _{ij}} - {1 \over P}} \right)\left( {{\delta _{ik}} - {1 \over P}} \right)} = \sum\limits_{i = 1}^P {\left( {{\delta _{ij}}{\delta _{ik}} - {1 \over P}\left( {{\delta _{ij}} + {\delta _{ik}}} \right) + {1 \over {{P^2}}}} \right)} \cr & \,\,\,\,\,\,\,\, = \sum\limits_{i = 1}^P {{\delta _{ij}}{\delta _{ik}}} - {1 \over P}\sum\limits_{i = 1}^P {\left( {{\delta _{ij}} + {\delta _{ik}}} \right)} + {1 \over {{P^2}}}\sum\limits_{i = 1}^P 1 \cr & \,\,\,\,\,\,\,\, = {\delta _{ik}} - {1 \over P}\left( 2 \right) + {1 \over {{P^2}}}\left( P \right) = {\delta _{ik}} - {2 \over P} + {1 \over P} \cr & \,\,\,\,\,\,\,\, = {\delta _{ik}} - {1 \over P} \cr}\tag{4} $$

Or specifically

$${A_{jk}} = \left\{ \matrix{ 1 - {1 \over P}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,j = k \hfill \cr - {1 \over P}\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,\,j \ne k \hfill \cr} \right.\tag{5}$$

---

Example

Let us consider the special case for $p=2$.

$$\begin{array}{l} Q = \sum\limits_{i = 1}^2 {{{\left( {{y_i} - \bar y} \right)}^2}} = {\left( {{y_1} - \frac{{{y_1} + {y_2}}}{2}} \right)^2} + {\left( {{y_2} - \frac{{{y_1} + {y_2}}}{2}} \right)^2}\\ \,\,\,\,\, = {\left( {\frac{{{y_1} - {y_2}}}{2}} \right)^2} + {\left( {\frac{{{y_2} - {y_1}}}{2}} \right)^2} = \frac{1}{2}{\left( {{y_1} - {y_2}} \right)^2}\\ \,\,\,\,\, = \frac{1}{2}\left( {y_1^2 + y_2^2 - 2{y_1}{y_2}} \right) \end{array}$$

Now you can easily check by looking at Eq.$(6)$ that

$$A = \left[ {\begin{array}{*{20}{c}} {\frac{1}{2}}&{ - \frac{1}{2}}\\ { - \frac{1}{2}}&{\frac{1}{2}} \end{array}} \right]\tag{6}$$

which is consistent with the general formula derived in Eq.$(5)$.