Simplification of a product of three matrices

334 Views Asked by At

Define $$\mathbf{c}_t = \begin{bmatrix} x_{1t} \\ x_{2t} \\ \vdots \\ x_{Nt} \end{bmatrix}\in \mathbb{R}^N$$ where all entries are in $\mathbb{R}$, $t = 1, 2, \dots, p+1$.

I am trying to simplify $$\sum_{j=0}^{p}\beta_j\mathbf{c}^{T}_{j+1}\mathbf{c}_{k+1} $$ where $k = 0, 1, \dots, p$ and $$\boldsymbol{\beta}=\begin{bmatrix} \beta_0 \\ \beta_1 \\ \vdots \\ \beta_p \end{bmatrix} \in \mathbb{R}^{p+1}$$ and the matrix $\mathbf{X}$ is the concatenation of the $\mathbf{c}$ vectors, i.e., $$\mathbf{X} = \begin{bmatrix} \mathbf{c}_1 & \mathbf{c}_2 & \cdots & \mathbf{c}_{p+1} \end{bmatrix}\in \mathbb{R}^{N \times (p+1)}\text{.}$$ To understand the context, this is (what I believe is) a simplification of one side of the so-called "normal equations:" $$\begin{align*} \sum_{i=1}^{N}\sum_{j=0}^{p}x_{ij}x_{ik}\beta_{j} &= \sum_{j=0}^{p}\sum_{i=1}^{N}x_{ik}x_{ij}\beta_{j} \\ &= \sum_{j=0}^{p}\beta_{j}\left(\sum_{i=1}^{N}x_{ij}x_{ik}\right) \\ &=\sum_{j=0}^{p}x_{ik}\begin{bmatrix} x_{1j} & x_{2j} & \cdots & x_{Nj} \end{bmatrix} \begin{bmatrix} x_{1k} \\ x_{2k} \\ \vdots \\ x_{Nk} \end{bmatrix} \\ &= \sum_{j=0}^{p}\beta_j\mathbf{c}^{T}_{j+1}\mathbf{c}_{k+1} \tag{1} \end{align*}$$ and from here, I'm stuck.

I know the answer should be $\mathbf{X}^{T}\mathbf{X}\boldsymbol{\beta}$ (if the expression above is correct), but I have no idea how this follows. It should be the case that $(1)$ above should end up being the $k$th entry of $\mathbf{X}^{T}\mathbf{X}\boldsymbol{\beta}$, i.e., $$\mathbf{X}^{T}\mathbf{X}\boldsymbol{\beta} = \begin{bmatrix} \sum_{j=0}^{p}\beta_j\mathbf{c}^{T}_{j+1}\mathbf{c}_{1} \\ \sum_{j=0}^{p}\beta_j\mathbf{c}^{T}_{j+1}\mathbf{c}_{2} \\ \vdots \\ \sum_{j=0}^{p}\beta_j\mathbf{c}^{T}_{j+1}\mathbf{c}_{p+1} \end{bmatrix}$$

1

There are 1 best solutions below

0
On

I think that your notation is a bit off, since you should probably be indicating the first subscript as rows and the second subscript as columns consistently throughout the problem.

My main recommendation would be to expand out $\mathbf{X}^T\mathbf{X}$ into a sample mean notation, and then multiply in by $\beta$; it may become clearer how these sample mean cells in $\mathbf{X}^T\mathbf{X}$ relate to the inner product of $\mathbf{c}$ vectors.