Assume that $x_{n}$, $\widetilde{x}_{n}$ and $\bar{x}$ are column vectors and $u_{i}$ are orthonormal basis vectors.
When considering equations (1) and (2), it is not clear to me how equation (3) follows from that.
I know that $||x_{n}-\widetilde{x}_{n}||^2 = (x_{n}-\widetilde{x}_{n})^{T}(x_{n}-\widetilde{x}_{n})$.
However, after plugging in eq (2), I don't end up with eq (3).
Is someone able to help me out with this? Thanks a lot!
On
We have $\|\sum_i\alpha_iu_i\|^2=\sum_i|\alpha_i|^2$ since $u_i$ is orthonormal.
Apply it with $\alpha_i=(x_n^Tu_i-\bar x_n^Tu_i) =(x_n-\bar x_n)^Tu_i$.
By the way, your notation is not consequent (or properly introduced), I guess you missed the index $n$ for $\bar x$, and the conditions suggests that $\tilde x_n$ is just the orthogonal projection of $\bar x_n$ to the subspace spanned by $u_{M+1},\dots,u_D$.
[Same as Berci, but with more details] Since $\mathbf u_i$ are orthonormal, for any pair of indices $(i,j)$, $$ \mathbf u_i^T \mathbf u_j = \delta_{ij} = \begin{cases} 1 & i = j \\ 0 & i \neq j \end{cases} $$ So \begin{align*} J &= \frac{1}{N} \sum_{n=1}^N(\mathbf{x}_n - \tilde{\mathbf{x}}_n)^T(\mathbf{x}_n - \tilde{\mathbf{x}}_n) \\ &= \frac{1}{N} \sum_{n=1}^N\left(\sum_{i=M+1}^N\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_i\right\}\mathbf{u}_i\right)^T\left(\sum_{j=M+1}^N\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_j\right\}\mathbf{u}_j\right) \\ &= \frac{1}{N} \sum_{n=1}^N\sum_{i,j=M+1}^N\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_i\right\}\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_j\right\}\mathbf{u}_i^T\mathbf{u}_j \\ &= \frac{1}{N} \sum_{n=1}^N\sum_{i,j=M+1}^N\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_i\right\}\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_j\right\}\delta_{ij} \\ &= \frac{1}{N} \sum_{n=1}^N\sum_{i=M+1}^N\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_i\right\}\left\{(\mathbf{x}_n-\bar{\mathbf{x}})^T \mathbf{u}_i\right\}\\ &= \frac{1}{N} \sum_{n=1}^N\sum_{i=M+1}^N\left\{\mathbf{x}_n^T\mathbf{u}_i-\bar{\mathbf{x}}^T\mathbf{u}_i \right\}^2\\ \end{align*}