Confused about how to get formula from (12.4) to (12.5), which derivative respect to $u_1$? I understand how to calculate derivative for a scalar parameter, but confused how to calculate derivative for a vector parameter like $u_1$, especially when there is both $u_1$ and transpose of $u^T_1$. If anyone could provide a bit more details, it will be great.
The context of the question is from PCA.
By “derivative” they mean the gradient with respect to $\mathbf{u}_1$. That is, if $\mathbf{u}_1 = (v_1,\dots,v_D)^T$, then you compute $(\partial f/\partial v_1,\dots,\partial f/\partial v_D)$.
To find the gradient of your particular function $f(v_1,\dots,v_D)$ given by (12.4), it's easiest to write out the matrix products explicitly in terms of the variables $v_k$. Since the matrix $\mathbf{S}$ is symmetric ($S_{ij}=S_{ji}$), you get $$ \begin{split} f(v_1,\dots,v_D) &= \sum_{i=1}^D \sum_{j=1}^DS_{ij} v_i v_j + \lambda_1 \left( 1-\sum_{i=1}^D v_i^2 \right) \\ &= S_{11} v_1^2 + 2S_{12}v_1 v_2+\dots + \lambda_1 (1-v_1^2-v_2^2-\dots) . \end{split} $$ Then just compute the partials as usual.