How is this second order derivative with respect to a vector a matrix?

Question

I have the following equation and I am wondering how it is a matrix?

As you can see, $\Theta$ is a vector, as is u. I know that the second order derivative of a vector should be a matrix but I am struggling to see why from this equation.

The bottom half when evaluated gives me a vector as does u uT. So isnt everything just a vector in this equation.

Apologies, I am sure I am being very stupid.

Answer 1

Notice that $$uu^T = \left[\begin{array}{c}u_1\\ u_2 \\ \vdots \\ u_n\end{array}\right] \left[\begin{array}{cccc} u_1 & u_2 & u_3 & u_4 \end{array}\right] = \begin{bmatrix} u_1u_1 & u_1u_2 & \cdots & u_1u_n\\ u_2u_1 & \ddots & \cdots & u_2u_n\\ \vdots & \ddots & \ddots & \vdots\\ u_nu_1 & u_nu_2 & \cdots & u_nu_n\end{bmatrix}$$ is a matrix, so the righthand side of your equation is indeed a matrix.

Answer 2

Your $u_t^{(ij)}$ is a vector, so $u_t^{(ij)}u_t^{(ij)^T}$. Denoting this matrix as $U_{ij} = u_t^{(ij)}u_t^{(ij)^T}$ and the scalars as \begin{equation} \alpha_{ij} = \frac{e^{-\theta_t^T u_t^{(ij)} +b}}{(1+e^{-\theta_t^T u_t^{(ij)} +b})^2} \end{equation} You will get $- \sum_{ij} \alpha_{ij} U_{ij} - \lambda I$