I have trouble understand the gradient of equation 3.12 with respect to $W$.
Tn is a scalar output variable, $\phi(x)$ and $W$ are $N \times 1$ dimensional.
According to the book, the gradient results in a $1 \times N$ dimensional vector, since $\phi(x)$ transpose is $1 \times N$ dimensional. The gradient is supposed to result in a vector with same dimension as $W$. But this is not the case. Can someone explain?
Perhaps this situation is analogous to the derivative versus directional derivative (as might be familiar from multi-variable calculus).
Recall, given a function say $f:R^n \to R$, the gradient is defined as $\nabla f = \sum_{i = 1}^n \frac{\partial f}{\partial x_i} dx_i$ or alternatively $\left(f_{x_1}, f_{x_2}, \dots, f_{x_n}\right)$ while the directional derivative is given as $D_\vec{u}\, f = \nabla f (\vec{u})$ which amounts to the inner product $\nabla f \cdot \vec{u}$ and can also be expressed as $\vec{u}^T\nabla f$.
In your case, considering the function $E_D$ you have as being evaluated in the $\vec{w}$ direction may very well be the reason you wind up with scalars. You'll notice that this error function does, in-fact, vary with different values of $\vec{w}$ as expected.