here (pageg 21) it says : It is very important to remember that the gradient of a function is only defined if the function is real-valued, that is, if it returns a scalar value. We can not, for example, take the gradient of Ax, A ∈ Rn×n with respect to x, since this quantity is vector-valued.
while here (section # : gradient of linear functions) and here (D.1.2 Product rules for matrix-functions) and many other places i have observed identities and formulae of taking gradient of vector and matrix valued functions. In fact in the second link3 they talk about gradient for product of matrices.
Can some one help with clearing up on the contradiction ? What is the correct way to understand gradient for matrices.