I have read about Hadamard products between two matrices. I am on a quest to find something very similar for computing average cross-entropy gradient given:
$$ \nabla E_j = (x_{1j}(y_j - \hat{y_j}), \dots, x_{nj}(y_j - \hat{y_j})) $$
and
$$X= \begin{bmatrix} x_{11} & x_{12} & x_{13} & \dots & x_{1n} \\ x_{21} & x_{22} & x_{23} & \dots & x_{2n} \\ \\ x_{m1} & x_{m2} & x_{m3} & \dots & x_{mn} \end{bmatrix} $$
I want to compose an equation such that $X$ times something else gives me:
\begin{bmatrix} x_{11}(y_1 - \hat{y_1}) & \dots & x_{1n}(y_1 - \hat{y_1}) \\ x_{21}(y_2 - \hat{y_2}) & \dots & x_{2n}(y_2 - \hat{y_2}) \\ \\ x_{31}(y_3 - \hat{y_3}) & \dots & x_{3n}(y_3 - \hat{y_3}) \end{bmatrix}
So basically, I want to compose a matrix whose rows are essentially the rows of X each multiplied by the scalar $y_j - \hat{y}_j$
I have found a way to do this with Numpy:
(X.T * (y - y_hat)).T
Which in mathy notation is essentially this:
$$ (X^T * (y - \hat{y}))^T $$
I am wondering if this * operation in Numpy has a name. It is not (I don't think) an inner product of any kind. Does it have a basis in standard linear algebra?
You could think of it as just the dot product between the vector X where all your columns are organized into a vector and the vector Y which is your $y - \hat{y}$ vector repeated m times, and clearly this space of vectors is a metric space. So just take your matrices and put them as vectors (this clearly is an isomorphism) and your $y - \hat{y}$ repeated m times (which will be an element of this space) and will end up working with the normal old dot product over the field you're working on.