I've been following an article on how Neural Networks work. In the article it goes over the math step by step to develop one.
This is the article in question: https://www.analyticsvidhya.com/blog/2017/05/neural-network-from-scratch-in-python-and-r/
I've been following a long in Excel. On Step 10 I am stumped. The formula reads:
wout = wout + matrix_dot_product(hiddenlayer_activations.Transpose, d_output)*learning_rate
Where Matrix_dot_Product() is a function for multiplying matrices. These are the two matrices. $$\begin{pmatrix} .814 & .916 & .814\end{pmatrix}$$ $$\begin{pmatrix} .035\\\ .032\\\ -.13 \end{pmatrix}$$
Wout = .3 and learning rate = .1.
Sorry I cant seem to figure out how to properly express this using MathJax, other than the first formula I posted.
When implementing this in Excel I found that it evaluates to a single value? I thought that because there is matrix multiplication involved, it would evaluate to a matrix, but it doesn't.
Why is this?
In general, if you have a matrix $A$, which is $M\times N$ ($M$ rows, $N$ columns), and a matrix $B$, which is $K\times L$.
Then you can only perform the matrix multiplication $C=AB\,$ if $\,N=K$. If so, then $C=AB$ is $M\times L$. I.e., $$ \underbrace{C}_{M\times L} = \underbrace{A}_{M\times N}\, \underbrace{B}_{K\times L} \;\;\;\text{ if}\;\;\;\; N\equiv K $$ One special case is when $N=K$, $M=1$, $L=1$, in which $$C=AB=(a_1,\ldots,a_N)(b_1,\ldots,b_N)^T = (a_1,\ldots,a_N)\begin{pmatrix}b_1 \\ \vdots \\ b_N\end{pmatrix} = \sum_ja_jb_j$$ so this particular matrix multiplication is just the vector dot product. (Meaning $C$ is $1\times 1$, i.e. $C\in\mathbb{R}$).
On the other hand, we get the outer product when $N=K=1$, which produces a matrix from two 1D matrices (i.e., vectors).