What is the correct formula for updating the weights in a 1 hidden layer neural network?

Question

I'm creating a neural network with 3 layers and no bias.

On internet I saw that the expression for the derivative of the weights between the hidden layer and the output layer was: $$\Delta W_{j,k} = (o_k - t_k) \cdot f'\left[\sum_j (W_{j,k} \ \cdot o_j)\right] \cdot o_j,$$ where $t$ is the target output, $o$ is the activated output layer and $f'$ the derivative of the activation function.

But the shape of these weights is $\text{output nodes}\times\text{hidden nodes}$, and $\text{hidden nodes}$ can be bigger than $\text{output nodes}$, so the formula is wrong because of $o$ has length $\text{output nodes}$.

1. What is the right formula to updating these weights?
2. What is the right formula for updating the weights between the input layer and the hidden layer?

Answer 1

1. The correct formula for updating the weights between the hidden layer and the output layer is: $$\Delta W_{j,k} = h_k \ \cdot \ o'_{j} \ \cdot \ (o_j - t_j),$$ where $h$ is the activated hidden layer and $o'$ is the derivative of output layer.