How to update the weights knowing the loss in Neural Network

Question

Question

How update the weight knowing the loss

Work explanation

I have a really simple network composed by two layers with one neurons in both layers.

Considering an input of 1, the final result of this NN is 0.50 and the expected is 1

Now to train this simple NN I've calculated the loss following the reasoning explained in this video: https://youtu.be/Zr5viAZGndE?t=729

As you can see the loss is -0.249 and I know that lowering this error mean get a result near the expected.

The problem that I'm encountering is how to update the the weight of 0.03 to something that let the Neural Network lower the loss and outputs 1 as result on the next iteration.

Answer 1

After some studies I understood that the loss calculated with the above formula is the delta weight that can be directly added to the weight.

Initially I got confused because the problem of this formula was the first term of the chain rule where the inputs are inverted and the adjustment was going in the opposite direction.

So the correct is 2(expected - gussed)(sig'(input))(input)(learning rate)