Gradient for a loss function

Question

I am relatively new to math-notations, and am currently trying to understand how to program an ik solver using gradient descent. I need to calculate the gradient of a simple loss function using partial derivatives, and implement it in python, but don't really know how with this function. I think I am confused to what to do with the symbols. For the loss function I have:

$\Vert x_{current} - x_{goal}\Vert^2$

where

$x_{current}, x_{goal} \in \Bbb R^2$

I am just confused on how to derive this function and then implement it into code, since I dont know what to do with the symbols.

Answer 1

So the goal with gradient descent is to adjust your function in the direction of the negative gradient of the loss function. For example with linear regression, your function is $ax+b$ and you can adjust a and b via gradient descent by looking for the derivative of the cost (or loss) function $\sum_{i=0}^n\Vert y_{i} - y_{guess_{i}}\Vert^2$ with respect to a and b. This will give you a sequential update of a and b via the chain and product rule as follows:

$ a_{new} = a_ {previous} + \Vert y_{current} - y_{guess}\Vert * x * learning rate $

$ b_{new}= b_{previous} + \Vert y_{current} - y_{guess}\Vert* learning rate $