How to calculate the gradient from a jacobian

Question

I am trying to do some least-squares fitting (trying to implement for learning). I can calculate the Jacobian. For 100 points, I have a jacobian of 100 rows of 4 columns (I have 4 parameters to fit). How do I calculate the gradient?

Answer 1

If your Jacobian has $100$ rows and $4$ columns you are looking at the partial derivatives of $100$ different functions relative to $4$ variables. Then your Jacobian looks like $$ \begin{bmatrix} \frac{\partial f_1}{\partial x_1}& ... &\frac{f_1}{\partial x_{4}}\\ \vdots & & \vdots\\ \frac{\partial f_{100}}{\partial x_{1}}& ... & \frac{\partial f_{100}}{\partial x_4}\\ \end{bmatrix}. $$ Then the gradient of the $i$th function $f_i$ is the $i$th row of this matrix, or $$ \langle \frac{\partial f_i}{\partial x_1}, \frac{\partial f_i}{\partial x_2}, \frac{\partial f_i}{\partial x_4}, \frac{f_i}{\partial x_4} \rangle. $$