What is the Jacobian Matrix conceptually?

Question

I am having trouble with understanding the Jacobian Matrix and what it means conceptually. Can anyone explain this? A concrete example would be greatly appreciated.

Answer 1

I like to think of the Jacobian matrix as a sort of local linear (really, affine) approximation of a function. Consider $f\colon\mathbb{R}^m\to\mathbb{R}^n$ and a point $x^*\in\mathbb{R}^m$. Then from the Taylor series expansion of $f$ about $x^*$, we can drop the second-order and higher terms to obtain the following approximation of $f$: \begin{equation*} f(x) \approx f(x^*) + Df(x^*)(x-x^*), \end{equation*} where $Df(x^*)$ is the Jacobian of $f$ evaluated at $x^*$. In my head, I typically visualize some nonlinear surface defined by $f$ and a plane that is tangent to the surface at the point $x^*$. This tangent plane captures the linear behavior of $f$ around $x^*$ if you zoom into a very small neighborhood of $x^*$. When we compute the Jacobian, we are determining this tangent plane.

A particularly useful example of Jacobian's and this idea of Jacobian linearization is found in the analysis and control of nonlinear dynamical systems. Consider the system of differential equations \begin{equation*} \dot{x} = f(x) = \begin{bmatrix} x_2 \\ -\sin x_1 \end{bmatrix}, \end{equation*} which is a commonly used model for the behavior of a simple pendulum without friction. The function $f\colon\mathbb{R}^2\to\mathbb{R}^2$ is nonlinear, so many traditional techniques from system analysis and control become difficult to apply without some extra work. One thing that is immediately clear is that the point $x_e = (\pi,0)$ is an equilibrium of the system, since $f(x_e)=f(\pi,0) = (0,0)$, indicating the system doesn't move since $\dot{x}=0$. This makes sense intuitively if you were to imagine setting a pendulum in its upright state without any velocity. Also based on our intuition, we expect this equilibrium to be unstable, as any slight movement of the pendulum away from this upright position is going to further move the system away from the upright equilibrium. However, based on the nonlinear model, it might be difficult to mathematically prove this instability. Therefore, let's apply Jacobian linearization and see if that helps us. Doing so gives us the following linear approximation of the system's dynamics around the point $x_e$: \begin{equation*} \dot{\delta x} = Df(x_e) \delta x = \begin{bmatrix}0 & 1 \\ -\cos \pi & 0 \end{bmatrix} \delta x = \begin{bmatrix} 0 & 1 \\ 1 & 0 \end{bmatrix}\delta x, \end{equation*} where $\delta x = x-x_e$. The eigenvalues of the Jacobian are $\lambda_1(Df(x_e)) = 1$ and $\lambda_2(Df(x_e)) = -1$. From linear system's analysis, it is now immediately clear that the equilibrium $x_e$ is unstable for the linearized system, and therefore we conclude that $x_e$ looks like an unstable saddle point locally where the linearization is a good approximation of the true nonlinear system (indeed, this conclusion can be made rigorous via Hartman-Grobman theorem).