second derivative vs. symmetric bilinear form

Question

I would like to understand what is the relationship between symetric bilinear form and second derivative $f''$ and square matrix $A$. What is the setting (the domain of the billinear form, say) so that these notions are well realted ?

Answer 1

If your function $f:\mathbb{R}^n\to\mathbb{R}$ is smooth enough (say $C^2$), then you will have that $$f(a+h)=f(a)+Df_a(h)+\frac{1}{2}D^2f_a(h,h)+o_a(||h||^2),$$ where:

• $Df_a\colon\mathbb{R}^n\to\mathbb{R}$ is the usual derivative of $f$ at the point $a$, i.e. the linear application which matrix in the canonical bases $\mathcal{C}^n$ of $\mathbb{R}^n$ and $\mathcal{C}$ of $\mathbb{R}$ is $$[Df_a]_{\mathcal{C}\mathcal{C}^n}=\left(\frac{\partial f}{\partial x^i}(a)\right)_{1\leq i\leq n},$$
• $D^2f_a\colon\mathbb{R}^n\times\mathbb{R}^n\to\mathbb{R}$ is the second derivative of $f$ at the point $a$, i.e. the bilinear form which matrix in the canonical basis $\mathcal{C}^n$ of $\mathbb{R}^n$ is
  $$[D^2f_a]_{\mathcal{C}^n}=\left(\frac{\partial^2f}{\partial x^i\partial x^j}(a)\right)_{1\leq i,j\leq n}.$$ Thus, the above formula links the three concepts you are listing, establishing the best second order approximation of $f$ can be expressed by the mean of $f$ and its first and second derivatives when $f$ is smooth enough.

Answer 2

One of the possible ways to think of the second derivative (most useful for higher dimensional equivalent of the Taylor series) is to define the second derivative as

$$ df^{(2)}(a)(h,k):=\partial_h(\partial_kf(a)) $$

where $h,k\in\mathbb R^n$ and $\partial$ are directional derivatives. This has the benefit that in the case where these directional derivatives exist for all directions, then the second derivative as defined above defines a bilinear form with respect to $(h,k)$ on $\mathbb R^n$. When $h=k$ this can be thought of as a homogenous polynomial of the second degree. From linear algebra we know that bilinear forms can be represented by a matrix. The one corresponding to the second derivative is denoted by $H_f(a)$ and is usually called the Hessian of $f$ at $a$ which means

$$ df^{(2)}(h,k) = h^TH_f(a)k $$

In case when all second partial derivatives exist and are continuous, you can show via Schwartz's theorem that the second derivative is symmetric which makes the Hessian also symmetric. The domain of the form is as stated above $\mathbb R ^n\times \mathbb R^n$