Given $h(\alpha) = f(x+\alpha v)$, find $h''(\alpha)$

Question

Suppose that $f$ is a twice differentiable function and $\alpha \in \mathbb{R}, x,v \in \mathbb{R}^n$ and $x+\alpha v$ lies in the domain of $f$.

Let $h(\alpha) = f(x + \alpha v)$ and assume $y(\alpha) = x+\alpha v$

Then

\begin{align*} \dfrac{dh}{d\alpha} &= \sum\limits_{i = 1} ^n\dfrac{\partial f(y(\alpha))}{\partial y_i} \dfrac{dy_i(\alpha)}{d \alpha} \\ & = \begin{bmatrix} v_1 & \ldots & v_n \end{bmatrix} \begin{bmatrix} \dfrac{\partial f(y(\alpha))}{\partial y_1} \\ \vdots \\ \dfrac{\partial f(y(\alpha)}{\partial y_n} \end{bmatrix} = v^T\nabla f(x + \alpha v)\end{align*}

However, I have trouble taking a second derivative,

\begin{align*} \dfrac{d^2h}{d\alpha^2} &= \dfrac{d}{d\alpha} \sum\limits_{i = 1} ^n\dfrac{\partial f(y(\alpha))}{\partial y_i} \dfrac{dy_i(\alpha)}{d \alpha} \\ & = \sum\limits_{i = 1} ^n\dfrac{\partial^2 f(y(\alpha))}{\partial^2 y_i} (\dfrac{dy_i(\alpha)}{d \alpha})^2\\ & = \begin{bmatrix} v_1^2 & \ldots & v_n^2 \end{bmatrix} \begin{bmatrix} \dfrac{\partial^2 f(y(\alpha)}{\partial^2 y_1} \\ \vdots \\ \dfrac{\partial^2 f(y(\alpha)}{\partial^2 y_n} \end{bmatrix} \neq v^T\nabla^2 f(x + \alpha v)v\end{align*} which is not equal to the expected answer.

Can someone please help me figure out what went wrong?

Answer 1

Apply chain rule for the second derivative, you must.

$$\frac{d^2h}{d\alpha^2}=\frac{d}{d\alpha}\sum_{i=1}^n\frac{\partial f(y)}{\partial y_i}\frac{\partial y_i}{\partial \alpha} = \sum_{i=1}^n\left[\frac{\partial f(y)}{\partial y_i}\frac{\partial^2y_i}{\partial\alpha^2}+\frac{\partial^2f(y)}{\partial y_i^2}\left(\frac{\partial y_i}{\partial\alpha}\right)^2\right]$$

Take it from here, can you?

Answer 2

When you use the chain rule in the computation of $2$-derivatives, precisely here: $\dfrac{d}{d\alpha}\left(\dfrac{\partial f(y(\alpha))}{\partial y_i}\right)$, you need to remember what's inside is still a function on $\mathbb{R}^n$. This means you will need to do the same thing you did with the first time. Let me know if you still have trouble with this.