Confusion regarding Ito Differentiation

94 Views Asked by At

Suppose $x_t \in \mathbb{R}^{n}$ is the stochastic process

$$ \mathrm{d}x_t = (Ax_t + Bu_t) \textrm{d} t + D\textrm{d}w_t, \quad t\in [0,T], $$ where $w_t$ is the standard Brownian motion. Suppose I also have a function $f(x_t) := \frac{1}{2}x_tP_tx_t$, where $P_t = P_t^\intercal > 0$ is a positive-definite, symmetric matrix that satisfies the Riccati equation

$$ -\frac{\textrm{d}P_t}{\textrm{d}t} = P_t A + A^\intercal P_t - P_t(BR^{-1}B^\intercal-\mu DD^\intercal)P_t + Q, \quad P_{T} = M. $$

I am currently reading a paper that states:

\begin{align*} f_T - f_0 &= \frac{1}{2}\int_{0}^{T}\left(x_t^\intercal\left[P_tBR^{-1}B^\intercal P_t -Q\right] x_t + 2u_t^\intercal B^\intercal P_t x_t\right)\textrm{d} t + \int_{0}^{T} x_t^\intercal P_t D \textrm{d} w_t \\ &- \frac{1}{2}\int_{0}^{T} \left(\mu x_t^\intercal P_t D D^\intercal P_T x_t + \textrm{tr}\left[P_t DD^\intercal\right]\right) \textrm{d} t, \end{align*}

and I would like to verify this myself but am having difficulties. The first step is to recognize that the endpoint difference $f_T - f_0$ can be computed from the Ito differential

$$ f_T - f_0 = \int_{0}^{T} \textrm df_t. $$

Next, Ito's lemma states

$$ \textrm{d} f(x_t) = \left[(\nabla_x f)^\intercal (Ax_t + Bu_t) + \frac{1}{2}\textrm{tr}(D^\intercal H_xf D)\right]\textrm{d}t + (\nabla_x f)^\intercal D \textrm{d} w_t, $$

where $\nabla_x f$ denotes the gradient and $H_x f$ denotes the Hessian. Computing the gradient gives $\nabla_x f = P_t x_t \in \mathbb{R}^n$ and the Hessian $H_x f = P_t \in \mathbb{R}^{n\times n}$. As a result,

$$ \textrm{d} f_t = \left[x_t^\intercal P_t(Ax_t + Bu_t) + \frac{1}{2}\textrm{tr}(D^\intercal P_t D)\right] \textrm{d} t + x_t^\intercal P_t D \textrm{d} w_t. $$

However, in the paper, it is clear that the differential $\textrm{d} P_t$ appears in the integrals, but I am not seeing where this differential comes from in Ito's lemma. Any help would be appreciated!

1

There are 1 best solutions below

3
On BEST ANSWER

Here $f$ is also a function of $t$ through $P_{t}$, you should use Itô's formula for $f\left(t,x_{t}\right)$, where the drift term is $$\begin{align*} &\quad\ \ \frac{\partial f}{\partial t}+\left(\nabla_{x}f\right)^{\top}\left(Ax_{t}+Bu_{t}\right)+\frac{1}{2}\mathrm{tr}\left(D^{\top}H_{x}fD\right)\\ &=\frac{1}{2}x_{t}^{\top}\left[-P_{t}A-A^{\top}P_{t}+P_{t}\left(BR^{-1}B^{\top}-\mu DD^{\top}\right)P_{t}-Q\right]x_{t}\\ &\ \ +x_{t}^{\top}P_{t}\left(Ax_{t}+Bu_{t}\right)+\frac{1}{2}\mathrm{tr}\left(D^{\top}P_{t}D\right)\\ &=\frac{1}{2}\left[x_{t}^{\top}\left(P_{t}BR^{-1}B^{\top}P_{t}-Q\right)x_{t}+2x^{\top}P_{t}Bu_{t}+\mathrm{tr}\left(D^{\top}P_{t}D\right)-\mu x_{t}^{\top}P_{t}DD^{\top}P_{t}x_{t}\right]. \end{align*}$$