I am reading D.Henry's book "Perturbation of the boundary in Boundary-Value Problems of Partial Differential Equations". At p.24, in the proof of Lemma 2.3 the following assertion is made:
Let $\Omega$ be a fixed $C^2$-regular domain in $\mathbb{R}^N$ and let $h(t,x) = x + t V(x) + o(t)$ as $t \to 0$ be a smooth function (say $C^3$) such that $h(t,\cdot)$ is a (topological) imbedding of $\Omega$ in $\mathbb{R}^N$ for all small $t$. Let $N$ be the outward unit normal to $\Omega$ and let us define $N_{h(t,\Omega)}$ by the formula:
$$ h^*N_{h(t,\Omega)}(x) = N_{h(t,\Omega)}(h(t,x)) = \frac{(D_xh(t,x))^{-T} N(x) }{||(D_xh(t,x))^{-T} N(x)|| } $$
for all $x\in \Omega$ for all small enough $t$. Then, the i-th component of $N_{h(t,\Omega)}$ is given by
$$ N_i - t \sum_j \frac{\partial V_j}{\partial x_i} N_j + t N_i \sum_{j,k} N_j N_k \frac{\partial V_j}{\partial x_k} + o(t)$$
as $t \to 0$. I do not understand how to get this formula, since it is not easy to write down the "contra-gradient" $D_x h(t,\cdot)^{-T}$ explicitly. I think one should try to get information on the structure of $D_x h(t,\cdot)^{-T}$ from the fact that $h(t,x)$ is a perturbation of the identity for small $t$.
Any suggestion would be appreciated. Thanks in advance.
$\def\dt{\frac \partial{\partial t}} \def\ddt#1{\frac {\partial #1}{\partial t}}$Since we're only interested in the first-order term, we don't need a full expression for $(D_x h)^{-T}$ - all we need to know is its time derivative at $t=0$, which is quite tractable since $h$ is a peturbation of the identity. To simplify the notation I'm going to write $Z = (D_x h)^{-T}N$, so that the expansion coefficient you're interested in is $$\dt\Big|_{t=0} \frac Z{|Z|}.$$
To begin with, let's forget about the normalization, so we just need to differentiate $(D_x h)^{-T} N$ in time. First let's compute the time derivative of $D_x h$: from the original formula
$$h(t,x) = x + tV(x) + o(t)$$we see that $$D_xh=I+tDV+o(t);$$so the time derivative at $t=0$ is $$\dt\Big|_{t=0}D_xh=DV.$$
Since $N$ does not depend on $t$, we have $$\dt Z=\dt \left((D_xh)^{-T}\right)N.$$
From the formula for the derivative of the matrix inverse and the fact that transposition is linear (and thus commutes with derivatives) we get
$$ \ddt{Z} = -(D_x h)^{-T} \left(\dt D_x h\right)^T(D_x h)^{-T}N,$$
which evaluated at $t=0$ gives
$$ \ddt Z\Big|_{t=0}= -DV^TN.$$
Now we just need to take the normalization in to account: from the product/chain rules we get $$\dt \frac{Z}{|Z|} = \frac 1 {|Z|} \ddt Z-\frac{1}{|Z|^2}\ddt{|Z|}Z.$$
Since $$\ddt{|Z|} = \frac1{2|Z|} \ddt{|Z|^2} = \frac1{2|Z|}(2 Z \cdot \ddt Z),$$
evaluating everything at $t=0$ gives
$$\dt \frac{Z}{|Z|} \Big|_{t=0} = -DV^TN + (N^T DV^T N) N,$$
which (after converting to index notation) is exactly the coefficient of $t$ in your desired expression for $N_{h(t,\Omega)}$.