Let $\mu$ and $\lambda$ be Radon measures on $\mathbb{R^n}$ such that $\mu << \lambda$. Prove that
$\displaystyle \int D(\mu,\lambda,x)^2 d\lambda x= \int D(\mu,\lambda,x)d\mu x$.
Is it possible to do
$\begin{align} \int D(\mu,\lambda,x)^2 d\lambda x &= \int D(\lambda, \mu,x)^{-1} D(\mu,\lambda,x) d\lambda x \\ &= \int D(\lambda, \mu,x)^{-1} d\mu x \\ &= \int D(\mu,\lambda,x) d\mu x \end{align}$
Otherwise I cannot see how you would do a substitution.
Ok, let me give you a detailed answer:
By definition of the Radon-Nikodym derivative, you have
$$ \int \chi_A (x) \, d\mu(x) = \mu(A) = \int_A D(\mu, \lambda, x) d\lambda(x) = \int \chi_A (x) \cdot D(\mu, \lambda, x) d\lambda(x) $$
for every measurable set $A$. By linearity, this just means
$$ \int f(x) \, d\mu(x) = \int f(x) \cdot D(\mu, \lambda, x)\,d\lambda(x) \qquad \qquad (\dagger) $$
for every simple (nonnegative) function $f : X \rightarrow [0,\infty]$. Using a standard monotone convergence argument, this implies that $(\dagger)$ is also true for all non-negative, measurable functions $f :X \to [0,\infty]$.
Now take $f = D(\mu, \lambda, x)$.