Let $I(a,b):= \int_0^{\pi/2}\ln(a^2\cos^2x+b^2\sin^2x)\,dx$
Calculate $I(a,b)$.
My attempt:
Define function $F(a,b,x)$ as following $\frac{\partial f}{\partial a}F(a,b,x)=\ln(a^2\cos^2x+b^2\sin^2x)$
Then $I(a,b):= \int_0^{\pi/2}\ln(a^2\cos^2x+b^2\sin^2x)dx= \int_0^{\pi/2}\frac{\partial f}{\partial a}F(a,b,x)= \frac{\partial f}{\partial a}\int_0^{\pi/2}F(a,b,x)$
The last equality is done by using Leibniz integral rule, I must prove that the function is well defined in a 3-dimension cube.
Calculating F:
$F(a,b,x)=\int \ln(a^2\cos^2x+b^2\sin^2x)\,da$
Using the integral by parts, $u'=1, u=a, v=\ln(a^2\cos^2x+b^2\sin^2x), v'=\frac{2a\cos^2x}{a^2\cos^2x+b^2\sin^2x}$
$F(a,b,x)=uv-v'u=a\ln(a^2\cos^2x+b^2\sin^2x)\space-\space \int\frac{2a^2\cos^2x}{a^2\cos^2x+b^2\sin^2x}\,da= a{\ln(a^2\cos^2x+b^2\sin^2x)} \space-\space \int\frac{2a^2\cos^2x}{a^2\cos^2x+b^2\sin^2x}da= a{\ln(a^2\cos^2x+b^2\sin^2x)}\space-\space \int\frac{2a^2\cos^2x+b^2\sin^2x-b^2\sin^2x}{a^2\cos^2x+b^2\sin^2x}da= a{\ln(a^2\cos^2x+b^2\sin^2x)}-\int 1-\frac{b^2\sin^2x}{(a^2\cos^2x+b^2\sin^2x)}da$
And I am stuck.. I don't know how to handle that integral. I am not even sure I am solving it correctly..
Any tips?
Hint. One may set $$ f(s):=\int_0^{\pi/2}\ln(s+\sin^2 x)dx, \qquad s\geq0. $$ Then differentiating under the integral sign with respect to $s$ you get $$ \begin{align} f'(s)&=\int_0^{\pi/2}\frac1{s+\sin^2 x}dx\\\\ &=\int_0^{\infty}\frac1{s+\dfrac{t^2}{t^2+1}}\dfrac{dt}{t^2+1}\quad (t=\tan x)\\\\ &=\int_0^{\infty}\frac1{(s+1)t^2+s}dt\\\\ &=\frac{\pi}2\frac{1}{\sqrt{s(s+1)}}\\\\ &=\pi \left.\left(\ln \left(\sqrt{s}+\sqrt{s+1}\right)\right)\right|_s^{'} \end{align} $$ Thus $$ \int_0^{\pi/2}\ln(s+\sin^2 x)dx=\pi \ln \left(\sqrt{s}+\sqrt{s+1}\right)+C $$ with $C=f(0)=-\pi \ln 2$ (this one is standard) giving
Assume without loss of generality that $b^2>a^2$, then your initial integral is obtained by writing $$ \ln\left(a^2\cos^2x+b^2\sin^2x\right)=\ln(b^2-a^2)+\ln\left(s+\sin^2x\right). $$ with $s=\dfrac{a^2}{b^2-a^2}$.