The values of $\lambda$ for which the following equation has a non-trivial solution $\phi(x)=\lambda\int_0^{\pi} K(x,t)\phi(t)dt$ where $0\leq x\leq \pi$ and $K(x,t)= \begin{cases} \sin x\cos t & 0\leq x\leq t \\ \cos x\sin t & t\leq x\leq \pi \end{cases} $ are
$(a)$ $\bigg(n+\frac{1}{2}\bigg)^2-1$, $n\in \mathbb{N}$.
$(b)$ $n^2-1$, $n\in \mathbb{N}$.
$(c)$ $\frac{1}{2}(n+1)^2-1$, $n\in \mathbb{N}$.
$(d)$ $\frac{1}{2}(2n+1)^2-1$, $n\in \mathbb{N}$.
I converted this integral equation into its equivalent ordinary second order linear differential equation $\phi''(x)+2\lambda\cos^2(x)\phi(x)=0$ with mixed boundary conditions $\phi(0)=0$ and $\phi'(\pi)=0$. Next I need to find the eigenvalues of this ODE to find the required answer. $\lambda=0$ will give us trivial solution, so this possibility is discarded. But cannot check for the other two cases $\lambda=\mu^2>0$ and $\lambda=-\mu^2<0$. Any help will be appreciated in this regard.
Unfortunately there seems to be a mistake in the above. Substituting the kernel in and given it's form, it is easy to prove that:
$$\phi(x)=\lambda\sin x\int_{x}^\pi dt \cos t~\phi (t)+\lambda\cos x\int_{0}^x dt \sin t~\phi (t)$$
Then applying derivatives successively we can also prove that:
$$\phi'(x)=\lambda\cos x\int_{x}^\pi dt \cos t~\phi (t)-\lambda\sin x\int_{0}^x dt \sin t~\phi (t)$$
$$\phi''(x)=-\lambda\sin x\int_{x}^\pi dt \cos t~\phi (t)-\lambda\cos x\int_{0}^x dt \sin t~\phi (t)-\lambda\cos^2 x~\phi(x)-\lambda\sin^2x~\phi(x)=-(1+\lambda)\phi(x)$$
We also obtain the constraints $\phi(0)=0,\phi'(\pi)=0$ as mentioned in the question.
and now we can solve the equation setting $\omega=\sqrt{1+\lambda}$:
$$\phi(x)=A\cos(\omega t)+B\sin(\omega t)$$
and applying the constraints we obtain
$$A=0~~,~~\omega\pi=(n+\frac{1}{2})\pi$$
and therefore the eigenvalues that produce legitimate solutions to the equation are given by
$$\lambda_n=(n+\frac{1}{2})^2-1$$