How can I verify that the differential equation:
$$y''_n +(2n\coth x)y'_n+(n^2 -1) y_n =0$$
has homogeneous solution
$$y_n = \left(\frac{1}{\sinh x} \frac{d}{dx}\right)^n (Ae^x + Be^{^-x})$$
for degree $n \in N$
How to start this problem? Should I calulate first and second derivative from d/dn * y_n and later try to insert into equation? I don´t really understand how it can be verified. I will be grateful for all help.
In order to prove the result for arbitrary values of $n$ we can use an induction argument. Assume that the hypothesis holds for a nominal value $n$ and then derive steps to prove that it also holds for $n+1$. This requires that we first verify the hypothesis for low values.
Proving for $n=0$
Suppose that $n=0$. The stated homogenous solution is, $$ y_0 = Ae^x + Be^{-x} \hspace{1cm} \tag{1} $$
and the stated differential equation is, $$ y_0'' - y_n = 0 \hspace{1cm} \tag{2} $$
Taking the first and second derivatives of (1) yields, $$ \begin{align} y_0' =& Ae^{x} - Be^{-x} \\ y_0'' =& Ae^{x} + Be^{-x} \end{align} $$ and so, $$ y_0'' - y_0 = 0 $$
This is that same as stated in (2), and so we can assert that the solution is true for $n=0$.
Proving for $n=1$
Suppose now that $n=1$. The differential equation is, $$ y_1'' + (2\coth{x})y_1' = 0 $$
the proposed solution is, $$ \begin{align} y_1 =& \frac{1}{\sinh{x}} \frac{d}{dx}(Ae^{x} + Be^{-x}) \\ =& \frac{1}{\sinh{x}} (Ae^{x} - Be^{-x}) \hspace{1cm} \tag{3} \end{align} $$
Differentiating (3) gives, $$ \begin{align} y_1' =& (B-A)\mbox{csch}^2{x} \\ y_1'' =& 2(A - B) \coth(x)\mbox{csch}^2(x) \end{align} $$
and so,
$$ \begin{align} y_1'' + (2\coth{(x)})y_1' =& 2(A - B) \coth(x)\mbox{csch}^2(x) + (2\coth{(x)})(B-A)\mbox{csch}^2{(x)} \\ =&2(A - B) \coth(x)\mbox{csch}^2(x) - 2(B-A)\coth{(x)}\mbox{csch}^2{(x)} \\ =& 0 \end{align} $$
as required.
Asserting the hypothesis for arbitrary $n$
Suppose that the hypothesis holds for $n$. We then have that, $$ y_n'' + (2n \coth)y_n' + (n^2 - 1)y_n = 0 \tag{4} $$ and also a solution, $$ y_n = \left(\mbox{csch} \frac{d}{dx}\right)^n (A e^{x} + Be^{-x}) \tag{5} $$
We are free to differentiate Equation (4) with respect to x. This will come in useful futher down. $$ y_n''' + (2n \coth(x)) y_n' - (2n\mbox{csch}^2(x)) y_n' + (n^2 - 1)y_n' = 0 \tag{6} $$
The hypothesis that we are tying to prove is that, $$ y_{n+1}'' + (2(n+1)\coth(x))y_{n+1}' + ((n+1)^2 - 1)y_{n+1} = 0 \tag{7} $$
In order to reach the hypothesis we will first compute the values of $y_{n+1}, y_{n+1}'$ and $ y_{n+1}''$, using the solution in Equation (5). $$ \begin{align} y_{n+1} =& \left(\mbox{csch}(x) \frac{d}{dx} \right) y_n = \mbox{csch}(x)y_n' \tag{8}\\ y_{n+1}' =& \left( \frac{d}{dx} \mbox{csch}(x) \right) y_n' + \mbox{csch}(x) y_n'' \tag{9}\\ y_{n+1}'' =& \left( \frac{d^2}{dx^2} \mbox{csch}(x) \right) y_n' + 2 \left( \frac{d}{dx} \mbox{csch}(x) \right) y_n'' + \mbox{csch}(x) y_n'''\tag{10} \end{align} $$
Notice that equation (10) has a triple derivative in, which needs to be dealt with. Rearranging equation (6) gives that, $$ y_n''' = (2n\mbox{csch}^2(x)) y_n' - (2n \coth(x)) y_n' - (n^2 - 1)y_n' \tag{11} $$
If we now put Equation (11) into Equation (10), and then combine that with Equations (9) and (8) to put into the left hand side of Equation (7); some rearranging gives, $$ \left((2n+1)\mbox{csch}^3(x) - (1 + 2n) \mbox{csch}(x)\coth^2(x) - (n^2 -1)\mbox{csch}(x) + n(n+2) \mbox{csch}(x) \right) y_n' = 0 \tag{12} $$
Which proves the result.
Note: I used Wolfram Alpha to verify Equation (12).