Find determinant $$D_n(a,b,c)= \begin{vmatrix} a & b & 0 & 0 & \cdots & 0 & 0 & 0 \\ c & a & b & 0 & \cdots & 0 & 0 & 0 \\ 0 & c & a & b & \cdots & 0 & 0 & 0 \\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots \\ 0 & 0 & 0 & 0 & \cdots & c & a & b \\ 0 & 0 & 0 & 0 & \cdots & 0 & c & a \end{vmatrix} $$ for $a,b,c \in \mathbb{R}$.
I used recurrence relations, $$D_n=aD_{n-1}-bcD_{n-2}$$ Now, we must consider these cases:
If $b=0 \lor c=0$ then $$D_n=a^{n-1}D_1=a^n\frac{1}{a}\times a=a^n$$
If $b\neq 0 \land c\neq 0$ then we solve quadratic equation $$x^2-px-r=0$$ where $p=a$ and $r=-bc$, so equation is $$x^2-ax+bc=0$$
Roots of this equation are $$x_1=\frac{a+\sqrt{a^2-4bc}}{2},x_2=\frac{a-\sqrt{a^2-4bc}}{2}$$
2.1. If $x_1\neq x_2$ then $$D_n=k_1x_1^n+k_2x_2^n$$ where $$k_1=\frac{D_2-x_2D_1}{x_1(x_1-x_2)},k_2=-\frac{D_2-x_1D_1}{x_2(x_1-x_2)}$$
Solving for $k_1$ and $k_2$, $$k_1=\frac{a^2-2bc+a\sqrt{a^2-4bc}}{a^2-4bc+a\sqrt{a^2-4bc}}$$ $$k_2=\frac{2bc-a^2-a\sqrt{a^2-4bc}}{4bc+a\sqrt{a^2-4bc}-a^2}$$
This gives $D_n=\frac{a^2-2bc+a\sqrt{a^2-4bc}}{a^2-4bc+a\sqrt{a^2-4bc}}\left(\frac{a+\sqrt{a^2-4bc}}{2}\right)^n+\frac{2bc-a^2-a\sqrt{a^2-4bc}}{4bc+a\sqrt{a^2-4bc}-a^2} \left(\frac{a-\sqrt{a^2-4bc}}{2}\right)^n$
2.2. If $x_1=x_2$ then $$D_n=x_1^{n-1}D_1+(n-1)x_1^{n-2}(D_2-x_1D_1)$$ $$D_n=a\left(\frac{a+\sqrt{a^2-4bc}}{2}\right)^{n-1}+(n-1)\left(\frac{a+\sqrt{a^2-4bc}}{2}\right)^{n-2}\times \frac{2bc-a^2-a\sqrt{a^2-4bc}}{2}$$
Could someone check this? Are there any more cases to consider?
Thanks for replies.
Your solution looks ok to me. You have all cases covered.
The determinant $D_n$ is a continuous function of $a,b,c$, and it is a good idea to check that your formulas obey this. If you let $c\to0$ in your last formula of 2.1, you get $D_n=a^n$ as you should. In the case $x_1=x_2$ you have $a^2-4bc=0$ and $x_1=x_2=a/2$, which simplifies your formula significantly: $$ D_n=a(a/2)^{n-1}+(n-1)(a/2)^{n-2}\times\frac{-a^2}4 = a^n2^{-n}(3-n). $$