I've got following recursive equation to solve:
$$a_n = \begin{cases} 5, & n = 0 \\ 3, & n = 1 \\ 5n^2 - 6n - 4 + a_{n-1}, & n \ge 2 \end{cases}$$
I also found the resulting generating function (I think that should be correct):
$$F(x) = z^2F(x) + \frac{5(x+1)}{(1-x)^3} - \frac{6}{(1-x)^2} + \frac{2}{1-x} +8x + 9$$
Edit: Which of course would give me: $$F(x) = \left(\frac{1}{1-x^2}\right)\times \left( \frac{5(x+1)}{(1-x)^3} - \frac{6}{(1-x)^2} + \frac{2}{1-x} +8x + 9\right)$$
How should I go forward?
Any help is greatly appreciated!
For $n\geq 1$, the solution of the linear recurrence of the first order $$a_n -a_{n-1}=5n^2 - 6n - 4\quad\mbox{for $n \ge 2$, with $a_1=3$}$$ should have the form $$a_n=An^3+Bn^2+Cn+D$$ where $A,B,C,D$ are constants to be determined. In fact, for $n\geq 1$, \begin{align*} a_n&=a_1+\sum_{k=2}^n(a_k -a_{k-1})=3+\sum_{k=2}^n (5k^2-6k-4)\\ &= 3+10\sum_{k=2}^n \binom{k}{2}-\sum_{k=2}^n k-4\sum_{k=2}^n1\\ &=3+10\binom{n+1}{3}-\binom{n+1}{2}+1-4(n-1)\\ &=\frac{5n^3}{3}-\frac{n^2}{2}-\frac{37n}{6}+8. \end{align*} The case $n=0$ should be considered separately: $$a_n = \frac{5n^3}{3}-\frac{n^2}{2}-\frac{37n}{6}+ \begin{cases} 5 & n = 0 \\ 8 & n \ge 1 \end{cases}$$