Ok, this is going to be a long one.
So, using generating functions I have to find a closed formula for the number of ways to get n dollars if I have infinite amounts of coins of 1, 2 and 5 dollars.
This is my procedure:
$(1+x+x^2+...)(1+x^2+x^4+...)(1+x^5+x^{10}+...)$
In this polynomial, the coefficient next to $x^n$ should be the solution to the problem. First, I convert the 3 factors into formal series
$\sum_{n=0}^{\infty}x^n\times\sum_{n=0}^{\infty}(x^2)^n\times\sum_{n=0}^{\infty}(x^5)^n$
Now, to multiply those, I need them to be of the format $a_nx^n$ so I split the second and third sum into the following:
$\displaystyle\sum_{n=0}^{\infty}(x^2)^n=\frac{\sum_{n=0}^{\infty}x^n+\sum_{n=0}^{\infty}(-1)^nx^n}{2}=\sum_{n=0}^{\infty}(\frac{1+(-1)^n}{2})x^n$
$\displaystyle\sum_{n=0}^{\infty}(x^5)^n=\sum_{n=0}^{\infty}x^n+\sum_{n=0}^{\infty}(\lfloor{\frac{n}{5}}\rfloor-\lceil{\frac{n}{5}}\rceil)x^n=\sum_{n=0}^{\infty}(1+\lfloor{\frac{n}{5}}\rfloor-\lceil{\frac{n}{5}}\rceil)x^n$
Then I can finally multiply them. I first multiply the first two resulting in
$\displaystyle\sum_{n=0}^{\infty}\sum_{k=0}^{n}\frac{1+(-1)^{n-k}}{2}x^n$
Then I multiply that with the third sum. The result is the following
$\displaystyle\sum_{n=0}^{\infty}\sum_{k=0}^{n}(\sum_{l=0}^{n}\frac{1+(-1)^{n-l}}{2})(1+\lfloor{\frac{n-k}{5}}\rfloor-\lceil{\frac{n-k}{5}}\rceil)x^n$
So, if I'm right, the coefficient next to $x^n$ should be
$a_n=\displaystyle(\sum_{k=0}^{n}\frac{1+(-1)^{n-k}}{2})(\sum_{k=0}^{n}(1+\lfloor{\frac{n-k}{5}}\rfloor-\lceil{\frac{n-k}{5}}\rceil))$
So, plugging in $n=3$ works. I get $2$. Either 2 dollar coins, or one two dollars coin. But when I try $n=5$ I get 6 and I think the result should be 4. What did I do wrong?
The usual way to do this would be to sum each series before you multiply, getting $\frac 1{(1-x)(1-x^2)(1-x^5)}$ and expand that, getting $1+x+2x^2+2x^3+3x^4+4x^5+\dots$ In your case, I believe the upper limit of the $l$ sum should be $k$, not $n$, to get the proper convolution.