Using generating functions, find the number of ways to divide $n$ coins into $5$ boxes that meet the condition that the number of coins in boxes $1$ and $2$ is an even number that does not exceed $10$, the remaining boxes contain $3$ to $5$ coins.
I have found that the generating function of this problem is $(1+x^2+x^4+x^6+x^8+x^{10})^2(x^3+x^4+x^5)^3$. I have tried to convert it into $\frac{(x^{12}-1)^2x^9(x^3-1)^3}{(x^2-1)^2(x-1)^3}$ however I can't improve any further more.
All that is needed is to multiply out your product and read of the answer:
$x^{35} + 3 x^{34} + 6 x^{33} + 7 x^{32} + 8 x^{31} + 9 x^{30} + 15 x^{29} + 20 x^{28} + 27 x^{27} + 29 x^{26} + 36 x^{25} + 39 x^{24} + 47 x^{23} + 46 x^{22} + 49 x^{21} + 45 x^{20} + 49 x^{19} + 46 x^{18} + 47 x^{17} + 39 x^{16} + 35 x^{15} + 26 x^{14} + 21 x^{13} + 13 x^{12} + 8 x^{11} + 3 x^{10} + x^{9}$
That reads there is 1 possible way to put 35 coins in the boxes (20 in the 2 boxes and 3*5 in the 3 boxes. There is 1 possible way to put in 9 coins (zero in the 2 boxes and 3 times 3 in the 3 boxes). There are zero ways to put more then 35 or less then 9. No need to "simplify" it into a fraction where I think you did it wrong.