I am new to generating functions / power series and I am struggling with expanding following generating function: $$F(x)=\frac{(x^2-x^{10})(1-x^{10})^3}{(1-x)^4}$$ I tried to pattern my solution on that answer and expand generating function in the same way, so i rewrote $F(x)$ as: $$F(x)=\frac{x^2(1-x^8)(1-x^{10})^3}{(1-x)^4}$$ and in order to obtain coefficient at $x^{20}$ I tried to calculate: $${(20-2)+3 \choose 3}-{(20-2)-8+3 \choose 3}-{(20-2)-10+3 \choose 3} \cdot 3$$ The result of this is $549$ which is smaller by $3$ than proper solution which is $552$. I don't know what I did wrong. In fact, I don't understand this method of expanding coefficients so if you could provide the name of this technique or some resources to read about I would be thankful. I mean, I see the pattern this technique follows, but I don't see why it works.
This generating function gives solution to one problem that I solved first in purely combinatoric way. My calculations were nearly identical to these presented above, but I had to exclude intersection of two sets which was of size $3$. So I understand that there should be $+3$ term in solution above, but I don't know where it would come from in case of expanding generating function.
I think that you missed one term. Notice that $$(1-x^8)(1-x^{10})^3=(1-x^8)(1-3x^{10}+O(x^{20}))=1-x^8-3x^{10}+3x^{18}+o(x^{18}).$$ Hence \begin{align} [x^{20}]\frac{x^2(1-x^8)(1-x^{10})^3}{(1-x)^4} &=[x^{18}]\frac{1-x^8-3x^{10}+3x^{18}+o(x^{18})}{(1-x)^4}\\ &=[x^{18}]\frac{1}{(1-x)^4}-[x^{10}]\frac{1}{(1-x)^4}-[x^{8}]\frac{3}{(1-x)^4}+[x^{0}]\frac{3}{(1-x)^4}\\ &=\binom{18+3}{3}-\binom{10+3}{3}-3\binom{8+3}{3}+3=552. \end{align}