Coefficient of a generating function

Question

I have the following generating function, $$f(x)=(1+x+x^2+x^3)^4$$ I want to find the $[x^5]$ coefficient, to do so I wrote, $$[x^5]f(x)=[x^5](1+x+x^2+x^3)^4=[x^5]\left(\frac{1-x^4}{1-x}\right)^4= [x^5](1-x^4)^4(1-x)^{-4} $$ I am wondering how to proceed from here?

Answer 1

We obtain \begin{align*} \color{blue}{[x^5]}&\color{blue}{(1-x^4)^4(1-x)^{-4}}\\ &=[x^5]\sum_{j=0}^\infty\binom{-4}{j}(-x)^{j}(1-x^4)^4\tag{1}\\ &=\sum_{j=0}^5\binom{3+j}{3}[x^{5-j}](1-x^4)^4\tag{2}\\ &=\sum_{j=0}^5\binom{8-j}{3}[x^{j}]\sum_{k=0}^4\binom{4}{k}(-1)^kx^{4k}\tag{3}\\ &=\binom{8}{3}\binom{4}{0}-\binom{4}{3}\binom{4}{1}\tag{4}\\ &\,\,\color{blue}{=40} \end{align*}

Comment:

• In (1) we apply the binomial series expansion to $(1-x)^{-4}$.

• In (2) we use the binomial identity $\binom{-p}{q}=\binom{p+q-1}{q}(-1)^q$ and apply the rule $[x^{p-q}]A(x)=[x^p]x^qA(x)$. We set the upper limit of the series to $5$ since other terms do not contribute to $[x^5]$.

• In (3) we change the order of summation $j\to 5-j$ and expand the binomial.

• In (4) we select the coefficients of $x^0$ and $x^4$. Other terms do not contribute.

Answer 2

Write the two binomials in terms of series: $$[x^5](1-x^4)^4(1-x)^{-4}=[x^5]\sum_{i=0}^{4} {4\choose i}(-x^4)^i \cdot \sum_{j=0}^{\infty}{-4\choose j}(-x)^j=\\ [x^5]{4\choose 0}(-x^4)^\color{green}{0}\cdot \color{red}{{-4\choose 5}}(-x)^\color{green}5+{4\choose 1}(-x^\color{green}4)^\color{green}1\cdot \color{blue}{{-4\choose 1}}(-x)^\color{green}1=\\ [x^5]\color{red}{{4+5-1\choose 5}(-1)^5}(-x)^5-4x^4\color{blue}{{4+1-1\choose 1}(-1)^1}(-x)^1=\\ {8\choose 5}-4{4\choose 1}=40,$$ where it was used negative binomial series: $$(x+a)^{-n} = \sum_{k=0}^{\infty} {-n\choose k}x^ka^{-n-k};\\ {-n\choose k}={n+k-1\choose k}(-1)^k.$$