The question asks:
a) Find a generating function for the number of ways to distribute identical pieces of candy to 3 children so that no child gets more than 4 pieces. Write this generating function in closed form, as a quotient of polynomials. b) Find the number of ways to distribute 10 pieces of candy using this generating function.
I figured out for part a that $(1 + x + x^2 + x^3 + x^4)^3$ is represented as the generating function: $$f(x) = \left( \frac{1-x^5}{1-x} \right) ^3$$ But I don't know how to find the $a_{10}$ term. Please help.
You need to manipulate first term of $(1-x^5)^3 (1-x)^{-3}$ in such way that would allow you to find the coefficients. The coefficients up to $x^{10}$, i.e. $a_{10}$, will reveal you the different ways you may distribute.
Hint: Binomial theorem.