Use the product rule three times to find a formula for (fg)''' and compare the result with the expansion (a+b)3. Then try to guess a general formula for (fg)(n).
I don't really understand what they are asking me here and it would be really helpful if someone could explain it to me.
Let's break this down.
This means to compute the third derivative of the function $h(x) = f(x)g(x)$. That is, find the derivative $h',$ then differentiate that again, and then differentiate the result. (Cf. second derivative.)
You can just do the ordinary FOIL technique, or, if you know the binomial theorem (which is a big plus here), use it. See what the two expansions have in common.
Can you find the pattern and find a formula for an $n$-th derivative of a product?
These are all the hints I'll give.