Let $f(x)=3x+2$ and $g(x)=ax+b$, for some constants $a$ and $b$. If $ab=20$ and $f(g(x))=g(f(x))$ for $x=0,1,2\ldots 9$, find the sum of all possible values of $a$.
I understand what function composition is, but it's the last condition that really trips me up. Help is greatly appreciated.
First the idea: you want one $a$ that satisfies two conditions: first you should have $f(g(x))=g(f(x))$, and second $ab = 20$. You can see the second requirement as a constraint that you will use somewhere in the calculation.
The first condition is a matter of using the definition of composition. Recall that if you know $f$, namely, for every number $y\in \mathbb{R}$ you know what $f(y)$ is, then to compute the composition $f\circ g$ you pick the formula for $f(y)$ and substitute $y$ everywhere by $g(x)$. In other words: you know what $f$ does to any number, then you know what it does specially to the numbers of the form $g(x)$. This gives you the composition.
In practice, to your problem: apply the definition of composition of functions to evaluate both sides and equate them. We have
$$f(g(x))=3g(x)+2=3(ax+b)+2=3ax+3b+2$$
$$g(f(x))=af(x)+b=a(3x+2)+b=3ax+2a+b$$
Now equate both sides, you have
$$3b+2=2a+b$$
Now this is one equation for two unknowns, but since $ab = 20$, multiplying the above equation by $a$ you have
$$3ab+2a=2a^2+ab.$$
Notice that the idea here is that multiplying by $a$ you make $ab$ appear on the equation and allows you to impose the constraint. From this
$$60+2a=2a^2+20$$
This is equivalent in turn to
$$2a^2-2a-40=0$$
Thus one $a$ compatible with $f\circ g = g\circ f$ and $ab=20$ is a solution to the equation
$$x^2-x-20=0$$
which can be solved with Bhaskara as
$$x=\dfrac{1\pm \sqrt{1-4(1)(-20)}}{2}=\dfrac{1\pm\sqrt{1+80}}{2}=\dfrac{1\pm \sqrt{81}}{2}=\dfrac{1\pm 9}{2}$$
this yields $x = 5$ and $x = -4$. Hence these are the possible values for $a$ and their sum is $a_1 + a_2 = 1$.