I'm trying to learn very basic group theory and was working through this example:
If $A$ and $B$ are symmetries of a shape, $A*B$ , or sometimes just $AB$ denotes the symmetry you get by applying $B$ first, and then $A$. We will call this new symmetry the product of $A$ and $B$.
This seems an awfully lot like composition in terms of functions. I'm wondering – is multiplication and composition essentially analogous in group theory? It seems much more natural to describe in regards to $A$ composed onto $B$, and I'm not quite sure why multiplication should behave like function composition.
Note that the "multiplication" [in group theory] that you refer to is simply a name for the binary operation of the group. Despite this name, you should not associate the operation with the usual notion of multiplication of numbers. (For example, in groups like $\mathbb{Z}$ or $\mathbb{Z}/n\mathbb{Z}$, the operation is more closely related to addition of numbers.) Even worse, this operation is often not commutative, unlike the usual notion of multiplication of numbers. You should just think of it as an abstract operation, which may resemble other familiar notions (addition, multiplication, function composition) depending on context.
Groups often are studied by their actions on other objects. For example, the group elements of symmetric groups are permutations of some set of objects. Another example is that group elements of dihedral groups represent symmetries (rotations, reflections) of regular polygons. In these contexts, group elements are like "functions" on the "other objects," and "multiplication" of group elements is indeed like function composition. (See the "compatibility" property in the definition of a group action.)