I'm just beginning to dabble into some category theory (from Aluffi's Algebra) and I have some difficulty with morphisms in a slice category. Particularly, I can't see why the diagram of a morphism should commute in the ambient category. If we have the category $C_A $ and two objects
$$ f:Z\longrightarrow A $$
and
$$ g:Y\longrightarrow A $$
why shouldn't any morphism $Z\longrightarrow Y$ count as valid, as long as there is a morphism from both $Y$ and $Z$ into $A$, not only those $\sigma$ such that $f=g\sigma$? If the answer is obvious or I have some fundamental misunderstanding you could just say so and I'll try to think more about it.
The question "why shouldn't" is a little bit difficult when talking about a definition. Just as "why shouldn't 1 be a prime number" or "why shouldn't the empty set be a vector space", the reason basically is that the definition would give something much less interesting. For instance, whenever the category has a terminal object $1$, and $A$ has a point, i.e., a morphism $1 \to A$, then the category you propose is equivalent to the category $C$ itself.
The point of the category $C_A$ is to contain something like "$A$-indexed objects of $C$". Think of this as analogous to a category of bundles (e.g. vector bundles) over a space $A$: the morphism $Z \to A$ is the projection onto the base space, and the commutativity constraint on morphisms ensures that a morphism between two bundles maps the fiber in one bundle over a point $a \in A$ into the fiber over the same point of the other bundle.