From Mac Lane's Category theory:
I'm trying to verify that $p_j$ is a product diagram but I'm having some difficulty.
I can see that I would have to show that for every $q_j : g \rightarrow f_j$ there exists a $q: g \rightarrow f$ such that $p_j \circ q = q_j$.
My idea is to let $q = r \circ g$, where $r : Ga_g \rightarrow G \prod a_j$ is taken from the fact that $Gp_j$ is a product diagram.
$p_j$ and $q_j$ should be defined similarly $(p_j(f) =Gp_j(f) \text{ and } q_j(g)=r_j(g))$ as compositions from $Gp_j$'s product diagram as $r_j : G a_g \rightarrow Ga_j$ and $Gp_j : G \prod a_j \rightarrow G a_j$, respectively.
But from here I'm having trouble showing that $p_j \circ q (g) = q_j(g)$ are equal.
Anyone have any ideas?
You seem to be misunderstanding what the category $(x\downarrow G)$ is. An object of $(x\downarrow G)$ is a pair $(a,s)$ where $a$ is an object of $A$ and $s$ is a morphism $x\to Ga$ in $X$. A morphism between two such objects $(a,s)$ and $(a',s')$ is a morphism $q:a\to a'$ in $A$ such that $(Gq)s=s'$.
So your setup is wrong from the start. You want to be starting not just with what you call $g$ and $q_j$ but a pair $(a,g)$ where $a$ is an object of $A$ and $g:x\to Ga$, and the morphisms $q_j$ are actually morphisms $a\to a_j$ in $A$ with the property that $(Gq_j)g=f_j$. Your goal is then to show there is a unique $q:a\to \prod a_j$ such that $(Gq)g=f$ and $p_jq=q_j$ for all $j$.
As you suggest, you want to find this $q$ using the universal property of the product (but of $\prod a_j$, not $G\prod a_j$, since our maps are in $A$, not $X$!). There is a unique $q:a\to\prod a_j$ such that $p_jq=q_j$ for each $j$. So, this is the only $q$ that could work, and we just need to verify that $(Gq)g=f$.
Now since the maps $Gp_j:G\prod a_j\to Ga_j$ form a product diagram, to verify that $(Gq)g=f$ it suffices to verify that $(Gp_j)(Gq)g=(Gp_j)f$ for each $j$. This is true because $$(Gp_j)(Gq)g=(G(p_jq))g=(Gq_j)g=f_j=(Gp_j)f.$$