There is a canonical isomorphism $\mathrm{Ext}(\mathbb Z/4,\mathbb Z)\cong\mathbb Z/4$ based on the fact that an extension $$0\to\mathbb Z\xrightarrow i G\xrightarrow{p}\mathbb Z/4\to 0$$ is determined by the number $n\in\mathbb Z$ such that $i(n)=4u$ where $u\in G$ is an element with $p(u)=1\in\mathbb Z/4$. This fact can be found in MacLane's book.
It follows that the underlying group of $2\in\mathrm{Ext}(\mathbb Z/4,\mathbb Z)$ is $\mathbb Z\oplus \mathbb Z/2$.
One can also verify this by multipliying the element $1\in\mathrm{Ext}(\mathbb Z/4,\mathbb Z)$ by $2\in\mathrm{Hom}(\mathbb Z,\mathbb Z)$, that is, by forming the pushout of $$ \begin{matrix} \mathbb Z & \xrightarrow{4} & \mathbb Z & \to & \mathbb Z/4 \\ \downarrow^2 \\ \mathbb Z. \end{matrix} $$
Here is what confuses me: I would like to check the equality $$ 1+1=2\in\mathrm{Ext}(\mathbb Z/4,\mathbb Z). $$
That means, I first have to form the pullback of the diagram $$ \begin{matrix} & & \mathbb Z\\ & & \downarrow\\ \mathbb Z & \to & \mathbb Z/4 \end{matrix} $$ where both maps are the usual projections. This group consists of all pairs $(a,b)\in\mathbb Z^2$ such that $a+b$ is a multiple of $4$. It should be free abelian with generators $(4,0)$ and $(1,-1)$. To finish the computation of the Baer sum, I must divide out the skew diagonal generated by $(4,-4)$. Isn't the resulting group isomorphic to $$ \mathbb Z\oplus\mathbb Z/4\;? $$ What am I missing?
The pullback is $\{(x_1,x_2) \in \Bbb Z^2 / x_1 \equiv x_2 \pmod 4\}$, is free and generated by $a = (4,0)$ and $b = (1,1)$.
The skew diagonal is the subgroup generated by $(4,0) - (0,4) = (4,-4) = 2a-4b$.
So the quotient is isomorphic to $\Bbb Z^2 / (2,-4)$, which is itself isomorphic to $\Bbb Z \times \Bbb Z/2 \Bbb Z$ via the map $(x_a,x_b) \mapsto (2x_a+x_b,x_a \pmod 2)$.
The map $G \to \Bbb Z/4 \Bbb Z$ is obtained by the compositions $(u,v) \mapsto (x_a = v, x_b = u-2v) \mapsto (x_1 = 4x_a+x_b = u+2v, x_2 = x_b = u-2v) \mapsto x_1 \equiv x_2 \equiv u+2v \pmod 4$
And the map $\Bbb Z \to G$ by
$n \mapsto (x_1 = 4n, x_2 = 0) \mapsto (x_a = (x_1-x_2)/4 = n, x_b = x_2 = 0) \mapsto (u = 2x_a+x_b = 2n, v = a = n \pmod 2)$
So you get the extension $0 \rightarrow \Bbb Z \xrightarrow{\begin{pmatrix} 2 \\ 1 \end{pmatrix}} \Bbb Z \times \Bbb Z/2 \Bbb Z \xrightarrow{\begin{pmatrix}1&2\end{pmatrix}} \Bbb Z/4\Bbb Z \rightarrow 0$, as you wanted.