Let $(X,A)$ be a relative CW-complex. Consider the inclusion $$i:\ \ X\times \{0,1\} \cup A\times I \rightarrow X\times I.$$
I was wondering if this is a cofibration.
I guess it is, for there is a push out diagram $\require{AMScd}$ \begin{CD} A\times (0,1) @>>> X\times \{0,1\} \cup A\times I\\ @VVV @VVV\\ X\times(0,1) @>>> X\times I \end{CD} the left vertical arrow is obviously a cofibration. So is the right vertical arrow.
Is the above argument correct?
Your argument is not correct because the diagram in your question is not a pushout diagram. If it were one, then for any two maps $\phi : X \times \{ 0, 1 \} \cup A \times I \to Y$ and $\psi : X \times (0,1) \to Y$ which agree on $A \times (0,1)$ we would get a unique map $\chi : X \times I \to Y$ whose restriction to $X \times \{ 0, 1 \} \cup A \times I$ is $\phi$ and whose restriction to $X \times (0,1)$ is $\psi$.
Counterexample 1.
Any CW-complex $X$ can be regarded as relative CW-complex $(X,A) = (X,\emptyset)$. Take $Y = I$ and define $\phi : X \times \{ 0, 1\} \to I, \phi(x,t) = 0$ and $\psi : X \times (0,1) \to I, \psi(x,t) = t$. These maps agree on $\emptyset \times (0,1)$ but do not give you the desired $\chi : X \times I \to I$.
Counterexample 2.
Let $(X,A) = (I,\{ 0\})$. Define $\phi : I \times \{ 0, 1 \} \cup \{ 0 \} \times I \to I, \phi(x,y) = y$ and $\psi : I \times (0,1) \to I, \psi(x,y) = \min (x+y,1)$. These maps agree on $\{ 0 \} \times I$ but again you do not the desired $\chi : I \times I \to I$.
A correct proof can be based on the observation that $(X \times I, X \times \{ 0, 1 \} \cup A \times I)$ is also a relative CW-complex.
By the way, it is a general theorem that if an inclusion $A \hookrightarrow X$ is a cofibration, then so is $X \times \{ 0, 1 \} \cup A \times I \hookrightarrow X \times I$. The proof requires some deeper properties of cofibrations. See Theorem 6 of
Strøm, Arne. "Note on cofibrations II." Mathematica Scandinavica 22.1 (1969): 130-142.