$\require{AMScd} \newcommand{\RP}{\mathbb{RP}}$ I want a converse of this fact specialized to the case where I am pushing out a map BY a fibration:
I.e., if I am given a diagram
$ \begin{CD} E_1 @>i>> E_2 \\ @V\text{fibration}Vp_1V @Vp_2V\text{fibration}V\\ B_1 @>cofibration>j> B_2 \\ \end{CD}$,
does it follow that $i$ is a cofibration?
I am doing this to prove that the inclusion $F_p E \to F_{p+1} E$ is a cofibration,for the filtration used in the serre spectral sequence.
Edit
When I made this question, I had the misconception that every pushout square was also a pullback square(I will conveniently blame it on a student lecturer in my class who said "this pushout square" when he was referring to a square that was just a pullback square.). Thus I did not realize that I needed to further specify that the above square be a pullback square. I am embarrassed to admit it. But I created a new question with this new condition.
In full generality, it's not true, even if $E_1 \to E_2$ is an inclusion. Take $E_1 \to E_2$ to be your favorite inclusion that's not a cofibration and let $B_1 = B_2 = *$. Then $E_i \to B_i= *$, $i = 1, 2$, are fibrations (every space is fibrant), the identity $B_1 \to B_2$ is a cofibration, and the diagram commutes, but $E_1 \to E_2$ is not a cofibration by construction.
However, if $E_1$ and $E_2$ are CW complexes, then it's probably true, since every (cellular) inclusion of CW complexes is a cofibration.
EDIT: The example I gave above ignores the "pushout" condition. If we assume (per OP's comment) that the square is a pullback, then the answer seems to be affirmative. See theorem 14.1 in Strom's Modern Classical Homotopy Theory.