Let $X$ be CW-complex and let $Y$ be a subcomplex. I have to show that $(X,Y)$ is a relative CW-complex. I thought this would follow quickly, but I got stuck. The case when $Y\neq X_i$ for all $i\ge-1$ is the only non-trivial case.
It seems to be most natural to define $\tilde X_i:=X_i\cup Y$ for all $i\ge-1$ for our new relative CW-complex. All checks out, except for one thing: let $i\ge-1$ be such that $X_i\subsetneq Y\subsetneq X_{i+1}$. We need that $\tilde X_{i+1}=X_{i+1}$ arises from $\tilde X_i=X_i\cup Y=Y$ by attaching $i$-cells. How does one show this? One only knows that $X_n\setminus X_{n-1}$ is a union of open $n$-cells, and that $Y_n\setminus Y_{n-1}$ is a union of open $n$-cells in $X_n\setminus X_{n-1}$, but I don't see how this implies the needed result.
To get $\tilde{X}_{i+1}$ from $\tilde{X}_i$, you just attach all the $(i+1)$-cells of $X$ which are not in $Y$. It is clear that this gives you $\tilde{X}_{i+1}$ as a subset of $X$. To verify that $\tilde{X}_{i+1}$ has the correct topology, you need to know that a map on $\tilde{X}_{i+1}$ is continuous iff it is continuous on $\tilde{X}_i$ and on each of the new $(i+1)$-cells. But a map in $\tilde{X}_{i+1}$ is continuous iff it is continuous on each cell of $\tilde{X}_{i+1}$, and those cells are exactly the cells of $\tilde{X}_i$ and the new $(i+1)$-cells. Since a map is continuous on $\tilde{X}_i$ iff it is continuous on each of its cells, we conclude that a map on $\tilde{X}_{i+1}$ indeed is continuous iff it is continuous on $\tilde{X}_i$ and each of the new cells.