Is $H=\{ x\in D_{4} \mid x^2=1\}$ is a subgroup of $D_4?$
My attempt : I think not
Take the elements $s$ and $rs$ of $D_4$
Here $s^2=1$ and $( rs)^2 =rsr^{-1}s=s^2=1$
But $s(rs)=rs^2=r \neq 1$
so the $H$ doesn't satisfy the closure property
Therefore $H$ is not a subgroup of $D_4$
Your conclusion is correct but your reasoning is slightly off. You need to show that s(rs) is not in H. For that you need to show that $(s(rs))^2$ is not 1.