I was wondering if the following theorem still holds if one only requires $\tau$ to be a homeomorphism and not a diffeomorphism: "Let M be a smooth manifold and $\tau:M\to M$ a differentiable function such that $\tau(\tau(x))=x$ and $\tau(x)\neq x$. Then the quotient space $M/\tau$ is a smooth manifold."
I don't see where the "differentiable" part is used in the proof. The Theorem can e.g. be found in "Jänich, Vektoranalysis" and if needed I can also give you the proof.
Thanks in advance for any help!
Such examples do not exist if $M$ has dimension $\le 3$, since in this range of dimensions all topological manifolds are (uniquely) smoothable. However, there are fixed-point free non-smoothable involutions $\tau: S^4\to S^4$ such that the fake $RP^4=S^4/\tau$ is a non-smoothable 4-manifold. See:
Daniel Ruberman, Invariant knots of free involutions of $S^4$, Topology Appl. 18, 217-224 (1984). ZBL0559.57016.