(a) Let $U$ map $H\to K$ for two Hilbert spaces. Prove that $|U\varphi|=|\varphi|$ for all $\varphi$ if and only if $U^{\ast}U=1$
(b) If $U$ is invertible and $U^{\ast}U=1$, prove that $UU^{\ast}=1$ (Hint: Multiply by $U$ and $U^{-1})$
For (a) i have this:
$|U\varphi|^2=|\varphi|^2\Leftrightarrow \langle U\varphi,U\varphi\rangle=\langle \varphi,\varphi\rangle \Leftrightarrow \langle\varphi, U^{\ast}U\varphi\rangle =\langle\varphi,\varphi\rangle \Leftrightarrow U^{\ast}U\varphi=\varphi \Leftrightarrow U^{\ast}U=1$
For (b) how prove (b)?
Because you don't mention it, you might be overlooking something in your argument for $(a)$. You don't say how you get from $$\tag1\langle \varphi, U^*U\varphi\rangle=\langle \varphi,\varphi\rangle$$ to $U^*U\varphi=\varphi.$ This is not hard, but it is not entirely immediate. You would usually use the polarization identity to get from $\langle \varphi,(U^*U-I)\varphi\rangle=0$ (for every $\varphi$) to $U^*U=I$. And the argument does not work without quantifiers: you cannot conclude, if you have $(1)$ for a single $\varphi$, that $U^*U\varphi=\varphi$. You need to have $(1)$ for all $\varphi\in H$.
For instance, with $H=K=\mathbb C^2$, let $\varphi=\begin{bmatrix} 1\\0\end{bmatrix}$, and $T=\begin{bmatrix} 1&0\\-1&0\end{bmatrix}$. Then $$ \langle T\varphi,\varphi\rangle=\left\langle\begin{bmatrix} 1\\-1\end{bmatrix},\begin{bmatrix} 1\\0\end{bmatrix}\right\rangle=1=\langle\varphi,\varphi\rangle, $$ but $T\varphi\ne\varphi$.
For part (b), if $U^*U=I$ and $U$ is invertible, then multiplying on the right by $U^{-1}$ you get $U^*=U^{-1}$. Now you have $I=UU^{-1}=UU^*$.