I want to prove $ \left\langle Av,v \right\rangle =0\;\forall v\implies A=0$. Here is a solution.
But what if we take $A$ to be rotation by $90$ degrees in the plane? It seems $ \left\langle Av,v \right\rangle =0$ is satisfied but $A\neq 0$... Is being a complex vector space crucial here? I'm confused..
Yes, the fact that $V$ is a complex vector space is crucial. If $V$ is a real inner-product space, then $$ \langle A v, v \rangle = 0 \quad \forall v \in V \iff A^* = -A $$ You may note that your $90^\circ$ rotation indeed satisfies $A^* = -A$.