I am trying to prove the inscribed angle theorem using vectors. I fixed the dots $A=(\cos\theta,\sin\theta)$, and $B=(\cos\varphi,\sin\varphi)$, and I took another point $C=(\cos\psi,\sin\psi)$ in the biggest arc $AB$.
My idea was to calculate $\dfrac{\langle A-C,B-C\rangle}{\lvert{A-B}\rvert\lvert{B-C\rvert}}$, what according to my calculations is $$\dfrac{1+\cos(\theta-\varphi)-\cos(\psi-\theta)-\cos(\varphi-\psi)}{2\sqrt{1+\cos(\theta-\psi)\cos(\varphi-\psi)-\cos(\theta-\psi)-\cos(\varphi-\psi)}}.$$
My main difficulty here is the square root, which I can't get rid of. Does someone know how to proceed from here?
Or maybe to solve the problem with vectors with a different approach?
You expanded your terms all the way. Take a few steps back: \begin{align} \langle A-C, B-C\rangle &= \langle (\cos\theta-\cos\psi,\sin\theta-\sin\psi),(\cos\phi-\cos\psi,\sin\phi-\sin\psi)\rangle\\ &= (\cos\theta-\cos\psi)(\cos\phi-\cos\psi)+(\sin\theta-\sin\psi)(\sin\phi-\sin\psi)\\ &= -2\sin\frac{\theta-\psi}2\sin\frac{\theta+\psi}2 \cdot(-2)\sin\frac{\phi-\psi}2\sin\frac{\phi+\psi}2 \\&\quad+ 2\cos\frac{\theta+\psi}2\sin\frac{\theta-\psi}2 \cdot 2\cos\frac{\phi+\psi}2\sin\frac{\phi-\psi}2\\ &= 4\sin\frac{\theta-\psi}2\sin\frac{\phi-\psi}2\left(\sin\frac{\theta+\psi}2\sin\frac{\phi+\psi}2 + \cos\frac{\theta+\psi}2\cos\frac{\phi+\psi}2\right)\\ &= 4\sin\frac{\theta-\psi}2\sin\frac{\phi-\psi}2\cos\left(\frac{\theta+\psi}2-\frac{\phi+\psi}2\right)\\ &= 4\sin\frac{\theta-\psi}2\sin\frac{\phi-\psi}2\cos\frac{\theta-\phi}2 \end{align}
\begin{align} |A-C|^2|B-C|^2 &= ((\cos\theta-\cos\psi)^2+(\sin\theta-\sin\psi)^2)((\cos\phi-\cos\psi)^2+(\sin\phi-\sin\psi)^2)\\ &= (2-2\cos\theta\cos\psi - 2\sin\theta\sin\psi)(2-2\cos\phi\cos\psi - 2\sin\phi\sin\psi)\\ &= 4(1-\cos(\theta-\psi))(1-\cos(\phi-\psi))\\ &= 4\cdot 2\sin^2\frac{\theta-\psi}2\cdot 2\sin^2\frac{\phi-\psi}2\\ &= 16 \sin^2\frac{\theta-\psi}2\sin^2\frac{\phi-\psi}2 \end{align}
and hence $$\cos\measuredangle ACB = \frac{\langle A-C, B-C\rangle}{|A-C||B-C|} = \cos\frac{\theta-\phi}2$$ which means that the angle $\measuredangle ACB$ does not depend on $C$ and is equal to half the central angle: $$\measuredangle ACB = \frac12(\theta-\phi) = \frac12 \measuredangle AOB.$$