This was a question in an exam:
Calcualte tan of the angle between a and b if:a = (4,3) and b = (5, -12)
There are two answers to this question:
Some students devided the dot product of a and b by the cross product of a and b. And their answer was $\frac{63}{16}$.this answer implies that $\theta \in [180, 270]$ because the dot product is negative,thus, $sin(\theta)$ is negative and the cross product is also negative,thus, $cos(\theta)$ is also negative.
Others used only the dot product and deduced from it that $\theta \in [90, 180]$ as its value is negative.And their answer was $\frac{-63}{16}$.
I think the difference between the two answers is that $\theta_1 = 360 - \theta_2$ ($\theta_2$ is the smaller angle).
So which answer is correct?
The angle between two nonzero vectors $u,v$ is defined to be the number $\theta \in [0,\pi]$ such that $$\cos(\theta)= \frac{u\cdot v}{\|u\|\|v\|}$$
Plugging in your vectors yields
$$\cos(\theta) = \frac{-16}{65}$$
The negative sign implies that $\theta \in (\frac {\pi}2,\pi]$. Drawing a triangle and using the Pythagorean theorem we find that the length of the "opposite" side is $63$. Thus $\tan(\theta) = \pm \frac{63}{16}$. Knowing that $\theta \in (\frac {\pi}2,\pi]$ gives us the negative answer.
If you wanted to use the cross product, here's how to get the same solution:
$$\require{cancel}\frac{a\cdot b}{\|a\times b\|} = \frac{\color{red}{\cancel{\color{black}{\|a\|}}}\color{red}{\cancel{\color{black}{\|b\|}}}\cos(\theta)}{\color{red}{\cancel{\color{black}{\|a\|}}}\color{red}{\cancel{\color{black}{\|b\|}}}\sin(\theta)} = \frac{\cos(\theta)}{\sin(\theta)} = \frac{-16}{63} = \frac 1{\tan(\theta)}$$
Note here that $\sin(\theta)\ge 0$ because we only allow $\theta \in [0,\pi]$, so we immediately get $\tan(\theta) = -\frac{63}{16}$.