O is the origin
OP and OQ are vectors, for simplicity lets denote P as vector with magnitude 1
facing angle of 45 degrees from positive x axis.
Let's denote Q as vector with magnitude 2 facing angle of 20 degrees from positive x axis.
We like to find angle between OP and OQ. Let's say angle is theta.
I mean I think understand the definition of the dot product of OP and OQ.
it's just when OP is projected to OQ, it is the distance that covers part of OQ.
Then distance should be mod(OP)*cosine of theta. However, I am not sure
how the cosine theta equals dot product of OP and OQ/[mod(OP)*mod(OQ)]
where did this Mod(OQ) come from???
Your understanding about the dot product is correct, it is the part of $OP$ which is in the direction of $OQ$, but it is also multiplied by the length of $OQ$, which is $|OQ|$. $|OQ|$ and $|OP|$ are the lengths of $OP$ and $OQ$.