If $\mathbf{a}$ and $\mathbf{b}$ are two vectors, we get the magnitude of the rotation or moment as $|\mathbf{a}||\mathbf{b}|\sin(\angle ab)$.
Now, we are multiplying that with the unit vector orthogonal to the $\mathbf{ab}$-plane. That's when we get $\mathbf{a}\times \mathbf{b}$ vector.
But my question is, why normal to the $\mathbf{ab}$-plane? Why not in any other direction? Why 90 degree and nothing other than that?
On
Or, you could think about it another way. Let's say you want to design a product between two vectors $a$ and $b$ with base space $R^3$ with the following properties (we'll use the generic $\times$ for now)
$$ a \times b \text{ is a vector} \\ a \times b = -b \times a \\ a \times (b + c) = a \times b + a \times c \\ (s a) \times b = s (a \times b) \quad \text{s is a scalar} $$
Properties 1,3 and 4 immediately suggest the introduction of a linear map $T^a(b) \equiv a \times b $.
Clearly, $T^a(b)$ is a vector, and the linearity properties fall (3) and (4) are seen to be satisfied trivially, e.g.
$$ T^a(b + c) = T^a(b) + T^a(c) \equiv a \times b + a\times c $$
The second property (anti-commutativity) is tricky. It is easier to assert that the linear map $T^a$ must have the form $$ T^a = \epsilon_{ijk} a_j \mathbf{e}_i \otimes \mathbf{e}_k $$
where $\otimes$ is the tensor product. Does this imply $a \times b = -b \times a$? Well for starters, $a \times a $ must be 0. With our definition, $a \times a $ is
$$ \epsilon_{ijk} a_j \mathbf{e}_i \otimes \mathbf{e}_k \cdot a_m \mathbf{e}_m \\ = \epsilon_{ijk} a_j a_m \delta_{km} \mathbf{e}_i \\ = \epsilon_{ijk} a_j a_k \mathbf{e}_i \\ = 0 $$ The last conclusion follows because $a_j a_k$ will always result in pairs of terms of opposite sign when contracted with the levi-civita symbol.
What about the generalization of this property? Does it include full anti-commutativity?
$$ T^a(b) = \epsilon_{ijk} a_j \mathbf{e}_i \otimes \mathbf{e}_k \cdot b_m \mathbf{e}_m \\ = \epsilon_{ijk} a_j b_k \mathbf{e}_i \\ \quad \text { (on simplification using } \mathbf{e}_k \cdot \mathbf{e}_m = \delta_{km}) $$
Similarly, $$ T^b(a) = \epsilon_{ijk} b_j \mathbf{e}_i \otimes \mathbf{e}_k \cdot a_m \mathbf{e}_m \\ = \epsilon_{ijk} a_k b_j \mathbf{e}_i \\ = \epsilon_{ikj} a_j b_k \mathbf{e}_i \\ = -T^a(b) $$ The last step follows because the indices of epsilon are permuted the other way.
So we have constructed a product which has all the properties of a cross product using a linear map.
Now, we can examine the angle between a vector and its cross product using these definitions.
$$ a \cdot (a \times b) \\ = a \cdot T^a(b) \\ = a_m \mathbf{e}_m \cdot \epsilon_{ijk} a_j b_k \mathbf{e}_i \\ = \epsilon_{ijk} a_i a_j b_k \\ = 0 $$
Again the last result is easily verified using the properties of the permutation symbol, because it results in pairwise terms of opposite signs like $\epsilon_{12k} a_1 a_2 b_k + \epsilon_{21k} a_2 a_1 b_k = 0$.
Since $a \cdot (a \times b)$ defined this way is 0, the vector $a \times b$ is perpendicular to $a$. (It will also be perpendicular to b, as is easily verified).
There are a few different ways to see this. If you want to define the cross product in $\mathbb R^3$ in a coordinate-free, geometric manner you define $v \times w$ to be equal to the vector with magnitude $\| v \| \cdot \| w\| \sin(\theta)$ (with $\theta$ the angle between $v$ and $w$) pointing in the direction determined by the right-hand rule. The usual coordinate description is then a consequence of this.
Orthogonality can be deduced from the coordinate description of the cross product via the triple scalar product. First recall that two non-zero vectors are orthogonal if and only if their inner product is zero. For $v, w \in \mathbb R^3$, also recall that $v \cdot (v \times w) = \det (v, v, w)$ and $w \cdot (v \times w) = \det (w, v, w)$. Here, $(v, v, w)$ and $(w, v, w)$ are taken to be $3 \times 3$-matrices whose rows are given by the indicated vectors. Both determinants are $0$ since the matrices have two identical rows and are hence singular. This shows that both $v$ and $w$ are orthogonal to $v \times w$, so that $v \times w$ is parallel to the normal vector determined by the span of $v$ and $w$.