The problem is as follows:
Find a vector which is perpendicular to the vectors $\vec{u}=\hat{j}+\sqrt{3}\hat{k}$ and $\vec{v}=\sqrt{3}\hat{j}+2\hat{k}$ whose magnitude is equal to the area of the parallelogram which is formed by $\vec{u}$ and $\vec{v}$.
The alternatives in my book are as follows:
$\begin{array}{ll} 1.&-2\hat{j}\\ 2.&-\hat{i}\\ 3.&3\hat{k}\\ 4.&5\hat{i}\\ 5.&3\hat{i}\\ \end{array}$
I'm totally lost at this question. What should I do to find the area?. Does it exist a formula which can be used to relate it with the fact that a vector is perpendicular to those two?. Can someone explain the solution step by step so I can understand it?.
The cross product of two vectors is a vector perpendicular to both of them, and its magnitude is the area of a parallelogram with the vectors for sides. It can be computed for $\vec{u}=\hat{j}+\sqrt{3}\hat{k}$ and $\vec{v}=\sqrt{3}\hat{j}+2\hat{k}$ as follows:
$$\vec u\times\vec v=\begin{vmatrix}\hat i&&\hat j&&\hat k\\0&&1&&\sqrt3\\0&&\sqrt3&&2\end{vmatrix}.$$
Here is additional information about the cross product, added per OP's request (see comments):
The cross product of two vectors will yield a vector perpendicular to both, whereas if two vectors are perpendicular then their dot product will be $0$; so, for example, $\vec a ⋅(\vec a ×\vec b)=0$.
The dot product is commutative, but the cross product is anti-commutative: $\,\vec b ×\vec a =−(\vec a ×\vec b ) $.
See this question for explanations why the magnitude of $\vec a\times\vec b$ gives the area of the parallelogram.