Someone can help me to define the rule of this task?
For which $x\in\mathbb{R}$ does the vector system rank third
$\{\begin{bmatrix}x\\1\\3\end{bmatrix}$,$\begin{bmatrix}1\\3\\-2x\end{bmatrix}$,$\begin{bmatrix}1\\2\\3\end{bmatrix}\}$
I found that if x=1 or x=0 the rank is 2. I don't understand how to define the whole set of the possible solution. Is there a proper way to find it out?
The rank is three if and only if the vectors are linearly independent. Name your vectors $v_1,v_2,v_3$. Then you can check for which choices of $x$ you fin $\lambda_1,\lambda_2,\lambda_3$ such that $$\lambda_1v_1+\lambda_2v_2+\lambda_3v_3=0 $$ and at least one them not being zero. Can you proceed from here? If not, I am happy to expand my answer.
Another way is to consider the determinant of the matrix $A=(v_1,v_2,v_3)$. It is non-vanishing if and only if $v_1,v_2,v_3$ are linearly independent; therefore if and only if the system has rank three. Now the determinant is $$\det(A)=4x^2+7x-6$$ so everything boils down to finding the roots of this polynomial.