Find the coordinates of the vector $x$, if it is known that is perpendicular to the vectors $a_1 = (4, -2, -3)$ and $a_2 = (0, 1, 3)$, forms an acute angle with a unit j and $|x| = 26$ (values must be an integer)
I tried but I can't understand what I should do with the angle.
I have that
$a_{11}x_1 + a_{12}x_2 + a_{13}x_3 = 0$
$a_{21}x_1 + a_{22}x_2 + a_{23}x_3 = 0$
$4x_1 - 2x_2 + 3x_2 = 0$
$x_2 + 3x_2 = 0$
$x_1 = -3/4x_3$
$x_2 = -3x_3$
$x_3 = -1/3x_2$
$26^2 = x_1^2 + x_2^2 + x_3^2$
and then solve this but my answers are not correct because of the misunderstanding what to do with the angle and how to solve it
So, you start by solving \begin{align*} (x_1, x_2, x_3) \cdot (4, -2, -3) &= 0 \\ (x_1, x_2, x_3) \cdot (0, 1, 3) &= 0. \end{align*} This will give you ever vector orthogonal to both $(4, -2, -3)$ and $(0, 1, 3)$. In expanding this, you make a small error, forgetting a minus sign: \begin{align*} 4x_1 - 2x_2 \color{red}- 3x_3 &= 0 \\ x_2 + 3x_3 &= 0. \end{align*} Despite the small error, you seem to have arrived at the correct solution. The second equation comes to $$x_2 = -3x_3$$ as you got, and the first equation therefore becomes $$x_1 = \frac{1}{4}(2x_2 + 3x_3) = \frac{1}{4}(2(-3x_3) + 3x_3) = -\frac{3}{4}x_3,$$ again agreeing with your working.
We don't need to solve for $x_3$ in terms of $x_2$. We have let $x_3$ be a free variable; it can be anything it wants, and the other variables are along for the ride. In particular, we get $$(x_1, x_2, x_3) = \left(-\frac{3}{4}x_3, -3x_3, x_3\right) = x_3\left(-\frac{3}{4}, -3, 1\right).$$ I just want a single non-zero solution to this, and I don't care about the length. I could choose $x_3 = 1$ and get a perfectly fine solution, but I'm going to choose $x_3 = -4$, just to give us some integers! When $x_3 = -4$, we get the solution: $$(x_1, x_2, x_3) = (3, 12, -4).$$ Take some time to demonstrate for yourself that this is perpendicular to the given vectors.
Next, find the length of this vector. If it has length $26$, we've lucked out! Otherwise, we may need to scale this vector, changing its length, but not its direction. Doing so will still mean the vector is orthogonal to the two given vectors. $$\|(3, 12, -4)\| = \sqrt{3^2 + 12^2 + 4^2} = \sqrt{169} = 13.$$ Pretty close! This vector is precisely half the length we want, so I'm going to double its length: $(6, 24, -8)$.
Lastly, we want an acute angle with $j$. Two vectors have an acute angle if their dot product is positive. So, we compute $(6, 24, -8) \cdot j = 24 > 0$. We therefore have the vector we wanted. If we had the dot product be negative, then I would simply have negated the whole vector (i.e. $(-6, -24, 8)$) to get a parallel vector of the same length, but pointing in the opposite direction. We didn't have to do this, so the final answer is $(6, 24, -8)$.