What is the unit normal vector to the line joining i - j and 2i+3j towards origin?
What i tried is that i added the vectors and took the unit vector of the added vector but could not get the right answer. I tried vector multiplication as well. I am not getting the point what do they mean by line joining two vectors.
Its answer is $\frac{4i-j} {\sqrt{17}}$
You have two vectors, $$\vec{a} = i - j = \left [ \begin{matrix} 1 \\ -1 \end{matrix} \right ]$$ and $$\vec{b} = 2 i + 3 j = \left [ \begin{matrix} 2 \\ 3 \end{matrix} \right ]$$ The line joining $\vec{a}$ and $\vec{b}$ is $$\vec{p}(t) = \vec{a} + t (\vec{b} - \vec{a})$$ so to find a vector perpendicular to that line, you need to find a vector perpendicular to $\vec{c} = (\vec{b}-\vec{a})$:
$$\vec{c} = \vec{b} - \vec{a} = (2 i - i) + (3j - - j) = i + 4 j = \left [ \begin{matrix} 1 \\ 4 \end{matrix} \right ]$$
In two dimensions, we can rotate any vector $\vec{v} = x i + y j = \left [ \begin{matrix} x \\ y \end{matrix} \right ]$ counterclockwise, $$\vec{v}_{cc} = y i - x j = \left [ \begin{matrix} y \\ -x \end{matrix} \right ]$$ or clockwise, $$\vec{v}_{c} = -y i + x j = \left [ \begin{matrix} -y \\ x \end{matrix} \right ]$$
In this particular case, rotating $\vec{c}$ counterclockwise gives us $$\vec{c}_{cc} = 4 j - i = \left [ \begin{matrix} 4 \\ -1 \end{matrix} \right ]$$ and $$\vec{c}_{c} = -4 j + i = \left [ \begin{matrix} -4 \\ 1 \end{matrix} \right ]$$ clockwise.
Now, let's consider the direction. We know that any two vectors $\vec{u}$ and $\vec{v}$, and the angle $\theta$ between them, fulfills $$\vec{u} \cdot \vec{v} = \lVert \vec{u} \rVert \lVert \vec{v} \rVert \cos \theta$$ Here, we have vectors $\vec{c}_{cc} = 4 i - j$ and $\vec{c}_{c} = -4 i + j$, and need to choose which one is more "towards origin" from the line. If we consider point $\vec{a}$, then $-\vec{a}$ is towards origin: $$\begin{aligned} -\vec{a} \cdot \vec{c}_{cc} &= (-1 \cdot 4) + (1 \cdot -1) = -4 - 1 = -5 \\ -\vec{a} \cdot \vec{c}_{c} &= (-1 \cdot -4) + (1 \cdot 1) = 4 + 1 = 5 \\ \end{aligned}$$ If we consider point $\vec{b}$, then $-\vec{v}$ is towards the origin: $$\begin{aligned} -\vec{b} \cdot \vec{c}_{cc} &= (-2 \cdot 4) + (-3 \cdot -1) = -8 - 3 = -11 \\ -\vec{b} \cdot \vec{c}_{c} &= (-2 \cdot -4) + (-3 \cdot 1) = 8 - 3 = 5 \\ \end{aligned}$$ In both cases, the dot product with $\vec{c}_{c}$ is larger, so the angle to $\vec{c}_{c}$ is smaller, and $\vec{c}_{c}$ is more towards origin.
So, the vector perpendicular to the line between $\vec{a}$ and $\vec{b}$ is $$\vec{c}_c = -4 i + j = \left [ \begin{matrix} -4 \\ 1 \end{matrix} \right ]$$ However, we still need to scale it to unit length. We do that by dividing it by the norm (length) of the vector: $$\hat{c} = \frac{\vec{c}_c}{\lVert\vec{c}_c\rVert} = \frac{-4 i + j}{\sqrt{4^2 + 1^2}} = \frac{- 4 i + j}{\sqrt{17}}$$
So, the correct answer is $$\hat{c} = \frac{-4 i + j}{\sqrt{17}}$$
If you inspect the following diagram, where $\hat{c}$ is drawn in blue near $\vec{a}$, you'll see it is correct:
Note that the vector $\frac{4 i - j}{\sqrt{17}}$ is opposite to the blue one in the above diagram, pointing down right, and definitely away from the origin.