The question I am trying to answer is shown below:
And I know that the answer requires me to find the reduced row echelon form of the following matrix :
$$ \begin{bmatrix} 2 & 1 & 9 & 7 & 4 \\ 8 & 3 & 10 & 5 & 12 \\ 7 & 6 & 2 & 11 & 7 & p_{mn} \end{bmatrix} $$
But I do not know when I have an entry 2 in the matrix and I want to make it 1, so I want to multiply by its inverse, shall I put its inverse according to the multiplication table or the addition table of $\mathbb{Z_{13}}$? or it does not matter?
The homogeneous system of linear equations can be written $Ax=0$ for some matrix $A$ and $x\in \Bbb F_{13}^5$. We can think of this as a map $f\colon \Bbb F_{13}^5\to \Bbb F_{13}^5$ where $f(x)=Ax$. Then the desired set of solutions to the equation is exactly $\ker f$, which is the same as the null space of $A$.
First let's simplify the vectors $v_1,v_2,v_3$ given. We may row reduce the matrix $$\begin{pmatrix} 2 & 1 & 9 & 7 & 4\\ 8 & 3 & 10 & 5 & 12\\ 7 & 6 & 2 & 11 & 7 \end{pmatrix}$$
to obtain
$$\begin{pmatrix} 1 & 0 & 0 & 1 & 6\\ 0 & 1 & 0 & 10 & 4\\ 0 & 0 & 1 & 11 & 3 \end{pmatrix}.$$
This is the desired null space of our matrix $A$. The main point is that $v_1,v_2,v_3$ are indeed linearly independent, so the set $v_1,v_2,v_3,e_4,e_5$ is a basis of $\Bbb F_{13}^5$, where $e_4=(0,0,0,1,0)$ and $e_5=(0,0,0,0,1)$. In this basis, the desired matrix is just
$$B=\begin{pmatrix} 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 0 & 0\\ 0 & 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0 & 1 \end{pmatrix}.$$
Presumably we want to express $B$ in the standard basis. The change of basis matrix is
$$P=\begin{pmatrix} 1 & 0 & 0 & 0 & 0\\ 0 & 1 & 0 & 0 & 0\\ 0 & 0 & 1 & 0 & 0\\ 1 & 10 & 11 & 1 & 0\\ 6 & 4 & 3 & 0 & 1 \end{pmatrix}.$$
Thus $A=PBP^{-1}$. The rest is computational.