I'm having difficulty determining the number of solutions of a system of equations.
Suppose I want to solve the following system of equations
$$\begin{bmatrix} 5x_1+3x_2+2x_3\\ 2x_1-8x_2+6x_3\\ 7x_1+4x_2+15x_3 \end{bmatrix} =\begin{bmatrix} 0\\0\\0 \end{bmatrix} $$
Solving by elimination
$$ \begin{split} 2R_1&:\space 10x_1 &+ 6x_2 &+4x_3 &=0\\ 5R_2&:\space 10x_1 &-40x_2 &+30x_3 &=0\\ 2R_1-5R_2&:\space &+46x_2 &-26x_3 &=0 \end{split} $$
A solution to this would be $x_2=\frac{13}{23}$, $x_3=1$
From $R_1$, $x_1=-\frac{17}{23}$
Since these 3 values satisfy $R_1$ and $R_2$, but don't satisfy $R_3$, does this automatically mean that the system only has the trivial solution $R_1=R_2=R_3=0$?
No, it means you picked the "wrong" solution of the reduced equation. Maybe there is a different value of $x_2,x_3$ which would force a different value for $x_1$...
UPDATE
To understand if the system will admit non-trivial solutions, eliminate the variables one by one from the equations in a systematic way. For example, you combined $R_1$ and $R_2$ to get $$46x_2 -26x_3 = 0$$ and in a similar way, compute $2R_3 - 7R_2$ to eliminate $x_1$ as well, getting the equation $$64x_2 -12x_3 = 0$$
Now note your new system (once you divide top equation by 2 and bottom one by 4 to simplify) is $$ \begin{split} 23x_2 &- 13x_3 &= 0\\ 16x_2 &- 3x_3 &= 0 \end{split} $$ It is now easy to eliminate $x_2$. Note from the last equation, $x_2 = 3/16 x_3$ and plugging that into the top equation: $$ 0 = 23x_2 - 13x_3 = 23 \times \frac{3}{16} x_3 - 13 x_3 = x_3\left(\frac{23 \cdot 3}{16} - 13\right) $$ which means $x_3=0$ is the only possible value. Hence, $$ x_2 = \frac{3}{16} x_3 = 0 $$ and you can plug into any of the original equations to prove $x_1=0$ as well, and the system only admits the trivial solution.