Given that I have an augmented matrix in Row Echelon Form or Reduced Row Echelon form, and the bottom row(s) contain only zeroes.
Generally speaking (disregarding fringe cases if any?):
Why do rows of only zero imply the presence of a free variable?
(Three unknowns, the last row of the matrix contains all zeros.)
Can you have a free variable without the bottom row(s) containing only zeroes?
Does the presence of a free variable always mean you have an infinite number of solutions?
This is not true, consider matrices with more rows than columns like $$\left(\begin{matrix}1 & 0 \\ 0 & 1 \\ 0 & 0\end{matrix}\right).$$ Free variables are implied by columns without pivot element.
Yes, consider $$\left(\begin{matrix}1 & 0 & 0 \\ 0 & 1 & 0\end{matrix}\right).$$
This depends on the size of the base field. If it is infinite (like $\mathbb{Q}$ or $\mathbb{R}$) then yes. Otherwise no.