My professor stated that if an augmented matrix is in the row echelon form and the bottom most row(s) is all $0$s, then there are infinitely many solutions.
However, after reading a bit more, I am starting to think that his statement is false. Isn't there only infinitely many solutions when there are free variables present?
You are right.
$$\begin{bmatrix} 1 & 0 \\ 0 & 1 \\ 0 & 0 \end{bmatrix}\begin{bmatrix} x_1 \\ x_ 2 \end{bmatrix}= \begin{bmatrix} 0 \\ 0 \\ 0 \end{bmatrix}$$
has exactly one solution.
Some conditions are needed to make his statement true, first we need the system to be consistent. Also, for a matrix $A \in \mathbb{R}^{m \times n}$, we need $m \le n$. In fact, if $n > m$ and it is consistent, then there will always be free variables. His statement is interesting when $m=n$.