Is a matrix invertible only when it is a square matrix?
What about a matrix of the order $m \cdot n$ with $m \gt n$ and such that it is row-equivalent to a row-reduced echelon matrix with more non-zero rows than columns?
What is the motivation behind the concepts of left and right inverse? Is it only useful when dealing with non-square matrix?
Let $A$ be a full rank $m\times n$ matrix. By full rank we mean $\DeclareMathOperator{rank}{rank}\rank(A)=\min\{m,n\}$.
If $m<n$, then $A$ has a right inverse given by $$ A^{-1}_{\text{right}}=A^\top(AA^\top)^{-1} $$
If $m>n$, then $A$ has a left inverse given by $$ A^{-1}_{\text{left}}=(A^\top A)^{-1} A^\top $$
Of course, a right inverse is useful for solving an equation of the form $XA=Y$ and a left inverse is useful for solving an equation of the form $AX=Y$.