The exact question I am trying to solve is as follows:
Let A: $\mathbb{R}^3\rightarrow\mathbb{R}^2$ and B: $\mathbb{R}^2\rightarrow\mathbb{R}^3$ be linear transformations, so BA: $\mathbb{R}^3\rightarrow\mathbb{R}^3$ and AB: $\mathbb{R}^2\rightarrow\mathbb{R}^2$.
a) Prove that BA cannot be invertible.
b) Give an example of A and B such that AB is invertible.
A couple of things I've tried so far: From a theorem in my textbook "A linear transformation: T: V$\rightarrow$W is invertible if and only if it is one-to-one and onto." I thought about trying to show that BA was either not one-to-one or not onto. Since $Ker(T)=0$ implies one-to-one, I thought about showing that there must be a non-zero vector in BA. Not really sure where to go with this.
I've tried searching the internet for similar questions but have not had any luck. The only thing I've come across that may be helpful is the following:
I found a proof showing that if UT is one-to-one then T is one-to-one. If UT is is onto, then U is onto. If U and T are one-to-one and onto, then UT must also be one-to-one and onto. Not sure if these really help because it says "Let V, W, Z be vector spaces and let $T : V \to W$ and $U : W \to Z$ be linear." So this is essentially proving the invertibility of AB, in my question, but I am interested in disproving the invertibility of BA, or TU from this proof (if I'm right).
I am also not sure where to start with coming up with an A and B such that AB is invertible.
I am only in an introductory linear algebra course and I am doing these problems to prepare for my exam, which may or may not include this question. Any suggestions would be appreciated :)
The rank of $A$ is at most $2$. So, the rank of $BA$ is at most $2$ and, since $BA$ is a linear map from $\mathbb R^3$ into itself, it cannot be surjective and so it canot be invertible.
On the other hand, you can take $B(x,y)=(x,y,0)$ and $A(x,y,z)=(x,y)$. So$$A\bigl(B(x,y,z)\bigr)=A(x,y,0)=(x,y).$$