For which values of $a, b$ does the system of equations not have any solutions?

Question

I am trying to solve the following problem:

For which values of $a$ and $b$ does the linear system represented by the augmented matrix not have any solution?

$$ \left[\begin{array}{ccc|c} 1&-2&3&-4\\ 2&1&1&2\\ 1&a&2&-b \end{array} \right] $$

---

Truthfully, I don't know where to start. Thus, any help is welcomed. Thank you very much!

EDIT: I tried to solve the problem on my own using Gaussian elimination, but I am not sure that the solution is the right one, or is that the right way to go.

EDIT2: OK, here is how I tried to solve it using Gaussian elimination: multiplication of the third row with -1, and adding with the first row, the I got $$ \left[ \begin{array}{ccc|c} 1&-2&3&-4\\ 2&1&1&2\\ 0&2a&-2&2b \end{array} \right] $$

Dividing the third row with 2: $$ \left[ \begin{array}{ccc|c} 1&-2&3&-4\\ 2&1&1&2\\ 0&a&-1&b \end{array} \right] $$

Now my problem is, that I have two variables in one equation. I have no idea how to go next.

Answer 1

After a bit of Gaussian elimination, we arrive at

$$\begin{bmatrix}1 & 0 & 1 & 0 \\ 0 & 1 & -1 & 2\\ 0 & 0 & a+1 & -b - 2a\\ \end{bmatrix}$$

Now, when $a = - 1$ and $-b-2a \neq 0 \iff b\neq 2$, there will be no solution to the system of equations. Why not?

Answer 2

By row reduction the system becomes (if I didnt make a mistake (highly likely))

$$\begin{pmatrix} 1&0&1&0\\ 0&1&-1&2\\ 0&0&a+1&-b-2a\\ \end{pmatrix}$$

In order for the rank to be less than $3$ we need that $a+1=0$, so $a=-1$ for no solution we then need $-b-2a \neq 0$ so $b\neq 2$