For which values of $a,b\in\mathbb{R}$ does the matrix have no solutions, one solution and infinite solutions.
I have the following functions: $$ax+4y +az= 0 \\ x + ay + 3z = b \\ x(a+1) + y(a+4) + z(a-b^2) = b-2 $$ I have reduced this to: $$\left(\begin{array}{ccc|c} 1& a& 3 & b\\ 0 & 1 & \frac{1}{2a} & \frac{b}{4-a^2} \\ 0 & 0 & 1 & \frac{2}{b^2+3} \end{array}\right) $$ (can't figure out how to make a line in the matrix).
Using this I get to the point where I have $4xb^2+12x-a^2b^2-4a^2 = 8b^3+24b-b^3a^2+3ba^2$ what can I conclude from this, and have I done anything wrong here?
First swap the first and second row. Then by subtracting the first and the second row from the third one and then subtracting $a$ times the first row from the second we get:
$$\left(\begin{array}{ccc|c} 1& a& 3 & b\\ 0 & 4-a^2 & -2a & -ab \\ 0 & 0 & -b^2-3 & -2 \end{array}\right) $$
If $a \not = \pm 2$ we have a unique solution, as the coefficient matrix would be invertible, considering that $-b^2 - 3 \le -3 < 0$
if $a=\pm2$ by substituting the matrix will reduce to:
$$\left(\begin{array}{ccc|c} 1& \pm 2& 3 & b\\ 0 & 0 & 1 & \frac b2 \\ 0 & 0 & 0 & -2 + \frac{b^3 + 3b}{2} \end{array}\right) $$
The last equation has infinitely many solutions if $-2 + \frac{b^3 + 3b}{2} = 0$ and none otherwise.