$$\left(\begin{array}{ccc|c} -1 & 2 & 1 & 3\\ 3 & \alpha & -2 & \beta\\ -1 & 5 & 2 & 9 \end{array}\right)$$
I am struggling to solve this system $Ax=b$. I understand the basics of Gauss elimination but am not sure how to handle it with the alpha and beta. It needs to be solved giving conditions for alpha and beta for no solutions, one solution and infinitely many solutions.
I also need to compute the determinant of A and give a condition on $\alpha$ such that $A$ is invertible.
On
$$\det\left( \begin{array}{rrr} -1 & 2 & 1 \\ 3 & a & -2 \\ -1 & 5 & 2 \\ \end{array} \right)=-3-\alpha$$ if $-3-\alpha\ne 0$ that is $\alpha\ne -3$ there exists one and only one solution
$$\left\{\frac{3 a-b+6}{a+3},\frac{b+3}{a+3},\frac{3 (2 a-b+3)}{a+3}\right\}$$
if $\alpha=-3$ the determinant of the matrix $A$ of coefficients is zero $\text{rank}(A)=2$, so we must consider the augmented matrix $A|B$ $$\left( \begin{array}{rrr|r} -1 & 2 & 1 & 3 \\ 3 & -3 & -2 & b \\ -1 & 5 & 2 & 9 \\ \end{array} \right)$$
if $\text{rank}(A|B)\ne \text{rank}(A)$ the system has no solutions, therefore to have infinite solutions we must have $\text{rank}(A|B)=2$ so all $3-$rd order determinants extracted from $A|B$ must be zero
$$ \det \left( \begin{array}{rrr} 2 & 1 & 3 \\ -3 & -2 & \beta \\ 5 & 2 & 9 \\ \end{array} \right) =\beta+3 $$
if $\beta\ne -3$ the system is impossible
if $\beta+3=0\to \beta=-3$ the determinant is zero the rank of the augmented matrix is equal to the rank of $A$ and we have infinite solutions
$\{t,\;t,\;3-t\}$
Let me start with some steps. Perhaps you'll be able to finish.
$$\left(\begin{array}{ccc|c} -1 & 2 & 1 & 3\\ 3 & \alpha & -2 & \beta\\ -1 & 5 & 2 & 9 \end{array}\right)\sim \left(\begin{array}{ccc|c} 1 &-2 &-1 &-3\\ -1 & 5 & 2 & 9\\ 3 & \alpha & -2 & \beta \end{array}\right)\sim \left(\begin{array}{ccc|c} 1 &-2 &-1 &-3\\ 0 & 3 & 1 & 6\\ 0 & \alpha+6 & 1 & \beta+9 \end{array}\right)\sim \left(\begin{array}{ccc|c} 1 &-2 &-1 &-3\\ 0 & 3 & 1 & 6\\ 0 & \alpha+3 & 0 & \beta+3 \end{array}\right) $$
Side note: Some comments to the strategy for this: It is good to avoid computations involving parameters if possible. That's the reason why I put the row with parameters into the last place. Similarly, that's the reason why in the last step I used third column rather than second. (It is much easier to subtract second row from the third, than to subtract $(\alpha+6)/3$ multiple of second row. At least when doing things by hand, it might help if we try to get simpler expressions.)
If you look at the above matrix, the last row corresponds to the equations $(\alpha+3)x_2=\beta+3$. What can you say about solutions of this equation depending on $\alpha$ and $\beta$. (What happens if $\alpha=-3$? What happens in $\alpha\ne-3$? Think about similar possibilities for $\beta$.)
We could as well introduce new parameters $c=\alpha+3$ and $d=\beta+3$ to simplify notation a bit. (In this way we have the equation $cx_2=d$ and we consider what happens depending on whether $c$, $d$ is zero or non-zero.)
If $c\ne0$, then the system will have unique solution. Let us denote $t=d/c$ to simplify the notation: $$\left(\begin{array}{ccc|c} 1 &-2 &-1 &-3\\ 0 & 3 & 1 & 6\\ 0 & c & 0 & d \end{array}\right)\sim \left(\begin{array}{ccc|c} 1 &-2 &-1 &-3\\ 0 & 3 & 1 & 6\\ 0 & 1 & 0 & t \end{array}\right)\sim \left(\begin{array}{ccc|c} 1 & 0 &-1 &-3+2t\\ 0 & 1 & 0 & t\\ 0 & 0 & 1 & 6-3t \end{array}\right)\sim \left(\begin{array}{ccc|c} 1 & 0 & 0 & 3-t\\ 0 & 1 & 0 & t\\ 0 & 0 & 1 & 6-3t \end{array}\right) $$ where $t=\frac dc = \frac{\beta+3}{\alpha+3}$. (You can check by direct computation that this is indeed a solution.)
If $c=0$, then we have $$\left(\begin{array}{ccc|c} 1 &-2 &-1 &-3\\ 0 & 3 & 1 & 6\\ 0 & 0 & 0 & d \end{array}\right).$$
What happens if $d\ne0$? What happens if $d=0$?
You mentioned that you also want to compute determinant of $A$. This can be done using basically the same row operations as above - you just need to know how row operations influence the determinant.