Let $X$ be a matrix with 2 rows and 2 columns.
Solve the following equation: $$ X^2 = \begin{pmatrix} 3 & 5\\ -5 & 8 \end{pmatrix} $$
Here is what I did:
Let $ X = \begin{pmatrix} a & b\\ c & d \end{pmatrix} $. After multiplying I got the following system:
$$ \left\{\begin{matrix} a^2 + bc = 3\\ ab + bd = 5\\ ac + cd = -5\\ d^2 + bc = 8 \end{matrix}\right. $$
At this point I got stucked.
If you know how to solve this please help me! Thank you!
We have the following criteria which you already stated correctly, but you missed one more information $(5)$ - still, you can solve this root problem without this additional knowledge by plugging in recursively - which comes from the determinant, we get then \begin{align} a^2 + bc &= 3 \tag1\\ ab + bd &= 5\tag2 \\ ac + cd &= -5\tag3 \\ d^2 + bc &= 8\tag4 \\ \det(X)=ad-bc&=7=\sqrt{\det(M)} \tag{5a} \end{align} Remark remember that we have $\det(AB)=\det(A)\det(B)$
This gives us
\begin{align} (1)-(4)&=a^2-d^2=(a-d)(a+d)=-5\\ (2)&=b(a+d)=5 \\ (3)&=c(a+d)=-5 \\ (1)+(5a)&=a(a+d)=10 \end{align} so we get from $(2)\wedge(3)$ $b=-c$ and further $a-d=c$ and $a=-2(a-d)\iff\frac32a=d$ therefore \begin{align} a(a+d)=10=a(a+\frac32a)=\frac52a^2\iff4=a^2 \end{align}
and thus we get for $a=2$ \begin{align} a=2,d=3,c=-1,b=1 \end{align} so $$ X_1=\begin{pmatrix}2 &1\\ -1&3\end{pmatrix} $$ and for $a=-2$ \begin{align} a=-2,d=-3,c=1,b=-1 \end{align} so $$ X_2=\begin{pmatrix}-2 &-1\\ 1&-3\end{pmatrix}=-X_1 $$ Remark due to Robert Israel:
Indeed we have to investigate the other possible determinant solution \begin{align} \det(X)=ad-bc&=-7 \tag{5b} \end{align} then we get \begin{align} (1)+(5b)&=a(a+d)=-4 \\ (1)-(4)&=a^2-d^2=(a-d)(a+d)=-5 \end{align} which gives us $\frac54a=(a-d)\iff-\frac14a=d$ and therefore \begin{align} a(a+d)=-4=a(a-\frac14a)\iff-4=\frac34a^2 \end{align} which leads, if we stay in the field of the real numbers, to a contradiction. However, one might find for example other complex solutions. For a more detailed discussion please check out the comment section.