I'm not sure how to find possible values of $a_{11}$, $a_{12}$, $a_{22}$, $a_{21}$.
$$\begin{align} \left(a_{11} a_{22}-a_{12}a_{21}\right)^2 &= \phantom{-}1 \\[6pt] a_{11}^2 + a_{21}^2 &= \phantom{-}2 \\[6pt] a_{11}a_{12}+a_{21}a_{22}&=-1 \\[6pt] a_{12}^2 + a_{22}^2 &= \phantom{-}1 \end{align}$$ One of the solutions is $(a_{11},a_{12},a_{21},a_{22})=(1,0,-1,1)$.
(original problem image, ignoring statistical terms on the left)
On
You may use a geometrical approach.
Given $$\begin{align} \left(a_{11} a_{22}-a_{12}a_{21}\right)^2 &= \phantom{-}1 \\[6pt] a_{11}^2 + a_{21}^2 &= \phantom{-}2 \\[6pt] a_{11}a_{12}+a_{21}a_{22}&=-1 \\[6pt] a_{12}^2 + a_{22}^2 &= \phantom{-}1 \end{align}$$ we know from the first equation that the area of the parallelogram spanned by $\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}$ and $\begin{pmatrix}a_{12}\\a_{22}\end{pmatrix}$ equals one and from the second equation that the length of equals $\sqrt2$ whereas from the fourth equation that the length of $\begin{pmatrix}a_{12}\\a_{22}\end{pmatrix}$ equals one.
First choose any vector $\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}$ of length $\sqrt2$. Then the vector $$\pm\frac{1}{\sqrt2}\begin{pmatrix}-a_{21}\\a_{11}\end{pmatrix}$$ is an orthogonal unit vector to $\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}$, hence $\pm1/2\begin{pmatrix}-a_{21}\\a_{11}\end{pmatrix}$ has length $1/\sqrt2$ so the rectangle spanned by $\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}$ and $\pm1/2\begin{pmatrix}-a_{21}\\a_{11}\end{pmatrix}$ has area one and that area won't change if we add some multiple of $\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}$ to it.
From the fourth equation we want to have the length of $$\pm1/2\begin{pmatrix}-a_{21}\\a_{11}\end{pmatrix}+\lambda\begin{pmatrix}a_{11}\\a_{21}\end{pmatrix}$$ to be one, which brings us to $\lambda=\pm\frac12$. But only $\lambda=-\frac12$ for $+$ and $\lambda=\frac12$ for $-$ fulfils the third equation.
Finally the solutions are $$a_{12}=\frac12(a_{11}-a_{21})\text{ and }a_{22}=\frac12(a_{11}+a_{21})$$ and $$a_{12}=-\frac12(a_{11}+a_{21})\text{ and }a_{22}=-\frac12(a_{11}+a_{21}).$$
Let
$$A=\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}$$
First method :
Let $$\vec{U}=(a_{11},a_{21}), \vec{V}=(a_{12},a_{22})\tag{*}$$
(column vectors of $A$).
Let $U$ and $V$ be their resp. norms, and $\theta=angle(\vec{U},\vec{V}).$
First equation : cross product :
$$(UV \sin \theta)^2=1 \ \ \iff \ \ UV \sin \theta=\pm 1\tag{1}$$
Third equation : dot product
$$UV \cos \theta=-1\tag{2}$$
Second and fourth equation :
$$U=\sqrt{2}, \ \ \ V=1\tag{3}$$
Using (2) and (3) we get
$$\cos \theta=-\frac{1}{\sqrt{2}}=\cos(\frac{3\pi}{4}) \ or \ \cos(\frac{5\pi}{4})$$
Using (1) and (3) we get :
$$\sin \theta =\pm \frac{1}{\sqrt{2}}=\sin(\frac{\pi}{4}+k \frac{\pi}{2}) $$
Therefore, there are two solutions for angle $\theta$:
$$\theta=\frac{3\pi}{4} \ \ or \ \ \theta=\frac{5\pi}{4}.$$
Let us take $\vec{U}=(\cos u,\sin u)$ where $u$ is an arbitrary angle.
Then, as $\theta$ is the angle between the two vectors, and $\vec{U}$ has its norm $V=1$ :
$$\vec{V}=(\cos (u+\theta),\sin(u+\theta)).$$
giving the general expressions of entries $a_{ij}$ (see (*)).
Otherwise said, the genral solutions are, for an arbitrary angle $u$ :
$$A=\begin{pmatrix}\sqrt{2}\cos u&\cos (u+3 \pi/4)\\ \sqrt{2}\sin u&\sin (u+3 \pi/4)\end{pmatrix} \ \ \text{or} \ \ A=\begin{pmatrix}\sqrt{2}\cos u&\cos (u+5 \pi/4)\\ \sqrt{2}\sin u&\sin (u+5 \pi/4)\end{pmatrix}$$
Taking $u=-\frac{\pi}{4}$ gives back the particular solution.
Second method :
The 3 last equations can be gathered in the following matrix equation :
$$\underbrace{\begin{pmatrix}a_{11}&a_{21}\\a_{12}&a_{22}\end{pmatrix}}_{A^T}\underbrace{\begin{pmatrix}a_{11}&a_{12}\\a_{21}&a_{22}\end{pmatrix}}_A=\underbrace{\begin{pmatrix}2&-1\\-1&1\end{pmatrix}}_B$$
wher $B$ is a positive definite matrix.
The first equation is naturally a consequence of this relationship (by equating the determinants of both sides).
How can be solved the matrix equation :
$$A^TA=B \ \ \text{ with unknown} \ \ A \ ?$$
The simplest way is to find a particular solution :
$$A_0^TA_0=B$$
then take account of the fact that $A^TA=A_0^TA_0 \ \ \iff \ \ (AA_0^{-1})^T(AA_0^{-1})=I$
means that $AA_0^{-1}$ is an orthogonal matrix. An orthogonal matrix in $\mathbb{R}^2$ can have two types :
$AA_0^{-1}=R$ or $AA_0^{-1}=S$,
where $R=\begin{pmatrix}\cos \alpha&-\sin \alpha\\\sin \alpha&\cos \alpha\end{pmatrix}$ and $S=\begin{pmatrix}\cos \alpha&\sin \alpha\\\sin \alpha&-\cos \alpha\end{pmatrix}$
where $\alpha$ is an arbitrary angle (resp. R = rotation matrix and S = symmetry matrix).
Therefore, the definitive answer is :
$$A=RA_0 \ \ \text{or} \ \ A=SA_0 \tag{4}$$
It remains to find the particular solution $A_0$ : it is given by the so-called "Cholesky factorization", a standard procedure, giving the upper triangular solution :
$$A_0=\begin{pmatrix}\dfrac{1}{\sqrt{2}}&-\dfrac{1}{\sqrt{2}}\\0&1\end{pmatrix}$$
It suffices now to replace $A_0$ by this expression in (4) to retrieve the two categories of solutions given by the first method.