We are asked to find the equation of the parabola in the xy-plane that passes through the origin, has a vertical axis parallel to the y-axis, and passes closest - in the sense of least squares - to the points (-1,3), (1,1), and (2,5).
I tried to solve it by searching for $$ y=a_{0}+a_{1}x+a_{2}x^2,$$ but then I noticed that the axis of the parabola needs to be parallel to y'y, so I need to search for a parabola of the form $$x=a_{0}+a_{1}y+a_{2}y^2.$$ How should I proceed with the method of least squares now?
Plotting the points should convince you that the equation of the parabola should involve $y$ depending on $x$:
Plug in the given points $(x,y)$ to produce a system of $4$ equations in $3$ variables:
$$\begin{cases} 0 = a_0 & (x,y)=(0,0) \\ 3 = a_0 - a_1 + a_2 & (x,y)=(-1,3) \\ 1 = a_0 + a_1 + a_2 & (x,y)=(1,1) \\ 5 = a_0 + 2a_1 + 4a_2 & (x,y)=(2,5) \end{cases}$$
In matrix form,
$$\mathbf{Ax}=\mathbf b \iff \begin{bmatrix}1 & 0 & 0 \\ 1 & -1 & 1 \\ 1& 1 & 1 \\ 1 & 2 & 4\end{bmatrix} \begin{bmatrix}a_0\\a_1\\a_2\end{bmatrix} = \begin{bmatrix}0\\3\\1\\5\end{bmatrix}$$
To get the least-squares solution, left-multiply both sides by $\mathbf A^\top$, then by $(\mathbf A^\top\mathbf A)^{-1}$. (i.e. multiply by the left-pseduoinverse)
$$\mathbf{Ax}=\mathbf b \implies \mathbf x = (\mathbf A^\top\mathbf A)^{-1}\mathbf A^\top\mathbf b$$
You should end up with a solution that looks like this: