After looking at the picture we are told to find the area enclosed by the total ellipse. See that the first quadrant contains one-fourth of the entire area.
The formula used as the equation of the ellipse is:
$$\frac{x^2}{a^2}+\frac{y^2}{b^2}=1$$ Solving for y:
$$y=\pm\frac{b}{a}\sqrt{a^2-x^2}$$ Then: $$y=\frac{b}{a}\sqrt{a^2-x^2}$$ Then: $$\frac{1}{4}A=\int^a_0\frac{b}{a}\sqrt{a^2-x^2}dx$$
What I don't understand is why a is chosen as the upper bound and not b?
Here you will find the complete method for this calculation. As for your specific question, I understand your confusion. To resolve it, make it a habit to write the variable of integration alongside the integration sign. Your correct step should be:
$$\frac{1}{4}A=\int^a_0\frac{b}{a}\sqrt{a^2-x^2}dx$$
Notice the $dx$ I added at the end. This makes it clear that the integration is taking place wrt $x$ (not $y$, as you must have originally thought). The upper limit for $x$ is clearly $a$ and not $b$.