Make a the subject, $$\sqrt{c-a^2}=a+b\tag1$$
$$c-a^2=(a+b)^2\tag2$$
$$c-a^2=a^2+b^2\tag3$$
$$c-b^2=a^2+a^2\tag4$$
$$a^2=\frac{c-b^2}{2}\tag5$$
$$a=\sqrt{\frac{c-b^2}{2}}\tag6$$
My teacher said I am wrong!
I trace all my steps but can't see where I went wrong.
You expanded incorrectly in $(3)$. Recall
$$\color{blue}{(a+b)^2 = a^2+2ab+b^2} \color{red}{\neq a^2+b^2}$$
You can try expanding it to confirm.
$$(a+b)^2 = (a+b)(a+b) = a^2+ab+ba+b^2 = a^2+2ab+b^2$$
So you can continue from $(2)$.
$$c-a^2 = (a+b)^2$$
$$c^2-a^2 = a^2+2ab+b^2$$
$$0 = 2a^2+2ab+b^2-c^2$$
You want to solve for $a$, so group the terms for solving more easily.
$$0 = 2a^2+2ba+\big(b^2-c^2\big)$$
Use the Quadratic Formula.
$$\color{blue}{a}x^2+\color{purple}{b}x+c = 0 \implies x = \frac{-\color{purple}{b}\pm\sqrt{\color{purple}{b}^2-4\color{blue}{a}c}}{2\color{blue}{a}}$$
$$0 = \color{blue}{2}a^2+\color{purple}{2b}a+\big(b^2-c^2\big) \implies a = \frac{-\color{purple}{2b}\pm\sqrt{(\color{purple}{2b})^2-4(\color{blue}{2})\big(b^2-c^2\big)}}{2(\color{blue}{2})}$$
Simplifying, you get
$$a = \frac{-2b\pm\sqrt{4b^2-8b^2+8c^2}}{4}$$
$$a = \frac{-2b\pm\sqrt{8c^2-4b^2}}{4}$$
$$a = \frac{-2b\pm\sqrt{4(2c^2-b^2)}}{4}$$
$$a = \frac{-2b\pm 2\sqrt{2c^2-b^2}}{4}$$
You can also complete the square, which yields the same answer.