This $y \in \{\sqrt{x}|x \in Q\}$
turned into this $∃x \in Q(y = \sqrt{x})$
In the second statement how would I read the stuff in the Q brackets. Q(...).
Also is there a different name for both of these statements notations used or are both of them set theory notation?
You're misreading the result. The brackets do not belong to the $\mathbb Q$ but to the entire $\exists x\in\mathbb Q$ prefix.
There are many ways to write punctuation for such quantified formulas, and the difference between them generally doesn't encode any meaning other than what the author thinks looks nicest. You can also find people would would write the same result as, for example $$ (\exists x\in \mathbb Q) \; y=\sqrt x \\ (\exists x\in \mathbb Q)(y=\sqrt x) \\ \exists x\in \mathbb Q : y=\sqrt x \\ \exists x\in \mathbb Q. y=\sqrt x $$ All of these variants pronounced, "There exists an $x$ in $\mathbb Q$ such that $y=\sqrt x$".
The $\exists$ symbol is called a quantifier -- more precisely, it is the existential quantifier. It is not, in particular, a "set theoretic" notation, but belongs instead to predicate logic.