If I want to define a set $A$ by an equation $\sqrt{x}$, but only for real numbers $R$, how it will be correct to define:
$$A = \{x:\sqrt(x),x\in R\}$$
or
$$A = \{x:\{x:\sqrt(x)\}∩R\}$$
or
$$A = \{x:\sqrt(x)\} ∩R$$
If I want to define a set $A$ by an equation $\sqrt{x}$, but only for real numbers $R$, how it will be correct to define:
$$A = \{x:\sqrt(x),x\in R\}$$
or
$$A = \{x:\{x:\sqrt(x)\}∩R\}$$
or
$$A = \{x:\sqrt(x)\} ∩R$$
An equation is a formula of the form $P=Q$, where $P$ and $Q$ are terms in some formal language. $\sqrt x$ is a term or expression, not an equation.
To define a set by an equation, you can use set operations like $\cap$, $\cup$, and $\setminus$, or set-builder notation. Given a logical formula $\phi$, the notation $\{x:\phi(x,x_1,\ldots,x_n)\}$ means "the set of all $x$ such that $\phi(x,x_1,\ldots,x_n)$ is true." For example, $\{x:\forall y\neg(y\in x)\}$ is the set of all sets which contain no elements - i.e. the set containing only the empty set. I would caution that the use of set-builder notation for arbitrary formulas tends to lead to paradoxes (the most famous example being Russell's paradox.) For this reason, mathematicians typically require the lefthand side of the expression to specify that $x$ is an element of some known set, as in $\{x\in X:\phi(x,x_1,\ldots,x_n)\}$. If $\{x:\phi(x,x_1,\ldots,x_n)\}$ is a set, then $\{x\in X:\phi(x,x_1,\ldots,x_n)\}=X\cap\{x:\phi(x,x_1,\ldots,x_n)\}$. For convenience, we sometimes omit bits of notation, such as writing $\{x:x^2<2\}$ in place of the more precise $\{x\in\Bbb R:x^2<2\}$.
Some general identities/conventions for setbuilder notation are:
$\{x\in X:x\in Y\}=X\cap Y$.
$\{x:\phi\land\psi\}=\{x:\phi\}\cap\{x:\psi\}$.
$\{x:\phi\lor\psi\}=\{x:\phi\}\cup\{x:\psi\}$.
For a function $f:X\to Y$, the set $\{y\in Y:(\exists x\in X)(\phi(x)\land y=f(x))\}$ is often shorthanded as $\{f(x)\in Y:\phi(x)\}$ or even $\{f(x):\phi(x)\}$.
Sometimes we'll write $\{y\in Y:y=g(x)\land x\in X\}$ instead of $\{y\in Y:(\exists x\in X)(y=g(x))\}$ (actually, that's shorthand for $\{y\in Y:\exists x(x\in X\land y=g(x))\}$.)
For logical formulas $\phi$ and $\psi$, we sometimes write $\{\phi(x):\psi(x)\}$ in place of $\{x:\phi(x)\land \psi(x)\}$. This is implicit in $\{x\in X:x\in Y\}=\{x:x\in X\land x\in Y\}$.
In your case, for maximum formality you would probably want to write $A=\{y\in \Bbb R:(\exists x\in\Bbb R)(y^2=x)\}$. However, as others have pointed out in comments, $\{\sqrt x:x\in\Bbb R\}$, $\{y\in\Bbb R:y=x^2\land x\in\Bbb R\}$, $\{y:y=\sqrt x,x\in\Bbb R\}$ all work.