Let $X$ be a Hilbert space and let $X'$ be the dual space of $X$ with respect to the duality pairing $\langle\cdot,\cdot\rangle$.
Let $A: X \mapsto X'$ be a bounded linear operator. We assume that $A$ is self-adjoint, i.e., we have $\langle Au,v\rangle = \langle u,Av\rangle $ for all $u,v \in X$
I am confused about the meaning of being self-adjoint, since if $\langle Au,v\rangle := (Au)(v) $ then what does $\langle u,Av\rangle $ mean?
In a Hilbert space, you have the Riesz Representation Theorem, which tells you that given any $f\in X'$, there exists $y\in X$ such $$\tag{1}f(y)=\langle y,x\rangle,\ \ \ x\in X.$$ And this assignment is isometric so $X'$ is isomorphic to $X$. In practice, one thinks that $X'=X$ via the duality $(1)$.
In summary, in the case of a Hilbert space $X$ the duality $\langle\cdot,\cdot\rangle$ is precisely the inner product of $X$.