In the short while that I've known what conjugation is, I've met both kinds of notation about equally often (and the slightly less common $x^g$ a couple of times as well), so I am curious as to what is the difference between the two and whether they're equivalent. Is it a question of a purely stylistic nature, as, say, the choice $f(x)$ or $(x)f$?
Edit: Originally my question was about which of the 2 kinds of notation is more prevalent among MSE users, as I thought that they're 2 ways of denoting the same concept. I hadn't realised that it's actually 2 different definitions altogether, as opposed to simply a matter of notation; my thought process was that $g^{-1}x(g^{-1})^{-1}$ is the same as $g^{-1}xg$, i.e. having to do with what you take to be the "original" element, but apparently it has to do with the distinction between left and right actions.
My thanks to the other users for clearing this up!
You generally want $x^g$ to represent $g^{-1}xg$. The reason is so that the "exponential notation" works: $$(x^g)^h = h^{-1}(x^g)h = h^{-1}g^{-1}xgh = (gh)^{-1}x(gh) = x^{gh}.$$ If you were to use $x^g$ to denote $gxg^{-1}$, however, you get the wrong expression: $x^{gh} = ghxh^{-1}g^{-1} = (x^h)^g$.
That is why one usually sees ${}^gx$ used to denote $gxg^{-1}$.
Now, a different question is: when you say "$x$ conjugated by $g$", do you mean $gxg^{-1}$, or do you mean $g^{-1}xg$? The answer, unfortunately, is that it could mean either, depending on whether you like your actions on the left or on the right. The map $x\mapsto g^{-1}xg$ gives you a right action of $G$ on itself, whereas the map $y\mapsto gyg^{-1}$ gives you a left action of $G$ on itself.
They are both fine; it's a matter of what you prefer (like the commutator definition). Sometimes, other concerns affects your choice (much like the issue of "extension of $N$ by $K$", where people who do actions and representations prefer one meaning, and people who do varieties prefer the other). So long as you are consistent, you're fine, since a left action can be turned into a right action and vice-versa through an appropriate G-set isomorphism.
Note that if "conjugation by $g$" means $\varphi_g\colon x\mapsto gxg^{-1}$, then the map $G\to\mathrm{Aut}(G)$ given by $g\longmapsto \varphi_g$ is a homomorphism, since $$\varphi_{gh}(x) = (gh)x(gh)^{-1} = g(hxh^{-1})g^{-1} = \varphi_g(\varphi_h(x)).$$ Whereas if "conjugation by $g$" means $\psi_g\colon x\mapsto g^{-1}xg$, then the map $g\longmapsto \psi_g$ is not a group homomorphism from $G$ to $\mathrm{Aut}(G)$, because you get $$\psi_{gh}(x) = (gh)^{-1}x(gh) = \psi_h(\psi_g(x)).$$ So you get a map to $(\mathrm{Aut}(G))^{\mathrm{op}}$. This, and the fact that many people prefer left actions, is why in my experience $x\mapsto gxg^{-1}$ is slightly more prevalent.