Consider the simplest example of simple Lie algebra $L=\mathfrak{sl}(2,\mathbb{C})$. In terms of generators and relations, it is given by $$\langle x,y,h:[h,x]=2x, [h,y]=-2y, [x,y]=h\rangle.$$ The root space decomposition is $$L=H\oplus L_{\alpha}\oplus L_{-\alpha} = \langle h\rangle \oplus \langle x\rangle \oplus \langle y\rangle.$$ Here root system is just $\Phi=\{\alpha,-\alpha\}$. This root system has an automorphism $\alpha\mapsto -\alpha$.
My aim is to understand how this automorphism of root system gives an automorphism of Lie algebra $L$.
As Humphreys in Lie algebra (Section 14.3, p. 76-77) describes:
(1) The induced map on $H$ sends $h$ to $-h$. [OK]
(2) To apply theorem 14.2 (in book above), we decree that $x$ should be sent to $-y$. [Why?]
My question is, how do we justify (2)? Or say, why can't we do the following: $$x\mapsto 2y, y\mapsto \frac{1}{2}x ? $$ or even more simple $$x\mapsto y, y\mapsto x?$$
Edit: I think, for any non-zero $k$, the map $h\mapsto -h$, $x\mapsto ky$ and $y\mapsto \frac{1}{k}x$ defines automorphism of $L$.
In general, does automorphism of a root system of Lie algebra uniquely determines automorphism of Lie algebra?
Reference: (Page 76-77 of Humphreys Lie algebra)
Here, Theorem 14.2 is the following one.
I think that there is some misunderstanding here. For a semisimple Lie algebra $\mathfrak g$, the automorphisms of $\mathfrak g$ form a Lie group whose Lie algebra is isomorphic to $\mathfrak g$. (This is a consequence of the fact that any derivation of $\mathfrak g$ is inner.)
So there is certainly no simple correspondence to automorphisms of the root system, which form a finite group. I don't have the book you are referring to at hand, so I am not sure what is done there. I suppose that it is shown that any automorphism of the root system is induced by an automorphism of $\mathfrak g$ which fixes the Cartan subalgebra $\mathfrak h$. There is certainly a huge freedom of choice in such an automorphism, in particular you can always apply an automorphism of $\mathfrak g$ which induces the identity on $\mathfrak h$. In that sense, the automorphism you are proposing (with $x\mapsto ry$ for $r\in\mathbb C\setminus\{0\}$) is the composition of the automorphism in Humphrey's book with the adjoint action of the matrix $\begin{pmatrix} s & 0 \\ 0 & s^{-1}\end{pmatrix}\in SL(2,\mathbb C)$, where $s^2=-r$. (This adjoint action induces the identity on $\mathfrak h$.) So your choice would be OK as well.
The important thing about automorphisms of the root system is that they give rise to automorphisms of the Dynkin diagram. These automorphisms describe the quotient of all automorphisms of $\mathfrak g$ by the inner automorphisms (i.e.~the adjoint actions of elements of a connected Lie group with Lie algebra $\mathfrak g$).