The definition of complexification for a Lie group $G$ is a complex Lie group $G_C$ with a continuous homomorphism $\phi: G\to G_C$ with the universal property that, if $f: G → H$ is an arbitrary continuous homomorphism into a complex Lie group $H$, then there is a unique complex analytic homomorphism $F: G_C \to H$ such that $f = F \circ \phi$.
What can be a simple proof of the fact that with this definition the complexification of $O(n,\mathbb{R})$ is $O(n,\mathbb{C})$?
I will answer the question myself, providing a reference where an answer to my question is provided as solution to an exercise and attempting a solution for such exercise. Amendments and comments are more than welcome.
On Daniel Bump Lie Groups an explicit complexification is given for unitary realizations of $SO(n,\mathbb{R})$ (exercise 24.1) and $O(n,\mathbb{R})$ (chapter 30).
The author introduces the matrix
$$ J =\begin{bmatrix} & & & 1\\ & & . &\\ & . & &\\ 1 & & &\\ \end{bmatrix} $$
and defines
As part of exercise 5.3 it is asked to show that $O(n,\mathbb{C})_J \cong O(n,\mathbb{C})$ and that $O(n,\mathbb{C})_J \cap U(n) \cong O(n,\mathbb{R})$. Exercise 24.1 asks then to show that $SO_J(n,\mathbb{C})$ is the complexification of $SO(n,\mathbb{R})$ and chapter 30 mentions that, for the same reasons of exercise 24.1, $O_J(n,\mathbb{C})$ is the complexification of $O(n,\mathbb{R})$.
The theorem which I quote below in section 4 defines the complexification of some particular unitary groups $K$ as the inclusion $i: K \to G$ where $G$ is a complex analytic group. In particular, we will derive the complexification $i: SO_J(n,\mathbb{C}) \cap U(n) \to SO_J(n,\mathbb{C})$. In section 3 we find an isomorphism $\phi_1:SO(n,\mathbb{R}) \leftrightarrow SO_J(n,\mathbb{C}) \cap U(n)$ and in section 2 another isomorphism $\phi_2:SO_J(n,\mathbb{C}) \leftrightarrow SO(n,\mathbb{C})$, so the desired homomorphism $\phi$ of my question is given by $$ \phi: \phi_2 \circ i \circ \phi_1 $$ We will also show that, restricted to $SO(n,\mathbb{R})$, $\phi_2 \circ \phi_1 = I_d$, so $\phi = i$.
Following is my attempt to solve part of exercise 5.3 and 24.1 to prove this line of reasoning.
In exercise 5.3, the following matrix $\sigma$ is introduced, where all the elements not belonging to the main diagonals are zero and the "central" element is $1$ in case the dimension $n$ is odd:
$$ \sigma =\begin{bmatrix} 1/\sqrt{2i} & \dots & 0 & \dots & -i/\sqrt{2i}\\ \vdots & 1/\sqrt{2i} & \dots & -i/\sqrt{2i}& \vdots \\ 0 & & & & 0\\ \vdots & i/\sqrt{2i} & & -1/\sqrt{2i}& \vdots \\ i/\sqrt{2i} &\dots & 0 & \dots & -1/\sqrt{2i}\\ \end{bmatrix} $$ By direct computation we can show that
We have that, for each $g \in O_J(\mathbb{C})$, the conjugate $h = \sigma^{-1}g\sigma$ is complex orthogonal, as $$hh^t=\sigma^{−1}g\sigma \sigma^tg^t(\sigma^{−1})^t =\sigma^{-1}gJg^t (\sigma^{-1})^t=\sigma^{-1}J(\sigma^{-1})^t=I$$ $$h^th=\sigma^t g^t (\sigma^{-1})^t \sigma^{-1} g \sigma=\sigma^t g^t J g \sigma=\sigma^t J \sigma=I$$ Moreover, if $det(g) = 1$ then also $det(h)=det(\sigma^{-1}g\sigma)=1$. This means that the following isomorphisms hold:
As before, let $g \in O_J(\mathbb{C})$ and $h \in O(\mathbb{C})$ with $h=\sigma^{-1}g\sigma$. If $g$ is unitary, we have that $$\overline{h}=\overline{\sigma^{-1}g\sigma}=\overline{\sigma^{-1}}\overline{g}\overline{\sigma}=\sigma^t(g^{-1})^t(\sigma^{-1})^t=(\sigma^{-1}g^{-1}\sigma)^t=(h^{-1})^t=h$$ as $h$ is orthogonal, hence $h$ is real and belongs to $O(\mathbb{R})$.
Conversely, if $h \in O(\mathbb{R})$ we have that
$$ gg^*=\sigma h \sigma^{-1} (\sigma^{-1})^* h^* \sigma^*=\sigma h h^* \sigma^*=\sigma h h^t \sigma^*= I $$ $$ g^*g=(\sigma^{-1})^* h^* \sigma^* \sigma h \sigma^{-1} =(\sigma^{-1})^* h^* h \sigma^{-1}=(\sigma^{-1})^* h^t h \sigma^{-1}= I $$ hence $g$ is unitary. We have then that the group $O_J(\mathbb{C}) \cap U(n) \cong O(\mathbb{R})$. Similarly, we have that $SO_J(\mathbb{C}) \cap U(n) \cong SO(\mathbb{R})$.
We have found two unitary groups isomorphic to $O(\mathbb{R})$ and $SO(\mathbb{R})$.
Exercise 24.1 asks to show that $SO_J(\mathbb{C})$ is the complexification of $SO(n)$ using the previous results. I have chosen to use the following theorem (which is right before this exercise, so is presumably what the author wants me to use):
In the theorem proof, $G$ is shown to have decomposition $PK$, where $K$ is a faithful unitary representation of the group, $P=\{e^{iX}|X \in \kappa\}$ and $\kappa$ is the Lie algebra of $K$. By a previous theorem, for compact Lie groups an unitary faithful representation $K$ always exists. In our case the unitary representation $K$ of $SO(n)$ is indeed $SO_J(\mathbb{C}) \cap U(n)$. I will now prove that $G$ coincides with $SO_J(\mathbb{C})$ by proving that $G \subseteq SO_J(\mathbb{C})$ and then $SO_J(\mathbb{C}) \subseteq G$.
$G \subseteq SO_J(\mathbb{C})$:
By virtue of the algebra isomorphism induced by the conjugation by $\sigma$ of section 3, which is a Lie group isomorphism $SO_J(\mathbb{C}) \cap U(n) \cong SO(\mathbb{R})$, we can represent the elements $X \in \kappa$ as $\sigma H \sigma^{-1}$ where $H$ is a real antisimmetric matrix (as in $\mathfrak{so(n)}$).
So we have $e^{iX}=e^{i\sigma H\sigma^{-1}}=\sigma e^{iH}\sigma^{-1}$ and an element $g \in G=PK$ can be written as $\sigma e^{iH}\sigma^{-1} k$ where $k \in SO_J(\mathbb{C}) \cap U(n)$.
We have that $$gJg^t = \sigma e^{iH}\sigma^{-1} k J k^t (\sigma^{-1})^t e^{iH^t} \sigma^t = \sigma e^{iH}\sigma^{-1} J (\sigma^{-1})^t e^{iH^t}\sigma^t = \sigma e^{iH} e^{iH^t} \sigma^t= \sigma^{-1} e^{iH}e^{-iH} \sigma^t =\sigma \sigma^t=J$$ and $$det(g)=det(\sigma e^{iH}\sigma^{-1})det(k)=det(e^{iH})det(k)=e^{tr(iH)}=1$$ as $H$ is real antisimmetric. So $g \in SO_J(\mathbb{C})$.
$SO_J(\mathbb{C}) \subseteq G$:
Conversely, lets suppose $g \in SO_J(\mathbb{C})$. As $g$ is non singular, there is a unique polar decomposition $g=PK$, where $P$ is a hermitian positive semidefinite matrix and $K$ is unitary. As $K$ is unitary and $P$ is hermitian, we have that $$gg^*=PKK^*P^*=PKK^{-1}P=P^2$$ so $P=\sqrt{gg^*}$.
As $gg^*$ is hermitian positive definite there is a unique $P$ such that $P^2=gg^*$ so the square root $\sqrt{ g g^* }$ is well defined. As $g=\sigma h \sigma^{-1}$ we have that $P=\sigma \sqrt{ h h^* }\sigma^{-1}$ and also $\sqrt{ h h^* }$ is well defined as $hh^*$ is hermitian positive definite. Now we prove that $P \in SO_J(\mathbb{C})$ in the following way: as $h$ is orthogonal, so is $hh^*$. $\sqrt{ h h^* }$ is orthogonal as well, as we have $$\sqrt{ h h^* }(\sqrt{ h h^* })^t=\sqrt{ h h^* }\sqrt{ (h h^*)^t }=\sqrt{ h h^* }\sqrt{ (h h^*)^{-1}}=\sqrt{ h h^* }\sqrt{ (h h^*)}^{-1}=I$$ The uniqueness of the square root for $h h^*$ guarantees that we can switch transposition with square root and inversion with square root. Then $P=\sigma \sqrt{ h h^* }\sigma^{-1} \in SO_J(\mathbb{C})$ and $k \in SO_J(\mathbb{C})$ as well for group axiom. The last thing we have to prove is that $\sqrt{ h h^* }$ can be written as $e^{iH}$ with $H$ a real antisymmetric matrix (hence $iH$ is hermitian imaginary). $\sqrt{ h h^* }$ has a unique hermitian logarithm, as it is positive definite (the exponential matrix from the space of hermitian matrixes to the space of positive definite hermitian matrixes is bijective). Moreover, we have that
$$\overline{log(\sqrt{ h h^* })}=log(\overline{\sqrt{ h h^* }})=log(\sqrt{ \overline{h h^*}})=log(\sqrt{ \overline{h \overline{h^t}}})=log(\sqrt{ \overline{h \overline{h^{-1}}}})=log(\sqrt{ \overline{h (\overline{h})^{-1}}})=log(\sqrt{ \overline{h} (h)^{-1}})=log(\sqrt{(h \overline{(h)^{-1}})^{-1}})=log((\sqrt{h \overline{(h)^{-1}}})^{-1})=-{log(\sqrt{ h h^* })}$$
hence $iH=log(\sqrt{ h h^* })$ is imaginary.
When trying to apply the theorem used in the previous section to $O_J(\mathbb{C}) \cap U(n)$, we find an obstruction, because such group is not connected. However, I think that the condition of connectedness is used only in a section of the proof, where any $G=PK$ where $K$ is compact connected is proved to be a closed subgroup of $GL(\mathbb{C})$. In our case, we are dealing with a specific $G$ and if we manage to find that $O_J(\mathbb{C})$ is a subgroup of $GL(\mathbb{C})$ without resorting to connectedness, we are done. But $O_J(\mathbb{C})$ has been shown to be a group diffeomorphic to $O(\mathbb{C},n)$ in section 2 above, so it is necessarily closed, and hence an analytic Lie subgroup of $GL(\mathbb{C},n)$. We can then apply verbatim the reminder of the proof, where we set $O_J(\mathbb{C}) \cap U(n) = K$ (removing the requirement that $det(K)=1$) $P=e^{i\sigma H\sigma^{-1}}$ and, by the same arguments of section 4, $O_J(\mathbb{C}) = PK = G$:
In a different question I asked for a clarification of this proof of the theorem, but that's a separate matter.