When defining ultraproduct it is defined to be product of domains of models $A_\xi$ modulus the equivalence relation by ultrafilter on index set. The relation on the product are defined by $(a_0/_{\mathcal U}, \dots, a_n/_{\mathcal U}) \in R$ iff $\{\xi : (a_0(\xi), \dots, a_n(\xi))\in R_\xi\}$ where $R_\xi$ is relation on model with domain $A_\xi$ and similar for the function.
I define the constant $C \in \prod_{\xi \in X}A_\xi / \mathcal U$ as $C = (\prod_\xi C_\xi)/\mathcal U$ where $C_\xi$ is the constant in model with the domain $A_\xi$. How to show it is well-defined?
There is no well-definedness issue at all. If $C$ is a constant symbol in the language $L$, then the interpretation $C_{\xi}$ of $C$ in the model with domain $A_{\xi}$ is uniquely defined. So the "sequence" $(C_{\xi})$, what you call $\prod_{\xi} C_{\xi}$, is uniquely determined, and hence so is $(\prod_{\xi} C_{\xi})/\mathcal{U}$
The situation is different with say a unary function symbol $f$ of the language $L$. For there we define $f$ at $(\prod_{\xi} a_{\xi})/\mathcal{U}$ is defined in terms of the $f(a_{\xi})$. It is conceivable that different choices of representatives $\prod_{\xi} a_{\xi}$ and $\prod_{\xi} a_{\xi}'$ yield different results. Easily, they don't, but this has to be checked.