For a given matrix $X=\left(x_{i,\,j}\right)_{(i,\,j)}$, I am searching for a mathematical operator that will give me another matrix $Y$ with the absolute values of $X$, i.e.: $Y=\left(|x_{i,\,j}|\right)_{(i,\,j)}$.
It would be nice, if I could write something like $Y=|X|$ but I dont think, that this is a valid notation. Is there any convention how to do so for matrices (or vectors)? Or do I have to define my own operator like $\tilde{X}:=\left(|x_{i,\,j}|\right)_{(i,\,j)}$?
Thank you! :)
Taking the elementwise absolute value of a matrix or vector does not, at first glance, seem like the kind of operation which is going to come up very often in mathematics—such an operation doesn't play nice with the underlying structures of the spaces being considered (i.e. the algebra structure of a set of matrices, or the vector space structure of a set of vectors). Hence I doubt that there is any kind of standard notation.
If $X = (x_{ij})$, then it is entirely reasonable to "decorate" the notation $X$ in order to denote the object you are interested in. Your suggestion of a tilde is a fine decoration, and I doubt that it will cause confusion, particularly if you clearly define the notation early on. Off the top of my head, the notations $$ X^{|\cdot|} \qquad\text{or}\qquad X^{\text{abs}} $$ might also be reasonable, and perhaps slightly more expressive.
Finally, there are some notations which I would avoid:
Don't use $|X|$. This notation is already somewhat overloaded in the context of matrices and vectors. As Nate Eldredge points out in a comment, it might be understood as the matrix square root of $X^T\!X$. Personally, I would assume that $|X|$ denotes the determinant of a matrix. And there are other meaningful notions of the "absolute value" of a matrix, hence even in "standard" contexts, this notation has the potential to be ambiguous.
Similarly, $|v|$ typically denotes the (Euclidean) length of a vector $v \in \mathbb{R}^n$.
Don't use $\|X\|$. This notation denotes the norm of a matrix, vector, or operator. This may mean precisely the same as $|X|$, above, or could imply a more general notion of norm, such as the $p$-norm of a vector, e.g. $$ \|v\| = \left( |v_1|^p + |v_2|^p + \dotsb + |v_n|^p\right)^{1/p}, $$ where $v \in \mathbb{R}^n$. Note that this might also be written as $\|v\|_p$. In any event, the notation $\|X\|$ is likely to cause confusion.
Don't use $X^*$. This is typically understood to be the adjoint of an operator (conjugate-transpose of a complex-valued matrix). The notation is very common in functional analysis / operator theory.
Don't use $\hat{X}$. If $v$ is a vector in $\mathbb{R}^n$, then $\hat{v}$ is often understood to mean a unit vector in the direction of $v$. There is also some possibility that $\hat{X}$ could be mistaken for a Fourier transform, though I think that this is unlikely.