I've come to know that,
For random variable $X$ ,with a Probability mass function P, $\phi_X(t)$ defined by :
$\phi_X(t) : \mathbb R \to \mathbb C$
$\phi_X(t) = E(e^{itX}) =E[\cos tX + i\sin tX]$
is called the characteristic function of $X$.
I know another notion of characteristic function which is basically a set theoretic notion and there the word characteristic makes more sense to me. But I'm unsure why here $\phi_X(t)$ is called Characteristic function.
According to Earliest Known Uses of Some of the Words of Mathematics, the term "characteristic function" in this sense (actually its French equivalent "fonction caractéristique") was first used by Poincaré in 1912 (except that with his notation, that function was what we now call the moment generating function).
To me as an analyst, the terminology never made a lot of sense: why not just call it the Fourier transform of the probability measure?(with $i$ replaced by $-i$ just for the sake of confusion). However, that same site says "Fourier transform" first appeared in English in 1923, so maybe the probabilists got there first!