How does the Z-transformation work?

Question

I understand that to transform any normal distribution to the standard normal distribution we use the Z-transformation: $$Z = \frac{x - \mu}{\sigma}$$

What I am looking for is a basic and intuitive explanation of how does this transformation work. i.e how does finding the Z-score of a value of a normal distribution means finding its equivalent in the standard normal distribution?

Answer 1

1. Let's $X\sim N(\mu;\sigma^2)$ that is,

$$f_X(x)=\frac{1}{\sigma\sqrt{2\pi}}e^{-\frac{1}{2\sigma^2}(x-\mu)^2}$$

with $x \in \mathbb{R}$, $\mu \in \mathbb{R}$, $\sigma >0$

Let's take the new random variable

$$Z=\frac{X-\mu}{\sigma}$$

Let's calculate the CDF of Z by definition:

$$F_Z(z)=\mathbb{P}[Z \leq z]=\mathbb{P}[\frac{X-\mu}{\sigma} \leq z]=\mathbb{P}[X \leq \sigma z+\mu]=F_X(\sigma z+\mu)=\int_{-\infty}^{\sigma z+\mu}f_X(x)dx$$

to get $f_Z(z)$ simply derive the CDF

$$f_Z(z)=\frac{d}{dz}F_Z(z)=f_X(\sigma z+\mu)\cdot \sigma=\frac{1}{\sqrt{2\pi}}e^{-\frac{z^2}{2}}$$

2. another way to see the problem is to observe that the Gaussian family is a Location-Scale Family, that is

$$g(x|\mu,\sigma)=\frac{1}{\sigma}\psi(\frac{x-\mu}{\sigma})$$

where $\psi(.)$ is the Standard Normal Density

Answer 2

This is due to the fact that the family of normal distributions is scaling and translation invariant. It not the only one, but its importance due to the central limit theorem makes it special.

In lay terms: if $Z$ is the normal $0$, standard deviation $1$, then one can generate normal distribution with a described mean $\mu$ and standard deviation $\sigma$ by setting $X=\mu + \sigma Z$. So, if one knows how to sample random numbers distributed according to $Z$, the transformation $Z\mapsto \mu+\sigma Z$ will give you obtain samples of the normal distribution $N(\mu,\sigma)$.

As you may be aware of this, a proof of why this works is based on a simple change of variables in some integral: $$ \frac{1}{\sqrt{2\pi\sigma^2}}\int_A e^{\tfrac{(x-\mu)^2}{2\sigma^2}}\,dx=\int_{\frac{1}{\sigma}(A-\mu)} e^{-\tfrac{z^2}{2}}\,dz $$

---

As I mentioned before, there are other scaling and/or translation invariant families. If one has measures of that type, using a single sample of a fixed distribution within the family, allows one to generate samples of any other distribution within the family by a simple transformation (translation, scaling, or both via the transformation $f(x-\mu)$, $\frac{1}{\sigma}(f(x/\sigma)$ or $\frac{1}{\sigma}f((x-\mu)/\sigma)$ respectively)