I've always calculated the $Z_{score}$ of a value $x$ using the mean and standard deviation of a sample including $x$. However I have been thinking whether it wouldn't be more correct to use the mean and standard deviation of the sample excluding $x$.
Example:
$x=5$
$S = {1,2,3,4}$
$Z_{score}=({x-\mu})/\sigma = 1.26 $ where $\mu = \sum_{i=1}^5 i/5 = 3$ and $\sigma = 1.58$
$Z^{'}_{score}=({x-\mu^{'}})/\sigma^{'} = 1.94$ where $\mu^{'}=\sum_{i=1}^4 i/4=2.5$ and $\sigma^{'} = 1.29$
In most practical cases this doesn't seem to be significantly different, but I'm no statistician.
What are the pros and cons of using each method?