do percentiles follow poisson distribution?

42 Views Asked by At

I don't have formal background in mathematics or stats, so I apologize for my non-technical language.

I am wondering what distribution, and is this Poisson, normal distribution percentiles follow. In other words – Let's say I generate a normal RVs and I count number of measurements in top 99 percentile. For $100$ measurements afaik I should expect one such event, for $1000$ there should be $10$. I wonder what would be the distribution of those counts will be if I run the experiment $N$ times.

So $N$ experiments, $1000$ measurements in each (where variable follows norm distribution), I count measurements that are in 99 percentiles.

I wrote some ugly code for that and I wonder if I made a bug somewhere, because this is how it looks:

def run_experiments(n=1000,alpha=0.01,N=10000):
    events=[]   # event - number of measurements that fall in given percentile 
    for i in range(N):
        df=pd.DataFrame({'pdf':norm().pdf(norm.rvs(size=n))}) # generate normal rvs 
        events.append(len(df.where(df['pdf']<alpha).dropna(how='any'))) # save how many fell into a percentile
    return events

df=pd.DataFrame({'events':run_experiments()}) # run experiment and save to dataframe 
fig,ax=plt.subplots(1,1)   # create plot
ax.hist(df,bins=max(df['events'])) # histogram plot 
ax.set_xticks([i for i in range(21)]) # make sure ticks are ok 
ax.grid(True) # grid is important in plotting 
plt.show() # showing the plot is also important 

The histogram I arrive at is following:

histogram of events for my experiment

Could anyone please let me know what am I missing – I was expecting a Poisson histogram with $\mu = 1000 \cdot 0.01 = 10$, and I believe I received Poisson with $\mu = 6$. When I ran the experiment with $\alpha = 0.005$ I received following histogram.

enter image description here

I couldn't find a coding error but maybe I am missing something obvious.

Edit: Ok, I think my error is in filtering which is applied on PDF, I should have applied it on sigma and RVs. However, the Poisson question still applies.