You decide to make a lottery with n tickets, where each ticket is numbered between 1 and n, and each ticket is unique. Each ticket costs $5, and the lottery works in the following manner. Once all n tickets have been purchased, a number x is selected at random between 1 and n and all the money is divided equally between people with tickets less than x. That way, if 1 is selected, you(as the organizer) get to keep the prize pool.
Everyone's number is randomized, the only case where the organizer wins the prize pool is if x=1, as no ticket number is less than 1, so no person wins anything and so the organizer automatically wins the prize pool. Also, the organizer does not hold the number 1, and does not win any money if the number is greater than 1.The only way where they earn the prize pool is if the chosen number is 1, other than that, there is no way they can win any money.
The first part is to calculate the expected winnings per lottery as the organizer. So there is only one case where you can win the prize pool, I multiplied the case where you win and the prize pool, so I get:
$$\frac{1}{n} \cdot 5n = 5$$
Now, the second part is to calculate the expected winnings if somebody purchases a ticket. I am not able to figure out the probability where somebody wins. For the prize pool, I am confused if it is 5n or 5p (where p is the person's ticket number).
A quick way to solve the problem:
As you have noted, the probability that no contestant gets any payout is $\frac 1n$ (as only the selection $1$ achieves that result). Thus there is a $\frac {n-1}n$ chance that the contestants (collectively) will get the entire pool of $5n$. But, of course, a priori, each contestant has the same expected gain here so the answer must be $$\frac {n-1}{n^2}\times 5n=\frac {5(n-1)}n=5-\frac 5n$$
Phrased differently: the expected payout to the organizer is $5$, and the expected total payout is $5n$ (that's the guaranteed payout). thus each contestant must expect to get $$\frac {5n-5}n=5-\frac 5n$$ which, of course, is the same result.
Sanity check: Let's do this in the case $n=2$. In that case a contestant either gets a $1$ or a $2$. If they get a $2$, they can not make any money. If they get a $1$, they have a $\frac 12$ chance of getting $10$ and a $\frac 12$ chance of getting $0$. Thus the answer is $\frac 12\times \frac 12\times 10=\frac 52$, as predicted.
Similarly, let's handle $n=3$. A contestant's outcome is determined by the pair $(i,j)$ where $i$ is the ticket value and $j$ is the chosen value. There are $9$ possible outcomes, each with probability $\frac 19$. The only pairs that have positive payouts are $(1,2)$ which has a payout of $15$ and $(2,3), (1,3)$ each of which has a payout of $\frac {15}2$. Thus the expected payout is $\frac 19\times \left( 15+\frac {15}2+\frac {15}2\right)=\frac {10}3$ as predicted.