I am trying to calculate the probability of a straight that ends on a specific number, e.g. 6:{2, 3, 4, 5, 6} or Q:{8, 9, 10, J, Q}.
Calculate Total number of straights:
Assume $x$ in [5,13(K)].
Because there are 4 suits and 5 cards in the straight, there are $4^5$ straights that will end $x$. After choosing the straight, there are 47 cards remaining. I cannot choose the cards whose number is $x+1$, so I subtract off those 4 cards to get 43. To choose the remaining 2 cards in the hand, I do ${43 \choose 2}=903$. So the total number of straights is $4^5 * 903=924,672$. (Note this number should include 6-card and 7-card straights)
Subtract off better hands:
Now I need to subtract off any "higher value" hands that are also straights.
- Royal Flush: 0 (Ace case: 4)
- Straight Flush: 4140 (moderately confident, see below)
- 4-of-a-kind: 0
- Full-house: 0
- Flush: ? (not confident, see below)
Straight Flush
After making the straight flush, can choose any 2 from the remaining deck, except the card right above, so I can choose from 52 - 5 - 1 = 46. ${46 \choose 2} = 1035$. With 4 suits, we get $4 * 1,035 = 4,140$.
Flush
I am already subtracting off the straight flushes, so I only need to subtract the flushes that are not fully included in the straights. I.e. There are 5 cards that are certainly part of the straight (IN), and 2 cards that are not necessarily (OUT). I can ignore cases where the 5 straight cards are also the 5 flush cards.
I split it up into 3 cases:
2 out, 3 in
(i.e. both cards outside the straight are part of the flush, 3 cards inside the straight are part of the flush)
2 of the inside cards are NOT part of the flush, chosen ${5 \choose 2}=10$ ways. 3 "other" suits for 2 cards makes $3^2=9$ more ways. 2 outside are chosen from 13 in suit, less the three that are inside. So that makes ${10 \choose 2}=45$. Then there are 4 suits, so I get $4 * 9 * 10 * 45 = 16,200$.
2 out, 4 in
(i.e. both cards outside the straight are part of the flush, 4 cards inside the straight are also part of the flush)
1 inside card is NOT part of the flush, chosen 5 ways. 3 "other" suits for 1 card makes 3 more ways. ${9 \choose 2}=36$ ways to choose outside. 4 suits, so I get $4 * 3 * 5 * 36 = 2,160$.
1 out, 4 in
1 inside card is NOT part of the flush, chosen 5 ways. 3 "other" suits for 1 card makes 3 more ways. 1 outside card is NOT part of the flush chosen 2 ways, with 38 possible options. 9 ways to choose outside flush card. 4 suits, so I get $5 * 3 * 2 * 38 * 4 * 9 = 41,040$.
Conclusion
So, in total: $924,672 - 4,140 - 16,200 - 2,160 - 41,040 = 861,132$
This is too high.
The correct total number of possible straights is 6,180,020 (from other tables).
I am getting 7,760,188 before even including the Ace-high case.
There are 9 "ending numbers" ([5, 13(K)]) that would, by my incorrect calculations, have 861,132, which sums to 7,760,188. Nearly 2 million hands too many.
Any advice on where I'm going wrong?