An urn contains $26$ colored balls: $12$ orange and $14$ blue. $11$ balls are selected at random without replacement.
(a) Probability of exactly $3$ of them being orange
(b) Probability that at least $5$ of them are orange
(c) Probability that the remaining balls are orange, given that at least $5$ of them are orange
My ideas for a solution:
(a) $\binom{26}{11}\times\frac{12}{26}\times\frac{11}{25}\times\frac{10}{24}\times\frac{14}{23}\times\frac{13}{22}\times\frac{12}{21}\times\frac{11}{20}\times\frac{10}{19}\times\frac{9}{18}\times\frac{8}{17}\times\frac{7}{16}$
(b) $\mathbb{P}(X \geq5) = 1 - \mathbb{P}(X \lt 5)$, where $\mathbb{P}(X \lt 5) = \mathbb{P}(X=0) + \mathbb{P}(X=1) + \mathbb{P}(X=2) + \mathbb{P}(X=3) + \mathbb{P}(X=4)$
(c) I'm not exactly sure how to set up this conditional probability
For some reason, I don't remember doing a lot of problems like this but here you go... Your answer for (a) is close but the binomial coefficient should be $\binom{11}{3}$ or equivalently $\binom{11}{8}$ instead and here is why: you are correct in multiplying the conditional probabilities together, namely your implied logic that the probability of drawing $3$ orange and $8$ blue balls from the urn is the same regardless of the order of the $11$ balls. But the number of outcomes is not $\binom{26}{11}$ but $\binom{11}{3}$ because you are choosing $3$ places out of $11$ for the orange balls. The other coefficient implies that you are selecting $11$ orange balls for example.
In general, if you have $N_{\mathbb{Or}}$ orange balls and $N_{B}$ blue balls (total of $N$) in an urn and you are selecting $n$ balls total with replacement, the probability of one possible order of draws where $k$ are orange is:
$$\frac{N_{\mathbb{Or}}}{N}\times\frac{N_\mathbb{Or}-1}{N-1}\times\cdots\times\frac{N_{\mathbb{Or}}-(k-1)}{N-(k-1)}\times\frac{N_B}{N-k}\times\cdots\times\frac{N_B-(n-k-1)}{N-(n-1)}$$
Then, letting $N_{\mathbb{Or},n}^{(S)}$ represent the number of orange balls that are selected out of $n$ total selected we have:
$$\mathbb{Pr}[N_{\mathbb{Or},n}^{(S)}=k]=\binom{n}{k}\prod_{a=0}^{n-1}\left(\frac{1}{N-a}\right)\prod_{b=0}^{k-1}(N_{\mathbb{Or}}-b)\prod_{c=0}^{n-k-1}(N_B-c)$$
This looks "messy" but hopefully it highlights how it is developed.
Now, for your part (b) you can use the above formula to get $\mathbb{Pr}[N_{\mathbb{Or},n}^{(S)}=i]$ for $i = 0,1,...,4$ and sum them up.
For part (c) you are being asked to compute $$\mathbb{Pr}[N_{\mathbb{Or},11}^{(S)}=11|N_{\mathbb{Or},11}^{(S)} \geq 5] = \frac{\mathbb{Pr}[N_{\mathbb{Or},11}^{(S)}=11 \bigcap N_{\mathbb{Or},11}^{(S)} \geq 5]}{\mathbb{Pr}[N_{\mathbb{Or},11}^{(S)} \geq 5]} = \frac{\mathbb{Pr}[N_{\mathbb{Or},11}^{(S)}=11]}{\mathbb{Pr}[N_{\mathbb{Or},11}^{(S)} \geq 5]}$$
You know the denominator from (b) and the numerator is easy.