A teacher gives a student a bag which contains white and black balls. The teacher asks the student to pick $5$ balls out of the bag. The students picks 5 white balls and the teacher says : "The probability of this event is $0.5$". Can you determine how many white and black balls are in the bag ?
I assume that the trials are independent and that the balls are picked from the bag without replacement (the problem does not mention this). Let $n_b$ (resp. $n_w$) denote the number of black (resp. white) balls in the bag. The probability of picking $5$ white balls from the bag (without replacement) is:
$$ p = \frac{1}{2} = \frac{n_w (n_w - 1) (n_w - 2) (n_w - 3) (n_w - 4)}{5!(n_w + n_b)^5} = \frac{n_w!}{(n_w - 5)!5!(n_w + n_b)^5}$$
[The probability of picking the $1$st ball from the bag is $n_w/(n_w+n_b)$. For the second, it is $(n_w - 1)/(n_w + n_b)$... and so on].
Is this the right way to go with this problem ?
That does not look right, since you keep dividing by $n_w + n_b$, which assumes that you pick with replacement.
With $B$ the number of black balls, and $W$ the number of white balls, you should simply get that:
$$P = \frac{W \choose 5}{{W + B} \choose 5}$$
Or, using your method:
$$ p = \frac{n_w (n_w - 1) (n_w - 2) (n_w - 3) (n_w - 4)}{(n_w + n_b)(n_w + n_b-1)(n_w + n_b-2)(n_w + n_b-3)(n_w + n_b-4)}$$
One solution is $B=1$ and $W=9$ ... I suspect there are no other solutions but don't know how to prove that ...