For conditional distribution $$f_{X|Y}(x|y) = \frac{f(x,y)}{f_Y(y)}$$
this is the basic definition I know about conditional distribution
Consider n + m trials having a common probability of success. Suppose, however, that this success probability is not fixed in advance but is chosen from a uniform (0, 1) population. What is the conditional distribution of the success probability given that then + m trials result inn successes?
But for this question
why for this problem conditional formula looks like
$$f_{X|N}(x|n) = \frac{P(N=n|X=x)f_X(x)}{P(N=n)}$$????

You say you understand $$f_{X \mid Y} (x \mid y) = \frac{f(x,y)}{f_Y(y)}.$$ Then you should also understand $$f_{Y \mid X} (y \mid x) = \frac{f(x,y)}{f_X(x)}$$ which can be rearranged as $$f(x,y) = f_{Y \mid X}(y \mid x) f_X(x).$$ Plugging this last expression for $f(x,y)$ into the first equation above yields $$f_{X \mid Y} (x \mid y) = \frac{f_{Y \mid X}(y \mid x) f_X(x)}{f_Y(y)}.$$ This is sometimes called Bayes's rule.