In a village with N people, M are Male and F are female. It was found that X of the male and Y of female are students. If a person is selected randomly, what is the probability that:
A) The person is student and he is a male.
Which formula should I use here?
P(S and M) = X/N
or
P(S/M) = P(S and M)/P(M)
There is no condition here. As it is stated in your question, you are looking for probability that a randomly selected person is both male and a student. Let's look at this from a counting standpoint.
From here, we can also extrapolate the following (might be unnecessary, but let's be thorough...)
The key here is the "given" condition. We were already told that $X$ is the number of villagers who are male and students. So the number of students who are both male and students is simply $X$ divided by the sample space, which in this case is all villagers, or cardinality wise is just $N$. Thus
$$P(\text{student and male}) = \frac{X}{N}$$
Conditional probability though puts a condition on the original sample space of $N$ villagers. So by looking at $P(\text{student} | \text{male})$, you are changing the sample space to only male students. Using the conditional probability formula,
$$P(\text{student} | \text{male})=\frac{P(\text{student and male})}{P(\text{male})}=\frac{X/N}{P(\text{male})}=\frac{X/N}{M/N}=\frac{X}{M}$$
But again, think of what the original problem is asking..... out of ALL villagers (not just the male).... this is why the first solution is correct.