In how many ways can $5$ men and $5$ women be arranged in a circle, if two particular women must not be next to a particular man?
May anyone explain why the answer is $864$:
If the separated man $O$ is the starting point and the separated women $W_1,W_2$ are to choose any seat except the two seats beside $O$, so they would have $7$ seats to choose from, implying $P^7_2$ arrangements. Then the other men and women would have $P^4_4 P^3_3$ choices respectively. Therefore the total arrangements would be $P^4_4 \times P^3_3 \times P^7_2$. However, my logic is incorrect since the actual answer is $P^4_4 \times P^3_3 \times P^3_2$ but this suggests $W_1,W_2$ only have $3$ seats to choose instead of $7$. Why is it so?
Judging by the book's answer, there is an unstated assumption that the men and women alternate seats.
Let's consider both the stated question and what appears to be the intended question.
Suppose the man in question is Oliver and the women who will not sit next to him are Nancy and Paula.
Seat Oliver. Seat one of the seven people who have not been named to Oliver's left and one of the six remaining people who have not been named to his right. That leaves seven people, including Nancy and Paula, to be seated in the remaining seven chairs. As we proceed clockwise around the table relative to Oliver, those seats can be filled in $7!$ ways. Hence, the number of distinguishable seating arrangements of ten people in which neither Nancy nor Paula sits next to Oliver is $7 \cdot 6 \cdot 7!$.
Let Oliver, Nancy, and Paula play the same roles as above.
Seat Oliver. Since men and women alternate seats, doing so determines which seats will be occupied by the men and which seats will be occupied by the women.
Since men and women alternate seats, only women can be placed next to Oliver. Since Nancy and Paula do not sit next to Oliver, that leaves three women available to be Oliver's neighbors. There are three ways to select the woman who sits to Oliver's left. Once that woman has been seated, there are two women who can be placed to his right. That leaves three seats for the three remaining women, who include both Nancy and Paula. The three women can be placed in those seats in $3!$ ways as we proceed clockwise around the table relative to Oliver. The remaining four seats must be occupied by the remaining four men. Those seats can be filled in $4!$ ways as we proceed clockwise around the table from Oliver. Thus, the number of distinguishable seating arrangements in which men and women alternate seats and neither Nancy nor Paula sits next to Oscar is $3 \cdot 2 \cdot 3! \cdot 4!$, which is equal to the book's answer.