I have ten professors, and need to pick four of them for a committee. It is a bad idea for Hatfield and McCoy to serve together. It is a bad idea for El and Luthor to serve together. How many possible committee assignments are there?
I was thinking 10 choose 4 for all the possible committee combinations and now subtract 2*10 choose 2 for the committees Hatfield and McCoy serve together as well as the committees that El and Luthor serve together. Then add one back on the end for the committee that all four would serve on together. Not sure if that will work to count the committees. Let me know your thoughts.
There are three major ways to approach this:
First is purely additive, finding and summing combinations that we know will be valid. There are 6 professors who can serve in any combination of four, and there are 15 such possible combinations of those 6 profs taken 4 at a time (the order in which the profs are chosen is irrelevant, so these are combinations, not permutations). On top of that, there are 20 combinations of these 6 profs taken 3 at a time, to which you can add any one of the four "restricted" profs, so there are 80 additional combinations of four professors which includes one restricted prof. Finally, there are 15 possible unique pairs of the 6 unrestricted profs, to which you could add either El and Hatfield, El and McCoy, Luthor and Hatfield or Luthor and McCoy, so there are 60 combinations with a valid pair of the restricted profs. 15 + 80 + 60 = 155.
You could also approach it from the opposite direction, starting with all possible combinations and removing those we know will be invalid. This was your chosen strategy if I read your OP right. There are 210 total possible combinations of the 10 profs taken 4 at a time. Of these, some would be invalid because they have one of the two invalid pairs: El and Luthor or Hatfield and McCoy. There are 15 possible pairs for the other two profs, and so 30 combinations that have an invalid pair. Other combinations would be invalid because they have an invalid pair plus another of the restricted profs; there are four of these invalid combinations (basically 4 nCr 3; E-L-H, E-L-M, H-M-E, H-M-L) and six possibilities for the last person, or 24 combinations. Finally, there is one combination with both invalid pairs comprising all four profs on the committee. 210 - 30 - 24 - 1 = 155.
One more way to build it up: if you take one valid pair of the restricted profs (Hatfield-El, Hatfield-Luthor, McCoy-El, McCoy-Luthor) and exclude the other pair, now you have 8 profs for which any combination of four is valid, or 70 possibilities. There are, as shown, four combinations of includable pairs (or excludable ones depending on your POV) and so you'd multiply by 4 (280); however, this total now includes certain overlapping sets. There are 15 combinations (6 nCr 4) that don't have any restricted profs, and these combinations will be present in all four groups, so we should remove three of those groups' duplicates from our total. Similarly, for each single restricted prof, there are two groups with 20 combinations each (6 nCr 3) that have that professor, and so for each of those four profs we subtract 20 duplicate possibilities. So, 280 - 3(15) - 4(20) = 155.
Obviously, all three strategies will get you to the right answer (and there are others); the "best" strategy will depend on how easy it is for you to ensure you have included all possibilities when including or excluding groups.