If P and Q are points of intersection of the circles $x^2+y^2+3x+7y+2p-5=0$ and $x^2+y^2+2x+2y-p^2=0$, then there is a circle passing through P, Q and (1,1) for how many values of p?
From the family of circles eqaution
$$S_1+\lambda S_2=0$$ The circle passes through (1,1)
So $$(7+2p)+\lambda (6-p^2)=0$$ $$\lambda =\frac{7+2p}{p^2-6}$$
Then it should be valid for values of p except for $\sqrt 6$. But the answer says -1 is not valid. What is correct answer?
On
You’ve made a slightly subtle, but common error. Observe that your “family of circles equation” omits $S_2$ itself: there’s no value of $\lambda$ for which $S_1+\lambda S_2$ is a multiple of $S_2$. That’s going to be a problem if $S_2$ passes through the point $(1,1)$, which it does when $p^2=6$.
You should instead start with the affine combination $(1-\lambda)S_1+\lambda S_2$, which doesn’t suffer from this flaw. It does omit the combination $S_1-S_2$, but that’s OK because that combination produces a line instead of a circle, anyway. If you work through the same calculations as before, you’ll end up with $\lambda(p+1)^2 = 2p+7$, which gives you a unique $\lambda$ unless $p=-1$, in agreement with the given solution.
That aside, I disagree with the given solution. The two circles only intersect when $p$ is in the (approximate) intervals $[-8.469,-1.744]$ and $[0.876,5.336]$†. Outside of these intervals there are no intersection points $P$ and $Q$ in the first place, so unless you’re allowing for imaginary intersection points, I would say that there are many more values of $p$ besides $-1$ for which there is no circle through $P$, $Q$ and $(1,1)$. Even taking this into consideration, though, $p=\pm\sqrt6$ do lie within the region in which the circles intersect, so your solution is still incorrect.
† More precisely, the endpoints of the intervals are the roots of $p^4+4p^3-44p^2-44p+69$, which are the values of $p$ for which the circles are tangent.
Hint
So, the equation becomes $$(1+\lambda)(x^2+y^2)+x(\cdots)+y(\cdots)+\cdots=0$$
What if $1+\lambda=0$