The equation of the plane which passes through the point of intersection of lines $\frac{x-1}{3}=\frac{y-2}{1}=\frac{z-3}{2}$ and $\frac{x-3}{1}=\frac{y-1}{2}=\frac{z-2}{3}$ and at greatest distance from the point $(0,0,0)$ is
$(A)4x+3y+5z=25\hspace{1cm}(B)4x+3y+5z=50\hspace{1cm}(C)3x+4y+5z=49\hspace{1cm}(D)x+7y-5z=2$
I found the point of intersection of two lines as $(4,3,5)$.Let the equation of the required plane is $ax+by+cz+d=0$.Squared distance of plane $ax+by+cz+d=0$ from $(0,0,0)$ is $\frac{d^2}{a^2+b^2+c^2}$
We need to maximize $\frac{d^2}{a^2+b^2+c^2}$ under the constraint $4a+3b+5c+d=0$.
I cannot solve it further.I am stuck here.Please help me.Thanks.
An alternative to your approach is to this geometrically. To be the furthest distance then the plane must be perpendicular to the vector $(4,3,5)$. So to find the points of the plane $(x,y,z)$ we have:
$$(4,3,5)\cdot((x,y,z)-(4,3,5))=0$$ $$4x+3y+5x-16-9-25=0$$ $$4x+3y+5x=50$$