A(1,-1) and B(2,-3) are two points and D is a point in AB produced such that AD=3AB. Find the coordinates of D.
After writing the equations of slope and applying distance formula I got two solutions for D : (4,-11) and (-2,13).
But the answer key says that the answer's (4,-11). Why should we reject (-2,13) ?
It's not clear, but if your problem is asking for you to solve a vector equation, then you haven't finished the problem — you've correctly solved for the points $D$ for which $\vec{AD}$ and $3\vec{AB}$ have the same magnitude, but you haven't actually checked the solutions to see whether they make the vectors equal.
As an aside, there's a simpler way to do the calculation: substitute $\vec{AD} = D - A$ and $\vec{AB} = B - A$ and solve the equation for $D$.