There is a problem here in Q. $5$ on the last page. It states to find coordinates of point $p$.
Taking point $a=(3,2,5,1), \ b=(3,4,7,1), \ c= (5,8,9,3)$.
Also, $b$ has two coordinates in common with $a$, and $p$ lies on the same line as $a,b$.
So, those two coordinates of $p$ are same as $a,b$. Hence, $p= (3,x,y,1)$; where $x,y\in \mathbb{R}$ are unknown.
Given that $\triangle acp, \triangle bcp$ are right-angled; get:
$1. \ \ \triangle acp:\ \ \ \ \ {ac}^2 = {ap}^2 + {cp}^2\implies({(-2)}^2+6^2+4^2+2^2) = ({(x-2)}^2 +{(y-5)}^2) + (2^2+{(8-x)}^2+{(9-y)}^2+{(-2)}^2 )$
$60 = 2x^2+2y^2-20x-28y+182\implies x^2+y^2-10x-14y+61=0$
$2. \ \ \triangle bcp:\ \ \ \ \ {bc}^2 = {bp}^2 + {cp}^2\implies(2^2+4^2+2^2+2^2) = ({(x-2)}^2 +{(y-5)}^2) + (2^2+{(8-x)}^2+{(9-y)}^2+{(-2)}^2 )$
$28 = 2x^2+2y^2-20x-28y+190\implies x^2+y^2-12x-16y+95=0$
From $1,2$, get: $-2x -2y +34 = 0\implies x +y -17=0$.
But, how to proceed it further to find coordinates of $p$ is unclear.
Having $p=(3,x,y,1)$ does not reflect that $p$ is on the line $ab$. However, parametric equation of an arbitrary line passing through arbitrary points $a,\,b$ in Euclidean space is $x=a+t(b-a)$ with some parameter $t\in \mathbb{R}$, hence $$p=a+t(b-a)=(3,2,5,1)+t\,(0,2,2,0)$$ or, more compactly $$p=(3,2+2t,5+2t,1)$$ and your further computations will succeed as we have only one equation $$(b-a)\cdot(p-c)=0\;(\Leftrightarrow (b-a)\perp (p-c))$$ in other words, the two equations in the OP solution are equivalent and do not give solutions to two variables. $$(0,2,2,0)\cdot((3,2+2t,5+2t,1)-(5,8,9,3))=0$$ $$(0,2,2,0)\cdot(-2,2t-6,2t-4,-2)=0$$ $$2\cdot(2t-6)+2\cdot(2t-4)=0$$ $$t=\frac{5}{2}$$ $$p=(3,7,10,1)$$ Verification, for a case of a typo: $(b-a)\cdot(p-c)=0$.