$$v_1 = 4i+2j$$
Firstly find alpha by taking dot product of velocity and y=x $$\cos\alpha = \frac{(4i + 2j)\cdot(i+j)}{\sqrt{(1+1)(4^2+2^2)}}$$ Then by pythagoras, $\sin\alpha = \frac{1}{\sqrt{10}}$
So $v=(|v_1|\cos\alpha)i + (|v_1|e\sin\alpha)j=(\sqrt{20}\cdot\frac{3}{\sqrt{10}})i + (\sqrt{20}\cdot \frac{1}{3} \cdot\frac{1}{\sqrt{10}})j=(3\sqrt{2})i + (\frac{\sqrt{2}}{3})j$
However, this is not correct in the textbook. I think I am wrong to apply the coefficient of restitution like this, since $X$ is not parallel to the line so $X$ of $v_1$ will not necessarily equal $X$ of $v$, i.e. some amount of the COR would have to be applied to each? Am I correct in my thinking here and how can I amend what I have done so far?(the textbook provides an entirely different approach, but I was able to solve problems where the line is flat with my current reasoning)
You need to decompose the velocity into one component $v_{\parallel}=a(i+j)$ parallel to the wall and one $v_\perp=b(i-j)$ perpendicular to the wall. This results in an easily solvable system $v_1=v_∥+v_⊥$ with solution $a=3$, $b=1$. The interaction with the wall only changes $v_\perp$, so that the reflected velocity is $v_2=v_∥-\frac13v_⊥=\frac83 i+\frac{10}3 j$.