I want to solve the equation for ridge regression:
\begin{align} f(\beta)=&\frac{1}{2}||X \beta-y||_2^2+\lambda_{1} \frac{1}{2}||\beta||^2_2 \\ \end{align}
Where $X\in \mathbb{R}^{n\times n}, \beta\in \mathbb{R}^{d\times1}, y\in \mathbb{R}^{n\times1}$.
Over a line segment
\begin{align} (a+\gamma(s-a)) \end{align}
Where $a\in \mathbb{R}^{d\times1}, s\in \mathbb{R}^{d\times1}$
And I need to solve this for gamma. So I tried solving this as I would a scalar problem by substituting this in for $\beta$ and setting equal to $0$ as such:
\begin{align} \nabla f(\beta)=&X^{\top}(X \beta-y)+\lambda_{1} \beta \\ \nabla f((a+\gamma(s-a)))=&X^{\top}(X(a+\gamma(s-a))-y)+\lambda_{1}(a+\gamma(s-a))\\ =&X^{\top}(X a+\gamma X(s-a)-y)+\ldots\\ =&X^{\top}Xa+\gamma X^{\top} X(s-a)-X^{\top} y+\lambda_1 a+\gamma \lambda_{1}(s-a)\\ =&\gamma\left(X^{\top} X(s-a)+\lambda_{1}(s-a)\right)+X^{\top} X a-X^{\top} y+\lambda_1 a\\ \\ 0=&\gamma\left(X^{\top} X(s-a)+\lambda_{1}(s-a)\right)+X^{\top} X a-X^{\top} y+\lambda_1 a\\ \gamma\left(X^{\top} X(s-a)+\lambda_{1}(s-a)\right)=&X^{\top} y-X^{\top} X a-\lambda_1 a \end{align}
However at this point I have 2 $d\times1$ matrices on each side of the equation and must take the inverse to get the answer. However this results in a 2x2 matrix and I am doubtful that a scalar can solve this. Can someone point out where I went wrong or what approach I should have taken to solve this?
As @Marekkk alluded to in his comment, the solution was to substitute the line segment into the main equation and find the derivative with respect to gamma.