I have a linear system where $x$ is unknown and $y$ is known, both vectors have proper dimensions such that:
$$A(\theta)x=y$$
matrix $A$ depends on a particular parameter $\theta$. I know that for a fixed $\theta_0$ i can solve the linear system by using projection methods (for example least squares which minimizes the euclidean norm). Now, I would like to find the optimal parameter $\theta$ that leads to $x$ with a minimum 2-norm. How can i proceed? My idea was to derive the entire system with respect to $\theta$ and then solve the derivatives with respect to $\theta$ but other suggestions are also appreciated.