I have a function $f(\vec{x})$ that converts a vector to a scalar. $f$ is relatively complex and thus this needs to be solved numerically. Maybe something like gradient descent.
I want to find the vectors $\vec{x}$ such that for a given value of $|\vec{x}|$, have the maximum possible value of $f(\vec{x})$
Is there a way to find these vectors numerically.
I tried using Lagrange multipliers and found this is equivalent to asking to find the vectors $\vec{x}$ such that $\nabla f(\vec{x}) = \lambda \vec{x}$. Is there any way to solve this numerically.
I tried using iteration to find fixed points of the gradient but this yield ony one non-trivial solution. I think solutions with small values of $\lambda$ are being drowned out. How can I solve this problem?
Yes, there is a simpler way.
If your vector is $2$ dimensional, then you can let $r=|\vec{x}|$, and also declare $\theta$, such that $\vec{x}=\begin{bmatrix}r\cos \theta \\ r\sin\theta \end{bmatrix}$.
Here, $r$ is a known constant, and $\theta$ is your input.
Then, make the substitution, in your function $f(\vec{x})$ such that the function is now in terms of $r$ and $\theta$ instead of $x_1$ and $x_2$, the components of $\vec{x}$.
Now, you can use Newton's Method (find the root of $f'(\theta)$ to find the $\theta$ that maximizes $f(\theta))$, and then use that $\theta$ and your $r$ to find $\vec{x}$.