I want to find all critical points of the function $f\colon \Bbb R^2 \to \Bbb R$ with $f(x,y) = x \sin y + ax^2 + by^2$ in dependence of $a,b \in \Bbb R$.
I computed
$$D_{(x,y)}f = \begin{bmatrix} \sin y+2ax & x\cos y+2by \end{bmatrix}$$
And from the equation $D_{(x,y)}f = \begin{bmatrix} 0&0 \end{bmatrix}$ I get the non-linear system of equations
$$\begin{cases} \sin y+2ax &= 0 \\ x\cos y+2by &=0 \end{cases}$$
I don't know how to start solving this, especially the Sine and Cosine seem to make it difficult. How would you approach it?
We have $$\sin y+2ax = 0\tag{1}$$ $$x\cos y+2by=0\tag{2}$$ so from $(1)$, $$x=-\frac{\sin y}{2a}\tag{3}$$ and putting $(3)$ into $(2)$, we get $$\sin2y=8aby\tag{4}$$ This cannot be solved algebraically since $\sin$ and $\cos$ are transcendental functions.