I have the following non-convex constraint in variables $x \geq 0$ and $\textbf{p}$
$$ax+bx\|\textbf{p}\|^2-c \leq 0$$
where $a,b,c$ are positive constants. We can see that the above constraint does not result in a convex set. How to handle such kind of constraints in the optimization problem?
We have the following nonlinear constraint in $x \geq 0$ and $\mathrm y \in \mathbb R^n$
$$a x + b x \| \mathrm y \|_2^2 - c \leq 0$$
where $a, b, c > 0$ are given. Using Michael Grant's suggestion, we divide both sides by $x > 0$
$$a + b \| \mathrm y \|_2^2 - c x^{-1} \leq 0$$
Let $z := x^{-1}$. Hence, we obtain the following inequality in $\mathrm y \in \mathbb R^n$ and $z > 0$
$$(c z - a) - \mathrm y^\top \left( b^{-1} \mathrm I_n \right)^{-1} \mathrm y \geq 0$$
Since $b > 0$, we can use the Schur complement to write the inequality above as a (convex) linear matrix inequality (LMI) in $\mathrm y \in \mathbb R^n$ and $z > 0$
$$\begin{bmatrix} b^{-1} \mathrm I_n & \mathrm y\\ \mathrm y^\top & c z - a\end{bmatrix} \succeq \mathrm O_{n+1}$$