Let suppose we have system
$$ \dot x=f1(x,y,\alpha,\beta),$$ $$ \dot y=f2(x,y,\alpha,\beta),$$
For determining the stability of the system, I use a small pertub from fixed point like
$$ x = x0+x1 $$ $$ y = y0+y1 $$
where $ x0, y0 $ is the fixed point of the system (when $ \dot x = 0, \dot y = 0$) we could reduced to the new system
$$ \left[ \begin{matrix}\dot x1 \\ \dot y1 \end{matrix} \right] = J \left[ \begin{matrix}x1 \\ y1 \end{matrix} \right]$$
Then calculate the eigenvalues for J which is Jacobian Matrix, we could determine the stable state of the fixed point. But this is only for one set of ($ \alpha, \beta $)
My problem here is when we are varying the parameter $ \alpha, \beta $, how could I obtain the stability boundary? (numerically, because the system a little bit complicate, not easy to find the analytical solution). I'm finding an algorithm to do that. Could you please guide me a little bit? (MatLab example is the best)