Consider the following dynamic system \begin{align} f(x,y) = \left(\frac{1}{4}\left(\frac{1}{2y} - x\right), \frac{10y^2 - 12xy + 3}{24y - 16x}\right). \end{align} There exists a steady state at $(x_\infty,y_\infty) = (\sqrt{30}/6,\sqrt{30}/10)$ such that $f(x_\infty,y_\infty) = (0,0)$. The jacobian at the steady state reads \begin{align} \mathrm J_f(x_\infty,y_\infty) = \begin{bmatrix} -\frac{1}{4} & -\frac{5}{12}\\ \frac{9}{2} & 0 \end{bmatrix}. \end{align} Now note that the determinante is positive $\mathrm D_\infty = \mathrm{det}(\mathrm J_f(x_\infty,y_\infty)) = 15/8$, the trace negative $\mathrm T_\infty = \mathrm{trace}(\mathrm J_f(x_\infty,y_\infty)) = -1/4$ and $\mathrm D_\infty - \mathrm T_\infty^2/4 = 119/64$ such that according to the Poincare diagram the steady state is a spiral sink.
When I consider a streamslice plot, however, it is evident that the steady state is unstable.
What am I missing?
The Matlab code reads:
clear all; clc;
syms x y
dx = (1/(2*y) - x)/4;
dy = (10*y^2 - 12*x*y + 3)/(24*y-16*x);
xss = sqrt(30)/6;
yss = sqrt(30)/10;
J = jacobian([dx,dy],[x,y]);
Jss = subs(J,[x,y],[xss,yss]);
D = det(Jss);
T = trace(Jss);
C = D - T^2/4;
dx_fun = matlabFunction(dx);
dy_fun = matlabFunction(dy);
X_max = 2;
Y_max = 2;
N = 50;
[X,Y] = meshgrid(linspace(0.01,X_max,N),linspace(0.01,Y_max,N));
M = 50;
[start_x,start_y] = meshgrid(linspace(0.01,X_max,M),linspace(0.01,Y_max,M));
dX = dx_fun(X,Y);
dY = dy_fun(X,Y);
hold on
figure(1)
streamslice(X,Y,dX,dY)
plot(xss,yss,'r*')
xlim([0 X_max])
ylim([0 Y_max])
%saveas(gcf,'poin.png')
The fixed point is stable as your analysis predicts—it's just not globally stable.
This is clear if you zoom in to the fixed point a little: