I have the following inequality that has to hold for all frquencies $w$
$${\left[ {\begin{array}{*{20}{c}} {H\left( {jw} \right)}\\ 1 \end{array}} \right]^*}\left[ {\begin{array}{*{20}{c}} { - 2mn}&{ - m - n}\\ { - m - n}&{ - 2} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {H\left( {jw} \right)}\\ 1 \end{array}} \right] \prec 0 \quad \quad(1)$$ where $H\left( {jw} \right) = \frac{{cb}}{{jw - a}} + d$. Now if we factorize the matrix
$$\left[ {\begin{array}{*{20}{c}} { - 2mn}&{ - m - n}\\ { - m - n}&{ - 2} \end{array}} \right] = \left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)}&{\frac{1}{2}\left( {m + n} \right)}\\ 0&1 \end{array}} \right]\left[ {\begin{array}{*{20}{c}} 2&0\\ 0&{ - 2} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)}&0\\ {\frac{1}{2}\left( {m + n} \right)}&1 \end{array}} \right]$$ Note that the matrix $$\left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)}&{\frac{1}{2}\left( {m + n} \right)}\\ 0&1 \end{array}} \right]$$ could be considered as a mobius transformation $$f\left( z \right) = \frac{{\frac{1}{2}\left( {m - n} \right)z}}{{\frac{1}{2}\left( {m + n} \right)z + 1}}$$ This mobius maps the circle symmetric about real axis and cuts it at $-1/m$ and $-1/n$ ($m\ge n>0$)to the unit circle. (1) is feasible if and only if the nyquist of $H(s)$ does not cut the circle symmetric about real axis and cuts it at $-1/m$ and $-1/n$. How could I prove that?
my attempt:
If I use $$\left[ {\begin{array}{*{20}{c}} { - 2mn}&{ - m - n}\\ { - m - n}&{ - 2} \end{array}} \right] = \left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)}&{\frac{1}{2}\left( {m + n} \right)}\\ 0&1 \end{array}} \right]\left[ {\begin{array}{*{20}{c}} 2&0\\ 0&{ - 2} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)}&0\\ {\frac{1}{2}\left( {m + n} \right)}&1 \end{array}} \right]$$
and plug it, the inequality becomes
$${\left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)H\left( {jw} \right)}\\ {\frac{1}{2}\left( {m + n} \right)H\left( {jw} \right) + 1} \end{array}} \right]^*}\left[ {\begin{array}{*{20}{c}} 2&0\\ 0&{ - 2} \end{array}} \right]\left[ {\begin{array}{*{20}{c}} {\frac{1}{2}\left( {m - n} \right)H\left( {jw} \right)}\\ {\frac{1}{2}\left( {m + n} \right)H\left( {jw} \right) + 1} \end{array}} \right] \prec 0$$
If I could somehow conclude that the inequality is feasible whenever the nyquist of $f\left( {H\left( {jw} \right)} \right) = \frac{{\frac{1}{2}\left( {m - n} \right)H\left( {jw} \right)}}{{\frac{1}{2}\left( {m + n} \right)H\left( {jw} \right) + 1}}$ lies outside the unit circle I can argue that the inverse mobius $f^{-1}(z)$ would mean the nyquist of $H(jw)$ lies outside the circle symmetric about real axis and cuts it at $-1/m$ and $-1/n$, but how could I conclude the former from the inequality in (2)?
From what I can see, this means that the graph of $H(j\omega)$ should live in some not-too-simple region of the complex plane.
To figure this out, let us consider the quadratic form
\begin{equation} \begin{bmatrix} s\\ 1 \end{bmatrix}^*\begin{bmatrix} -2mn & -(m+n)\\ -(m+n) & -2 \end{bmatrix}\begin{bmatrix} s\\ 1 \end{bmatrix} \end{equation} and check in what region of the complex plane this inequality is negative. Letting $s=\sigma+j\omega$ we get the inequality $$-2mn(\sigma^2+\omega^2)-2(m+n)\sigma-2<0.$$
Clearly, $\omega\ne0$ can only make this inequality more negative. So, the first idea would be to check for what values for $\sigma$ we have that the inequality holds for all $\omega\in\mathbb{R}$. Analyzing this polynomial, we find that it takes negative values for all $\omega\in\mathbb{R}$ provided that $$\sigma\notin[-1/m,-1/n].$$ This is described by the set
$$\mathcal{S}:=((-\infty,-1/m)\cup(-1/n,+\infty))\times\mathbb{R}.$$
Now we can look at when the right-hand side of $$mn\omega^2>-mn\sigma^2-(m+n)\sigma-1>0$$ is positive. This happens when $\sigma\in[-1/m,-1/n]$ and in this case, the term $\omega^2$ needs to be chosen such that the above inequality holds. This region is described as $$\mathcal{P}=\{(\sigma,\omega)\in[-1/m,-1/n]\times\mathbb{R}:mn\omega^2>-mn\sigma^2-(m+n)\sigma-1\}.$$
In summary, the region is described by the quadratic form is $\mathcal{S}\cup\mathcal{P}$. Getting back to your original problem, you will have to make sure that $H(j\omega)\in\mathcal{S}\cup\mathcal{P}$ for all $\omega\in\mathbb{R}$. There are many possible cases. Some are simple as they are structural, some others need more care.
For instance, if $a,b,c,d$ are such that $\min\{d,d-bc/a\}>-1/n$ or $\max\{d,d-bc/a\}<1/m$, then this is fine.
More complex cases arise when say $-bc/a+d>-1/n$ and $d<-1/m$. In such a case, we need to make sure that the curve does not enter the forbidden region.