How to prove that the following function:
Is $$f: \mathbb R^n\rightarrow \mathbb R,\qquad f(x) = \frac{||Ax-b||_2^2}{1-x^Tx}, \qquad \text{ dom }f=\{x|||x||_2<1\}$$ convex? Here $\|x\|_2$ is the $2$-norm.
I tried to use the Jensen inequality or the second-order condition, but I can't figure them out. I think that's the wrong direction.
A convex model of the epigraph $$\left\{(t,x)\in\mathbb{R}^{n+1}~:~t\geq \frac{\|Ax-b\|_2^2}{1-\|x\|_2^2}, \ \|x\|_2\leq 1\right\}$$ is given by $$t(1-s)\geq \|Ax-b\|_2^2,$$ $$s\geq \|x\|_2^2,$$ $$t\geq 0,\ s\leq 1$$ and these constraints are convex because they describe two rotated quadratic cones in the sense of second-order conic optimization, see for instance https://docs.mosek.com/modeling-cookbook/cqo.html#rotated-quadratic-cones.