On visualizing the spaces $\Bbb{S}_{++}^n$ and $\Bbb{R}^n\times\Bbb{S}_{++}^n$ for $n=1,2,\ldots$

401 Views Asked by At

Let $\Bbb{S}_{++}^n$ denote the space of symmetric positive-definite $n\times n$ real matrices. I am looking for hints concerning the visualization of such spaces for $n=1,2,\ldots$. I know that $\Bbb{S}_{++}^n$ is a convex cone, but I am not sure how it does "look like". How could I compute the equation of the cone analytically? My idea is to define $\Sigma\in\Bbb{S}_{++}^2$, as $$ \Sigma=\pmatrix{\sigma_{11} &\sigma_{12} \\ \sigma_{12} &\sigma_{22}}, $$ and demand $\lvert\Sigma\rvert>0\implies\sigma_{11}\sigma_{22}-\sigma_{12}^2>0$, but then what?

Additionally, is we consider the space $\Bbb{R}^n\times\Bbb{S}_{++}^n$, what would it look like for $n\geq2$? For $n=1$, $\Bbb{R}\times\Bbb{S}_{++}$ should be the half-space $\{(x,\sigma)\in\Bbb{R}^2\colon\sigma>0\}$.

Any advice on how might I visualize spaces such as the above would be much appreciated! The visualization methods do not need to be strictly rigor, any intuitive solution would be appreciated too. Thanks in advance!

EDIT

Let me add another question associated with my original one. Let $f\colon\Bbb{R}^n\times\Bbb{S}_{++}^n\to\Bbb{R}$ given by $$ f(\mathbf{x},\Sigma) = \sum_{i=1}^{l} \alpha_i \exp \Bigg( - (\mathbf{x}-\mathbf{x}_i)^\top\Bigg(\frac{\Sigma+\Sigma_i}{2}\Bigg)^{-1}(\mathbf{x}-\mathbf{x}_i) - \ln \Bigg( \frac{\rvert\frac{\Sigma+\Sigma_i}{2}\lvert}{\sqrt{\rvert\Sigma\lvert\rvert\Sigma_i\lvert}} \Bigg) \Bigg) , $$ where $\alpha_i\in\Bbb{R}$, $\mathbf{x}_i\in\Bbb{R}^n$ and $\Sigma_i\in\Bbb{S}_{++}^n$ is the mean vector and the covariance matrix, respectively, of the $i$-th Gaussian distribution from a set of $l$ distributions $\{\mathcal{N}(\mathbf{x}_i,\Sigma_i)\}_{i=1}^{l}$.

I am interested in visualizing $f(\mathbf{x},\Sigma)=0$. I know that the space $\Bbb{R}^n\times\Bbb{S}_{++}^n$ has dimensionality equal to $n+\frac{n(n+1)}{2}=\frac{n^2+3n}{2}$. I am particularly interested in visualizing $f(\mathbf{x},\Sigma)=0$ in the case of $n=2$. Is that possible?

Yet Another EDIT

Concerning my second question (the first "EDIT" above), this is what I have come up with. I am thinking of drawing the $2$D curve that arises when one sets $\Sigma$ to be the zero matrix of order $2$. Then, I guess that I have the projection of the original space ($\Bbb{R}^n\times\Bbb{S}_{++}^n$) onto the Euclidean space ($\Bbb{R}^n$) for $n=2$. Is this correct? How could I denote that? For instance, could I state that I will give the visualization of $f(\mathbf{x},\Sigma)=0$ in the space $\Bbb{R}^n\times\{\mathbf{0}\}$, where $\mathbf{0}$ denotes the zero matrix of order $n$?

2

There are 2 best solutions below

12
On BEST ANSWER

The space of all symmetric, positive definite $2\times 2$ matrices consists of all matrices of the form $$ \begin{bmatrix}x & z \\ z & y\end{bmatrix} $$ satisfying the inequalities $$ xy - z^2 > 0 \qquad\text{and}\qquad x>0. $$ This comes from the fact that a matrix is positive definite if and only if its leading principal minors are all positive.

Geometrically, this is the open cone in $\mathbb{R}^3$ obtained by rotating the first quadrant $x>0$, $y>0$ in the $xy$-plane around the line $y=x$. The vertex of the cone is the origin $(0,0,0)$, and the axis is the line $y=x$ in the $xy$-plane. One way to see this is to note that the first inequality above can be written $$ \left(\frac{x-y}{\sqrt{2}}\right)^2 + z^2 < \left(\frac{x+y}{\sqrt{2}}\right)^2 $$ where $(x-y)/\sqrt{2}$, $(x+y)/\sqrt{2}$, and $z$ are an orthonormal system of coordinates on $\mathbb{R}^3$.

The intersection of this cone with the $xy$-plane is the open first quadrant. This represents the matrices for which $z=0$, i.e. the diagonal matrices with positive entries. Note that the eigenspaces of such a matrix are precisely the $x_1$ and $x_2$ axes in the $x_1x_2$-plane. Conceptually, rotating this quadrant around the line $y=x$ corresponds to rotating the eigenspaces in the $x_1x_2$-plane. However, note that a rotation by an angle of $\theta$ in $xyz$ space corresponds to a rotation of the eigenspaces by an angle of $\theta/2$ in the $x_1x_2$-plane.

In general, if we think of the space of all symmetric $n\times n$ matrices as $\mathbb{R}^{n(n+1)/2}$, then the matrices with determinant $0$ form some sort of hypersurface in this space, and the positive definite matrices will be one component of the complement of this hypersurface (which can be defined analytically using the leading principal minors). The positive definite matrices can always be obtained by "rotating" the space of positive diagonal matrices, although in general the rotation is more complicated than a simple rotation around an axis. This rotation corresponds geometrically to rotating the perpendicular frame of eigenspaces in $\mathbb{R}^n$.

11
On

Not sure what you intend with the second question. This is what I think: we don't really need to consider all the terms do we...$f(\mathbf x,\Sigma)=0$ is equivalent to one of the terms blowing up to infinity:

$$ (\mathbf x-\mathbf x_i)^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}(\mathbf x -\mathbf x_i)=\infty $$

One naturally resorts to homogenization $\mathbf x\mapsto\mathbf x/\mathrm x_0$: $$ (\mathbf x/\mathrm x_0 -\mathbf x_i)^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}(\mathbf x/\mathrm x_0 -\mathbf x_i)=\infty $$ being also equivalent to $$ \begin{split} (\mathbf x-\mathbf x_i\mathrm x_0)^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}(\mathbf x-\mathbf x_i\mathrm x_0)=\mathbf x^T\left(\frac{\Sigma+\Sigma_i}{2}\right)^{-1}\mathbf x&=0\\ \mathrm x_0&=0 \end{split} $$ This does have the geometric meaning of a quadric induced in a hyperplane if you fix $\Sigma$. When you change $\Sigma$, you end up with a family of quadrics. But of course if $\Sigma+\Sigma_i$ is positive definite, the quadric is an empty set so long as $\mathbb R^n$ is considered. I'm not sure how that might help.