Let $k\in \mathbb{N}$. Consider the set $S^k$ of $k\times k$ symmetric matrices, and the set $S^k_+$ of $k\times k$ positive definite symmetric matrices. Let the inner product on $S^k$ be entry-wise inner product. Then what is the out normal of $S^k_+$ as a subset of $S^k$. My guess is that the outer normal is negative semi-definite.
Edit: By choosing an orthonormal basis for $S^k$, we can identity $S^k$ with $\mathbb{R}^{k(k+1)/2}$ isometrically. Then, viewing $S^k_+$ as a subset of $\mathbb{R}^{k(k+1)/2}$, we can make sense of the outer normal (hopefully almost everywhere when endowed with an appropriate Hausdorff measure on $\partial S^k_+$.
The boundary of $S^k_+$ consists of the set of all matrices that are positive semidefinite but fail to be strictly positive definite. Note that this set is not a smooth manifold. Rather, it is a variety with many singular points at which there is no "normal vector".
As it turns out, the singular points of the boundary are precisely the positive semidefinite matrices that have rank strictly less than $k-1$.
If $A$ is positive semidefinite with rank $k-1$, then the normal vector is defined. In particular, if $x$ is a non-zero vector in the (one-dimensional) kernel of $A$, then $xx^T$ is an inward-facing normal vector on the boundary of $S_+^k$. So indeed, the outer normal $-xx^T$ is negative semidefinite (more specifically, with rank $1$).
As it turns out, this rank one vector is a multiple of the adjugate $\operatorname{adj}(A)$ of $A$.
It's interesting to specifically consider the $2 \times 2$ case. Relative to the basis $$ \pmatrix{1&0\\0&0}, \quad \frac 1{\sqrt{2}}\pmatrix{0&1\\1&0}, \quad \pmatrix{0&0\\0&1}, $$ we find that the set boundary to the set of positive definite matrices is given by $$ B = \{(a,b,c) : a \geq 0 \text{ and } 2ac - b^2 = 0\}. $$ We can compute the gradient of $f(a,b,c) = 2ac - b^2$ at any point to find the normal vector $$ \nabla f = (2c,-2b,2a). $$ In other words: if the matrix $$ A = \pmatrix{a & b/\sqrt{2}\\ b/\sqrt{2} & c} $$ is positive semidefinite and non-zero, then the normal vector at $a$ will point along the matrix $$ B = \pmatrix{c & -b/\sqrt{2}\\ -b/\sqrt{2} & a}. $$ We indeed see that this is a multiple of the adjugate of $A$.