I cannot understand what it formally means when one says that a subset of the cone of positive semidefinite (PSD) matrices is bounded.
I found a pretty general definition of a bounded set, i.e., that every element of that set has a norm bounded by a finite constant. In the case of PSD matrices, I think it would mean the following
$$\Big(\forall X \in A \subseteq \mathbb{S}^n_+ \Big) \Big (\|X\| := \sup_{\|h\|=1} \|Xh\| < \alpha < +\infty \Big)$$
However, I also found a definition that a set is bounded if it can be surrounded by an Euclidean ball of a finite radius. I guess we can think of that radius to be this $\alpha$ above and these definitions are equivalent. Please correct me if I am wrong. What is more, my problem is that this ball can be centered not necessarily at $0$ but at some other point (i.e. at some non-zero PSD matrix). How do I account for this translation in the radius then? How do I measure a distance between an origin (a zero matrix) and a non-zero PSD matrix at which a ball may be centered?
Can a condition for a bounded set be somehow stated in terms of an inner product of matrices? This kind of condition would seem most useful for me in practice but I could not find any such formulation. Thank you for your help.
The inner product of matrices is just the (restriction of) the standard inner product in $\mathbb{R}^{n(n+1)/2}$, so it defines a norm, and you can just as well use that norm to define bounded sets. This will be exactly the one you describe as "surrounded by Euclidean ball..". Since on a finite-dimensional vector space all norms are equivalent, it doesn't matter if you take this or the spectral norm - the bound $\alpha$ will only differ by a constant factor (depending on $n$).