Let's say we have a set of points $P$, and let the mean of those points be $\bar{p}$, which is also the point that minimizes the sum of squared distances to the points of $P$. Let $opt$ denote the sum of squares distances from $\bar{p}$ to $P$. We're given a point $q$ such that the sum of squared distances from $q$ to $P$ is at most $(1+\epsilon)opt$ for some small number $\epsilon$. How can we bound the squared distance from $\bar{p}$ to $q$?
What I've tried:
A simple but untight bound would be to use the weak triangle inequality for squared distance which states that for every $a,b,c$: $$d^2(a,c)\leq 2(d^2(a,b)+d^2(b,c))$$
To get $$d^2(q,\bar{p})\cdot|P|\leq 2(opt+(1+\epsilon)opt)=(4+2\epsilon)opt\implies\\d^2(q,\bar{p})\leq\frac{(4+2\epsilon)opt}{|P|}$$
We can also instead use the strong triangle inequality and square it to get a tighter version of the weak triangle inequality for square distances and then we can show the following tighter bound:
$$d^2(q,\bar{p})\leq\frac{(1+\epsilon+2\sqrt{1+\epsilon})opt}{|P|}\approx \frac{(2+2\epsilon)opt}{|P|}$$
The problem with both bounds is that they aren't tight enough. At least half of the points $p\in P$ satisfy $$d^2(p,\bar{p})\leq \frac{2\cdot opt}{|P|},$$ otherwise $opt$ would have been larger.
This is a better bound than the bounds I was able to prove for $q$ even if $\epsilon$ approaches 0, which shows how much those bounds are not tight. So I am trying to find a better bound than this.
Numerically, $opt$ is equivalent to the moment of inertia about an axis passing through $\bar{p}$ (i.e., $I_{\bar{p}}$), assuming each point in $P$ has mass $1$ and we embed the $P$ in a Euclidean space of dimension $\dim(P)+1$. This allows our axis of rotation to be perpendicular to the hyperplane that contains $P$.
For example, in two dimensions ($x,y$ axes), the sum of squared distances from a point $p$ is equivalent to the moment of inertia about $p$ if we take the axis of rotation to be parallel to the $z$ axis and passing through $p$
Also, since $\bar{p}$ is the minimizer of the moment of inertia, it also makes $\bar{p}$ the "center of mass" of this group of points.
Therefore, if we define $\bar{p}$ as the origin of our expanded (Cartesian) coordinate system, then we can use the Parallel Axis Theorem:
$$I' = I_{cm} + md^2$$
With $m=|P|$ and $I_{\bar{p}} = I_{cm}$ and $d$ being the parallel displacement of the new "axis of rotation" relative to the axis going through center of mass we can your condition on the new point $q$ resulting in sum of squares $opt*(1-\epsilon)$ in terms of displacement of the parallel axis:
$$I_q = I_{\bar{p}} + |P|d^2 = (1+\epsilon)I_{\bar{p}}$$
Doing the algebra, we get:
$$I_{\bar{p}} + |P|d^2 = (1+\epsilon)I_{\bar{p}} \implies |P|d^2 = (1+\epsilon)I_{\bar{p}} - I_{\bar{p}} = \epsilon I_{\bar{p}} \implies d = \sqrt{\frac{\epsilon I_{\bar{p}}}{|P|}}$$
So the new point is at a distance $\sqrt{\frac{\epsilon \cdot opt}{|P|}}$ from $\bar{p}$.