In A First Course in Optimization Theory by Sundaram I read on page 51:
Given a quadratic form $A$ and any $t\in\mathbb R$, we have $(tx)'A(tx)=t^2x'Ax$, so the quadratic form has the same sign along lines through the origin. Thus, in particular, $A$ is positive definite (resp. negative definite) if and only if it satisfies $x'Ax>0$ (resp. $x'Ax<0$) for all $x$ in the unit sphere $\mathcal C=\{u\in\mathbb R^n\mid\lVert u\rVert=1\}$ We will use this observation to show that if $A$ is a positive definite (or negative definite) $n\times n$ matrix, so is any other quadratic form $B$ which is sufficiently close to $A$:
I don't understand the the part where it says "for all $x$ in the unit sphere $\mathcal C=\{u\in\mathbb R^n\mid\lVert u\rVert=1\}$".
Shouldn't the $u$ in the set $\mathcal C$ be in $\mathbb{R}^3$ since $\mathcal C$ is supposed to describe a unit sphere?
Would be great if there is a diagram that depicts the thing being described in this paragraph.
“Sphere” is a concept which applies to any dimension. It is the set of points which have the same distance (radius) from a given point (center). For the unit sphere, the radius is $1$ and the center is the origin. So the set of all vectors with unit norm (i.e. length one) forms the unit sphere, in any dimension.
If people wish to distinguish spheres of different dimensions, they often include the dimension of the surface in the name. This is one less than the dimension of the containing space. So your common sphere in 3D would be a two-sphere. A circle in the plane would be a one-sphere, and a pair of points on a line would be a zero-sphere. Visualizing objects of higher dimension, like the three-sphere, is inherently tricky. So a diagram would likely have been of little use: either it would have depicted a low-dimensional special case, or it would have been very hard to understand.