Let $Q(x)$ be a smooth symetric matrix with constant signature $(p,q,k)$ where $x$ belong in $\mathbb{R}^n$ and $p+q+k=m$.
Question: Locally around $x_0$, does an invertible matrix $P(x)$ of size $m$ exists such that,
$$ P^T(x) Q(x) P(x) = \begin{pmatrix} Id_p &0&0\\0&-Id_q&0\\ 0&0&0_k\end{pmatrix} $$
/!\ I do not require that $P(x)$ is the jacobian of some diffeormorphism.
Proposition of Proof:
- If $Q(x_0)$ is invertible and all its eignevalue are simple, then this property is locally preserved. Hence the basis of orthonormal vectors depend smoothly on the point.
- If $Q(x_0)$ has a multiple eigenvalue, then locally the eigen hypersurface can split into multiple eigen hypersurfacesof smaller dimension. However because the bilinear form asociated with $Q$ is symmetric the vectors spanning the multiple smaller hypersurfaces will converge to a basis of the eigenhypersurface. Hence the transformation is still smooth.
- If $det(Q(x_0))=0$, then locally we have $\mathbb{R}^n=\ker Q \oplus^{\perp}rg(Q)$. The kernel of $Q$ is determined by a set of equations, by the implicit functions theorem we can express the vector spanning this kernel by smooth functions. As the image of $Q$ is orthogonal to the kernel, then it is also spanned by smooth functions and we can express $Q$ in this subspace and repeat the previous argument.
A necessary condition for local-existence of a $C^{\infty}$ function $P(x)$ is that $k$ is locally constant. -indeed, by continuity, $p,q$ are locally constant-
Conversely, if $k$ is locally constant, it suffices that the eigenvalues and a basis of (unit length) eigenvectors of $Q(x)$ are globally $C^{\infty}$ parametrizable.
When $Q$ is only smooth, that is -in general- false. Yet, it's true when $Q$ is analytic; in this case, $P$ is an analytic function.
cf. my post in
Do eigenvalues depend smoothly on the matrix elements of a diagonalizable matrix?
EDIT. To the OP. I see that you didn't understand my post. More precisely, if we do the following reasoning: If $Q(x)$ is ANALYTIC, then
i) There is an analytic parameterization of the eigen-elements of $Q(x)$: $Q(x)=U^T(x)diag(\lambda_i(x))U(x)$ where the $U(x)$ are orthogonal matrices.
ii) Let $P(x)=U(x)diag(\mu_i(x))$, where $\mu_i(x)=|\lambda_i(x)|^{-1/2}$ if $\lambda_i(x)\not= 0$ and $=1$ otherwise. Then $P(x)$ is ANALYTIC and answers the question.