Let $A$ be a $m\times n$ matrix with full rank (and $m\geq n$, so that it has no null space). I understand that if $S$ is a positive definite symmetric matrix, then $A^TSA$ also has full rank (since $\langle A^TSAx,x\rangle = \langle S Ax,Ax\rangle > 0$ when $Ax\not=0$).
I am asking about the case that $S$ is not positive definite, but it is symmetric and invertible. How big can the null space of $A^TSA$ be? Is it possible that $A^TSA$ has a two dimensional nullspace? A two hundred dimensional nullspace? It seems to me that there should exist such examples, but I am unable to find them.
I'll give the answer in terms of rank, with scalars in $\mathbb R$. It's worth pointing out that if we had special structure, e.g. commutativity where $AA^T S = SAA^T$ then $\text{rank}\big(A^TSA\big) = n$. Outside of special structure, we can only bound the rank.
applying Frobenius Rank Inequality
$n + (n-m) =\text{rank}\Big(A^T S\Big) + \text{rank}\Big(SA\Big)- \text{rank}\Big(S\Big)\leq \text{rank}\Big(A^T SA\Big)$
so there is a lower bound on rank of $\max\big(0,2n -m\big)$.
Unfortunately one cannot improve on this inequality in general. E.g. consider bipartite
$S := \begin{bmatrix}\mathbf 0 &B \\ B^T &\mathbf 0\end{bmatrix}$ with invertible $B$ and suppose $A:=\begin{bmatrix}\mathbf I_n \\ \mathbf 0\end{bmatrix}$. Notice $2n=m$. Then
$\mathbf 0 = A^TBA$ and
$0 = n + (n-2n) = n + (n-m) \leq \text{rank}\Big(A^T SA\Big)$
which is to say that Frobenius's Rank Inequality is met with equality.