I have a matrix equation of the form:
$$AXA'=0$$
Where $A$ is of dimension $(p \times n)$ and $X$ is of dimension $(n \times n)$ with $n>p$. I know $A$, I do not know $X$. Can there be any non-trivial solutions to this equation for $X$? And if so, when do they exist and how can I find them?
Edit: There are definitely situations in which non-trivial solutions exist. For example, if:
$$A=\begin{bmatrix}1 & 1\end{bmatrix}$$
A possible solution for $X$ exists as:
$$\begin{bmatrix}1 & 0 \\ 0 & -1\end{bmatrix}$$
Thoughts on the problem:
I will stick to the case where $X$ is symmetric.
First, the case where $A$ is of dimension $n \times n$, and $A$ is invertible. In this case, we have Sylvester's law of inertia, which states that $AXA'$ will necessarily have the same "inertia" as $X$. For instance, if $X$ is positive or negative semidefinite, then $AXA'$ will also be positive or negative semidefinite of the same rank.
With that, break this square matrix $A$ into blocks $$ A = \pmatrix{A_1\\A_2} $$ where $A_1$ is $p \times n$. We then find that we can write $$ AXA' = \pmatrix{A_1XA_1' & A_1 XA_2'\\ A_2XA_1' & A_2XA_2'}. $$ With that, we can now address the case where $A$ is $p \times n$ of full row-rank: we see that if $A_1$ is such a matrix, then it must be that $A_1XA_1'$ is a $p \times p$ submatrix of some matrix whose signature is equal to that of $X$.
With that, the question can be reframed as follows: if $X$ is an $n \times n$ matrix of the form $$ X = \pmatrix{0 & B\\B^T & C}, $$ then what can its signature be? If $C$ is invertible, then with the Schur complement we find that $X$ is congruent to the matrix $$ \pmatrix{-BC^{-1}B^T & 0\\0 & C}. $$ In other words, if $X$ is invertible, then $X$ can have positive signature at most $n-p$.
For the case where $A$ does not have full row-rank, we can use a rank-factorization $A = FG$ to reduce to the case where $A$ has fewer rows but full rank (i.e. we can replace $A$ with $G$).