I am trying to solve the following matrix equation
\begin{equation} \mathbf{XYX}' = \mathbf{Z} \end{equation}
where, $\mathbf{X}$ and $\mathbf{Y}$ are unknown and $\mathbf{Z}$ is known. Matrices $\mathbf{X}$ and $\mathbf{Y}$ have the following structure
$$\mathbf{X} = \left[\begin{array}{ccccc} \mathbf{P} & \mathbf{0} & \mathbf{0} & \cdots & \mathbf{0}\\ \mathbf{PQ} & \mathbf{P} & \mathbf{0} & \cdots & \mathbf{0}\\ \mathbf{PQ}^2 & \mathbf{PQ} & \mathbf{P} & \cdots & \mathbf{0}\\ \cdots & \cdots & \cdots & \cdots & \cdots\\ \mathbf{PQ}^{\alpha-1} & \mathbf{PQ}^{\alpha-2} & \mathbf{PQ}^{\alpha-3} & \cdots & \mathbf{P} \end{array}\right]$$
$$\mathbf{Y} = \left[\begin{array}{ccccc} \mathbf{RI}_n & \mathbf{0} & \mathbf{0} & \cdots & \mathbf{0}\\ \mathbf{0} & \mathbf{RI}_n & \mathbf{0} & \cdots & \mathbf{0}\\ \mathbf{0} & \mathbf{0} & \mathbf{RI}_n & \cdots & \mathbf{0}\\ \cdots & \cdots & \cdots & \cdots & \cdots\\ \mathbf{0} & \mathbf{0} & \mathbf{0} & \cdots & \mathbf{RI}_n \end{array}\right]$$
where, $\mathbf{P}\in\mathrm{R}^{m\times n}$, $\mathbf{Q}\in\mathrm{R}^{n\times n}$ and $\mathbf{R}\in\mathrm{R}^{n\times n}$. $\mathbf{I}_n$ is an identity matrix of size $n$. So given that the matrix $\mathbf{Z}$ and $m$, $\alpha$ are known, we have to solve the equation for the following two cases:
Case 1: Find $n$, $\mathbf{P}$, $\mathbf{Q}$, $\mathbf{R}$.
Case 2: Assume that $n$ is known. Find $\mathbf{P}$, $\mathbf{Q}$, $\mathbf{R}$.
In my opinion, we have to solve some least squares problem. But I have trouble formulating that problem. Another thing to notice is that $\mathbf{Z}$ is symmetric. So, we do not need to use all the entries in $\mathbf{Z}$ in formulating the least square problem. Another idea that comes to my mind is to substitute $\mathbf{T} = \mathbf{XY}^{1/2}$. Then $\mathbf{Z} = \mathbf{TT}^T$. If we do the Cholesky decomposition of $\mathbf{Z}$ as $\mathbf{Z} = \mathbf{AA}^T$ then $\mathbf{T}$ and $\mathbf{A}$ can be compared. But this comparison, again, requires some kind of least squares formulation with which I am having problem.
Any help is greatly appreciated.