I have an application where I am using an unscented Kalman filter to process data.
While the unscented transformation eliminates the linearization assumption used with the typical state-transition matrix in the extended Kalman filter, there are still areas where the state-transition matrix is useful.
Is there a way to derive a state-transition matrix from a set of propagated sigma points from an unscented transformation?
May I ask why you want to obtain the state transition matrix? The unscented filter is used to bypass an inaccurate linearization assumption, which may not reflect what is actually going on and thus throws off linearization techniques like the EKF.
Anyway, one way I would do it is to use the posterior densities $P_{k}$ and $P_{k+1}$ (which you already have thanks to the unscented transform) to weight your sigma points $x_{k}$ and $x_{k+1}$. Next, let $A \in \mathbb{R}^{n \times n}$ be the matrix of unknown coefficients, and define $$ x_{k+1} = Ax_{k}. $$ A least squares fit should then give you $A$, the state transition matrix.