In the Equivariant Adaptive Source Separation paper I am wondering what the intermediate steps for eq. 23 on page 5 are? I am not sure where to even begin with this and haven't been able to find any information online.
Also, this paper introduced relative gradient to me (and I think introduced it in general) which seems like a really profound thing, but the author does not dwell on it at all which leaves me wondering. Is there anything that goes more into depth on it?
In the paper, we have the following: $$ \delta R_z = \mathcal{E}R_z+R_z\mathcal{E}^T \tag{21} $$ and $$ K(R_z+\delta R_z) = K(R_z) + \mathrm{Trace}\{(I-R_z^{-1})\delta R_z\}+o(\delta R_z) \tag{22} $$
If you replace (21) in (22), and use the fact that $R_z = R_z^T$ (symmetry), you get for what's inside the trace: $$ (I-R_z^{-1})\delta R_z = (I-R_z^{-1})(\mathcal{E}R_z+R_z\mathcal{E}^T) = (I-R_z^{-1})\mathcal{E}R_z+ (I-R_z^{-1}) R_z^T\mathcal{E}^T) = (I-R_z^{-1})\mathcal{E}R_z+ (I-R_z^{-1})(\mathcal{E}R_z)^T. $$ Using the trace properties, and $(I-R_z^{-1}) = (I-R_z^{-1})^T$ (thanks again to symmetry of $R_z$) you get \begin{equation} \mathrm{Trace}\{(I-R_z^{-1})\mathcal{E}R_z+ (I-R_z^{-1})(\mathcal{E}R_z)^T\} = \mathrm{Trace}\{(I-R_z^{-1})\mathcal{E}R_z\}+\mathrm{Trace}\{(I-R_z^{-1})^T(\mathcal{E}R_z)^T\} = \mathrm{Trace}\{(I-R_z^{-1})\mathcal{E}R_z\}+\mathrm{Trace}\{(\mathcal{E}R_z)^T(I-R_z^{-1})^T\} = \mathrm{Trace}\{(I-R_z^{-1})\mathcal{E}R_z\}+\mathrm{Trace}\{((I-R_z^{-1})(\mathcal{E}R_z))^T\} = 2\mathrm{Trace}\{(I-R_z^{-1})\mathcal{E}R_z\} = \mathrm{Trace}\{2R_z(I-R_z^{-1})\mathcal{E}\} = \mathrm{Trace}\{2(R_z-I)\mathcal{E}\} = \mathrm{Trace}\{2(R_z-I)^T\mathcal{E}\} =<2(R_z-I)|\mathcal{E}> \end{equation}
Identifying this with Eq. (14), we find $$ \nabla\phi = 2(R_z-I) \tag{23} $$