I want to differentiate $f_{\theta}(\nabla_{\theta}(X))$ with respect to $\theta$, where $\theta$ is orthogonal $n \times p$ matrix (lies on Stiefel manifold) and $\nabla_{\theta}(X)$ is the derivative of $X$ with respect to $\theta$ and also is an $n \times p$ matrix and
$f_{\theta}(\nabla_{\theta}(X))= \nabla_{\theta}(X)-\theta\frac{\theta^{T}\nabla_{\theta}(X)-\nabla_{\theta}^{T}(X) \theta}{2}$
I am not so familiar with algebra, but I tried the below way:
using differential:
$d(f)=d(\nabla_{\theta} (X))- 0.5d(\theta\theta^{T}\nabla_{\theta} (X))-0.5d(\theta\nabla_{\theta}^{T}(X)\theta)$
as $\theta$ is orthogonal and $\theta\theta^{T}=I$, the second term will be $d((\nabla_{\theta} (X))$. Thus
$d(f)=0.5d(\nabla_{\theta} (X))-0.5d(\theta)\nabla_{\theta}^{T}(X)\theta-0.5\theta d(\nabla_{\theta}^{T}(X))\theta-0.5\theta\nabla_{\theta}^{T}(X)d(\theta)$
Since $d(\theta)=I$, then: $d(f)=0.5d(\nabla_{\theta} (X))-0.5\nabla_{\theta}^{T}(X)\theta-0.5\theta d(\nabla_{\theta}^{T}(X))\theta-0.5\theta\nabla_{\theta}^{T}(X)$
in this step, can I replace $d(\nabla_{\theta} (X))$ with $\nabla_{\theta}^2 (X)$?