Suppose $(M,g)$ and $(N,h)$ are Riemannian manifolds, and suppose $f:M\to N$ is smooth. Then for $p\in M$, we have the differential $df_p:T_pM\to T_{f(p)}N$ as a linear operator.
I'm curious if it makes sense to define a "reverse differential" $rf_p:T_{f(p)}N\to T_pM$
such that for any $\xi\in T_pM$ and any $\zeta\in T_{f(p)}N$ the following equality holds
$$g(rf_p(\zeta),\xi)=h(\zeta,df_p(\xi)).$$
This seems to me to be one natural way of extending the gradient. Indeed, if $(N,h)=(\mathbb{R},\delta)$, then we can define the gradient of $f$ at $p$, $\text{grad}(f)_p$ via $$g(\text{grad}(f)_pp,\xi)=df_p(\xi),$$ for any $\xi\in T_pM$. If we let $\left.\tau\sim\tau\frac{d}{dt}\right|_{f(p)}=:v$ be the usual identification of $\mathbb{R}$ and $T_{f(p)}\mathbb{R}$, then we see that the above definition for the reverse differential results in \begin{align} g(rf_p(v),\xi)&=\delta(v,df_p(\xi))\\ &=\tau df_p(\xi)\\ &=\tau g(\text{grad}(f)_p,\xi)\\ &=g(\tau\text{grad}(f)_p,\xi), \end{align} and hence that $$\text{grad}(f)_p = rf_p\left(\left.\frac{d}{dt}\right|_{f(p)}\right).$$
This motivation stems from a reading through Reverse differential categories, and I'm curious if this type of derivative works on arbitrary Riemannian manifolds. The authors use the category of smooth functions on Euclidean space as an example, which leads me to believe that any finite-dimensional, inner product space could handle this definition. This in turn has to led me to the (semi-?)Riemannian setting. However, I feel that this would be found somewhere in the literature already, and I haven't been able to find any such sources dealing with this "reverse differential" in this setting.
Sounds like the adjoint of a linear map between different inner product spaces. Let $(V_i,g_i)$ be inner-product spaces for $i=1,2$, and let $T:V_1\to V_2$ be a linear map. We can always consider the dual map $T':V_2^*\to V_1^*$. But now since we have inner products, we can consider the musical isomorphisms $V_i\cong V_i^*$ induced by the inner products (i.e $g_i^{\flat}:V_i\to V_i^*$, $g_i^{\flat}(x):=g_i(x,\cdot)$, and the inverse is denoted $g_i^{\sharp}:V_i^*\to V_i$). Now, we can define the adjoint of $T$ (with respect to $g_1,g_2$) to be the linear map $T^*:V_2\to V_1$, \begin{align} T^*:=g_1^{\sharp}\circ T'\circ g_2^{\flat}. \end{align} (i.e certain diagram commutes). Conceptually this should make sense: you're given a map $T:V_1\to V_2$, which always induces a mapping between dual spaces in the opposite direction $T':V_2^*\to V_1^*$ (because of how composition works). Using the inner products, we can then force this into a map between the vector spaces themselves rather than their duals.
Then, by unwinding the definitions, you can show that for any $v_1\in V_1, v_2\in V_2$, we have \begin{align} g_1(T^*(v_2),v_1)&=g_2(v_2,T(v_1)). \end{align} Notice that positive-definiteness of inner products wasn't used, only non-degeneracy so this construction extends to the indefinite case as well. You now just have to apply this idea to each tangent space.
The standard use of the adjoint of a linear map consists of using a single inner product space so for $T:V\to V$, we get a map $T^*:V\to V$ such that for all $x,y\in V$, $\langle T(x),y\rangle=\langle x, T^*(y)\rangle$.