Jacobian transpose times a vector

Question

It is possible to evaluate Jacobian times a vector:

$$ J v = \lim_{\varepsilon \to 0} \frac{ f(x + \epsilon v) - f(x) } {\epsilon }$$

Is it possible to calculate $ J^T v$ and $ J^T J v$ in similar fashion?

Answer 1

There is no similar formula for $J^T v$. For example consider the case when $f\colon\mathbb R^n \to \mathbb R$, then $J$ is $1\times n$ and so $J^T v$ is just a scalar multiple of $J^T$.

This is the reason why for example transpose free iterative methods are so important for things like large scale Newton methods, cf. chapter 7.4 in Iterative Methods for Sparse Linear Systems by Yousef Saad

Answer 2

What you can say is that $J(x)^T v$ is the gradient of $\phi(x) := f(x)^T v$. Therefore, $$ (J(x)^T v)_i = \lim_{h \to 0} \frac{(f(x + h e_i) - f(x))^T v}{h}, $$ where $e_i$ is the $i$-th column of the identity matrix.