What is the derivative of a vector with respect to its transpose?

Question

I've already looked at Vector derivative w.r.t its transpose $\frac{d(Ax)}{d(x^T)}$, but I wasn't able to find the direct answer to my question in that question. What is the value of $$\frac{d}{dx} x^T\text{ ?}$$ My initial intuition is that it is $1$, but I'm not exactly sure of why that would be so.

Answer 1

What sort of object can be the derivative of a vector-valued function whose values are row vectors and whose arguments are column vectors? Generally, what kind of object can be the derivative of a function whose values are members of one vector space $W$ and whose arguments are members of another vector space $V$?

$$ f: V\to W $$

The answer is that the value of such a derivative at any point in $V$ is a linear transformation from $V$ into $W$, and it may be a different linear transformation at each point in $V$. But if $f$ is itself linear, then it's the same linear transformation at each point in $V$: it's $f$ itself.

Transposition is linear. Therefore the value of its derivative at each point in its domain is itself.

Often one represents a linear transformation by a matrix. What would be the matrix in this case? No matter what basis you pick for the domain $V$, it seems natural to pick as a basis of $W$ the set of transposes of the basis vectors you chose for $V$. In that case, the matrix would be the identity matrix.

Answer 2

That depends on how you define vector derivative. There are generally two ways. One is applying abstract index notation, then $$\frac{d}{dx}x^T=\left(\frac{dx_i}{dx^j}\right)=(\delta_{ij})=(e_1\otimes\cdots\otimes e_n)^T$$ where $e_i$s are unit vector whose $i$ th component is one and zero otherwise.

Another way to look at it is to regard as directional derivative, then $$\frac{d}{dx}x^T=\lim_{h\to0}\frac{(x+hx)^T-x^T}{h}=x^T$$