Definition of Derivative of Matrix

Question

Let's assume A is $n\times 1$ constants, $X$ is $n\times 1$ vector. Does derivative of transpose(A)* X on X should be transpose(A) instead of A?

I saw both transpose(A) and A from different resources and would like to confirm the right answer.

Answer 1

For simplicity, let me write by $A^t$ the transpose of $A$.

I'm assuming that you're asking about the derivative of the function: $\mathbb{R}^n \to \mathbb{R} : X \mapsto A^t X$ ?

Short answer: the derivative is the same map: $(\mathbb{R}^n \to \mathbb{R} : X \mapsto A^t X)$, in other words, $A^t$

Slightly longer answer: for any linear map $B : \mathbb{R}^n \to \mathbb{R}^m$, its derivative is itself (ie, it's the same map $\mathbb{R}^n \to \mathbb{R}^m$). The reason for that is because the derivative $Df_x$ of a map $f$ at $x \in \mathbb{R}^n$ is defined as the (unique) linear map which gives a first-order Taylor approximation, ie, such that:

$f(x + \varepsilon) = f(x) + Df_x(\varepsilon) + O(\varepsilon^2)$ for any small $\varepsilon \in \mathbb{R}^n$.

When $f = B$ is linear, then we can choose $Df$ to be $B$ itself, and unicity guarantees that it's the only choice, so that the derivative of any linear map is itself.