Special case: Derivative of a vector with respect to another vector

98 Views Asked by At

I have that

$$ x = [a \ \ B]^{-1}c $$

where $x$, $a$, and $c$ are all $p \times 1$, and $B$ is $p \times (p-1)$, so that the invertible matrix $[a \ \ B]$ is $p \times p$.

I would like to compute the partial derivatives of the elements of $x$ with respect to the elements of $a$, which I believe can be expressed in a $p \times p$ matrix.

I have been working through the matrix cookbook. So far I have that the matrix of partial derivatives is something like

$$ [a \ \ B]^{-1} diag(c) [a \ \ B]^{-1} $$

where $diag(c)$ is the diagonal matrix whose main diagonal is $c$.

This matches my expectations in special cases, such as where $p=1$ and where $ [a \ \ B]$ is diagonal. But I am not sure it is right in general and I could use some help finding my way.

Thanks in advance.

1

There are 1 best solutions below

3
On BEST ANSWER

Define the matrix $$G = [\,a\,\,B\,]$$ and rewrite the equation to clear the inverse, then find the differential, and then the gradient. $$\eqalign{ c &= Gx \cr 0 &= dG\,x + G\,dx \cr dx &= -G^{-1}\,dG\,x \cr &= -G^{-1}\,[\,da\,\,0\,]\,x \cr &= -G^{-1}\,(x_1)\,da \cr &= -(x_1)G^{-1}\,da \cr \frac{\partial x}{\partial a} &= -(x_1)G^{-1} &\implies \frac{\partial x_i}{\partial a_j} = -(x_1)\,e_i^TG^{-1}e_j \cr }$$ This can be extended to columns beyond the first.
Denoting the $k^{th}$ column of $G$ by $G_k$, an analogous derivation yields $$\eqalign{ \frac{\partial x}{\partial G_k} = -(x_k)G^{-1} \,\,\,\implies \frac{\partial x_i}{\partial G_{jk}} = -(x_k)\,e_i^TG^{-1}e_j }$$