Derivative of multiplied vectors and matrix with respect to single variable in vector

Question

Let v be a vector $v = (a + bc_1, ..., a + bc_n)$. Thus, we can say $v = a1 + bc$, where $1 = (1,..., 1)$ and $c = (c_1,...c_n)$.

Let $M$ be a symmetric and quadratic matrix of proportions $n * n$, which elements we note $m_{ij}$.

What is the derivative with respect to $a$ of $v^TM v$? What is the derivative with respect to $b$ of $v^TM v$?

Context for the interested

I have tried to optimize the following problem in finance (specifically modern portfolio theory)

min $w^TΣw$ s.t. $w^T1 = 1$, $w^Tμ = μ_*$, $w^Tθ \geq θ_*$, where θ is ESG-score.

The problem is the basic mean-variance optimization with an added θ-inequality. From this problem I reduced it to a simpler form using KKT-conditions, and now use Lagrange multipliers (a and b in the problem above are the multipliers) to finally optimize the problem.

Answer 1

As far as I understand, your $b$ is a real parameter (or complex parameter), but in any case you are looking at a vector function of one variable. You can simply expand $v^T M v$ to get a quadratic in $b$ and differentiate in a straightforward manner this quadratic: $$\begin{align*} \dfrac{d}{db}v^T M v &= \dfrac{d}{db}\left( \alpha^T M \alpha + b(\alpha^T M c + c^T M \alpha) + b^2(c^T M c) \right) \\ &= \alpha^T M c + c^T M \alpha + 2b c^T M c \end{align*} $$ where $\alpha = a \cdot 1$.

Alternatively, as hinted to in the comments: define $B: \Bbb R^n \times \Bbb R^n \to \Bbb R$ by $B(u,v) = u^T M v$.

This $B$ is a bilinear form, and therefore is of class $C^{\infty}$, with derivative $DB(u,v)(h,k) = B(u,k) + B(h,v)$, i.e. $$DB(u,v)(h,k) = u^T M k + h^T M v$$ Next, apply the chain rule: $$\dfrac{d}{db}(v^T M v) = D[B \circ (I,I) \circ (f)](b)$$ where $I$ is the identity map and $f$ is $b \mapsto \alpha + bc$. Then, $$\begin{align*} \dfrac{d}{db}(v^T M v) &= DB(f(b),f(b))(f'(b),f'(b)) \\ &= DB(\alpha + bc, \alpha + bc)(c,c) \\ &= (\alpha^T + bc^T)Mc + c^T M(\alpha + bc) \\ &= \alpha^T Mc + bc^T Mc + c^t M \alpha + b c^TMc \\ &= \alpha^T M c + 2 b c^T M c + c^T M \alpha \end{align*}$$ Of course this second way of doing it is an overkill, but it helps to think this way when the situation isn't as straightforward.

Added: I now noticed that you asked for both the $a$ and $b$ derivatives. To get the $a$ derivative, follow the exact same procedure(s) outlined here.

Answer 2

Let's use Greek letters for the scalar variables, lowercase Latin for vectors, and uppercase Latin for matrices.

First, calculate the differential of $v$ $$\eqalign{ \def\a{\alpha} \def\b{\beta} \def\c{c} \def\O{{\large\cal\Omega}} \def\o{{\tt1}} v &= \o\a + \c\b \\ dv &= \o\,d\a + c\:d\b \\ }$$ Then the differential of the quadratic form $$\eqalign{ \O &= v^TMv \\ d\O &= v^TM\,dv + dv^TMv \\ &= 2v^TM\:dv \\ &= 2v^TM\,(\o\,d\a + c\:d\b) \\ &= 2v^TM\o\,d\a \;+\; 2v^TMc\,d\b \\ }$$ From this, one can identify the desired derivatives $$\eqalign{ \frac{d\O}{d\a} &= 2v^TM\o, \qquad \frac{d\O}{d\b} = 2v^TMc \\ }$$