Derivative of Hadamard Product Multiply by Summing Vector (each element is 1)

65 Views Asked by At

Given $\mathbf{A} \in \mathbb{R}^{k \times n}$, diagonal matrix $\mathbf{W} \in \mathbb{R}^{k \times k}$, $$ \mathbf{F}(\mathbf{X}) = \mathbf{W}(\mathbf{AX} \odot \mathbf{AX}) \mathbf{1}_{3} $$

where $\mathbf{X} \in \mathbb{R}^{n \times 3}$, $\mathbf{1}_{3}\in \mathbb{R}^{3 \times 1}$ is a vector with all the elements equal to 1, $\odot$ is Hadamard product (elements-wise product).

How to compute $\frac{\partial \mathbf{F}}{\partial \mathbf{X}}$? Sorry for my poor tensor understanding, matrix form would be better.

Many thanks!

1

There are 1 best solutions below

2
On

I think I have solved it.

$$ \begin{aligned} d \mathbf{F} &= d\left[\mathbf{W\left(AX \odot AX\right)\mathbf{1}_{3}} \right] \\ &= d \mathbf{W} \left[\left(\mathbf{AX \odot AX}\right)\mathbf{1}_{3}\right] + \mathbf{W} d \left[\left(\mathbf{AX \odot AX}\right)\mathbf{1}_{3}\right] \\ &= \mathbf{W} d \left[\left(\mathbf{AX \odot AX}\right)\mathbf{1}_{3}\right] \\ &= \mathbf{W} d \left(\mathbf{AX \odot AX}\right) \mathbf{1}_{3} + \mathbf{W} \left(\mathbf{AX \odot AX}\right) d \mathbf{1}_{3} \\ &= \mathbf{W} d \left(\mathbf{AX \odot AX}\right) \mathbf{1}_{3} \\ &= \mathbf{W} \left[2\mathbf{\left(AX\right)} \odot d \left(\mathbf{AX}\right)\right] \mathbf{1}_{3} \end{aligned} $$

$$ \begin{aligned} vec\left(d\mathbf{F}\right) &= vec \left(\mathbf{W} \left[2\mathbf{\left(AX\right)} \odot d \left(\mathbf{AX}\right)\right] \mathbf{1}_{3}\right) \\ &= \left(\mathbf{1}_{3}^{T} \otimes \mathbf{W}\right) vec\left[2\mathbf{\left(AX\right)} \odot d \left(\mathbf{AX}\right)\right] \\ &= \left(\mathbf{1}_{3}^{T} \otimes \mathbf{W}\right) \left [vec \left(2\mathbf{AX}\right) \odot vec \left(\mathbf{A} d \mathbf{X}\right) \right] \\ &= \left(\mathbf{1}_{3}^{T} \otimes \mathbf{W}\right) diag\left(2 \mathbf{vec \left(AX\right)}\right) vec \left(\mathbf{A} d \mathbf{X}\right) \\ &= \left(\mathbf{1}_{3}^{T} \otimes \mathbf{W}\right) diag\left(2 \mathbf{vec \left(AX\right)}\right) vec \left(\mathbf{A} \left(d \mathbf{X}\right) \mathbf{I}_{3} \right) \\ &= \left(\mathbf{1}_{3}^{T} \otimes \mathbf{W}\right) diag\left(2 \mathbf{vec \left(AX\right)}\right) \left(\mathbf{I}_{3}^{T} \otimes \mathbf{A}\right) vec \left(d \mathbf{X}\right) \end{aligned} $$

$$ \therefore \frac{\partial \mathbf{F}}{\partial \mathbf{X}} = \left[\left(\mathbf{1}_{3}^{T} \otimes \mathbf{W}\right) diag\left(2 \mathbf{vec \left(AX\right)}\right) \left(\mathbf{I}_{3}^{T} \otimes \mathbf{A}\right)\right] ^{T} $$

Is that right? Feel free to let me know if I made any mistake.