I have an optimization function in the following form:
$E = \operatorname*{argmin}_{A} \sum_j \| A\bf{x}_j - B \|_2^2 + \mu\sum_i a_{ii}^2$
Where, A is an unknown diagonal matrix with elements $a_{ii}$. $B$ and $\bf{x}$ are known.
My question is how can I differentiate this function to find $A$ ?
I am unclear about how to differentiate $E$ with respect to $A$ as $a_{ii}$ is also occurs in the function.
Thank you.
In this case, it's probably easier just to avoid Matrix notation and just consider it a function of $n$ variables. To simplify, let's write $a_i$ rather $a_{ii}$. Also, write $x_{ij}$ for the $i$th component of $x_j$. Then in summation notation, the objective is: $$ \begin{aligned} E &= \sum_j \sum_i (a_{i} x_{ij} - B_i)^2 + \mu \sum_i a_{i}^2\\ \end{aligned} $$ By rearranging the sum, each variable can be minimized independently: $$ \begin{aligned} E &= \sum_i \left(\mu a_i^2 + \sum_j (a_{i} x_{ij} - B_i)^2\right ) \end{aligned} $$ $$ \frac{\partial E}{\partial a_i} = 2 \sum_j (a_{i} x_{ij} - B_i)x_{ij} + 2 \mu a_{i} $$ This gives a minimizer of: $$ a_i^* = \frac{\sum_j B_i x_{ij}} {\mu + \sum_j x_{ij}^2} $$