Putting Gradient of a quadratic expression equal to zero and getting vector subtraction from a scalar

100 Views Asked by At

I have encountered this problem in my research and I want to take its derivative using closed form equation. But I get some scalar subtracting a vector in the end which I don't understand. Kindly help me with this. The problem is here: $$\alpha_px^2+\beta_px+v_p^Tx+\rho/2||D_px||^2$$ Here alpha, beta are scalars, v is a vector and D is a matrix. The last squared term is 2-norm. I need to find extremal values i.e. put gradient equal to zero and then solve the equation for x which is a vector too. I get $ x = \frac{-\beta_p-v_p^T}{2\alpha_p+\rho D_p^TD_p}$. Here $\beta_p$ is a constant(scalar) whereas $v_p^T$ is a row vector. How can they be subtracted? Am I doing something wrong? Would appreciate any help! Thanks

Here is the original problem if that is helpful: enter image description here Here $f_p(x_p) = \alpha_px_p^2+\beta_px_p$ which is a quadratic function and $v_p = \gamma_p^k - \rho \!\sum_{j \in \mathcal{N}_p}W_j x_j^k$.

As each node p belonging to $C_1$ can solve the problem separately and then add so there is no need of this summation $\sum_{p \in \mathcal{C}_1}$ for individual nodes.

That's how I am trying to solve it.

Thats how I am doing it

1

There are 1 best solutions below

7
On BEST ANSWER

I will try to put some of your information together, even though your question still does not fully add up.

So let us fix $\alpha\in\mathbb R$, $v\in\mathbb R^n$ and $D\in\mathbb R^{n\times n}$ and define the function $$ f:\mathbb R^n\to\mathbb R,\ x\mapsto \alpha \|x\|^2+\|Dx\|^2+v^Tx.$$ One can easily calculate (as you correctly did) $$ \nabla f = 2\alpha x + 2 D^TDx + v. $$ What you are looking for are solutions of the vector equation $$ -v = 2(\alpha I + D^TD )x, $$ where $I$ is the identity matrix. If the matrix $\alpha I + D^TD$ is invertible, you are easily done.

In your question you have another parameter $\beta\in\mathbb R$. However, $x\mapsto \beta x$ is a function $\mathbb R^n\to\mathbb R^n$, so your function is not well defined. If you instead mean something like $x\mapsto \beta \|x\|$, my answer can easily be adapted to that case.

Edit to include the above case: We want to differentiate a term of the type $\beta\|x\|$. The gradient of $\|x\|$ of course highly depends on the norm you use, I simplify by assuming we are using the Euclidean norm. There, you have the gradient vector $$ \nabla \|x\| = \frac{x}{\|x\|},$$ so adding $\beta\|x\|$ to the above function definition results in $$ \nabla(\alpha \|x\|^2+\beta\|x\|+\|Dx\|^2+v^Tx) = 2\alpha x + 2 D^TDx + v + \beta \frac{x}{\|x\|}.$$ Putting this equal to zero and solving this equation is considerably harder than its linear equivalent from above, which is due to the fact that the term $\beta\|x\|$ is a non-linear contribution to the value of your function.