I am studying the paper called Algorithms for Non-negative Matrix Factorization which introduced the Multiplicative update rules. In lemma 2 on page 5, it is introducing an auxiliary function $G(h, h^t)$. I have a few questions regarding the function and its proof.
(1) First, what is $\delta_{ab}$, is that a element-wise operation or just a constant?
(2) In the proof of lemma 2, how line 19 transformed to line 20. Specifically, I tried to substitute $M_{ab}(h^t)$ in 19 but I could only match the second part of line 20 (which is $v_ah^t_a(W^tW)_{ab}h^t_bv_b$. I could not understand where the first part,$h^t_a(W^tW)_{ab}h^t_bv_a^2$ , coming from?
(3) after that how it is transformed from line 20 to line 21.
I would appreciate any helps on this.
Thanks in advance.
$\delta_{ab}$ is the Kronecker delta symbol which is one when $a=b$ and zero otherwise. (Thus $K$ is a diagonal matrix, like they say).
The first part comes from plugging in the form for $K$ and doing the sum over one of the indices giving $$ \sum_{ab}v_a h_a h_b\delta_{ab}(W^TWh)_a/h_a = \sum_a v_a^2 h_a (W^TWh)_a$$ Then you can expand out the matrix multiplication $(W^TWh)_a$ as a sum $\sum_b(W^TW)_{ab}h_b$ to get $$ \sum_{ab}v_a^2 h_a(W^TW)_{ab} h_b$$
The second term that you calculated is represented by the $-v_av_b$ term in parentheses. The first term, as we expressed it above, would correspond to a $v_a^2.$ The reason we can replace this with $\frac{1}{2}(v_a^2+v_b^2)$ is because $h_a (W^T W)_{ab}h_b$ is symmeric under interchange of $a$ and $b.$ So we can think of it as just writing $v_a^2 = \frac{1}{2}(v_a^2+v_a^2)$ and then relabeling, switching $a$ and $b$ in one of the terms.