How to introduce weights for Jensen's inequality into $log\sum\sum$

127 Views Asked by At

For log sum matrix function, I saw the example introducing the weight after sum regarding all matrices in summation for Jensen's inequality

\begin{align*} f(W,H) &= \sum_{ij}[-V_{ij}log \sum_{k}W_{ik}H_{kj}]\\ &= \sum_{ij}[-V_{ij}log \sum_{k}\color{red}{\pi_{ijk}}\frac{W_{ik}H_{kj}}{\color{red}{\pi_{ijk}}} ]\\ &\leq \sum_{ij}[-V_{ij}\sum_{k}\color{red}{\pi_{ijk}}log \frac{W_{ik}H_{kj}}{\color{red}{\pi_{ijk}}}] \end{align*}

where $V,W,H$ are matrix, and $\pi_{ijk} = \frac{W_{ik}H_{kj}}{\sum_{k}W_{ik}H_{kj}}$ and $\sum_{k}\pi_{ijk} = 1$

But if I have two sums after $-log$. For example

$$f(W,H,\sigma,\phi) = \sum_{ij}[-V_{ij}log \color{blue}{\sum_{l}}\sigma_{l}\color{blue}{\sum_{k}}\phi_{lk}W_{lik}H_{lkj}]$$

where $\sigma$ is vector and $\phi$ is matrix

I wonder where I should introduce the weight $\color{red}\pi$? Which dimensions do I have to concern about? Is it ok to concern only $l$ dimension like this following?; $$f(W,H,\sigma,\phi) = \sum_{ij}[-V_{ij}log \sum_{l}\color{red}{\pi_{l}}\frac{\sigma_{l}}{\color{red}{\pi_{l}}}\sum_{k}\phi_{lk}W_{lik}H_{lkj} ]$$

where $\pi_{l} = \frac{\sigma_{l}}{\sum_{l}\sigma_{l}}$

or all the dimensions after sum should have individual weight;

$$f(W,H,\sigma,\phi) = \sum_{ij}[-V_{ij}log \sum_{l}\sigma_{l}\sum_{k}\color{red}{\pi_{lijk}}\frac{\phi_{lk}W_{lik}H_{lkj}}{\color{red}{\pi_{lijk}}}]$$

where $\pi_{lijk} = \frac{\sigma_{l}\phi_{lk}W_{ik}H_{kj}}{\sum_{l}\sigma_{l}\sum_{k}\phi_{lk}W_{ik}H_{kj}}$