I would like to differentiate the following equation wrt. matrix $H_{kj}$ equal to $0$:
\begin{align} D(W, H) = \sum_{ij}[\sum_{k}\log \frac{W_{ik}H_{kj}}{\alpha} + \sum_{k}W_{ik}H_{kj}] + \sum_{kj}[\frac{\beta}{2} \vert H_{kj} \vert^{2}] \end{align}
where $W_{ik}$ and $H_{kj}$ are positive matrix with dimension $i*k$ and $k*j$, respectively. $\alpha$ and $\beta$ are constant.
Since $H_{kj}$ is positive, $\vert H_{kj}\vert = H_{kj}$:
\begin{align} 0 &= \frac{\partial}{\partial H_{kj}}[\sum_{ij}[\sum_{k}\log \frac{W_{ik}H_{kj}}{\alpha} + \sum_{k}W_{ik}H_{kj}] + \sum_{kj}[\frac{\beta}{2} H_{kj} ^{2}]]\\ &= \sum_{ij}[\sum_{k}\frac{\alpha}{W_{ik}H_{kj}} . \frac{W_{ik}}{\alpha} + \sum_{k}W_{ik}] + \sum_{kj}\beta H_{kj}\\ &= \sum_{kj}\frac{1}{H_{kj}} + \sum_{ik}W_{ik} + \sum_{kj}\beta H_{kj}\\ \end{align}
For the fixed $k, j$: \begin{align} 0 &= \frac{1}{H_{kj}} + \sum_{i}W_{ik} + \beta H_{kj}\\ \end{align}
Multiply both sides of equation with $\frac{H_{kj}}{\beta}$:
\begin{align} 0 &= (\frac{1}{H_{kj}}.\frac{H_{kj}}{\beta}) + ( \sum_{i}W_{ik}.\frac{H_{kj}}{\beta}) + (\beta H_{kj}.\frac{H_{kj}}{\beta})\\ 0&= \frac{1}{\beta} + \frac{ (\sum_{i}W_{ik})H_{kj}}{\beta} + H^{2}_{kj}\\ 0 &= (H_{kj} + \frac{ \sum_{i}W_{ik}}{2\beta})^{2} - \frac{ (\sum_{i}W_{ik})^{2}}{4\beta^{2}} + \frac{1}{\beta}\\ \end{align} Rearrange: \begin{align} (H_{kj} + \frac{ \sum_{i}W_{ik}}{2\beta})^{2}&= \frac{ (\sum_{i}W_{ik})^{2}}{4\beta^{2}} - \frac{1}{\beta}\\ \end{align} Take square root both sides: \begin{align} H_{kj} + \frac{ \sum_{i}W_{ik}}{2\beta} &= \sqrt {\frac{(\sum_{i}W_{ik})^{2}}{4\beta^{2}} - \frac{1}{\beta}}\\ H_{kj} &= \sqrt {\frac{(\sum_{i}W_{ik})^{2}}{4\beta^{2}} - \frac{1}{\beta}} - \frac{\sum_{i}W_{ik}}{2\beta}\\ \end{align}
I wonder if this is correct? Since the equation has matrices, I am not sure for multiplying/dividing them together.
The function can be written in matrix notation as $$\eqalign{ \phi &= {\mathbf 1}:\Big(WH+\log(WH)\Big) + \tfrac{\beta}{2}H:H \cr }$$ where a colon denotes the trace/Frobenius product, i.e. $\,A\!:\!B={\rm tr}(A^TB),\,$ the matrix ${\mathbf 1}$ has the same shape as $WH$, and the $\log()$ function is applied element-wise.
The differential and gradient of this function is $$\eqalign{ d\phi &= {\mathbf 1}:\Big(W\,dH+\frac{W\,dH}{WH}\Big) + \beta H:dH \cr &= \Big(W^T\Big(\frac{{\mathbf 1}+WH}{WH}\Big) + \beta H\Big):dH \cr \frac{\partial\phi}{\partial H} &= W^T\Big(\frac{{\mathbf 1}+WH}{WH}\Big) + \beta H \cr }$$ Now set the gradient to zero and solve for $H$ iteratively $$\eqalign{ H_+ &= -\beta^{-1}W^T\Big(\frac{{\mathbf 1}+WH}{WH}\Big) \cr }$$ where the notation $\big(\frac{1}{WH}\big)$ represents element-wise division.