My original question is Assumption on Mirror Descent convergence? but I realized that it boils down into the following question:
Suppose $\sum a_n =\infty$ as $n \rightarrow \infty$ where $(a_n)$ is positive and $a_n \rightarrow 0$. Also, $(b_n)$ is non-negative and bounded.
What can we say about convergence of $ \frac{\sum_{s=1}^k a_s^2b_s}{\sum_{s=1}^k a_s} $?