I'm stuck on part of a proof for Theorem 2.1 in Prediction, Learning, and Games by Cesa-Bianchi.
Consider $$ \Phi(\mathbf{u}) = \psi \left( \sum_{i=1}^{N} \phi(u_i) \right), $$
and $\xi$ as some vector in $\mathbf{R}^N$. A section of the proof gives an equality that I am not quite able to follow:
\begin{align*} \sum_{i=1}^{N} \sum_{j=1}^{N} \frac{\partial^2 \Phi}{\partial u_i \partial u_j} \Bigg|_{\xi}^{} r_{i,t}r_{j,t} = \psi''&\left( \sum_{i=1}^{N} \phi(\xi_i) \right) \sum_{i=1}^{N} \sum_{j=1}^{N} \phi'(\xi_i)\phi'(\xi_j)r_{i,t}r_{j,t} \\ &+ \psi'\left( \sum_{i=1}^{N} \phi(\xi_i) \right)\sum_{i=1}^{N}\phi''(\xi_i)r_{i,t}^2. \end{align*}
My attempt so far has been to unpack the LHS, and this is where I end up:
\begin{align*} \sum_{i=1}^{N}& \sum_{j=1}^{N} \frac{\partial^2 \Phi}{\partial u_i \partial u_j} \Bigg|_{\xi}^{} r_{i,t}r_{j,t} \\ &= \sum_{i=1}^{N} \sum_{j=1}^{N} \frac{\partial}{\partial u_i}\left( \frac{\partial \Phi}{\partial u_j} \right) \Bigg|_{\xi}^{} r_{i,t}r_{j,t} \\ &= \sum_{i=1}^{N} \sum_{j=1}^{N} \frac{\partial}{\partial u_i}\left( \frac{\partial \Phi}{\partial \psi} \frac{\partial \psi}{\partial u_j} \right) \Bigg|_{\xi}^{} r_{i,t}r_{j,t} \\ &= \sum_{i=1}^{N} \sum_{j=1}^{N} \frac{\partial}{\partial u_i} \left( \psi'\left( \sum_{i=1}^{N} \phi(u_i) \right)\phi'(u_j) \right) \Bigg|_{\xi}^{} r_{i,t}r_{j,t}. \\ \end{align*}
However, from here I am not sure how to continue.
A hint that I have received is to consider 2 cases depending on the values $i=j$ and $i \neq j$. My idea for that would be to do something like this:
\begin{align*} \frac{\partial}{\partial u_i}& \left( \psi'\left( \sum_{i=1}^{N} \phi(u_i) \right)\phi'(u_j) \right) \\ &= \begin{cases} \psi''\left( \sum_{i=1}^{N} \phi(u_i) \right) \phi'(u_i) + \psi'\left( \sum_{i=1}^{N} \phi(u_i) \right) \phi''(u_i)\: \:, \: i=j \\ \psi''\left( \sum_{i=1}^{N} \phi(u_i) \right) \phi'(u_j) \quad \quad\quad\quad\quad\quad\quad\quad\quad \:\:\: , i \neq j, \end{cases} \end{align*}
though I'm not sure if this is correct or why we have to consider the two different cases. Some guidance as to how to derive the equality would be greatly appreciated.
You can use the Kronecker delta to write
$$ \frac{\partial}{\partial u_i} \left( \psi'\left( \sum_{k=1}^{N} \phi(u_k) \right)\phi'(u_j) \right) =\psi''\left( \sum_{k=1}^{N} \phi(u_k) \right) \phi'(u_i) \phi'(u_j) + \psi'\left( \sum_{k=1}^{N} \phi(u_k) \right) \phi''(u_j) \delta_{ij} $$ Can you finish from that?