Here is my attempt at computing the diagonal elements of the Hessian of the Pearson Correlation with respect to the weights, $w_k$, as defined below. I start by deriving the gradient and then move on to the Hessian from there. It all entails using the chain rule and quotient rule applied to these expressions. I am hoping someone can help me verify if I've got it all correct. In the calculations I use the apostrophe ($'$) to denote the partial derivative with respect to $w_k$.
Pearson Correlation
$$r(x,w)=\frac{ \sum_{i=1}^d(x_i-\bar{x})(w_i-\bar{w}) }{ \sqrt{ \sum_{i=1}^d(x_i-\bar{x})^2 \sum_{i=1}^d(w_i-\bar{w})^2 } } = \frac{B}{\sqrt{CD}}$$
Gradient
$$\frac{ \partial r(x,w) }{ \partial w_k } = \frac{ B'\sqrt{CD} - B\sqrt{CD}' }{ (\sqrt{CD})^2 }$$
where
$$B' = \frac{\partial}{\partial w_k} \sum_{i=1}^d(x_i-\bar{x})(w_i-\bar{w}) = x_i-\bar{x}$$
and
$$\sqrt{CD}'=\frac{\partial}{\partial w_k} \left( CD \right)^{1/2} = \frac{1}{2} \left( CD \right)^{-1/2} 2 C (w_k-\bar{w}) = ( CD )^{-1/2} C (w_k-\bar{w})$$
so, all together
$$\frac{ \partial r(x,w) }{ \partial w_k } = \frac{ (x_i-\bar{x})\sqrt{CD} - B ( CD )^{-1/2} C (w_k-\bar{w}) }{ (\sqrt{CD})^2 }$$
We notice that $( CD )^{-1/2} C = \frac{(CD)^{1/2}}{D}$, so then $$\frac{ \partial r(x,w) }{ \partial w_k } =\frac{ (x_i-\bar{x})\sqrt{CD} - B \frac{(CD)^{1/2}}{D} (w_k-\bar{w}) }{ (\sqrt{CD})^2 } = \frac{ (x_i-\bar{x}) - \frac{B}{D} (w_k-\bar{w}) }{ \sqrt{CD} }$$
This is the same result described in Gradients of Pearson Correlation for the Analysis of Biomedical Data, Strickert et al.
Hessian Approximation
Here we are going to calculate the elements along the diagonal of the Hessian matrix based on the gradient.
$$\frac{ \partial r(x,w) }{ \partial w_k } = \frac{ (x_i-\bar{x}) - \frac{B}{D} (w_k-\bar{w}) }{ \sqrt{CD} }$$
Here is the expression for the elements of the Hessian using the quotient rule.
$$\frac{ \partial^2 r(x,w) }{ \partial w_k^2 } = \frac{ ((x_i-\bar{x}) - \frac{B}{D} (w_k-\bar{w}))'(\sqrt{CD}) - ((x_i-\bar{x}) - \frac{B}{D} (w_k-\bar{w}))(\sqrt{CD})' }{(\sqrt{CD})^2} $$
where
$$\frac{ \partial }{ \partial w_k }((x_i-\bar{x}) - \frac{B}{D} (w_k-\bar{w})) = - ( (\frac{B}{D})'(w_k-\bar{w}) + (\frac{B}{D})(w_k-\bar{w})' )$$ $$ = - \left( \frac{B'D - BD'}{D^2}(w_k-\bar{w}) + \frac{B}{D}(w_k-\bar{w})' \right)$$ $$ = - \left( \frac{(x_k - \bar{x})D - 2B(w_k-\bar{w})}{D^2}(w_k-\bar{w}) + \frac{B}{D} \right)$$
rearrange a bit
$$ = \frac{2B(w_k-\bar{w})^2}{D^2} - \frac{(x_k - \bar{x})(w_k-\bar{w}) + B}{D}$$
And $(\sqrt{CD})'$ from the previous section, so the entire expression looks like this
$$\frac{ \partial^2 r(x,w) }{ \partial w_k^2 } = \frac{ \left( \frac{2B(w_k-\bar{w})^2}{D^2} - \frac{(x_k - \bar{x})(w_k-\bar{w}) + B}{D} \right)(\sqrt{CD}) - ((x_i-\bar{x}) - \frac{B}{D} (w_k-\bar{w}))( CD )^{-1/2} C (w_k-\bar{w}) }{(\sqrt{CD})^2}$$
Again substituting for $( CD )^{-1/2} C= \frac{(CD)^{1/2}}{D}$ like above we get rid of one part of the denominator.
$$\frac{ \partial^2 r(x,w) }{ \partial w_k^2 } = \frac{ \frac{2B(w_k-\bar{w})^2}{D^2} - \frac{(x_k - \bar{x})(w_k-\bar{w}) + B}{D} - ((x_i-\bar{x}) - \frac{B}{D^2} (w_k-\bar{w}))(w_k-\bar{w}) }{\sqrt{CD}}$$
And consolidate some terms, we get the final expression for the diagonal elements of the Hessian.
$$\frac{ \partial^2 r(x,w) }{ \partial w_k^2 } = \frac{ \frac{3B(w_k-\bar{w})^2}{D^2} - \frac{(x_k - \bar{x})(w_k-\bar{w}) + B}{D} - (x_i-\bar{x})(w_k-\bar{w}) }{\sqrt{CD}}$$
Fellow Numerati here. I think you've made a mistake in the second to last line. Your $(x_i - \bar{x})(w_k - \bar{w})$ at the end of the numerator needs to be divided by $D$ as well. Then, when consolidating, you simply end up with $-2(x_i - \bar{x})(w_k - \bar{w})/D$ at the end of the numerator. This gives you \begin{equation} \frac{ \partial^2 r(x,w) }{ \partial w_k^2 } = \frac{ \frac{3B(w_k-\bar{w})^2}{D^2} - \frac{B + 2(x_k - \bar{x})(w_k-\bar{w})}{D}}{\sqrt{CD}}. \end{equation} I think you also didn't take into account that $\partial \bar{w}/\partial w_k \neq 0$. However as $N$ grows large, this should not make a meaningful difference.
(I tried using correlation as a loss function in LightGBM, but I never really got it to perform better than MSE.)