Lagrangian, but the one variable is inside sums

44 Views Asked by At

Context: Given a vector $Y$ with $N$ elements indexed $i=1,2,...,N$, provide a vector $X^*$ that has a sample correlation of exactly $\rho$ to $Y$, i.e. $corr(Y,X^*)=\rho$.

Procedure: There are probably many and perhaps easier ways to do this, but the following occurred to me because I saw it applied to a more difficult problem in a paper years ago. The paper, however, skips over the calculations and it still keeps me up at night that I was never able to figure out that Lagrangian.

Start with a vector $X$ where every element $x_i=1$, and an initial vector of weights $\omega$ where again every element $\omega_i=1$. Then find $\omega^*$, the vector of weights that (1) is Kullback-Leibler closest to $\omega$ and (2) satisfies $corr(Y,\omega^* X)=\rho$. In other words, minimize the Kullback-Leibler distance between $\omega^*$ and $\omega$, subject to $corr(Y,\omega^* X)=\rho$:

$\displaystyle \min_{\omega^*} \quad \sum_{i=1}^{N}\omega^*_iln\left(\frac{\omega^*_i}{\omega_i} \right)$

$s.t. \quad \frac{cov(Y,\omega^* X)}{sd(Y)sd(\omega^* X)}=\frac{\frac{1}{N-1} \sum_{i=1}^{N}(y_i-\frac{1}{N}\sum_{i=1}^{N}y_i)(\omega^*_ix_i-\frac{1}{N}\sum_{i=1}^{N}\omega^*_ix_i)}{\sqrt{\frac{1}{N-1} \sum_{i=1}^{N}(y_i-\frac{1}{N}\sum_{i=1}^{N}y_i)^2}\sqrt{\frac{1}{N-1} \sum_{i=1}^{N}(\omega^*_ix_i-\frac{1}{N}\sum_{i=1}^{N}\omega^*_ix_i)^2}} = \rho$

The Lagrangian is then:

$L(\omega^*,\lambda)= \sum_{i=1}^{N}\omega^*_iln\left(\frac{\omega^*_i}{\omega_i} \right) - \lambda(cov(Y,\omega^* X)-\rho \times sd(Y)sd(\omega^* X))$

And it is not difficult to take derivatives to come up with first-order conditions:

$\frac{\partial L}{\partial \omega^*}= \sum_{i=1}^{N} \left[ ln\left(\frac{\omega^*_i}{\omega_i} \right)+\frac{1}{\omega_i} \right] - \lambda \frac{\partial}{\partial \omega^*}cov(Y,\omega^* X) + \lambda \rho\times sd(Y)\frac{\partial}{\partial \omega^*}sd(\omega^*X)$

$\quad \quad = \sum_{i=1}^{N} \left[ ln\left(\frac{\omega^*_i}{\omega_i} \right)+\frac{1}{\omega_i} \right] -\lambda cov(Y,X)+\lambda\rho \times sd(Y) \\ \quad \quad \quad \quad\times \frac{2}{\sqrt{\omega^*_i}(N-1)}\sum_{i=1}^{N} \left[ \omega^*_ix_i^2 - \frac{x_i}{N}\sum_{i=1}^{N} \omega^*_ix_i - \frac{\omega^*_ix_i}{N} + \frac{2}{N^2}(\sum_{i=1}^{N}x_i) (\sum_{i=1}^{N}\omega^*_ix_i) \right] \stackrel{!}{=} 0$

$\frac{\partial L}{\partial \lambda}=cov(Y,\omega^* X)-\rho \times sd(Y)sd(\omega^* X) \stackrel{!}{=} 0$

So now I somehow need to solve the $\frac{\partial L}{\partial \omega^*}\stackrel{!}{=} 0$ for $\omega^*_i(\lambda)$, then I can substitute that expression for $\omega^*_i$ in $\frac{\partial L}{\partial \lambda}$ and, finally, give $\frac{\partial L}{\partial \lambda(\omega^*)}\stackrel{!}{=} 0$ to Newton-Raphson or whatever optimizer to compute the optimal $\lambda$.

My question is how in the world could I ever hope to solve $\frac{\partial L}{\partial \omega^*}\stackrel{!}{=} 0$ for $\omega^*_i(\lambda)$, given that the $\omega^*_i$ appears inside sums in there? Any hints and links to similar (solved) problems will be greatly appreciated. Thanks.