Ok, so I have this question:
For the linear model (matrix notation) $y=X\beta+\epsilon$, show that a generalized ridge regression estimator, $\hat\beta_{R}=(X'X+\lambda A)^{-1}X'y$ can be obtained as a solution of minimizing $SS_{res}(\beta)$ subject to the elliptical constraint $\beta'A\beta$, where $A$ is known, positive-definite symmetric matrix. Assume that both $X'X$ and $X'y$ are in correlation form.
So I've got:
$SS_{res}(\beta)=(y-X\hat\beta_{R})'(y-X\hat\beta_{R})$ and I should minimize this w.r.t. $\hat\beta_{R}$ taking into consideration the constraint $\beta'A\beta$. So I think I've got:
$min_{\hat\beta_{R}} (y-X\hat\beta_{R})'(y-X\hat\beta_{R})+\lambda\beta'A\beta$ (should there be a $\lambda$ there?).
But I can't seem to get the right answer, can someone please help?
Recall that $\frac{\partial}{\partial \beta} ||Y- X\beta||_2^2 = 2X'(Y-X\beta)$ and $\frac{\partial}{\partial x}(x'Ax) = 2Ax$ for symmetric (real) $A$. Your problem is $$ \arg \min_{\beta \in \mathbb{R}^p} ||Y-X\beta||_2^2 $$ s.t. $$ \beta'A\beta = \sum\sum a_{ij}\beta_i \beta_j < c, $$ or $$ \arg \min_{\beta \in \mathbb{R}^p} S(\beta)= \arg \min_{\beta \in \mathbb{R}^p} ||Y-X\beta||_2^2 + \lambda\beta'A\beta, $$ thus $$ \frac{\partial}{\partial \beta} S(\hat{\beta}) = -2X'(Y-X\hat\beta)+2\lambda A\hat\beta = 0, $$ $$ (X'Y+ \lambda A )\hat\beta = X'Y $$ $$ \hat \beta = (X'Y + \lambda A)^{-1}X'Y . $$