Suppose we want to choose $x$ to minimise the following (generalized) Tikhonov regularized least squares objective:
$$(Ax-b)^\top (Ax-b) + \lambda [(x-c)^\top W (x-c)],$$
where $W$ is symmetric, but we do not impose that $W$ is positive definite. (Here: $A$ is a real matrix with at least as many rows as columns, $W$ is a square real matrix, and $x$, $b$ and $c$ are real vectors.) We assume that $A^\top A$ is (strictly) positive definite.
For small enough $\lambda$, the solution has:
$$x^* = (A^\top A + \lambda W )^{-1} (A^\top b + \lambda W c ),$$
so:
$$x^*-c = (A^\top A + \lambda W )^{-1} [ A^\top ( b - A c ) ],$$
(see https://en.wikipedia.org/wiki/Tikhonov_regularization#Generalized_Tikhonov_regularization).
For small $\lambda$, $A^\top A + \lambda W$ is positive definite. As $\lambda$ increases though, the minimum eigenvalue of $A^\top A + \lambda W$ eventually hits zero. Call the value of $\lambda$ at which this happens $\lambda_0$.
My questions are the following (the final is the most important):
What happens to $x^*$ and $(x^*-c)^\top W (x^*-c)$ as $\lambda \rightarrow \lambda_0$?
Numerical experiments suggest that $(x^*-c)^\top W (x^*-c)$ is negative for $\lambda$ near $\lambda_0$ with $\lambda<\lambda_0$. Is there guaranteed to be a value of $\lambda$ for which $(x^*-c)^\top W (x^*-c)=0$? What is this value?
\begin{equation} (A^TA + \lambda W)e_{i} = \lambda_{i}e_i \end{equation} Since $\lambda_i$ is zero when $\lambda = \lambda_0$, then: \begin{equation} (A^TA + \lambda_0 W)e_{i} = 0 \end{equation}
\begin{equation} A^TAe_i = -\lambda_0 We_{i} \end{equation} \begin{equation} \lambda_0 = -A^TAe_{i}(We_{i})^{-1} \end{equation} \begin{equation} \lambda_0 = -A^TAW^{-1} \end{equation}
If we replace $\lambda$ in the expression $x^{*} - c$ with $\lambda_0$, we can see that $(A^TA + \lambda W)$ is zero. For the second part, one solution might be $x^{*} = c$. This suggests that
\begin{equation} (A^TA + \lambda W)c = A^Tb + \lambda Wc \end{equation}
If we solve for c, then \begin{equation} c = (A^TA)^{-1}A^Tb \end{equation} which does not depend on $\lambda$. So, there is some lambda which satisfies the equation.