$\textbf{Background and context}$
Let $\beta:\Omega\rightarrow\mathbb{R}$, be an unknown spatially distributed parameter (over the spatial domain $\Omega$). Let $\mathcal{F}$ represent a parameter-to-observable mapping, which maps parameter to a solution of a model equation (e.g., a partial differential equation model) and then extracts the PDE solution at various points which correspond to observable quantities. Suppose we have noisy measurements, $d\in\mathbb{R}^{n}$ of said observables, then we have the following
\begin{align*} d=\mathcal{F}(\beta)+\eta \end{align*}
where $\eta$ is the unknown noise that has corrupted the data $d$.
To determine the spatially distributed parameter $\beta$ one would solve the following regularized optimization problem
\begin{align*} \beta^{\star}=\arg\min_{\beta}\left(\frac{1}{2}\|d-\mathcal{F}(\beta)\|_{2}^{2}+\mathcal{R}(\beta)\right), \end{align*}
with regularizing functional $\mathcal{R}$. This regularizing functional is designed to ensure that the solution of the optimization problem is not excessively sensitive to the noise $\eta$. For example, one could choose the regularization as
\begin{align*} \mathcal{R}(\beta)=\frac{\gamma}{2}\int_{\Omega}\nabla\beta\cdot\nabla\beta\,\mathrm{d}x \end{align*}
in order to promote smoothness of $\beta^{\star}$, where the regularization parameter $\gamma>0$ penalizes the square norm of the gradient of $\beta$. A natural question is then what numerical value should we choose for $\gamma$. The Morozov discrepancy principle says to choose $\gamma$ so that the solution of the optimization problem $\beta^{\star}(\gamma)$ is such that
\begin{align*} \|d-\mathcal{F}(\beta^{\star}(\gamma))\|_{2}=\|\eta\|_{2} \end{align*}
provided that, one knowns the noise level $\|\eta\|_{2}$ (or an estimate). The idea is that the data discrepancy $\|d-\mathcal{F}(\beta^{\star}(\gamma))\|_{2}$ is an increasing function of $\gamma$, and if $\gamma$ is such that the data discrepancy is less than the noise then we are essentially overfitting.
$\textbf{My question -- complications from additional bound-constraints}$
I am working with regularized optimization problems with additional bound constraints, that is I would like to solve
\begin{align*} \beta^{\star}&=\arg\min_{\beta}\left(\frac{1}{2}\|d-\mathcal{F}(\beta)\|_{2}^{2}+\mathcal{R}(\beta)\right), \\ \text{ such that }& \beta \geq \beta_{\ell} \end{align*}
for a given lower-bound $\beta_{\ell}$ and an appropriately chosen regularization parameter $\gamma$. The complication is that $\mathcal{F}$ and $d$ can be such that all such $\beta$ for which $\|\mathcal{F}(\beta)-d\|_{2}\leq \|\eta\|_{2}$ are not feasible, that is they do not satisfy the constraint $\beta\geq \beta_{\ell}$.
I would like to determine $\gamma$ by choosing an alternative data set $d_{\text{alternative}}$, which is corrupted by similar noise $\eta$, but so that there are feasible $\beta$ such that $\|d_{\text{alternative}}-\mathcal{F}(\beta)\|_{2}=\|\eta\|_{2}$, which $\gamma$, will ensure that the parameter reconstruction $\beta^{\star}$ will not be overly sensitive to the random noise in $\eta$.
My question to you is
- Does this seem like a reasonable computational strategy?
- Have you seen other strategies in regularized bound constrained optimization?
Any feedback would be greatly appreciated.