Proximal Operator / Proximal Mapping of the Huber Loss Function

Question

Given the Scalar Huber Loss Function:

$$ {L}_{\delta} \left( x \right) = \begin{cases} \frac{1}{2} {x}^{2} & \text{for} \; \left| x \right| \leq \delta \\ \delta (\left| x \right| - \frac{1}{2} \delta) & \text{for} \; \left| x \right| > \delta \end{cases} $$

For the vector case one should apply the scalar function in a component wise manner and then sum all components:

$$ {H}_{\delta} \left( x \right) = \sum_{i} {L}_{\delta} \left( {x}_{i} \right) $$

What is the Proximal Operator for the vector function?
Namely what's $ \operatorname{prox}_{\lambda {H}_{\delta} \left( \cdot \right)} \left( y \right) = \arg \min_{x} \frac{1}{2} {\left\| x - y \right\|}_{2}^{2} + \lambda {H}_{\delta} \left( x \right) $?

Could anyone implement it in MATLAB?

Answer 1

From @dohmatob's answer to Proximal Operator of the Huber Loss Function we know the solution for the case $ \delta = 1 $:

$$ {\left( \operatorname{prox}_{\lambda {H}_{1} \left( \cdot \right)} \left( y \right) \right)}_{i} = {y}_{i} - \frac{\lambda {y}_{i}}{\max \left( \left| {y}_{i} \right|, \lambda + 1 \right)} $$

Since $ {H}_{\delta} \left( x \right) = {\delta}^{2} {H}_{1} \left( \frac{x}{\delta} \right) $ one could use the Scaling Property of the Proximal Operator:

$$\begin{aligned} \operatorname{prox}_{\lambda {H}_{\delta} \left( \cdot \right)} \left( y \right) & = \operatorname{prox}_{ {\delta}^{2} \lambda {H}_{1} \left( \frac{\cdot}{\delta} \right)} \left( y \right) \\ & = \delta \operatorname{prox}_{ \frac{{\delta}^{2} \lambda}{ {\delta}^{2} } {H}_{1} \left( \cdot \right)} \left( \frac{y}{\delta} \right) \\ & = \delta \operatorname{prox}_{ \lambda {H}_{1} \left( \cdot \right)} \left( \frac{y}{\delta} \right) \end{aligned}$$

Hence it is given by:

$$ {\left( \operatorname{prox}_{\lambda {H}_{\delta} \left( \cdot \right)} \left( y \right) \right)}_{i} = {y}_{i} - \frac{\lambda {y}_{i}}{\max \left( \left| \frac{{y}_{i}}{\delta} \right|, \lambda + 1 \right)} $$

A MATLAB implementation is given in my answer to Proximal Operator of Huber Loss Function (For $ {L}_{1} $ Regularized Huber Loss of a Regression Function).