If $f$ is a continuous differentiable function, then the gradient of $f$ at $\bar{u}$ is a subgradient of $f$ at $\bar{u}$.

153 Views Asked by At

I’m looking at the following:

Definition 4. Let $f: \mathbb{R}^{m} \rightarrow \mathbb{R}$ be a convex function, $u \in \mathbb{R}^{m} .$ Subgradient of $f$ at $u$ is a vector $\gamma(u) \in \mathbb{R}^{m}$ such that $$ f(v) \geq f(u)+\gamma(u)^{T}(v-u) \forall v \in \mathbb{R}^{m} $$

Example: If $f$ is a continuous differentiable function, then the gradient of $f$ at $\bar{u}$ is given by $$ \nabla f(\bar{u}):=\left.\left(\frac{\partial f}{\partial u_{1}}, \ldots, \frac{\partial f}{\partial u_{m}}\right)\right|_{u=\bar{u}} $$

Exercise. If $f$ is a continuous differentiable function, then the gradient of $f$ at $\bar{u}$ is a subgradient of $f$ at $\bar{u}$

Let $v \in \mathbb{R}^{m}$. I think I should look at the quantity $f(\bar{u})+\nabla f(\bar{u})^{T} \cdot(v-\bar{u})$ which is: $$ f(\bar{u})+\nabla f(\bar{u})^{T} \cdot(v-\bar{u})=f(\bar{u})+\left(\frac{\partial f(\bar{u})}{\partial u_{1}} \cdot\left(v_{1}-\bar{u}_{1}\right)+\ldots+\frac{\partial f(\tilde{u})}{\partial u_{m}} \cdot\left(v_{m}-u_{m}\right)\right). $$

The RHS of the long-form expression above is the linear approximation to $f$ at $\bar{u}$, i.e. the tangent to $f$ at $\bar{u}$. Since $f$ is convex (it doesn’t say so in the question but then otherwise it wouldn’t make sense?), we’d have $f(\bar{u}) \geq$ RHS?