Proof that generalized directional derivative is upper semicontinuous

307 Views Asked by At

In "Nonsmooth Optimization" by Mäkela and Neittaanmäki the definition of the generalized directional derivative is given as follows:

Definition 3.1.1 (Clarke). Let $f: \mathbf{R}^{n} \rightarrow \mathbf{R}$ be locally Lipschitz at a point $x \in \mathrm{R}^{n} .$ The generalized directional derivative of $f$ at $x$ in the direction of $v \in \mathrm{R}^{n}$ is defined by $$ f^{\circ}(x ; v)=\limsup _{y \rightarrow x, t \rightarrow 0} \frac{f(y+t v)-f(y)}{t} \, . $$

Then in Theorem 3.1.2. on page 30 they proof that $f^{\circ}(x ; v)$ is upper semicontinuous as a function of $(x ; v)$.

The proof starts with:

Let $\left(x_{i}\right),\left(v_{i}\right) \subset \mathbf{R}^{n}$ be sequences such that $x_{i} \rightarrow x$ and $v_{i} \rightarrow v$. By definition of upper limit, there exist sequences $\left(y_{i}\right) \subset \mathbf{R}^{n}$ and $\left(t_{i}\right) \subset \mathrm{R}$ such that $t_{i}>0$ \begin{equation} \left\|y_{i}-x_{i}\right\|+t_{i}< \frac{1}{i} \end{equation} and \begin{equation} f^{\circ}\left(x_{i} ; v_{i}\right) - \frac{1}{i}\leq \frac{f\left(y_{i}+t_{i} v_{i}\right)-f\left(y_{i}\right)}{t_{i}} \end{equation} for all $i \in \mathrm{N}$.

Why do such sequences $\left(y_{i}\right) \subset \mathbf{R}^{n}$ and $\left(t_{i}\right) \subset \mathrm{R}$ exist?

I mean by the above definition of the generalized directional derivative we have

\begin{align} f^{\circ}\left(x_{i} ; v_{i}\right) - \frac{1}{i} &= \limsup _{y \rightarrow x_{i}, t \rightarrow 0} \frac{f(y+t v_{i})-f(y)}{t} - \frac{1}{i}\\ &\leq \frac{f\left(y_{i}+t_{i} v_{i}\right)-f\left(y_{i}\right)}{t_{i}} \\ \end{align}

I would appreciate a lot if someone could explain to me what the thoughts behind this are.

1

There are 1 best solutions below

2
On BEST ANSWER

This is tricky! For me, the appearance of $1/i$ strongly suggested an approach that ultimately leads nowhere. Here's one way to work it out:

Define a family of functions, one for each $i \in \mathbb{N}$, by

$$ h_{i}\left(w,s\right) := \frac{f\left(w+sv_{i}\right) - f\left(w\right)} {s} $$

and consider the limit superior,

$$ f^{\circ}\left(x_{i},v_{i}\right) := \limsup\limits_{w \to x_{i}, s \to 0} \frac{f\left(w+sv_{i}\right) - f\left(w\right)} {s} = \limsup\limits_{w \to x_{i}, s \to 0} h_{i}\left(w,s\right). $$

Since the $\limsup$ is the largest of the limit points of the sequences that have the form $h_{i}\left(w_{j},s_{j}\right)$ and where $\left(w_{j},s_{j}\right)$ approaches $\left(x_{i},0\right)$, there must be at least one such sequence $\left(w,s\right)_{j} = \left(w_{j},s_{j}\right)$ making $h_{i}\left(w_{j},s_{j}\right)$ approach $f^{\circ}\left(x_{i},v_{i}\right)$. For that particular sequence, we have

$$ \lim_{j \to \infty} h_{i}\left(w_{j},s_{j}\right) = f^{\circ}\left(x_{i},v_{i}\right). $$

First note that we can go far enough down the sequence $\left(w,s\right)_{j}$ so that $\left\lVert w_{j} - x_{i}\right\rVert + s_{j}$ is as close to zero as we'd like; this works because $\left(w,s\right)_{j}$ was a sequence approaching $\left(x_{i},0\right)$. We can also go far enough down the sequence to make $h_{i}\left(w_{j},s_{j}\right)$ as close to its limit $f^{\circ}\left(x_{i},v_{i}\right)$ as we'd like.

In particular, for each $k \in \mathbb{N}$, we can let $j_{k}$ be the first index such that

\begin{gather*} \left\lVert w_{j_{k}} - x_{i}\right\rVert+ s_{j_{k}} < \frac{1}{k}\\ \text{ and }\\ f^{\circ}\left(x_{i},v_{i}\right) - \frac{1}{k} \le h_{i}\left(w_{j_{k}},s_{j_{k}}\right). \end{gather*}

By taking $k=i$, this gives

\begin{gather*} \left\lVert w_{j_{i}} - x_{i}\right\rVert+ s_{j_{i}} < \frac{1}{i}\\ \text{ and }\\ f^{\circ}\left(x_{i},v_{i}\right) - \frac{1}{i} \le h_{i}\left(w_{j_{i}},s_{j_{i}}\right). \end{gather*}

Now all we have to do is define new sequences by $y_{i} := w_{j_{i}}$ and $t_{i} := s_{j_{i}}$. Then

\begin{gather*} \left\lVert y_{i} - x_{i}\right\rVert + t_{i} < \frac{1}{i}\\ \text{ and }\\ f^{\circ}\left(x_{i},v_{i}\right) - \frac{1}{i} \le h_{i}\left(y_{i},t_{i}\right) = \frac{f\left(y_{i}+t_{i}v_{i}\right) - f\left(y_{i}\right)} {t_{i}}. \end{gather*}