I'm reading a paper about Gradient Descend Method in which there is a paragraph:
IMHO, $\partial_{w} f\left(w^{t}, x^{t}\right)$ is the partial derivative of $f$ w.r.t $w$, so it's a number. I don't understand what the symbol $\in$ in the expression $g\left(w^{t}, x^{t}\right) \in \partial_{w} f\left(w^{t}, x^{t}\right)$.
Could you please elaborate on this point?
A subgradient of a function $f:U \rightarrow \mathbb{R}$ at $x_0$ is any vector $v$ such that $$ f(x)-f(x_0) \ge v(x-x_0), \quad \forall x \in U $$ and the set of such vectors $v$ is $\partial f(x_0)$.
For example, $|x|$ is not continuous at zero, but if you compute the left- and right-hand derivatives, you'll notice that any value $d \in [-1,1]$ characterizes a tangent line to $|0|$. The idea here is that the derivative, if it exists, is unique, and that is too restrictive for most purposes. We have an entire set of potential derivatives. So let's study that object, and give up on uniqueness, and call it the subgradient.
For convex functions, the subgradient always exists and is always a convex set. Convex functions are a.e. differentiable so a.e. the subgradient is equal to the gradient, but at "kinks", the subgradient is set valued.
Notice from the $|0|$ example that the standard FONCs for minimization carry over to the subgradient: $0 \in [-1,1]$, so $|x|$ achieves a local minimum at zero.
Look up Clarke and Rockafellar for more. It's a huge field, often called nonsmooth optimization.