Here is the situation: I am trying to implement an algorithm called ProxASAGA. We have a set of functions $f_i: R^n \rightarrow R$ and we are trying to minimize their mean
$$ \frac{1}{n}\sum_{i=1}^nf_i(x). $$
On every iteration, there are the following instructions at some point:
- Sample $i$ uniformly in $\{1,..,n\}$,
- $S_i := $ support of $\Delta f_i$,
- ...
- $[\delta \alpha]_{S_i} = [\Delta f_i(\hat{x})]_{S_i} - [\hat{\alpha_i}]_{S_i}$, where $\hat{x}$ is an n-dimensional vector and $[x]_{B}$ denotes the vector $x$ restricted to the coordinates in block $B$.
First off, I do not understand how I could possibly compute the support of a function so I assume it is "given" to the algorithm. But then, by the definition I have read of a function's support, it is the set of points where the function is non zero, thus I can't find a sense to the term $[\Delta f_i(\hat{x})]_{S_i}$.
$S_i$ is something of the form $\{x_1, ..., x_n\}$ with $x_i \in R^n$ right ? How can it restrict a term that is also in $R^n$ ($\Delta f_i(\hat{x})$)?
I finally got an answer from a researcher of my school:
"The support $S$ of a vector (note: $\nabla f(x)$ is just a vector), is simply the set of all coordinates that are non-zero.
$$S=\{i | x_i \neq 0\}$$
The notation $x|S$ then usually means the (smaller) vector where all zeros have been deleted."