When we have a universal approximator, can we approximate any nonnegative function by passing the approximator to ReLU activation?

34 Views Asked by At

Consider the domain of all functions that will be mentioned is $X$, a compact subspace of $\mathbb{R}^n$.

Suppose that, given a continuous function $f$, $\forall \epsilon > 0$, $\exists \hat{f} \in \mathcal{F}$ such that $\lVert f - \hat{f} \rVert_{\infty} < \epsilon$.

Then, for any nonnegative continuous function $g$, for every $\epsilon > 0$, does there exist $\hat{f} \in \mathcal{F}$ such that $\lVert g - \max(0, \hat{f})\rVert_{\infty} < \epsilon$?

That is, when we have a universal approximator $\mathcal{F}$ for continuous functions, can we approximate any nonnegative function by the form of $\max(0, \hat{f})$ where $\hat{f} \in \mathcal{F}$?

1

There are 1 best solutions below

0
On BEST ANSWER

Please see the comments; I don't know whether it is possible to change this question as answered without an answer, so I posted this answer.