Let's say we are given a two dimensional ReLU-FFNN $f_N: \mathbb{R^2} \rightarrow \mathbb{R}^2$ with hidden dimension $N$: $$f_N(x)=W_1\phi(W_2x+b_2)+b_1$$ with $W_2 \in \mathbb{R}^{N \times 2},b_2 \in \mathbb{R}^{N},W_1 \in \mathbb{R}^{2 \times N},b_1 \in \mathbb{R}^{ 2}$.
Every hidden dimension $N$ defines a line $l_i: \mathbb{R} \rightarrow \mathbb{R}, \quad i=1,...,N$ given by $W_2 x+b_2=0$. These lines define polygons and on these polygons the function $f_N$ is linear.
Now we assume that each line has to intersect the 2-disc $D^2_r(0,0)$ centered at the origin $(0,0)$ with radius $r$. Let's take another disk centered at $(x',y')$ with radius $r'$ $D^2_{r'}(x',y')$ such that both disk do not intersect: $$D^2_r(0,0) \cap D^2_{r'}(x',y') = \emptyset$$
I have illustrated the situation in the picture below
My question is the following: Do we use the universal approximation property of $f_N$ in $D^2_{r'}(x',y')$ with this restriction on the weights? I.e. given a function $g \in \mathcal{C}(\mathbb{R}^2)$ can we still find an $N \in \mathbb{N}$ such that $\|f_N-g\|_{D^2_{r'}(x',y')}< \varepsilon, \quad \forall \varepsilon >0$
My intuition: I think we use the universal approximation property. The UAT is proven since we can "rewrite" any NN as sum of indicator functions and in the limit of vanishing inidcator site we can approximate any continuous function. However, by restricting the weights in the way we do, the linear subregions cannot have an arbitrary form. We can still create infinitely many of them in ${D^2_{r'}(x',y')$ but the all have a special form such that they cannot approximate any function.
I.e. in the picture below, take a ReLU-network with one hidden layer such that the linear subregion is parallel to the $r'$-arrow and activated on the right side of it.