Can a neural network learn a function with an asymptote? For example $lim_{x \to 0}{1 \over x^2}= \infty$, so can we use a neural network to approximate ${1\over x^2}= \infty$ ?
2026-03-27 18:08:35.1774634915
Can a neural network learn a function with asymptote?
121 Views Asked by Bumbble Comm https://math.techqa.club/user/bumbble-comm/detail At
1
A (feedforward) neural network is a repeated application of a linear (or affine) transformation and a nonlinear elementwise "saturation" function (i.e. a function that is finitely bounded above and below). For example,
$$ f(x) = W_3\sigma(W_2 \sigma(W_1 x)) $$
where the $W_i$ are weight matrices and $\sigma$ is an elementwise saturation (neuron) function. Your question is: can we find some finite $W_i$ such that $\underset{x \to c}{\lim} f(x) = \infty$ for some finite $c$.
Obviously since $\sigma$ is a bounded function, and the last $W_i$ is just a linear (or affine) transformation, this is not possible. I mean, consider the function $z=Ay$. Can you pick $A$ such that $z$ is infinite for some bounded $y$?
A more rigorous way to give you the answer "no" is to explain the Universal Approximation Theorem.