I don't know if it's just me but I have a very hard time accepting continuous functions, and would much prefer discretization. I am not a math expert but I do work with software and have the need to implement LSTM. You'll notice that all of the variables used in an LSTM are elements of $\mathbb{R}$. As a computer person I believe the universe is ultimately discrete like a graph or cellular automaton, and so am wondering if there is anything exists in terms of research or ideas in regards to discretizing neural networks to formulate them without the use of $\mathbb{R}$. Sorry if this quesiton is a bit vague, I just keep getting my mind wrapped up in the idea that maybe there is a way to map the real numbers used in a neural network onto a discrete hyperbolic surface (the graph of points of a hyperbolic tessellation for example), or some other interesting mapping/discretization.
For example, the sigmoid function is a value ranging from 0 to 1 over $\mathbb{R}$. But given computers can only handle floating point numbers up to a certain precision, there is inherently some amount of error (I don't understand how this error works or impacts the final result enough yet). But my mind keeps forcefully trying to imagine instead of 0.02481751875 being a value in your function, you would instead just move the decimal place and have it be 24817518750 or something. That would to some degree discretize it in my mind. But then I keep thinking, yeah the real numbers used by floating point computers is limited to say $N$ values, so perhaps if I just find a graph with $N$ nodes I can map the values of the neural network functions to this graph and in addition to discretizing it, give it some interesting new properties. Discrete difference quotients come to mind, but they use division which results in $\mathbb{R}$ from my limited understanding.
I imagine that the brain, with its "activation functions", appears continuous and like we're dealing with a fluid. But deep down it is ultimately discrete at a much smaller scale. How can I take advantage of this?
Does anything like this exist? Or am I way out in left field? If nothing else, where would be a good place to start a better search? I would like clean discretization without any real numbers or continuity. What to do?
The problem I initially see with the "discretization" topic like on Wikipedia is that it is about rounding or approximating continuous functions, like the Finite Difference Method. Maybe I am wrong, but I want to start from the opposite direction, and start with discreteness as a first-class citizen, rather than it approximating continuity which is taken as the "real" source of truth.