Let us start with some background and motivation. My main question is very simple and it is available few paragraphs further and it is written in bold.
My problem is based from the emerging theory of deep learning and one of its most interesting paper https://arxiv.org/abs/2106.14587 This paper suggests to consider deep neural network layer-wise and to assigned the complex of sets to each layer - this complex consists mainly from the set of possible activities of the neurons of the layer to which this set is assigned.
What this means? It means that layer U can have n neurons and each neuron can have activity level in some real interval, it may be [-1, 1]. So, the set of possible activities of n neurons is the Cartesian product of sets of individual intervals and the resulting set of possible activities is subset of n-dimensional real Euclidean space (vector space).
The approach of this article if generalizing comparing to the usual approach. The usual approach of deep learning (deep neural networks) is to assign one point/vector (n-dimensional vector) to some concept, some value, some decision. It is called the embedding of this concept or value and usually it is point.
The referenced article is open to the generalization and it allows to assign subset of n-dimensional Euclidean space as the representative of the concept. Even more - it goes further. Some theory (crisp or fuzzy) is assigned to each layer U. The theory is the set of statements in some logic. Of course, each theory can have one or more semantic models and usually those models are built using the set and its subsets. This happens in the references article as well. The n-dimensional Euclidean space is the space in which the models for the theory can be built. Each model of each possible theory is the complex of subsets and the usual set operations can be used to model the usual logical connectives.
Of course, this article is very detailed. It considers the following aspects as well:
- It denotes the semantic models/set-systems by F (when considered in the general case) and X (when considered as a set of activations) and it also considers the languages overs the models, denoted by A.
- This article use the full power of category theory: it considers theory and logics from the categorical perspective which involves the classifying objects of the topos and those classifying objects assign truth values to the objects (of the category of sets).
- This article considers the machine learning of theory from date: learning corresponds to the transformation of theory into more elaborated theory or into more crisp (less fuzzy) theory. Toposic view on logics and theories are used and that is why such transformations could be considered with ease. Otherwise such transformations may seem to be a formidable task.
Now - we can state the main problem. If we want to make this approach computable (implement on computer), the we should be ready to describe any possible subset of n-dimensional real Euclidean space (or more specifically: any subset of only the subspace [-1,1]*[-1,1]*...*[-1,1] of real Euclidean space) in some meaningful way. E.g. it is very easy do describe convex regions, some subspaces that can be defined as points on some algebraic curve, some list of specific points. But my question is: can and how we can describe any subscapce of Euclidean space?
If we can describe any subset, then we can consider the system of such subsets and posit that such system can be a model for some theory. And then we can try to see in some computational experiments on deep neural networks whether or not exactly such kind of system of subsets emerges in some layer of deep neural networks. Again - this seems to be formidable task (and the companion article https://arxiv.org/abs/2108.04751 is one example), but category theory is the framework i which we can try to manage this task.
I am afraid, that maybe there are no meaningful tools to express particularly nasty subsets of n-dimensional (finite-dimensional) Euclidean space?
If we can express any subset of Euclidean set (e.g. as a set of formulas or lists of individual points) then we can also express any system of subsets and that also means that we can express any model for any logic.
Of course, I would be happy to arrive at some finite and compact framework (some calculi), but it may not be possible. I am a bit confused now, but I hope that there are people with enough theory knowledge to give me some illumination about this.
I am imagining that a "description" is a string over some finite alphabet (symbol set).
If this is the case, it seems the answer to your question is no.
The cardinality of the set of all finite descriptions (finite strings over a finite alphabet) is the same as the cardinality of the natural numbers.
The cardinality of the set of countable descriptions (not that one would necessarily want to allow these) is the same as the cardinality of the real numbers.
However the set of all subsets of the reals (or of $\mathbb{R}^n$) is a greater cardinality. Hence you don't have enough descriptions to cover all subsets of the reals.