There are efforts to automate proof discovery using deep neural networks. There are varied approaches how the mathematical objects can be embedded into the layer of neural networks.
E.g. one can consider the formal expression of object in some language (Isabelle, Coq, Lean) and create n-dimensional real vector from it using the same approach that generates n-dimensional vector/embedding for the sentence of natural language.
Another approach is to create Abstract Syntax Tree graph of the mathematical expression in the same language and then create graph neural network for this expression and this neural network can create embedding for the expression.
These approaches are about the syntactic approach to neural embedding: embedding is created from the syntactic form of the mathematical expression. And this is fairly established practice.
My question is about the semantic approach of this embedding process. Can we take some mathematical object - e.g. manifold, e.g. particular model $M$ of some First Order theory $T$ and then embed this object into n-dimensional Euclidean space that, of course, could be formed as the space of all the possible points which can be generated by the activity values of n-dimensional neural network layer? Of course, such embedding could be approximate. But n-dimensional Euclidean spaces with its system of subspaces and mappings among them can be quite rich, it can be even considered as category and then this embedding could be functor from the category of objects under consideration to the category of the subspaces of Euclidean space?
I am aware of the work that uses quiver representations (https://arxiv.org/abs/2207.12773) to model neural networks and this can be valid approach indeed, but I feel that the activity in one layer of neural network is already reach enough to be modeled and that is why embedding into n-dimensional Euclidean space could be interesting.
So - my question is kind of reference request - are there any thoughts, research efforts, results, thinking about semantic (approximate) embedding/mapping of mathematical objects into n-dimensional vector spaces which are used in contemporary deep learning?
Notes added. The first commend suggested a look into Nash embedding theorems and rightly ascknowledged that problem is wildly general. But here are 3 notes in order:
The problem is general. Indeed, it becomes more general when one takes into account the reinstatement of Higher Order Logic, that is made possible with the development of contemporary tools, e.g. Benzmuller group http://page.mi.fu-berlin.de/cbenzmueller/ But this generality is the one that is required by contemporary machine learning applications. So, it may be justified just to conceive this line of thought.
When trying to find the right level of abstraction to handle this, one can take the inspiration from the homotopy hypothesis of $\infty$-category theory https://math.ucr.edu/home/baez/homotopy/homotopy.pdf $\infty$-groupoids are subclass of $\infty$-categories and there may little left outside this subclass that can not be considered as space (with interesting musings what it physically means - not to be a space?)
Nash embeddings are about isometrical embeddings, but machine learning may be not so strict and be satisfied by embeddings that just preserve some topological or homotopical properties, nonetheless, any kind of embeddings could be interesting for different kinds of applications.
p.s. it is sad, that there is no tag "homotopy-hypothesis".