The last days of June had revolutionary event in the theory of Deep Neural Networks - Arxiv has published the preprint https://arxiv.org/abs/2106.14587v1 which formalises the Deep Neural Networks in the language of categories, topoi and stacks and hence - enables the categorical reasoning over neural architectures and learning algorithms and hence - enables the discovery (or choice of already discovered) of task-optimal neural architectures and learning algorithms. This is truly historic moment in the realm and deep machine learning and artificial intelligence/artificial general intelligence, because up to now the deep machine learning (neural networks) was mostly trial-and-error black box exploration. This is historic moment in the civilization which can lead to the Singularity. This preprint will certainly go through improvements and it is part or larger Huawei Research effort, preprint references other articles yet to be published, including whole volume of papers. This my question, hence, starts the series of questions about digesting this preprint. The title of those questions will start with the phrase Topos and stacks of neural networks until relevant tag will be created.
This question concerns the sentence in page 15:
A toposic manner to encode such a situation consists to consider contra-variant functors from the category C of the network with values in the topos of G-sets.
My question is - how to understand this sentence?
Here is the subquestions that are guiding my semantic exploration of this sentence:
- What is topos of G-sets? Topos usually (in the context of this paper) is the category of functors from C (category of graphs) to Set (category of sets of real numbers, each set of real number corresponds to some collection of neural activation and/or weights of synapses). So, is this
topos of G-setsthe same category of functors: graphs->activations/weights with just constraints on the co-domain category of Sets. Does this sentence tries to say that category of Sets of constrained to subcategory that contain only those sets that are invariant under the action of some group? - It was my temptation to believe that my understanding in point 1 is correct, but I am confused by this sentence, because it does mentions some extra functors from C to topos of G-sets. My understanding is that this sentence had to speak about functors (with domain in C) which belong to the topos of G-sets and the construction of the topos of G-sets already involve C.
So - how to understand this sentence in the context of this paper?