Title:
Endomorphisms of Integer Valued Neural Networks with ReLU_t
Year of Publication:
2024
Authors:
Eric R Dolores, Aldo Guzmán-Sáenz, Sangil Kim, Susana Lopez Moreno, Jose L Mendoza-Cortes
Journal:
2025 Joint Mathematics Meetings (JMM 2025)
Abstract:
In order theory, the lexicographic sum construction associates to every finite poset with points an endomorphism of posets, that is . The language of operads allows us to study other objects with the property , where is the endomorphism operad of a set . Examples of possible sets are the Stanley order polynomials, zeta values, and order polytopes.
The paper “Tropical Geometry of Deep Neural Networks” by L. Zhang et al. introduces an equivalence between integer valued neural networks (IVNNs) with activation and tropical rational functions, which come with a map to polytopes. Here, IVNN refers to a network with integer weights but real biases, and is defined as for . The authors of the above-mentioned paper ask about the consequences we can infer from connecting the study of IVNN with with the theory of polytopes. We lift the structure of an algebra over an operad of posets from order polytopes to IVNNs with . This implies that we have a (algebra) set of neural networks indexed by arbitrary finite posets, and that we have a family of associative operations indexed by posets.
We then explain how the neural networks associated to the N poset () can be interpreted as a convolutional filter. When adapted to an implementation of the shallow ConvNet quaternion neural network classifier introduced by X. Zhu et al. in the paper “Quaternion convolutional neural networks", we reduce the number of trainable parameters from to , while also improving the performance of the neural network from a testing accuracy of to . We will also report ongoing experiments with state-of-the-art quaternion convolutional neural networks, as well as experiments with general convolutional neural networks and the study of the downsampling properties of poset neural networks.
URL: