Title: 

Endomorphisms of Integer Valued Neural Networks with ReLU_t

 

Year of Publication:

2024

 

Authors:

Eric R Dolores, Aldo Guzmán-Sáenz, Sangil Kim, Susana Lopez Moreno, Jose L Mendoza-Cortes

 

Journal:

2025 Joint Mathematics Meetings (JMM 2025)

 

Abstract:

In order theory, the lexicographic sum construction associates to every finite poset with  points an endomorphism of posets, that is . The language of operads allows us to study other objects  with the property , where  is the endomorphism operad of a set . Examples of possible sets  are the Stanley order polynomials, zeta values, and order polytopes.

 

The paper “Tropical Geometry of Deep Neural Networks” by L. Zhang et al. introduces an equivalence between integer valued neural networks (IVNNs) with  activation and tropical rational functions, which come with a map to polytopes. Here, IVNN refers to a network with integer weights but real biases, and  is defined as  for . The authors of the above-mentioned paper ask about the consequences we can infer from connecting the study of IVNN with  with the theory of polytopes. We lift the structure of an algebra over an operad of posets from order polytopes to IVNNs with . This implies that we have a (algebra) set of neural networks indexed by arbitrary finite posets, and that we have a family of associative operations indexed by posets.

 

We then explain how the neural networks associated to the N poset () can be interpreted as a  convolutional filter. When adapted to an implementation of the shallow ConvNet quaternion neural network classifier introduced by X. Zhu et al. in the paper “Quaternion convolutional neural networks", we reduce the number of trainable parameters from  to , while also improving the performance of the neural network from a testing accuracy of  to . We will also report ongoing experiments with state-of-the-art quaternion convolutional neural networks, as well as experiments with general convolutional neural networks and the study of the downsampling properties of poset neural networks.

 

 

URL:

https://urldefense.com/v3/__https://scholar.google.com/citations?view_op=view_citation&hl=en&user=HZttDwwAAAAJ&sortby=pubdate&citation_for_view=HZttDwwAAAAJ:qPeb-qHga9sC__;!!HXCxUKc!0ErcBeVHyMXlVK5S6hbxLiAKGyk0vCExZTy0JNeDTHSUlDqSUHnS5f8H6jNyPNf44LOjab63LialCUdG8MLTrw$