Neural and Evolutionary Computing

Date: Fri, 19 Jul 2024 | Total: 7

#1 Fundamental Visual Navigation Algorithms: Indirect Sequential, Biased Diffusive, & Direct Pathing [PDF] [Copy] [Kimi]

Authors: Patrick Govoni ; Pawel Romanczuk

Effective foraging in a predictable local environment requires coordinating movement with observable spatial context - in a word, navigation. Distinct from search, navigating to specific areas known to be valuable entails its own particularities. How space is understood through vision and parsed for navigation is often examined experimentally, with limited ability to manipulate sensory inputs and probe into the algorithmic level of decision-making. As a generalizable, minimal alternative to empirical means, we evolve and study embodied neural networks to explore information processing algorithms an organism may use for visual spatial navigation. Surprisingly, three distinct classes of algorithms emerged, each with its own set of rules and tradeoffs, and each appear to be highly relevant to observable biological navigation behaviors.

Subjects: Neural and Evolutionary Computing ; Artificial Intelligence

Publish: 2024-07-18 14:07:44 UTC

#2 Accurate Mapping of RNNs on Neuromorphic Hardware with Adaptive Spiking Neurons [PDF] [Copy] [Kimi]

Authors: Gauthier Boeshertz ; Giacomo Indiveri ; Manu Nair ; Alpha Renner

Thanks to their parallel and sparse activity features, recurrent neural networks (RNNs) are well-suited for hardware implementation in low-power neuromorphic hardware. However, mapping rate-based RNNs to hardware-compatible spiking neural networks (SNNs) remains challenging. Here, we present a ${\Sigma}{\Delta}$-low-pass RNN (lpRNN): an RNN architecture employing an adaptive spiking neuron model that encodes signals using ${\Sigma}{\Delta}$-modulation and enables precise mapping. The ${\Sigma}{\Delta}$-neuron communicates analog values using spike timing, and the dynamics of the lpRNN are set to match typical timescales for processing natural signals, such as speech. Our approach integrates rate and temporal coding, offering a robust solution for the efficient and accurate conversion of RNNs to SNNs. We demonstrate the implementation of the lpRNN on Intel's neuromorphic research chip Loihi, achieving state-of-the-art classification results on audio benchmarks using 3-bit weights. These results call for a deeper investigation of recurrency and adaptation in event-based systems, which may lead to insights for edge computing applications where power-efficient real-time inference is required.

Subjects: Neural and Evolutionary Computing ; Audio and Speech Processing

Publish: 2024-07-18 14:06:07 UTC

#3 Neuromorphic Circuit Simulation with Memristors: Design and Evaluation Using MemTorch for MNIST and CIFAR [PDF] [Copy] [Kimi]

Authors: Julio Souto ; Guillermo Botella ; Daniel García ; Raúl Murillo ; Alberto del Barrio

Memristors offer significant advantages as in-memory computing devices due to their non-volatility, low power consumption, and history-dependent conductivity. These attributes are particularly valuable in the realm of neuromorphic circuits for neural networks, which currently face limitations imposed by the Von Neumann architecture and high energy demands. This study evaluates the feasibility of using memristors for in-memory processing by constructing and training three digital convolutional neural networks with the datasets MNIST, CIFAR10 and CIFAR100. Subsequent conversion of these networks into memristive systems was performed using Memtorch. The simulations, conducted under ideal conditions, revealed minimal precision losses of nearly 1% during inference. Additionally, the study analyzed the impact of tile size and memristor-specific non-idealities on performance, highlighting the practical implications of integrating memristors in neuromorphic computing systems. This exploration into memristive neural network applications underscores the potential of Memtorch in advancing neuromorphic architectures.

Subjects: Neural and Evolutionary Computing ; Computational Physics

Publish: 2024-07-18 11:30:33 UTC

#4 Studying the Performance of the Jellyfish Search Optimiser for the Application of Projection Pursuit [PDF] [Copy] [Kimi]

Authors: H. Sherry Zhang ; Dianne Cook ; Nicolas Langrené ; Jessica Wai Yin Leung

The projection pursuit (PP) guided tour interactively optimises a criteria function known as the PP index, to explore high-dimensional data by revealing interesting projections. The optimisation in PP can be non-trivial, involving non-smooth functions and optima with a small squint angle, detectable only from close proximity. To address these challenges, this study investigates the performance of a recently introduced swarm-based algorithm, Jellyfish Search Optimiser (JSO), for optimising PP indexes. The performance of JSO for visualising data is evaluated across various hyper-parameter settings and compared with existing optimisers. Additionally, this work proposes novel methods to quantify two properties of the PP index, smoothness and squintability that capture the complexities inherent in PP optimisation problems. These two metrics are evaluated along with JSO hyper-parameters to determine their effects on JSO success rate. Our numerical results confirm the positive impact of these metrics on the JSO success rate, with squintability being the most significant. The JSO algorithm has been implemented in the tourr package and functions to calculate smoothness and squintability are available in the ferrn package.

Subjects: Computation ; Neural and Evolutionary Computing

Publish: 2024-07-18 16:39:01 UTC

#5 Novel Deep Neural Network Classifier Characterization Metrics with Applications to Dataless Evaluation [PDF] [Copy] [Kimi]

Authors: Nathaniel Dean ; Dilip Sarkar

The mainstream AI community has seen a rise in large-scale open-source classifiers, often pre-trained on vast datasets and tested on standard benchmarks; however, users facing diverse needs and limited, expensive test data may be overwhelmed by available choices. Deep Neural Network (DNN) classifiers undergo training, validation, and testing phases using example dataset, with the testing phase focused on determining the classification accuracy of test examples without delving into the inner working of the classifier. In this work, we evaluate a DNN classifier's training quality without any example dataset. It is assumed that a DNN is a composition of a feature extractor and a classifier which is the penultimate completely connected layer. The quality of a classifier is estimated using its weight vectors. The feature extractor is characterized using two metrics that utilize feature vectors it produces when synthetic data is fed as input. These synthetic input vectors are produced by backpropagating desired outputs of the classifier. Our empirical study of the proposed method for ResNet18, trained with CAFIR10 and CAFIR100 datasets, confirms that data-less evaluation of DNN classifiers is indeed possible.

Subjects: Machine Learning ; Artificial Intelligence ; Neural and Evolutionary Computing

Publish: 2024-07-17 20:40:46 UTC

#6 Generalisation to unseen topologies: Towards control of biological neural network activity [PDF] [Copy] [Kimi]

Authors: Laurens Engwegen ; Daan Brinks ; Wendelin Böhmer

Novel imaging and neurostimulation techniques open doors for advancements in closed-loop control of activity in biological neural networks. This would allow for applications in the investigation of activity propagation, and for diagnosis and treatment of pathological behaviour. Due to the partially observable characteristics of activity propagation, through networks in which edges can not be observed, and the dynamic nature of neuronal systems, there is a need for adaptive, generalisable control. In this paper, we introduce an environment that procedurally generates neuronal networks with different topologies to investigate this generalisation problem. Additionally, an existing transformer-based architecture is adjusted to evaluate the generalisation performance of a deep RL agent in the presented partially observable environment. The agent demonstrates the capability to generalise control from a limited number of training networks to unseen test networks.

Subjects: Neurons and Cognition ; Artificial Intelligence ; Machine Learning ; Neural and Evolutionary Computing

Publish: 2024-06-17 13:53:39 UTC

#7 gFlora: a topology-aware method to discover functional co-response groups in soil microbial communities [PDF1] [Copy] [Kimi]

Authors: Nan Chen ; Merlijn Schram ; Doina Bucur

We aim to learn the functional co-response group: a group of taxa whose co-response effect (the representative characteristic of the group) associates well statistically with a functional variable. Different from the state-of-the-art method, we model the soil microbial community as an ecological co-occurrence network with the taxa as nodes (weighted by their abundance) and their relationships (a combination from both spatial and functional ecological aspects) as edges (weighted by the strength of the relationships). Then, we design a method called gFlora which notably uses graph convolution over this co-occurrence network to get the co-response effect of the group, such that the network topology is also considered in the discovery process. We evaluate gFlora on two real-world soil microbiome datasets (bacteria and nematodes) and compare it with the state-of-the-art method. gFlora outperforms this on all evaluation metrics, and discovers new functional evidence for taxa which were so far under-studied. We show that the graph convolution step is crucial to taxa with low abundance, and the discovered bacteria of different genera are distributed in the co-occurrence network but still tightly connected among themselves, demonstrating that topologically they fill different but collaborative functional roles in the ecological community.

Subject: Machine Learning

Publish: 2024-07-04 12:43:53 UTC