Simple Classification Task Research Articles

Comparative Analysis of Classification and Segmentation Performance of Different Dog Breeds Based on Mask R-CNN

The classification of different image data is a common task in real life, which has been achieved as early as convolutional neural networks. With the emergence of R-CNN (Region CNN), simple classification tasks began to develop towards object detection tasks, followed by various segmentation tasks. After Faster R-CNN [4], Mask R-CNN [3] achieved instance segmentation while detecting objects. This algorithm model needs to distinguish different target objects in the image panel and learn a large number of characteristics to represent the details of each target object. In dog data images, there are differences and similarities in the characteristics of dog species, such as fur color, texture, and body shape. So in this article, the author selected and created two different dog datasets, representing targets with significant differences and similarities in features. Compare and analyze the performance of Mask R-CNN object detection and instance segmentation using these two dog datasets.

The underlying mechanisms of alignment in error backpropagation through arbitrary weights

Understanding the mechanisms by which plasticity in millions of synapses in the brain is orchestrated to achieve behavioral and cognitive goals is a fundamental question in neuroscience. In this regard, insights from learning methods in artificial neural networks (ANNs) and in particular supervised learning using backpropagation (BP) seem inspiring. However, the implementation of BP requires exact matching between forward and backward weights, which is unrealistic given the known connectivity pattern in the brain (known as the “weight transport problem”). Notably, it has been shown that under certain conditions, error BackPropagation Through Arbitrary Weights (BP-TAW) can lead to a partial alignment between forward and backward weights (weight alignment or WA). This learning algorithm, which is also known as feedback alignment (FA), can result in surprisingly good degrees of accuracy in simple classification tasks. However, the underlying mechanisms and mathematical basis of WA are not thoroughly understood. In this work, we demonstrate the mathematical basis of WA and answer the question of why and in what conditions WA occurs. We show that the occurrence of WA in ANNs is induced by statistical properties of the output and error signals of neurons, such as autocorrelation and cross-correlation, and can happen even in the absence of learning or reduction of the loss function. Moreover, we show that WA can be improved significantly by limiting the norm of input weights to neurons and that such a weight normalization (WN) method can improve the classification accuracy of BP-TAW. The findings presented can be used to further improve the performance of BP-TAW and open new ways for exploring possible learning mechanisms in biological neural networks without exact matching between forward and backward weights.

Spiking-NeRF: Spiking Neural Network for Energy-Efficient Neural Rendering

Artificial Neural Networks (ANNs) have achieved remarkable performance in many artificial intelligence tasks. As the application scenarios become more sophisticated, the computation and energy consumption of ANNs are also constantly increasing, which poses a challenge for deploying ANNs on energy-constrained devices. Spiking Neural Networks (SNNs) provide a promising solution to build energy-efficiency neural networks. However, the current training methods of SNNs cannot output values as precise as ANNs. This limits the applications of SNNs to relatively simple image classification tasks. In this article, we extend the application of SNNs to neural rendering tasks and propose an energy-efficient spiking neural rendering model, called Spiking-NeRF (Spiking Neural Radiance Fields). We first analyze the ANN-to-SNN conversion theory and propose an output scheme for SNNs to obtain the precise scene property values. Then we customize the parameter normalization method for the special network architecture of neural rendering. Furthermore, we present an early termination strategy (ETS) based on the discrete nature of spikes to reduce energy consumption. We evaluate the performance of Spiking-NeRF on both realistic and synthetic scenes. Experimental results show that Spiking-NeRF can achieve comparable rendering performance to ANN-based NeRF with up to \(2.27\times\) energy reduction.

Human-centred physical neuromorphics with visual brain-computer interfaces

Steady-state visual evoked potentials (SSVEPs) are widely used for brain-computer interfaces (BCIs) as they provide a stable and efficient means to connect the computer to the brain with a simple flickering light. Previous studies focused on low-density frequency division multiplexing techniques, i.e. typically employing one or two light-modulation frequencies during a single flickering light stimulation. Here we show that it is possible to encode information in SSVEPs excited by high-density frequency division multiplexing, involving hundreds of frequencies. We then demonstrate the ability to transmit entire images from the computer to the brain/EEG read-out in relatively short times. High-density frequency multiplexing also allows to implement a photonic neural network utilizing SSVEPs, that is applied to simple classification tasks and exhibits promising scalability properties by connecting multiple brains in series. Our findings open up new possibilities for the field of neural interfaces, holding potential for various applications, including assistive technologies and cognitive enhancements, to further improve human-machine interactions.

Dynamical stability and chaos in artificial neural network trajectories along training

The process of training an artificial neural network involves iteratively adapting its parameters so as to minimize the error of the network’s prediction, when confronted with a learning task. This iterative change can be naturally interpreted as a trajectory in network space–a time series of networks–and thus the training algorithm (e.g., gradient descent optimization of a suitable loss function) can be interpreted as a dynamical system in graph space. In order to illustrate this interpretation, here we study the dynamical properties of this process by analyzing through this lens the network trajectories of a shallow neural network, and its evolution through learning a simple classification task. We systematically consider different ranges of the learning rate and explore both the dynamical and orbital stability of the resulting network trajectories, finding hints of regular and chaotic behavior depending on the learning rate regime. Our findings are put in contrast to common wisdom on convergence properties of neural networks and dynamical systems theory. This work also contributes to the cross-fertilization of ideas between dynamical systems theory, network theory and machine learning.

Gravity Spy: lessons learned and a path forward

The Gravity Spy project aims to uncover the origins of glitches, transient bursts of noise that hamper analysis of gravitational-wave data. By using both the work of citizen-science volunteers and machine learning algorithms, the Gravity Spy project enables reliable classification of glitches. Citizen science and machine learning are intrinsically coupled within the Gravity Spy framework, with machine learning classifications providing a rapid first-pass classification of the dataset and enabling tiered volunteer training, and volunteer-based classifications verifying the machine classifications, bolstering the machine learning training set and identifying new morphological classes of glitches. These classifications are now routinely used in studies characterizing the performance of the LIGO gravitational-wave detectors. Providing the volunteers with a training framework that teaches them to classify a wide range of glitches, as well as additional tools to aid their investigations of interesting glitches, empowers them to make discoveries of new classes of glitches. This demonstrates that, when giving suitable support, volunteers can go beyond simple classification tasks to identify new features in data at a level comparable to domain experts. The Gravity Spy project is now providing volunteers with more complicated data that includes auxiliary monitors of the detector to identify the root cause of glitches.

Artificial Intelligence-Enhanced Neurocritical Care for Traumatic Brain Injury : Past, Present and Future.

In neurointensive care units (NICUs), particularly in cases involving traumatic brain injury (TBI), swift and accurate decision-making is critical because of rapidly changing patient conditions and the risk of secondary brain injury. The use of artificial intelligence (AI) in NICU can enhance clinical decision support and provide valuable assistance in these complex scenarios. This article aims to provide a comprehensive review of the current status and future prospects of AI utilization in the NICU, along with the challenges that must be overcome to realize this. Presently, the primary application of AI in NICU is outcome prediction through the analysis of preadmission and high-resolution data during admission. Recent applications include augmented neuromonitoring via signal quality control and real-time event prediction. In addition, AI can integrate data gathered from various measures and support minimally invasive neuromonitoring to increase patient safety. However, despite the recent surge in AI adoption within the NICU, the majority of AI applications have been limited to simple classification tasks, thus leaving the true potential of AI largely untapped. Emerging AI technologies, such as generalist medical AI and digital twins, harbor immense potential for enhancing advanced neurocritical care through broader AI applications. If challenges such as acquiring high-quality data and ethical issues are overcome, these new AI technologies can be clinically utilized in the actual NICU environment. Emphasizing the need for continuous research and development to maximize the potential of AI in the NICU, we anticipate that this will further enhance the efficiency and accuracy of TBI treatment within the NICU.

Outperforming RBM Feature-Extraction Capabilities by "Dreaming" Mechanism.

Inspired by a formal equivalence between the Hopfield model and restricted Boltzmann machines (RBMs), we design a Boltzmann machine, referred to as the dreaming Boltzmann machine (DBM), which achieves better performances than the standard one. The novelty in our model lies in a precise prescription for intralayer connections among hidden neurons whose strengths depend on features correlations. We analyze learning and retrieving capabilities in DBMs, both theoretically and numerically, and compare them to the RBM reference. We find that, in a supervised scenario, the former significantly outperforms the latter. Furthermore, in the unsupervised case, the DBM achieves better performances both in features extraction and representation learning, especially when the network is properly pretrained. Finally, we compare both models in simple classification tasks and find that the DBM again outperforms the RBM reference.

Creative thinking does not promote dishonesty.

We assessed the relation of creativity and unethical behaviour by manipulating the thinking style of participants (N = 450 adults) and measuring the impact of this manipulation on the prevalence of dishonest behaviour. Participants performed one of three inducer tasks: the alternative uses task to promote divergent thinking, the remote associates task to promote convergent thinking, or a simple classification task for rule-based thinking. Before and after this manipulation, participants conducted the mind game as a straightforward measure of dishonesty. Dishonest behaviour increased from before to after the intervention, but we found no credible evidence that this increase differed between induced mindsets. Exploratory analyses did not support any relation of trait creativity and dishonesty either. We conclude that the influence of creative thinking on unethical behaviour seems to be more ambiguous than assumed in earlier research or might be restricted to specific populations or contexts.

Shallow and deep learning of extreme rainfall events from convective atmospheres

Abstract. Our subject is a new catalogue of radar-based heavy rainfall events (CatRaRE) over Germany and how it relates to the concurrent atmospheric circulation. We classify daily ERA5 fields of convective indices according to CatRaRE, using an array of 13 statistical methods, consisting of 4 conventional (“shallow”) and 9 more recent deep machine learning (DL) algorithms; the classifiers are then applied to corresponding fields of simulated present and future atmospheres from the Coordinated Regional Climate Downscaling Experiment (CORDEX) project. The inherent uncertainty of the DL results from the stochastic nature of their optimization is addressed by employing an ensemble approach using 20 runs for each network. The shallow random forest method performs best with an equitable threat score (ETS) around 0.52, followed by the DL networks ALL-CNN and ResNet with an ETS near 0.48. Their success can be understood as a result of conceptual simplicity and parametric parsimony, which obviously best fits the relatively simple classification task. It is found that, on summer days, CatRaRE-convective atmospheres over Germany occur with a probability of about 0.5. This probability is projected to increase, regardless of method, both in ERA5-reanalyzed and CORDEX-simulated atmospheres: for the historical period we find a centennial increase of about 0.2 and for the future period one of slightly below 0.1.

Microcomb-Driven Optical Convolution for Car Plate Recognition

The great success of artificial intelligence (AI) calls for higher-performance computing accelerators, and optical neural networks (ONNs) with the advantages of high speed and low power consumption have become competitive candidates. However, most of the reported ONN architectures have demonstrated simple MNIST handwritten digit classification tasks due to relatively low precision. A microring resonator (MRR) weight bank can achieve a high-precision weight matrix and can increase computing density with the assistance of wavelength division multiplexing (WDM) technology offered by dissipative Kerr soliton (DKS) microcomb sources. Here, we implement a car plate recognition task based on an optical convolutional neural network (CNN). An integrated DKS microcomb was used to drive an MRR weight-bank-based photonic processor, and the computing precision of one optical convolution operation could reach 7 bits. The first convolutional layer was realized in the optical domain, and the remaining layers were performed in the electrical domain. Totally, the optoelectronic computing system (OCS) could achieve a comparable performance with a 64-bit digital computer for character classification. The error distribution obtained from the experiment was used to emulate the optical convolution operation of other layers. The probabilities of the softmax layer were slightly degraded, and the robustness of the CNN was reduced, but the recognition results were still acceptable. This work explores an MRR weight-bank-based OCS driven by a soliton microcomb to realize a real-life neural network task for the first time and provides a promising computational acceleration scheme for complex AI tasks.

Explanatory machine learning for sequential human teaching

The topic of comprehensibility of machine-learned theories has recently drawn increasing attention. Inductive logic programming uses logic programming to derive logic theories from small data based on abduction and induction techniques. Learned theories are represented in the form of rules as declarative descriptions of obtained knowledge. In earlier work, the authors provided the first evidence of a measurable increase in human comprehension based on machine-learned logic rules for simple classification tasks. In a later study, it was found that the presentation of machine-learned explanations to humans can produce both beneficial and harmful effects in the context of game learning. We continue our investigation of comprehensibility by examining the effects of the ordering of concept presentations on human comprehension. In this work, we examine the explanatory effects of curriculum order and the presence of machine-learned explanations for sequential problem-solving. We show that (1) there exist tasks A and B such that learning A before learning B results in better comprehension for humans in comparison to learning B before learning A and (2) there exist tasks A and B such that the presence of explanations when learning A contributes to improved human comprehension when subsequently learning B. We propose a framework for the effects of sequential teaching on comprehension based on an existing definition of comprehensibility and provide evidence for support from data collected in human trials. Our empirical study involves curricula that teach novices the merge sort algorithm. Our results show that sequential teaching of concepts with increasing complexity (a) has a beneficial effect on human comprehension and (b) leads to human re-discovery of divide-and-conquer problem-solving strategies, and (c) allows adaptations of human problem-solving strategy with better performance when machine-learned explanations are also presented.

QNLP in Practice: Running Compositional Models of Meaning on a Quantum Computer

Quantum Natural Language Processing (QNLP) deals with the design and implementation of NLP models intended to be run on quantum hardware. In this paper, we present results on the first NLP experiments conducted on Noisy Intermediate-Scale Quantum (NISQ) computers for datasets of size greater than 100 sentences. Exploiting the formal similarity of the compositional model of meaning by Coecke, Sadrzadeh, and Clark (2010) with quantum theory, we create representations for sentences that have a natural mapping to quantum circuits. We use these representations to implement and successfully train NLP models that solve simple sentence classification tasks on quantum hardware. We conduct quantum simulations that compare the syntax-sensitive model of Coecke et al. with two baselines that use less or no syntax; specifically, we implement the quantum analogues of a “bag-of-words” model, where syntax is not taken into account at all, and of a word-sequence model, where only word order is respected. We demonstrate that all models converge smoothly both in simulations and when run on quantum hardware, and that the results are the expected ones based on the nature of the tasks and the datasets used. Another important goal of this paper is to describe in a way accessible to AI and NLP researchers the main principles, process and challenges of experiments on quantum hardware. Our aim in doing this is to take the first small steps in this unexplored research territory and pave the way for practical Quantum Natural Language Processing.

Scalable Nanophotonic-Electronic Spiking Neural Networks

Spiking neural networks (SNN) provide a new computational paradigm capable of highly parallelized, real-time processing. Photonic devices are ideal for the design of high-bandwidth, parallel architectures matching the SNN computational paradigm. Furthermore, the co-integration of CMOS and photonic elements combineslow-loss photonic devices with analog electronics for greater flexibility of nonlinear computational elements. We designed and simulated an optoelectronic spiking neuron circuit on a monolithic silicon photonics (SiPh) process that replicates useful spiking behaviors beyond the leaky integrate-and-fire (LIF). Additionally, we explored two learning algorithms with the potential for on-chip learning using Mach-Zehnder Interferometric (MZI) meshes as synaptic interconnects. A variation of Random Backpropagation (RPB) was experimentally demonstrated on-chip and matched the performance of a standard linear regression on a simple classification task. In addition, we applied the Contrastive Hebbian Learning (CHL) rule to a simulated neural network composed of MZI meshes for a random input-output mapping task. The CHL-trained MZI network performed better than random guessing but did not match the performance of the ideal neural network (without the constraints imposed by the MZI meshes). Through these efforts, we demonstrate that co-integrated CMOS and SiPh technologies are well-suited to the design of scalable SNN computing architectures.

Weakly Supervised Audio-Visual Violence Detection

Violence detection in video is very promising in practical applications due to the emergence of massive videos in recent years. Most previous works define violence detection as a simple video classification task and use the single modality of small-scale datasets, e.g., visual signal. However, such solutions are undersupplied. To mitigate this problem, we first introduce a large-scale audio-visual XD-Violence dataset, which is composed of 4754 untrimmed videos with a total duration of 217 hours. Then we introduce two complementary tasks, i.e., coarse-grained violent frame detection and fine-grained violent event detection, to advance the simple violence video classification to frame-level violent event localization, which aims to accurately locate the violent events on untrimmed videos. We propose a novel network that takes as input audio-visual information and contains three parallel branches to capture different relationships among video snippets and further integrate features, where similarity branch and proximity branch capture long-range dependencies using similarity prior and proximity prior respectively, and score branch dynamically captures the closeness of predicted score. Our method outperforms other state-of-the-art methods on two benchmarks. Moreover, experiment results also show the positive effect of audio-visual input and relationship modeling. The dataset and code are released in https://roc-ng.github.io/XD-Violence/.

Improving Pretrained Language Model Fine-Tuning With Noise Stability Regularization.

The advent of large-scale pretrained language models (PLMs) has contributed greatly to the progress in natural language processing (NLP). Despite its recent success and wide adoption, fine-tuning a PLM often suffers from overfitting, which leads to poor generalizability due to the extremely high complexity of the model and the limited training samples from downstream tasks. To address this problem, we propose a novel and effective fine-tuning framework, named layerwise noise stability regularization (LNSR). Specifically, our method perturbs the input of neural networks with the standard Gaussian or in-manifold noise in the representation space and regularizes each layer's output of the language model. We provide theoretical and experimental analyses to prove the effectiveness of our method. The empirical results show that our proposed method outperforms several state-of-the-art algorithms, such as [Formula: see text] norm and start point (L2-SP), Mixout, FreeLB, and smoothness inducing adversarial regularization and Bregman proximal point optimization (SMART). In addition to evaluating the proposed method on relatively simple text classification tasks, similar to the prior works, we further evaluate the effectiveness of our method on more challenging question-answering (QA) tasks. These tasks present a higher level of difficulty, and they provide a larger amount of training examples for tuning a well-generalized model. Furthermore, the empirical results indicate that our proposed method can improve the ability of language models to domain generalization.

Binding time: Investigations on the integration of visual stimulus duration.

The perception of and reaction to objects creates bindings of (object) features and responses, also called event files. In this context, time is a so far understudied feature. We conducted four experiments to investigate whether the duration of visual stimuli is also integrated into such event files. Experiments 1, 2, and 4 used a simple colour classification task and in Experiment 3 the location of a stimulus had to be classified. In all Experiments, the presentation duration of the stimuli (coloured circles) was either short (20 ms) or long (300 ms). We expected partial repetition costs as an indicator of binding. That is, performance should be better when both colour (Experiment 3: location) and duration repeat or alternate relative to partial repetitions. Results showed no partial repetition costs in Experiments 1 and 3, indicating no integration of duration into visual event files. Experiments 2 and 4 revealed partial repetition costs. Performance was better when Colour and Duration repeated compared with a partial repetition. What distinguishes the latter two experiments from the former is that the coloured stimuli could change their presentation location. The results of all four experiments show a pattern that duration can be integrated into visual event files depending on two criteria: The experimental context holds the possibility of a location change of the target stimulus (Experiments 2 and 4) and the location itself is not response relevant (Experiment 3). The role of location changes for the integration of temporal stimulus features into visual event files is discussed.

Spiking Neural Network Based on Multi-Scale Saliency Fusion for Breast Cancer Detection

Deep neural networks have been successfully applied in the field of image recognition and object detection, and the recognition results are close to or even superior to those from human beings. A deep neural network takes the activation function as the basic unit. It is inferior to the spiking neural network, which takes the spiking neuron model as the basic unit in the aspect of biological interpretability. The spiking neural network is considered as the third-generation artificial neural network, which is event-driven and has low power consumption. It modulates the process of nerve cells from receiving a stimulus to firing spikes. However, it is difficult to train spiking neural network directly due to the non-differentiable spiking neurons. In particular, it is impossible to train a spiking neural network using the back-propagation algorithm directly. Therefore, the application scenarios of spiking neural network are not as extensive as deep neural network, and a spiking neural network is mostly used in simple image classification tasks. This paper proposed a spiking neural network method for the field of object detection based on medical images using the method of converting a deep neural network to spiking neural network. The detection framework relies on the YOLO structure and uses the feature pyramid structure to obtain the multi-scale features of the image. By fusing the high resolution of low-level features and the strong semantic information of high-level features, the detection precision of the network is improved. The proposed method is applied to detect the location and classification of breast lesions with ultrasound and X-ray datasets, and the results are 90.67% and 92.81%, respectively.

Dynamic noise filtering for multi-class classification of beehive audio data

Honeybees are the most specialized insect pollinators and are critical not only for honey production but, also, for keeping the environmental balance by pollinating the flowers of a wide variety of crops.Recording and analyzing bee sounds became a fundamental part of recent initiatives in the development of so-called smart hives. The majority of researches on beehive sound analytics are focusing on swarming detection, a relatively simple binary classification task (due to the obvious difference in the sound of a swarming and a non-swarming bee colony) where machine learning models achieve good performance even when trained on small data.However, in the case of more complex tasks of beehive sound analytics, even modern machine learning approaches perform poorly. First, training such models would need a large dataset but, according to our knowledge, there is no publicly available large-scale beehive audio data. Second, due to the specifics of beehive sounds, efficient noise filtering methods would be required, however, we could not find a noise filtering method that would increase the performance of machine learning models substantially.In this paper, we propose a dynamic noise filtering method applicable on spectrograms (image representations of audio data) which is superior to the most popular image noise filtering baselines. Further, we introduce a multi-class classification task of bee sounds and a large-scale dataset consisting of 10.000 beehive audio recordings. Finally, we provide the results of a large-scale experiment involving various combinations of audio feature extraction and noise filtering methods together with various deep learning models. We believe that the contributions of this paper will facilitate further research in the area of (beehive) sound analytics.

Binary image classification using collective optical modes of an array of nanolasers

Recent advancements in nanolaser design and manufacturing open up unprecedented perspectives in terms of high integration densities and ultra-low power consumption, making these devices ideal for high-performance optical computing systems. In this work, we exploit the symmetry properties of the collective modes of a nanolaser array for a simple binary classification task of small digit images. The implementation is based on a 8 × 8 nanolaser array and relies on the activation of a collective optical mode of the array—the so-called “zero-mode”—under spatially modulated pump patterns.