Binary Network Research Articles

SBNN: A Searched Binary Neural Network for SAR Ship Classification

The synthetic aperture radar (SAR) for ocean surveillance missions requires low latency and light weight inference. This paper proposes a novel small-size Searched Binary Network (SBNN), with network architecture search (NAS) for ship classification with SAR. In SBNN, convolution operations are modified by binarization technologies. Both input feature maps and weights are quantized into 1-bit in most of the convolution computation, which significantly decreases the overall computational complexity. In addition, we propose a patch shift processing, which can adjust feature maps with learnable parameters at spatial level. This process enhances the performance by reducing the information irrelevant to the targets. Experimental results on the OpenSARShip dataset show the proposed SBNN outperforms both binary neural networks from computer vision and CNN-based SAR ship classification methods. In particular, SBNN shows a great advantage in computational complexity.

An algorithm for reconstructing level-2 phylogenetic networks from trinets

Evolutionary histories for species that cross with one another or exchange genetic material can be represented by leaf-labelled, directed graphs called phylogenetic networks. A major challenge in the burgeoning area of phylogenetic networks is to develop algorithms for building such networks by amalgamating small networks into a single large network. The level of a phylogenetic network is a measure of its deviation from being a tree; the higher the level of a network, the less treelike it becomes. Various algorithms have been developed for building level-1 networks from small networks. However, level-1 networks may not be able to capture the complexity of some data sets. In this paper, we present a polynomial-time algorithm for constructing a rooted binary level-2 phylogenetic network from a collection of 3-leaf networks or trinets. Moreover, we prove that the algorithm will correctly reconstruct such a network if it is given all of the trinets in the network as input. The algorithm runs in time O(t⋅n+n4) with t the number of input trinets and n the number of leaves. We also show that there is a fundamental obstruction to constructing level-3 networks from trinets, and so new approaches will need to be developed for constructing level-3 and higher level-networks.

Binarized Aggregated Network With Quantization: Flexible Deep Learning Deployment for CSI Feedback in Massive MIMO Systems

Massive multiple-input multiple-output (MIMO) is one of the key techniques to achieve better spectrum and energy efficiency in 5G system. The channel state information (CSI) needs to be fed back from the user equipment to the base station in frequency division duplexing (FDD) mode. However, the overhead of the direct feedback is unacceptable due to the large antenna array in massive MIMO system. Recently, deep learning is widely adopted to the compressed CSI feedback task and proved to be effective. In this paper, a novel network named aggregated channel reconstruction network (ACRNet) is designed to boost the feedback performance with network aggregation and parametric rectified linear unit (PReLU) activation. The practical deployment of the feedback network in the communication system is also considered. Specifically, the elastic feedback scheme is proposed to flexibly adapt the network to meet different resource limitations. Besides, the network binarization technique is combined with the feature quantization for lightweight and practical deployment. Experiments show that the proposed ACRNet outperforms loads of previous state-of-the-art networks, providing a neat feedback solution with high performance, low cost and impressive flexibility.

Equal Bits: Enforcing Equally Distributed Binary Network Weights

Binary networks are extremely efficient as they use only two symbols to define the network: {+1, −1}. One can make the prior distribution of these symbols a design choice. The recent IR-Net of Qin et al. argues that imposing a Bernoulli distribution with equal priors (equal bit ratios) over the binary weights leads to maximum entropy and thus minimizes information loss. However, prior work cannot precisely control the binary weight distribution during training, and therefore cannot guarantee maximum entropy. Here, we show that quantizing using optimal transport can guarantee any bit ratio, including equal ratios. We investigate experimentally that equal bit ratios are indeed preferable and show that our method leads to optimization benefits. We show that our quantization method is effective when compared to state-of-the-art binarization methods, even when using binary weight pruning. Our code is available at https://github.com/liyunqianggyn/Equal-Bits-BNN.

Identification of circulating lncRNA in chronic kidney disease based on bioinformatics analysis.

Chronic kidney disease (CKD) is a high mortality disease and generally remains asymptomatic in the early stages. Long non-coding RNA (lncRNA) is defined as a non-protein-coding transcript more than 200 nucleotides which participate in numerous biological processes and have been identified as novel diagnostic markers for many diseases. Detection of circulating lncRNAs is a rapidly evolving, new area of molecular diagnosis. The purpose of our research was to identify circulating lncRNA expression profiles and possible molecular mechanisms involved in CKD. Blood samples were obtained from patients with CKD and healthy volunteers, and high-throughput sequencing was performed to identify differentially expressed (DE) lncRNAs and mRNAs. DE lncRNAs and mRNAs in peripheral blood mononuclear cells (PBMCs) were confirmed by quantitative reverse transcription polymerase chain reaction (qRT-PCR) to ensure the reliability and validity of RNA-seq data. Bioinformatics analysis was used to obtain biological functions and key pathways related to the pathogenesis of CKD. The interaction and co-expression functional networks for DE lncRNAs and mRNAs were also constructed. Our data showed that of the 425 DE lncRNAs detected, 196 lncRNAs were upregulated, while that of 229 lncRNAs were downregulated. A total of 433 DE mRNAs were identified in patients with CKD compared to healthy individuals. GO analysis revealed that DE lncRNAs were highly correlated with binding and pathway regulation. KEGG analysis suggested that DE lncRNAs were obviously enriched in regulatory pathways, such as antigen processing and presentation. We successfully constructed a potential DE lncRNA-mRNA co-expression network and analyzed the target genes of DE lncRNAs to predict cis- and trans-regulation in CKD. 100 lncRNAs that corresponded to 14 transcription factors (TFs) were identified in the TF-lncRNA binary network. Our findings on the lncRNA expression profiles and functional networks may help to interpret the possible molecular mechanisms implied in the pathogenesis of CKD; the results demonstrated that lncRNAs could potentially to be used as diagnostic biomarkers in CKD.

PB-GCN: Progressive binary graph convolutional networks for skeleton-based action recognition

Skeleton-based action recognition is an essential yet challenging visual task, whose accuracy has been remarkably improved due to the successful application of graph convolutional networks (GCNs). However, high computation cost and memory usage hinder their deployment on resource-constrained environment. To deal with the issue, in this paper, we introduce two novel progressive binary graph convolutional network for skeleton-based action recognition PB-GCN and PB-GCN*, which can obtain significant speed-up and memory saving. In PB-GCN, the filters are binarized, and in PB-GCN*, both filters and activations are binary. Specifically, we propose a progressive optimization, i.e., employing ternary models as the initialization of binary GCNs (BGCN) to improve the representational capability of binary models. Moreover, the center loss is exploited to improve the training procedure for better performance. Experimental results on two public benchmarks (i.e., Skeleton-Kinetics and NTU RGB + D) demonstrate that the accuracy of the proposed PB-GCN and PB-GCN* are comparable to their full-precision counterparts and outperforms the state-of-the-art methods, such as BWN, XNOR-Net, and Bi-Real Net.

Robust correlation filter tracking based on response map analysis network

Conventional discriminative-correlation-filter-based (DCF-based) visual tracking methods always update model at a fixed frequency and learning rate. Without evaluating the tracking confidence scores, the response map generated by filter is the only evidence for locating. Thus, most of the existing DCF-based methods suffer from the model contamination caused by drastic appearance variations, which leads to tracking drift even failure. And excessively frequent update will increase the computational redundancy and risk of over-fitting. In addition, these methods cannot recover target from heavy occlusion neither. Based on the observation that the shape of response maps reflects the matching degree between filter and target, we design and train a small-scale binary network named as response map analysis network (RAN) to evaluate the confidence scores of filters. Further, we propose to learn multiple filters to exploit different kinds of features, and adaptively adjust the update parameters according to the corresponding confidence scores. Moreover, we build a simple occlusion event model to detect heavy occlusion and recover target. Extensive experimental results validate the effectiveness of RAN and demonstrate that the proposed tracker performs favorably against other state-of-the-art (SOTA) DCF-based trackers in terms of precision, overlap rate and efficiency.

Integrating nocturnal and diurnal interactions in a Neotropical pollination network

Abstract Plants establish pollination interactions with different groups of animals, including nocturnal ones that establish interactions with economically valuable and culturally important crops, as well as wild plants of conservation concern. Despite the considerable number of studies addressing the structure and dynamic of pollination networks, nocturnal interactions have been relatively overlooked. Using a multilayer network approach and considering diurnal and nocturnal interactions, we aimed to understand how interactions at different periods of the day are integrated and contribute to the network structural pattern. We also aimed to highlight how multilayer networks may give a more nuanced assessment of species importance across layers. We assembled a pollination network of an intensively studied Neotropical area by standardizing interaction data from 16 previous studies into a presence/absence (binary) network. Then, we used a multilayer network approach to evaluate the network modularity and plant species' roles in these different temporal layers. Plants were classified as nocturnal or diurnal according to the onset of floral opening and pollinators were classified according to their foraging period. The network consisted of 178 pollinator species and 158 plant species, with 870 links. Among plant species, 135 species have diurnal floral opening while 23 species are nocturnal. The multilayer network was significantly modular, and these modules differed in the composition of pollinator groups (e.g., hawk moths, bats, bees, hummingbirds), as well as of diurnal and nocturnal plants. We show that diurnal and nocturnal interactions are organized into interconnected modules in the multilayer network. Nocturnal plants had higher values of versatility and multidegree than diurnal plants, due to their role in connecting the two temporal layers. Synthesis. Our study highlights the importance of integrating different pollination systems to understand the importance of distinct components that structure pollination networks. We also illustrate the value of tapping into existing information, particularly species interaction data, from well‐studied biodiversity hotspot areas, to gain a better understanding of how communities are structured. Finally, despite the relative scarcity of nocturnal pollination network studies, we showed nocturnal plants, which often make complementary use of diurnal pollinators, to be important in connecting the temporal layers.

Using tensor network states for multi-particle Brownian ratchets.

The study of Brownian ratchets has taught how time-periodic driving supports a time-periodic steady state that generates nonequilibrium transport. When a single particle is transported in one dimension, it is possible to rationalize the current in terms of the potential, but experimental efforts have ventured beyond that single-body case to systems with many interacting carriers. Working with a lattice model of volume-excluding particles in one dimension, we analyze the impact of interactions on a flashing ratchet's current. To surmount the many-body problem, we employ the time-dependent variational principle applied to binary tree tensor networks. Rather than propagating individual trajectories, the tensor network approach propagates a distribution over many-body configurations via a controllable variational approximation. The calculations, which reproduce Gillespie trajectory sampling, identify and explain a shift in the frequency of maximum current to higher driving frequency as the lattice occupancy increases.

Programming and training rate-independent chemical reaction networks

Embedding computation in biochemical environments incompatible with traditional electronics is expected to have a wide-ranging impact in synthetic biology, medicine, nanofabrication, and other fields. Natural biochemical systems are typically modeled by chemical reaction networks (CRNs) which can also be used as a specification language for synthetic chemical computation. In this paper, we identify a syntactically checkable class of CRNs called noncompetitive (NC) whose equilibria are absolutely robust to reaction rates and kinetic rate law, because their behavior is captured solely by their stoichiometric structure. In spite of the inherently parallel nature of chemistry, the robustness property allows for programming as if each reaction applies sequentially. We also present a technique to program NC-CRNs using well-founded deep learning methods, showing a translation procedure from rectified linear unit (ReLU) neural networks to NC-CRNs. In the case of binary weight ReLU networks, our translation procedure is surprisingly tight in the sense that a single bimolecular reaction corresponds to a single ReLU node and vice versa. This compactness argues that neural networks may be a fitting paradigm for programming rate-independent chemical computation. As proof of principle, we demonstrate our scheme with numerical simulations of CRNs translated from neural networks trained on traditional machine learning datasets, as well as tasks better aligned with potential biological applications including virus detection and spatial pattern formation.

EfficientNet embedded with spatial attention for recognition of multi-label fundus disease from color fundus photographs

Color fundus photographs enable the observation of numerous critical biomarkers and early-onset lesions associated with illnesses. Due to its non-invasive and cost-effective nature, this approach can be used for large-scale screening of fundus disorders. In recent years, most applications of data-driven deep learning approaches to fundus illnesses have been based on color fundus photographs. However, current screening methods for fundus illnesses are mainly focused on identifying particular diseases. The majority of published models for multi-disease recognition are ensembles of several binary classification networks, and an ensemble network consumes more resources than a single network. Additionally, many fundus illness recognition models make direct use of established networks for natural picture processing without performing structural optimization, which may reduce disease classification accuracy. To address these issues, we optimized the network structure to build a single multi-label fundus disease recognition model. Specifically, we began by selecting EfficientNet-B4 as a backbone network, then modified the output layer to create a multi-label recognition model. Based on this model, the spatial attention structure was extended to improve the network’s feature extraction capability. The focal loss was used to increase classification accuracy by mitigating the training effects of data imbalance. Our proposed model significantly improves recognition performance on a publicly available dataset compared to other baseline networks. When spatial attention and focal loss function are introduced into the backbone model, the baseline values of all relevant evaluation indicators are improved without significant increase in computational cost. Additionally, this article provides two error correction strategies for multi-label classification issues: mutual exclusion and super label space. When a more robust error correction strategy is employed with the model, F1 increases by 2.86% in comparison to the original performance of EfficientNet-B4.

Identity Recognition in Sanitary Facilities Using Invisible Electrocardiography.

This article proposes a new method of identity recognition in sanitary facilities based on electrocardiography (ECG) signals. Our team previously proposed a novel approach of invisible ECG at the thighs using polymeric electrodes, leading to the creation of a proof-of-concept system integrated into a toilet seat. In this work, a biometrics pipeline was devised, which tested four different classifiers, varying the population from 2 to 17 subjects and simulating a residential environment. However, for this approach to be industrially viable, further optimization is required, particularly regarding electrode materials that are compatible with industrial processes. As such, we also explore the use of a conductive silicone material as electrodes, aiming at the industrial-scale production of a toilet seat capable of recording ECG data, without the need for body-worn devices. A desirable aspect when using such a system is matching the recorded data with the monitored user, ideally using a minimal sensor set, further reinforcing the relevance of user identification through ECG signals collected at the thighs. Our approach was evaluated against a reference device for a population of 17 healthy and pathological individuals, covering a wide age range (24–70 years). With the silicone composite, we were able to acquire signals in 100% of the sessions, with a mean heart rate deviation between a reference system and our experimental device of 2.82 ± 1.99 beats per minute (BPM). In terms of ECG waveform morphology, the best cases showed a Pearson correlation coefficient of 0.91 ± 0.06. For biometric detection, the best classifier was the Binary Convolutional Neural Network (BCNN), with an accuracy of 100% for a population of up to four individuals.

Detecting mesoscale structures by surprise

The importance of identifying mesoscale structures in complex networks can be hardly overestimated. So far, much attention has been devoted to detect modular and bimodular structures on binary networks. This effort has led to the definition of a framework based upon the score function called ‘surprise’, i.e. a p-value that can be assigned to any given partition of nodes. Hereby, we make a step further and extend the entire framework to the weighted case: six variants of surprise, induced by just as many variants of the hypergeometric distribution, are, thus, considered. As a result, a general, statistically grounded approach for detecting mesoscale network structures via a unified, suprise-based framework is presented. To illustrate its performances, both synthetic benchmarks and real-world configurations are considered. Moreover, we attach to the paper a Python code implementing all variants of surprise discussed in the present manuscript.

Research on an Optimal Path Planning Method Based on A* Algorithm for Multi-View Recognition

In order to obtain the optimal perspectives of the recognition target, this paper combines the motion path of the manipulator arm and camera. A path planning method to find the optimal perspectives based on an A* algorithm is proposed. The quality of perspectives is represented by means of multi-view recognition. A binary multi-view 2D kernel principal component analysis network (BM2DKPCANet) is built to extract features. The multi-view angles classifier based on BM2DKPCANet + Softmax is established, which outputs category posterior probability to represent the perspective recognition performance function. The path planning problem is transformed into a multi-objective optimization problem by taking the optimal view recognition and the shortest path distance as the objective functions. In order to reduce the calculation, the multi-objective optimization problem is transformed into a single optimization problem by fusing the objective functions based on the established perspective observation directed graph model. An A* algorithm is used to solve the single source shortest path problem of the fused directed graph. The path planning experiments with different numbers of view angles and different starting points demonstrate that the method can guide the camera to reach the viewpoint with higher recognition accuracy and complete the optimal observation path planning.

Memristive KDG-BNN: Memristive binary neural networks trained via knowledge distillation and generative adversarial networks

With the increasing requirements for the combination of software and hardware, network compression and hardware deployment have become hot research topics. In network compression, binary neural networks (BNNs) are widely applied in artificial intelligence chips because of memory saving, high computational efficiency, and hardware friendliness. However, there is a performance gap between BNNs and full-precision neural networks (FNNs). This paper proposes a BNN training framework called KDG-BNN, consisting of three modules: a full-precision network, a 1-bit binary network, and a discriminator. The full-precision network guides the 1-bit binary network to train through distillation loss in this framework. Meanwhile, the 1-bit binary network acts as a generator and conducts adversarial training with the discriminator. By simultaneously optimizing the adversarial loss and distillation loss, the 1-bit binary network can learn the feature distribution of the full-precision network more accurately. Then, the generative adversarial network (GAN) is replaced by Wasserstein GAN with gradient penalty (WGAN-GP) to deal with gradient disappearance, and KDG-BNN is developed into KDWG-BNN. Experiments show that AdamBNN trained with KDWG-BNN can achieve 85.89% and 70.7% accuracy on CIFAR-10 and ImageNet, respectively, exceeding 0.76% on CIFAR-10 and 0.2% on ImageNet. The memristor has many features for hardware deployment, such as memory functions, continuous input and output, nanoscale size, etc., making it an ideal device for deploying neural networks. Therefore, this paper further proposes a memristor-based KDG-BNN implementation scheme by levering the merits of memristors and the lightweight BNNs in the hope of realizing and promoting end-side intelligent applications.

Categorization of Common Pigmented Skin Lesions (CPSL) using Multi-Deep Features and Support Vector Machine.

The skin is the main organ. It is approximately 8 pounds for the average adult. Our skin is a truly wonderful organ. It isolates us and shields our bodies from hazards. However, the skin is also vulnerable to damage and distracted from its original appearance: brown, black, or blue, or combinations of those colors, known as pigmented skin lesions. These common pigmented skin lesions (CPSL) are the leading factor of skin cancer, or can say these are the primary causes of skin cancer. In the healthcare sector, the categorization of CPSL is the main problem because of inaccurate outputs, overfitting, and higher computational costs. Hence, we proposed a classification model based on multi-deep feature and support vector machine (SVM) for the classification of CPSL. The proposed system comprises two phases: First, evaluate the 11 CNN model's performance in the deep feature extraction approach with SVM, and then, concatenate the top performed three CNN model's deep features and with the help of SVM to categorize the CPSL. In the second step, 8192 and 12,288 features are obtained by combining binary and triple networks of 4096 features from the top performed CNN model. These features are also given to the SVM classifiers. The SVM results are also evaluated with principal component analysis (PCA) algorithm to the combined feature of 8192 and 12,288. The highest results are obtained with 12,288 features. The experimentation results, the combination of the deep feature of Alexnet, VGG16 and VGG19, achieved the highest accuracy of 91.7% using SVM classifier. As a result, the results show that the proposed methods are a useful tool for CPSL classification.

From Coherence to Multivariate Causal Estimators of EEG Connectivity.

The paper concerns the development of methods of EEG functional connectivity estimation including short overview of the currently applied measures describing their advantages and flaws. Linear and non-linear, bivariate and multivariate methods are confronted. The performance of different connectivity measures in respect of robustness to noise, common drive effect and volume conduction is considered providing a guidance towards future developments in the field, which involve evaluation not only functional, but also effective (causal) connectivity. The time-varying connectivity measure making possible estimation of dynamical information processing in brain is presented. The methods of post-processing of connectivity results are considered involving application of advanced graph analysis taking into account community structure of networks and providing hierarchy of networks rather than the single, binary networks currently used.

Analysis of the Football Transfer Market Network

Using publicly available data from the football database transfermarkt.co.uk, it is possible to construct a trade network between football clubs. This work regards the network of the flow of transfer fees between European top league clubs from eight countries between 1992 and 2020 to analyse the network of each year’s transfer market. With the transfer fees as weights, the market can be represented as a weighted network in addition to the classic binary network approach. This opens up the possibility to study various topological quantities of the network, such as the degree and disparity distributions, the small-world property and different clustering measures. This article shows that these quantities stayed rather constant during the almost three decades of transfer market activity, even despite massive changes in the overall market volume.

Cost Effectiveness of 18F-FET PET for Early Treatment Response Assessment in Glioma Patients After Adjuvant Temozolomide Chemotherapy.

In light of increasing health-care costs, higher medical expenses should be justified socioeconomically. Therefore, we calculated the effectiveness and cost effectiveness of PET using the radiolabeled amino acid O-(2-18F-fluoroethyl)-l-tyrosine (18F-FET) compared with conventional MRI for early identification of responders to adjuvant temozolomide chemotherapy. A recently published study in isocitrate dehydrogenase wild-type glioma patients suggested that 18F-FET PET parameter changes predicted a significantly longer survival already after 2 cycles whereas MRI changes were not significant. Methods: To determine the effectiveness and cost effectiveness of serial 18F-FET PET imaging, we analyzed published clinical data and calculated the associated costs from the perspective of the German Statutory Health Insurance system. Based on a decision-tree model, the effectiveness of 18F-FET PET and MRI was calculated-that is, the probability to correctly identify a responder as defined by an overall survival of at least 15 mo. To determine the cost effectiveness, the incremental cost effectiveness ratio (ICER) was calculated-that is, the cost for each additionally identified responder by 18F-FET PET who would have remained undetected by MRI. The robustness of the results was tested by deterministic and probabilistic Monte Carlo sensitivity analyses. Results: Compared with MRI, 18F-FET PET increased the rate of correctly identified responders to chemotherapy by 26%; thus, 4 patients needed to be examined by 18F-FET PET to identify 1 additional responder. Considering the respective costs for serial 18F-FET PET and MRI, the ICER resulted in €4,396.83 for each additional correctly identified responder by 18F-FET PET. Sensitivity analyses confirmed the robustness of the results. Conclusion: In contrast to conventional MRI, the model suggests that 18F-FET PET is cost-effective in terms of ICER values. Considering the high cost of temozolomide, the integration of 18F-FET PET has the potential to avoid premature chemotherapy discontinuation at reasonable cost.

Learning to autofocus in whole slide imaging via physics-guided deep cascade networks.

Whole slide imaging (WSI), is an essential technology for digital pathology, the performance of which is primarily affected by the autofocusing process. Conventional autofocusing methods either are time-consuming or require additional hardware and thus are not compatible with the current WSI systems. In this paper, we propose an effective learning-based method for autofocusing in WSI, which can realize accurate autofocusing at high speed as well as without any optical hardware modifications. Our method is inspired by an observation that sample images captured by WSI have distinctive characteristics with respect to positive / negative defocus offsets, due to the asymmetry effect of optical aberrations. Based on this physical knowledge, we develop novel deep cascade networks to enhance autofocusing quality. Specifically, to handle the effect of optical aberrations, a binary classification network is tailored to distinguish sample images with positive / negative defocus. As such, samples within the same category share similar characteristics. It facilitates the followed refocusing network, which is designed to learn the mapping between the defocus image and defocus distance. Experimental results demonstrate that our method achieves superior autofocusing performance to other related methods.