Unitary Network Research Articles

An improved log-cosine variation slime mold - simplified gated recurrent neural network for the high-precision state of charge estimation of lithium-ion batteries

Accurate estimation of SOC is crucial for the safe and efficient utilization of electric vehicles, portable devices and renewable energy systems. Neural networks have become a powerful tool in the field of SOC estimation, but they have a complex structure, difficult parameter tuning, and high performance requirements. For this reason, this paper proposes a simplified gated recurrent neural network (SGRN) that greatly simplifies the unitary network structure and parameters. A Nadam gradient optimizer is introduced to update the network weight parameters, and the network hyperparameters are optimized using the log-cosine variant slime mold algorithm (LVSM). The algorithm is validated under HPPC, BBDST, and DST operating conditions under 25 °C, 10 °C, and − 5 °C. The results show that the LVSM-SGRN algorithm has higher accuracy, lower performance consumption, and better temperature adaptability than the conventional LSTM algorithm as well as the EKF algorithm. This contributes to the integration and large-scale industrialization of the algorithm, and comprehensively improves the safety of lithium-ion batteries and accuracy of SOC estimation.

Combinatorial CNN-Transformer Learning with Manifold Constraints for Semi-supervised Medical Image Segmentation

Semi-supervised learning (SSL), as one of the dominant methods, aims at leveraging the unlabeled data to deal with the annotation dilemma of supervised learning, which has attracted much attentions in the medical image segmentation. Most of the existing approaches leverage a unitary network by convolutional neural networks (CNNs) with compulsory consistency of the predictions through small perturbations applied to inputs or models. The penalties of such a learning paradigm are that (1) CNN-based models place severe limitations on global learning; (2) rich and diverse class-level distributions are inhibited. In this paper, we present a novel CNN-Transformer learning framework in the manifold space for semi-supervised medical image segmentation. First, at intra-student level, we propose a novel class-wise consistency loss to facilitate the learning of both discriminative and compact target feature representations. Then, at inter-student level, we align the CNN and Transformer features using a prototype-based optimal transport method. Extensive experiments show that our method outperforms previous state-of-the-art methods on three public medical image segmentation benchmarks.

Design of a photonic unitary neural network based on MZI arrays

In recent years, optical neural networks have attracted widespread attention, due to their advantages of high speed, high parallelism, high bandwidth, and low power consumption. Photonic unitary neural network is a kind of neural networks that utilize the principles of unitary matrices and photonics to perform computations. In this paper, we design a photonic unitary neural network based on Mach–Zehnder interferometer arrays. The results show that the network has a good performance on both triangular and circular binary classification datasets, where most of the data points are correctly classified. The accuracies achieve 97% and 95% for triangular and circular datasets, with the loss function values of 0.023 and 0.046, respectively.

Efficient training of unitary optical neural networks.

Deep learning has profoundly reshaped the technology landscape in numerous scientific areas and industrial sectors. This technology advancement is, nevertheless, confronted with severe bottlenecks in digital computing. Optical neural network presents a promising solution due to the ultra-high computing speed and energy efficiency. In this work, we present systematic study of unitary optical neural network (UONN) as an approach towards optical deep learning. Our results show that the UONN can be trained to high accuracy through special unitary gradient descent optimization, and the UONN is robust against physical imperfections and noises, hence it is more suitable for physical implementation than existing ONNs.

A feedforward unitary equivariant neural network

We devise a new type of feedforward neural network. It is equivariant with respect to the unitary group U(n). The input and output can be vectors in ℂn with arbitrary dimension n. No convolution layer is required in our implementation. We avoid errors due to truncated higher order terms in Fourier-like transformation. The implementation of each layer can be done efficiently using simple calculations. As a proof of concept, we have given empirical results on the prediction of the dynamics of atomic motion to demonstrate the practicality of our approach.

Decohering tensor network quantum machine learning models

Tensor network quantum machine learning (QML) models are promising applications on near-term quantum hardware. While decoherence of qubits is expected to decrease the performance of QML models, it is unclear to what extent the diminished performance can be compensated for by adding ancillas to the models and accordingly increasing the virtual bond dimension of the models. We investigate here the competition between decoherence and adding ancillas on the classification performance of two models, with an analysis of the decoherence effect from the perspective of regression. We present numerical evidence that the fully decohered unitary tree tensor network (TTN) with two ancillas performs at least as well as the non-decohered unitary TTN, suggesting that it is beneficial to add at least two ancillas to the unitary TTN regardless of the amount of decoherence may be consequently introduced.

University research networks as spaces for collaboration and social capital: The case of REUNI+D

Collaborative networks are considered a critical factor in developing the human species and any living being. Life would not be possible without the dynamic interaction between plants, animals, micro-organisms, non-living beings, and their environment. Nor would human life be possible without considering its physical, social, cultural, and technological environments as a unitary network living within a historical context. In science and academia, the importance of networks has grown exponentially. Based on the experience of a network of excellence with institutional recognition for more than ten years, the purpose of this article is to highlight the value and contributions of this way of working for the advancement of scientific knowledge and public policy. It argues the importance and the necessity of networks and the advantages of network thinking. To illustrate the importance of networks, we analyse the trajectory of the REUNI+D network of excellence, the contributions and tensions of this collaborative structure, the lengthy processes required for its constitution, the social capital that it provides to its members and some conclusions that show the challenges of research networks in universities.

An Efficient Multiresolution Network for Vehicle Reidentification

In general, vehicle images have varying resolutions due to vehicles’ movements and different camera settings. However, most existing vehicle re-identification models are single-resolution deep networks trained with pre-uniformly resizing vehicle images, which underestimates adverse effects of varying resolutions and leads to unsatisfactory performance. A straightforward solution for dealing with varying resolutions is to train multiple vehicle re-identification models. Each model is independently trained with images of a specific resolution. However, this straightforward solution requires significant overhead and ignores intrinsic associations among different resolution images. For that, an efficient multi-resolution network (EMRN) is proposed for vehicle re-identification in this paper. Firstly, EMRN embeds a newly-designed multi-resolution feature dimension uniform module (MR-FDUM) behind a traditional backbone network (i.e., ResNet-50). As a result, the whole model can extract fixed dimensional features from different resolution images so that it can be trained with one loss function of fixed dimensional parameters rather than training multiple models. Secondly, a multi-resolution image randomly feeding strategy is designed to train EMRN, making each mini-batch data of a random resolution during the training process. Consequently, EMRN can implicitly learn collaborative multi-resolution features via only a unitary deep network. Experiments on three large scale datasets, i.e., VeRi776, VehicleID, and VRIC, demonstrate that EMRN is superior to state-of-the-art vehicle re-identification methods.

Core-Centered Connection Abnormalities Associated with Pathological Features Mediate the Progress of Cognitive Impairments in Alzheimer’s Disease Spectrum Patients

Background: Abnormal default mode network (DMN) was associated with the progress of Alzheimer’s disease (AD). Rather than treat the DMN as a unitary network, it can be further divided into three subsystems with different functions. Objective: It remains unclear the interactions of DMN subsystems associated with the progress of cognitive impairments and AD pathological features. Methods: This study has recruited 187 participants, including test data and verification data. Firstly, an imaging analysis approach was utilized to investigate disease-related differences in the interactions of DMN subsystems in test data (n = 149), including 42 cognitively normal subjects, 43 early mild cognitive impairment (EMCI), 32 late mild cognitive impairment (LMCI), and 32 AD patients. Brain-behavior-pathological relationships regarding to the interactions among DMN subsystems were then further examined. Secondly, DMN subsystems abnormalities for classifying AD spectrum population in the independent verification data (n = 38). Results: This study found that the impaired cognition relates to disturbances in the interactions between DMN subsystems but preferentially in core subsystem, and the abnormal regulatory processes of core subsystem were significantly associated with the levels of cerebrospinal fluid Aβ and tau in AD-spectrum patients. Meantime, the nonlinear relationship between dysfunctional core subsystem and impaired cognition was observed as one progresses through the stages of MCI to AD. Importantly, this classification presented a higher sensitivity and specificity dependent on the core-centered connection abnormalities. Conclusion: The abnormal interaction patterns of DMN subsystems at an early stage of AD appeared and presented as core-centered connection abnormalities, which were the potential neuroimaging features for monitoring the development of AD.

Coupled Layer-wise Graph Convolution for Transportation Demand Prediction

Graph Convolutional Network (GCN) has been widely applied in transportation demand prediction due to its excellent ability to capture non-Euclidean spatial dependence among station-level or regional transportation demands. However, in most of the existing research, the graph convolution was implemented on a heuristically generated adjacency matrix, which could neither reflect the real spatial relationships of stations accurately, nor capture the multi-level spatial dependence of demands adaptively. To cope with the above problems, this paper provides a novel graph convolutional network for transportation demand prediction. Firstly, a novel graph convolution architecture is proposed, which has different adjacency matrices in different layers and all the adjacency matrices are self-learned during the training process. Secondly, a layer-wise coupling mechanism is provided, which associates the upper-level adjacency matrix with the lower-level one. It also reduces the scale of parameters in our model. Lastly, a unitary network is constructed to give the final prediction result by integrating the hidden spatial states with gated recurrent unit, which could capture the multi-level spatial dependence and temporal dynamics simultaneously. Experiments have been conducted on two real-world datasets, NYC Citi Bike and NYC Taxi, and the results demonstrate the superiority of our model over the state-of-the-art ones.

Unitary learning for diffractive deep neural network

Realization of deep learning with coherent diffraction has achieved remarkable development nowadays, which benefits on the fact that matrix multiplication can be optically executed in parallel with high band-with and low latency. Coherent optical field in the form of complex-valued entity can be manipulated into a task-oriented output. In this paper, a modulation mechanism is established by implementing the equivalence between a digital deep unitary neural network and optical coherent diffraction. We present a unitary learning avenue on diffractive deep neural network, meeting the physical unitary prior in coherent diffraction. The Unitary learning is a Backpropagation serving to unitary weights update through the gradient translation from Euclidean to Riemannian space. The temporal-space evolution characteristics in unitary learning are formulated and elucidated. And a compatible condition on how to select the nonlinear activation in complex space is unveiled, encapsulating the fundamental sigmoid, tanh and quasi-ReLu in complex space available in a single channel training. The performance of phase-ReLu is particularly emphasized. As a preliminary application, diffractive deep neural network with unitary learning is tentatively implemented on the 2D classification and verification tasks.

Assessing psychological well-being in early adulthood: Empirical evidence for the structure of daily well-being via network analysis

The transitional years of early adulthood, with key tasks of identity and intimacy development, engender both opportunities and risks for well-being. We propose that the conceptualization and measurement of early adults’ well-being can be improved through (a) an integration of ideas from developmental and psychological science on well-being, (b) the use of short, daily momentary assessments of well-being, and (c) a developmentally-informed examination of the structure of well-being within (and not just across) time. We developed a daily assessment of well-being based on the PERMA model (Seligman, 2011) and used network analysis to gain understanding from this data. Using Ecological Momentary Assessments, we assessed the five PERMA elements in college students’ daily life and their network properties. Consistent with the PERMA model, network analysis showed items clustered around theorized elements and formed a unitary network of well-being. Consistent with developmental theory, we found that having positive relationships and positive emotion were most central to early adults’ daily well-being.

Bipartite Functional Fractionation within the Default Network Supports Disparate Forms of Internally Oriented Cognition.

Our understanding about the functionality of the brain’s default network (DN) has significantly evolved over the past decade. Whereas traditional views define this network based on its suspension/disengagement during task-oriented behavior, contemporary accounts have characterized various situations wherein the DN actively contributes to task performance. However, it is unclear how different task-contexts drive componential regions of the DN to coalesce into a unitary network and fractionate into different subnetworks. Here we report a compendium of evidence that provides answers to these questions. Across multiple analyses, we found a striking dyadic structure within the DN in terms of the profiles of task-triggered fMRI response and effective connectivity, significantly extending beyond previous inferences based on meta-analysis and resting-state activities. In this dichotomy, one subset of DN regions prefers mental activities “interfacing with” perceptible events, while the other subset prefers activities “detached from” perceptible events. While both show a common “aversion” to sensory-motoric activities, their differential preferences manifest a subdivision that sheds light upon the taxonomy of the brain’s memory systems. This dichotomy is consistent with proposals of a macroscale gradational structure spanning across the cerebrum. This gradient increases its representational complexity, from primitive sensory-motoric processing, through lexical-semantic representations, to elaborated self-generated thoughts.

Complex Unitary Recurrent Neural Networks Using Scaled Cayley Transform

Recurrent neural networks (RNNs) have been successfully used on a wide range of sequential data problems. A well known difficulty in using RNNs is the vanishing or exploding gradient problem. Recently, there have been several different RNN architectures that try to mitigate this issue by maintaining an orthogonal or unitary recurrent weight matrix. One such architecture is the scaled Cayley orthogonal recurrent neural network (scoRNN) which parameterizes the orthogonal recurrent weight matrix through a scaled Cayley transform. This parametrization contains a diagonal scaling matrix consisting of positive or negative one entries that can not be optimized by gradient descent. Thus the scaling matrix is fixed before training and a hyperparameter is introduced to tune the matrix for each particular task. In this paper, we develop a unitary RNN architecture based on a complex scaled Cayley transform. Unlike the real orthogonal case, the transformation uses a diagonal scaling matrix consisting of entries on the complex unit circle which can be optimized using gradient descent and no longer requires the tuning of a hyperparameter. We also provide an analysis of a potential issue of the modReLU activiation function which is used in our work and several other unitary RNNs. In the experiments conducted, the scaled Cayley unitary recurrent neural network (scuRNN) achieves comparable or better results than scoRNN and other unitary RNNs without fixing the scaling matrix.

Machine learning by unitary tensor network of hierarchical tree structure

The resemblance between the methods used in quantum-many body physics and in machine learning has drawn considerable attention. In particular, tensor networks (TNs) and deep learning architectures bear striking similarities to the extent that TNs can be used for machine learning. Previous results used one-dimensional TNs in image recognition, showing limited scalability and flexibilities. In this work, we train two-dimensional hierarchical TNs to solve image recognition problems, using a training algorithm derived from the multi-scale entanglement renormalization ansatz. This approach introduces mathematical connections among quantum many-body physics, quantum information theory, and machine learning. While keeping the TN unitary in the training phase, TN states are defined, which encode classes of images into quantum many-body states. We study the quantum features of the TN states, including quantum entanglement and fidelity. We find these quantities could be properties that characterize the image classes, as well as the machine learning tasks.

Social Network Structures in African American Churches: Implications for Health Promotion Programs.

The prevalence of obesity among African Americans is higher than among other racial/ethnic groups. African American churches hold a central role in promoting health in the community; yet, church-based interventions have had limited impact on obesity. While recent studies have described the influence of social networks on health behaviors, obesity interventions informed by social network analysis have been limited. We conducted a cross-sectional study with 281 African American men and women from three churches in northeast urban cities in the USA. Data were collected on sociodemographic and clinical factors and anthropometrics. Using a social network survey applying a name generator, we computed network level metrics. Exponential random graph models (ERGM) were performed to examine whether each structural property found in the empirical (observed) networks occurred more frequently than expected by chance by comparing the empirical networks to the randomly simulated networks. Overall, church friendship networks were sparse (low density). We also found that while friendship ties were more reciprocated between dyads in church networks, and there were more tendencies for clustering of friendships (significant positive transitive closure) than in random networks, other characteristics such as expansiveness (number of actors with a great number of friends) did not differ from what would be expected by chance in random networks. These data suggest that interventions with African American churches should not assume a unitary network through which a single intervention should be used.

Quantum de Moivre–Laplace theorem for noninteracting indistinguishable particles in random networks

The asymptotic form of the average probability to count N indistinguishable identical particles in a small number of binned-together output ports of a M-port Haar-random unitary network, proposed recently in Shchesnovich (2017 Sci. Rep. 7 31) in a heuristic manner with some numerical confirmation, is presented with mathematical rigour and generalised to an arbitrary (mixed) input state of N indistinguishable particles. It is shown that, both in the classical (distinguishable particles) and quantum (indistinguishable particles) cases, the average counting probability into r output bins factorises into a product of counting probabilities into two bins. This fact relates the asymptotic Gaussian law to the de Moivre–Laplace theorem in the classical case and similarly in the quantum case where an analogous theorem can be stated. The results have applications to the setups where randomness plays a key role, such as the multiphoton propagation in disordered media and the scattershot Boson sampling.

Extracting Entanglement Geometry from Quantum States.

Tensor networks impose a notion of geometry on the entanglement of a quantum system. In some cases, this geometry is found to reproduce key properties of holographic dualities, and subsequently much work has focused on using tensor networks as tractable models for holographic dualities. Conventionally, the structure of the network-and hence the geometry-is largely fixed apriori by the choice of the tensor network ansatz. Here, we evade this restriction and describe an unbiased approach that allows us to extract the appropriate geometry from a given quantum state. We develop an algorithm that iteratively finds a unitary circuit that transforms a given quantum state into an unentangled product state. We then analyze the structure of the resulting unitary circuits. In the case of noninteracting, critical systems in one dimension, we recover signatures of scale invariance in the unitary network, and we show that appropriately defined geodesic paths between physical degrees of freedom exhibit known properties of a hyperbolic geometry.

Asymptotic Gaussian law for noninteracting indistinguishable particles in random networks

For N indistinguishable bosons or fermions impinged on a M-port Haar-random unitary network the average probability to count n1, … nr particles in a small number r ≪ N of binned-together output ports takes a Gaussian form as N ≫ 1. The discovered Gaussian asymptotic law is the well-known asymptotic law for distinguishable particles, governed by a multinomial distribution, modified by the quantum statistics with stronger effect for greater particle density N/M. Furthermore, it is shown that the same Gaussian law is the asymptotic form of the probability to count particles at the output bins of a fixed multiport with the averaging performed over all possible configurations of the particles in the input ports. In the limit N → ∞, the average counting probability for indistinguishable bosons, fermions, and distinguishable particles differs only at a non-vanishing particle density N/M and only for a singular binning K/M → 1, where K output ports belong to a single bin.

Learning Unitary Operators with Help From u(n)

A major challenge in the training of recurrent neural networks is the so-called vanishing or exploding gradient problem. The use of a norm-preserving transition operator can address this issue, but parametrization is challenging. In this work we focus on unitary operators and describe a parametrization using the Lie algebra u(n) associated with the Lie group U(n) of n × n unitary matrices. The exponential map provides a correspondence between these spaces, and allows us to define a unitary matrix using n2 real coefficients relative to a basis of the Lie algebra. The parametrization is closed under additive updates of these coefficients, and thus provides a simple space in which to do gradient descent. We demonstrate the effectiveness of this parametrization on the problem of learning arbitrary unitary operators, comparing to several baselines and outperforming a recently-proposed lower-dimensional parametrization. We additionally use our parametrization to generalize a recently-proposed unitary recurrent neural network to arbitrary unitary matrices, using it to solve standard long-memory tasks.