Two-layer Network Research Articles

The influence of cross-platform and spread sources on emotional information spreading in the 2E-SIR two-layer network

To explore the influence of cross-platform and spread sources on emotional information spreading, in this paper, a novel two-layer emotion-based susceptible–infected–recovered network (2E-SIR) is proposed and the state equations for emotional information spreading are constructed. Then, to verify the feasibility and effectiveness of the 2E-SIR network and the state equations, taking the negative and positive events on Sina Weibo and TikTok as examples and using two spread sources which are official media (OM) and key opinion leaders (KOL), we have the following conclusions. (1) Users are more willing to spread emotional information similar to the original events’, especially those posted by KOL. (2) The cumulative posting quantities simulated of all the emotional information almost always are consistent with their real data. (3) The outburst rate of positive emotional information, especially posted by OM, is faster than the negative’s. (4) Negative emotional information is more likely to trigger the emotional chaos. Further, parameter sensitivity analysis shows that the cumulative posting quantity on each platform is mainly affected by its own internal parameters. The more initially susceptible people, the more serious the chaos generated by the original information. In addition, increasing the average user circulation probability on the 2E-SIR network will slow down the outbreak rate of emotional information, and enhance the degree of emotion chaos.

Synaptic delay plasticity based on frequency-switched VCSELs for optical delay-weight spiking neural networks.

In this Letter, we propose an optical delay-weight spiking neural network (SNN) architecture constructed by cascaded frequency and intensity-switched vertical-cavity surface emitting lasers (VCSELs). The synaptic delay plasticity of frequency-switched VCSELs is deeply studied by numerical analysis and simulations. The principal factors related to the delay manipulation are investigated with the tunable spiking delay up to 60 ns. Moreover, a two-layer spiking neural network based on the delay-weight supervised learning algorithm is applied to a spiking sequence pattern training task and then a classification task of the Iris dataset. The proposed optical SNN provides a compact and cost-efficient solution for delay weighted computing architecture without considerations of extra programmable optical delay lines.

Nonlinear decision-making with enzymatic neural networks.

Artificial neural networks have revolutionized electronic computing. Similarly, molecular networks with neuromorphic architectures may enable molecular decision-making on a level comparable to gene regulatory networks1,2. Non-enzymatic networks could in principle support neuromorphic architectures, and seminal proofs-of-principle have been reported3,4. However, leakages (that is, the unwanted release of species), as well as issues with sensitivity, speed, preparation and the lack of strong nonlinear responses, make the composition of layers delicate, and molecular classifications equivalent to a multilayer neural network remain elusive (for example, the partitioning of a concentration space into regions that cannot be linearly separated). Here we introduce DNA-encoded enzymatic neurons with tuneable weights and biases, and which are assembled in multilayer architectures to classify nonlinearly separable regions. We first leverage the sharp decision margin of a neuron to compute various majority functions on 10 bits. We then compose neurons into a two-layer network and synthetize a parametric family of rectangular functions on a microRNA input. Finally, we connect neural and logical computations into a hybrid circuit that recursively partitions a concentration plane according to a decision tree in cell-sized droplets. This computational power and extreme miniaturization open avenues to query and manage molecular systems with complex contents, such as liquid biopsies or DNA databases.

HVAE: A deep generative model via hierarchical variational auto-encoder for multi-view document modeling

With the widespread development of the internet, multi-view text documents have become increasingly common, which has led to extensive research on multi-view text document modeling. As opposed to traditional single-view document modeling, which treats each document independently and learns each document as a single topic representation, the views of multi-view text documents have complicated correlation relationships that include both the global and local underlying topical information. In this study, we introduce a deep generative model for multi-view document modeling known as Hierarchical Variational Auto-Encoder (HVAE), which combines the advantages of the probability generative model for learning interpretable latent information and the deep neural network for efficient parameter inference. Specifically, a set of hierarchical topic representations is learned for each multi-view document to capture the document-level global topical information and view-level local topical information for each view. A two-level hierarchical topic inference network is investigated as the encoder network of HVAE, which is designed using an aligned variational auto-encoder, to learn the hierarchical topic representations. Subsequently, multi-view documents are generated through a two-layered generation network, considering both the view-level local and document-level global topic representations. Experiments on three real datasets of different scales for various tasks demonstrate the satisfactory results of the proposed method.

Dynamic Energy-Saving Offloading Strategy Guided by Lyapunov Optimization for IoT Devices

In the Internet of Everything era, various Internet of Things (IoT) devices have become popular, and the number of computing-intensive applications has increased substantially. As an emerging technology, mobile-edge computing (MEC) gives network edge nodes stronger computing and storage capabilities, bringing users a good Quality of Experience (QoE). By offloading some computing tasks to the edge for processing, the burden on IoT devices can be effectively reduced. However, this approach exacerbates the computing and storage resource depletion of the MEC server and the bandwidth and transmission cost of the wireless link used to offload computing tasks. Additionally, making an offloading decision online without future system status information is a considerable challenge. Therefore, we should study and design a reasonable offloading strategy to reduce the additional overhead, which is of significance. We establish a virtual queue model to describe the workload offloading problem of IoT devices in a two-layer MEC network. This is a stochastic optimization problem. Based on Lyapunov optimization, we transform the research problem into a deterministic optimization problem. A Lyapunov online energy consumption optimization algorithm (LOECOA) is proposed to effectively balance the system’s queue backlog and energy consumption. Based on theoretical analysis and a large number of experimental and numerical results, our algorithm performs better on energy consumption while satisfying the system constraints under a dynamic task arrival rate.

A patient-informed approach to predict iodinated-contrast media enhancement in the liver

ObjectiveTo devise a patient-informed time series model that predicts liver contrast enhancement, by integrating clinical data and pharmacokinetics models, and to assess its feasibility to improve enhancement consistency in contrast-enhanced liver CT scans. MethodsThe study included 1577 Chest/Abdomen/Pelvis CT scans, with 70–30% training/validation-testing split. A Gaussian function was used to approximate the early arterial, late arterial, and the portal venous phases of the contrast perfusion curve of each patient using their respective bolus tracking and diagnostic scan data. Machine learning models were built to predict the Gaussian parameters of each patient using the patient attributes (weight, height, age, sex, BMI). Pearson’s coefficient, mean absolute error, and root mean squared error were used to assess the prediction accuracy. ResultsThe integration of the pharmacokinetics model with a two-layered neural network achieved the highest prediction accuracy on the test data (R2 = 0.61), significantly exceeding the performance of the pharmacokinetics model alone (R2 = 0.11). Applying the model demonstrated that adjusting the contrast administration directed by the model may reduce clinical enhancement inconsistency by up to 40 %. ConclusionsA new model using a Gaussian function and supervised machine learning can be used to build liver parenchyma contrast enhancement prediction model. The model can have utility in clinical settings to optimize and improve consistency in contrast-enhanced liver imaging.

Modeling for external stakeholder risk assessment of project portfolios

PurposeExternal stakeholder risks (ESRs) caused by unfavorable behaviors hinder the success of project portfolios (PPs). However, due to complex project dependency and numerous risk causality in PPs, assessing ESRs is difficult. This research aims to solve this problem by developing an ESR-PP two-layer fuzzy Bayesian network (FBN) model.Design/methodology/approachA two-layer FBN model for evaluating ESRs with risk causality and project dependency is proposed. The directed acyclic graph (DAG) of an ESR-PP network is first constructed, and the conditional probability tables (CPTs) of the two-layer network are further presented. Next, based on the fuzzy Bayesian network, key variables and the impact of ESRs are assessed and analyzed by using GeNIe2.3. Finally, a numerical example is used to demonstrate and verify the application of the proposed model.FindingsThe proposed model is a useable and effective approach for ESR assessment while considering risk causality and project dependency in PPs. The impact of ESRs on PP can be calculated to determine whether to control risk, and the most critical and heavily contributing risks and project(s) in the developed model are identified based on this.Originality/valueThis study extends prior research on PP risk in terms of stakeholders. ESRs that have received limited attention in the past are explored from an interaction perspective in the PP domain. A new two-layer FBN model considering risk causality and project dependency is proposed, which can synthesize different dependencies between projects.

Design of marine propellers with prescribed and optimal spanwise circulation distributions based on genetic algorithms and neural network

Lifting-surface design methods are proposed for marine propellers based on genetic algorithms (GAs) and a two-layer back-propagation neural network (BPNN). For propellers with prescribed spanwise circulation distributions, the design problem is solved as GA-based optimization of camber surface geometry and pitch distribution which produce a circulation distribution to best fit the prescribed one. An in-house code based on the vortex lattice method (VLM) is employed to simulate circulation distribution and hydrodynamic performance of the propeller. Computer codes are developed in this research based on existing genetic algorithms, with a measure devised to accelerate convergence and improve the quality of solution. To optimize the spanwise circulation distribution, GAs are utilized again to explore the BPNN model established via the MATLAB toolbox and the in-house VLM code. Then the optimal circulation distribution is taken as the prescribed one for the GA-based design method mentioned above. Numerical tests are conducted to determine proper ranges of modeling parameters, such as the population size for GAs and the number of vortex lattices, and to assess the performance of the BPNN established. To numerically validate the proposed methods, viscous-flow simulation by solving the Reynolds-averaged Navier-Stokes equations is carried out for the propeller designed by using both methods mentioned above. The predicted pressure distributions over blade surfaces correlate consistently with the circulation distributions used to design the propeller, thus indicating that present design methods are effective and reasonably accurate.

Generative and interpretable machine learning for aptamer design and analysis of in vitro sequence selection.

Selection protocols such as SELEX, where molecules are selected over multiple rounds for their ability to bind to a target of interest, are popular methods for obtaining binders for diagnostic and therapeutic purposes. We show that Restricted Boltzmann Machines (RBMs), an unsupervised two-layer neural network architecture, can successfully be trained on sequence ensembles from single rounds of SELEX experiments for thrombin aptamers. RBMs assign scores to sequences that can be directly related to their fitnesses estimated through experimental enrichment ratios. Hence, RBMs trained from sequence data at a given round can be used to predict the effects of selection at later rounds. Moreover, the parameters of the trained RBMs are interpretable and identify functional features contributing most to sequence fitness. To exploit the generative capabilities of RBMs, we introduce two different training protocols: one taking into account sequence counts, capable of identifying the few best binders, and another based on unique sequences only, generating more diverse binders. We then use RBMs model to generate novel aptamers with putative disruptive mutations or good binding properties, and validate the generated sequences with gel shift assay experiments. Finally, we compare the RBM’s performance with different supervised learning approaches that include random forests and several deep neural network architectures.

Hybrid encryption technique: Integrating the neural network with distortion techniques.

This paper proposes a hybrid technique for data security. The computational model of the technique is grounded on both the non-linearity of neural network manipulations and the effective distortion operations. To accomplish this, a two-layer feedforward neural network is trained for each plaintext block. The first layer encodes the symbols of the input block, making the resulting ciphertext highly uncorrelated with the input block. The second layer reverses the impact of the first layer by generating weights that are used to restore the original plaintext block from the ciphered one. The distortion stage imposes further confusion on the ciphertext by applying a set of distortion and substitution operations whose functionality is fully controlled by random numbers generated by a key-based random number generator. This hybridization between these two stages (neural network stage and distortion stage) yields a very elusive technique that produces ciphertext with the maximum confusion. Furthermore, the proposed technique goes a step further by embedding a recurrent neural network that works in parallel with the first layer of the neural network to generate a digital signature for each input block. This signature is used to maintain the integrity of the block. The proposed method, therefore, not only ensures the confidentiality of the information but also equally maintains its integrity. The effectiveness of the proposed technique is proven through a set of rigorous randomness testing.

Prognosis of LED lumen degradation using Bayesian optimized neural network approach

Light Emitting Diodes (LEDs) are among the most widely used electronic devices for everyday lighting applications due to their durability over incandescent lamps. However, there are no standardized approaches to predict the reliability of LEDs as they are manufactured and tested as per the user requirements and applications. This dramatically limits developing generic prognostic algorithms pertaining to predicting the remaining useful life (RUL) of LEDs. In this study, we propose a Bayesian optimized neural network approach to predict the lumen degradation trends of LEDs. The proposed method does not require an accurate physical model representing the LED degradation behavior and does not require a large amount of degradation data. We have used a particle filter algorithm to train a simple two-layer feedforward neural network model and use the trained model to predict the lumen degradation of LEDs. Also, the weight decay issues commonly encountered in particle filter algorithm are addressed using three different resampling strategies and particle roughening method. To evaluate the effectiveness of the proposed approach, Root Mean Squared Error (RMSE) and Relative Accuracy (RA) were used as the prognostic metrics.

Recognition ability of untrained neural networks to symbolic numbers.

Although animals can learn to use abstract numbers to represent the number of items, whether untrained animals could distinguish between different abstract numbers is not clear. A two-layer spiking neural network with lateral inhibition was built from the perspective of biological interpretability. The network connection weight was set randomly without adjustment. On the basis of this model, experiments were carried out on the symbolic number dataset MNIST and non-symbolic numerosity dataset. Results showed that the model has abilities to distinguish symbolic numbers. However, compared with number sense, tuning curves of symbolic numbers could not reproduce size and distance effects. The preference distribution also could not show high distribution characteristics at both ends and low distribution characteristics in the middle. More than half of the network units prefer the symbolic numbers 0 and 5. The average goodness-of-fit of the Gaussian fitting of tuning curves increases with the increase in abscissa non-linearity. These results revealed that the concept of human symbolic number is trained on the basis of number sense.

Research on stock volatility risk and investor sentiment contagion from the perspective of multi-layer dynamic network

In the post-epidemic period, the international economic structure has been readjusted, with risks contagious across financial and economic systems. This paper primarily uses the high-frequency TENET network and the Granger-causality network to describe the interconnectedness between the tail risk of stock volatility and investor sentiment, then the two-layer network is constructed by the generalized variance decomposition method to examine the inter-layer connectedness. Based on the two-layer network, the heterogeneity frequency response of network connectedness and dynamic network structure are further analyzed from the perspective of frequency domain. The study found that the tail risk of high-frequency stock volatility displays industry heterogeneity and time-varying property, and investor sentiment contagion network provides information transmission medium for stock risk. The double-layer network study found that stock volatility in consumer goods industry exhibits higher risk spillover to investor sentiment. The diversified financial industry, real estate industry and energy industry in the two-layer network are systemically important industries. In addition, the study of the frequency domain dynamic network found that the connectedness volatility in the short-term risk network of stock volatility was significantly higher than that of the investor sentiment network, and the short-term risk spillover effect of the network played a leading role in the total risk spillover. The research conclusions provide reference for preventing systemic risks from the perspective of systemically important industries and cyclical fluctuations.

Transmission dynamics of disease spreading in multilayer networks with mass media.

On the basis of existing disease spreading research, in this paper we propose a Hesitant-Taken-Unaware-Aware-Susceptible-Asymptomatic-Symptomatic-Recovered (HTUA-SI^{a}I^{s}R) model with mass media in a two-layer network, which consists of a virtual communication layer and a physical contact layer. Based on the UAU-SIR model, we additionally consider three practical factors, including whether individuals will disseminate information or not, the influence of unaware individuals on aware individuals, and the direct recovery of asymptomatic infected individuals. Based on the microscopic Markov chain approach (MMCA), for the proposed HTUA-SI^{a}I^{s}R model, MMCA equationsare generated and the analytical expression of the epidemic threshold is obtained. Compared with Monte Carlo techniques, numerical simulations show the feasibility and effectiveness of the MMCA equations, as well as the HTUA-SI^{a}I^{s}R model theoretically. Meanwhile, extensive simulations demonstrate that the acceleration of the awareness dissemination in the virtual communication layer can effectively block the epidemic spreading and raise the epidemic threshold. However, under certain conditions, the increasing of T-state individuals will increase the U-state individuals because the T-state and U-state individuals can influence the A-state individuals losing their awareness of protection, and then promote the epidemic spreading and decrease the epidemic threshold. In addition, reducing asymptomatic infections can hinder the epidemic spreading. But, when the recovery rate of asymptomatic infections is greater than that of symptomatic infections, decreasing the tendency of individuals acquiring asymptomatic infections will lower the epidemic threshold.

A convergence analysis of Nesterov’s accelerated gradient method in training deep linear neural networks

As the training process of deep neural networks involves expensive computational cost, speeding up the convergence is of great importance. Nesterov’s accelerated gradient (NAG) is one of the most popular accelerated optimizers in the deep learning community, which often exhibits improved convergence performance over gradient descent (GD) in practice. However, theoretical investigations of NAG mainly focus on the convex setting. Since the optimization landscape of the neural network is non-convex, little is known about the convergence and acceleration of NAG. Nowadays, some works make progress towards understanding the convergence of NAG in training over-parameterized neural networks, where the number of the parameters exceeds that of the training instances. Nonetheless, previous studies are limited to the two-layer neural network, which are far from explaining the remarkable success of NAG in optimizing deep neural networks. In this paper, we investigate the convergence of NAG in training two architectures of deep linear networks: deep fully-connected linear neural networks and deep linear ResNets. Based on the over-parameterization regime, we first analyze the residual dynamics induced by the training trajectory of NAG for a deep fully-connected linear neural network under random Gaussian initialization. Our results show that NAG can converge to the global minimum at a (1-O(1/κ))t rate when the width is near-linear in the depth of the network, where t is the number of iterations and κ>1 is a constant depending on the condition number of the feature matrix. Compared to the (1-O(1/κ))t rate of GD, NAG achieves an acceleration over GD. For deep linear ResNets, we utilize the same analytical approach and obtain a similar convergence result, while the width requirement is independent of the depth. To the best of our knowledge, these are the first theoretical guarantees for the convergence and acceleration of NAG in training deep neural networks. Numerical results show the acceleration of NAG compared to GD in terms of iterations. In addition, we conduct experiments to evaluate the effect of the depth on the convergence rate of NAG, which validate our derived conditions of the width. We hope our results may shed light on understanding the optimization behavior of NAG for modern deep neural networks.

Return Instruction Classification in Binary Code Using Machine Learning

Binary code analysis is vital in source code unavailable cases, such as malware analysis and software vulnerability mining. Its first step could be function identification. Most function identification methods are based on function prologs/epilogs. However, functions may not have standard prologs/epilogs. To identify these functions, we need to use other methods. One approach is to identify return instructions first and then identify the start of a function. Currently, the multi-layer perceptron model is exploited to identify and validate a return instruction at a specific location. On this basis, a new approach is proposed to improve accuracy and provide more details. Specifically, a return instruction is classified into three classes: (1) false return instruction, (2) true return instruction inner a function but not the last instruction, and (3) true return instruction at the end of a function. The evaluation is performed on 5782 real-world binaries. Meanwhile, common classifiers including fully connected neural network, Two-layer Bidirectional Recurrent Neural Network (TBRNN), Two-layer Bidirectional Gate Recurrent Unit (TBGRU), Two-layer Bidirectional Long Short-term Memory Network (TBLSTM), Decision Tree, Random Forest, XGBoost, and Support Vector Machine (SVM) are evaluated on the same data set. The result shows that TBLSTM achieves an accuracy of 99.78%, which is higher than that of other classifiers in the evaluation, including the state-of-the-art tool IDA Pro 7.7.

TriATNE: Tripartite Adversarial Training for Network Embeddings.

Existing network embedding algorithms based on generative adversarial networks (GANs) improve the robustness of node embeddings by selecting high-quality negative samples with the generator to play against the discriminator. Since most of the negative samples can be easily discriminated from positive samples in graphs, their poor competitiveness weakens the function of the generator. Inspired by the sales skills in the market, in this article, we present tripartite adversarial training for network embeddings (TriATNE), a novel adversarial learning framework for learning stable and robust node embeddings. TriATNE consists of three players: 1) producer; 2) seller; and 3) customer. The producer strives to learn the representation of each sample (node pair), making it easy for the customer to differentiate between the positive and the negative, while the seller tries to confuse the customer by selecting realistic-looking samples. The customer, a biased evaluation metric, provides feedback for training the producer and the seller. To further enhance the robustness of node embedding, we model the customer as a two-layer neural network, where each unit in the hidden layer can be regarded as a customer with different preferences. TriATNE also plays against the producer by adjusting the weight of each customer. We test the performance of TriATNE on two common tasks: classification as well as link prediction. The experimental results on various publicly available datasets show that TriATNE can exploit the network structure well.

A Comparative Study on Prediction of Cutting Force using Artificial Neural Network and Genetic Algorithm during Machining of Ti-6Al-4V

The purpose of this comparative study is to improve the predictive accuracy of the cutting force during the turning of Ti-6Al-4V on a lathe machine. By optimizing the machining process parameters such as cutting speed, feed rate, and depth of cut, the cutting force in the machining process can be improved significantly. Cutting force is one of the crucial characteristics that must be monitored during the cutting process in order to enhance tool life and the surface finish of the workpiece. This paper is based on the experimental dataset of cutting forces collected during the turning of titanium alloy under the Minimum Quantity Lubrication (MQL) condition. To predict the cutting forces, two machine learning techniques are explored. Firstly, a black-box model called an Artificial Neural Network (ANN) is proposed to predict cutting force. Using the Levenberg-Marquardt algorithm, a two-layered feedforward neural network is built in MATLAB to predict cutting force. The second model to be implemented was the Genetic Algorithm (GA), a white-box model. GA is an optimization technique which is based on Darwinian theories. It is a probabilistic method of searching, unlike most other search algorithms, which require definite inputs. Using symbolic regression in HeuristicLab, a GA model is developed to estimate cutting force. The anticipated values of cutting forces for both models were compared. Since the ANN model had fewer errors, it was ascertained that the particular model is preferable for machining process optimization.

Locating Sources in Multiplex Networks for Linear Diffusion Systems

Accurately locating sources with limited resources plays an important role in suppressing and even predicting the spread of harmful information. Due to the existence of the coupling between layers, compared with simplex networks, many studies have revealed that multiplex networks present many interesting phenomena. However, almost all the works involved in source(s) localization concentrate on simplex networks, while it is unclear how the coupling strength between layers of a multiplex network affects source(s) localization. Here we focus on a linear diffusion process and propose a general framework of multiple sources localization in multiplex networks with minimum measurements by combining the observability theory and compressed sensing theory. One surprise finding is that, for undirected two-layer networks with fully coupling between layers, the messengers placed at the same layer is sufficient to accurately locate all the sources. Different from simplex networks, the diffusion coefficients of linear diffusion process in multiplex networks can affect the minimum number of messengers. Simulation results from various network structures show that the minimum number of messengers is fewer when the coupling strength between layers is higher. Besides, we find the larger the network size the fewer measurements are required for sources localization. We further provide simulations from synthetic networks and one real network demonstrates our framework of sources localization is robust against small amount of data, noise and the coupling strength between layers.

Benign Overfitting and Noisy Features

Modern machine learning models often exhibit the benign overfitting phenomenon, which has recently been characterized using the double descent curves. In addition to the classical U-shaped learning curve, the learning risk undergoes another descent as we increase the number of parameters beyond a certain threshold. In this article, we examine the conditions under which benign overfitting occurs in the random feature (RF) models, that is, in a two-layer neural network with fixed first layer weights. Adopting a novel view of random features, we show that benign overfitting emerges because of the noise residing in such features. The noise may already exist in the data and propagates to the features, or it may be added by the user to the features directly. Such noise plays an implicit yet crucial regularization role in the phenomenon. In addition, we derive the explicit tradeoff between the number of parameters and the prediction accuracy, and for the first time demonstrate that overparameterized model can achieve the optimal learning rate in the minimax sense. Finally, our results indicate that the learning risk for overparameterized models has multiple, instead of double descent behavior, which is empirically verified in recent works. Supplementary materials for this article are available online.