Single-layer Network Research Articles

Nonlinear subspace clustering by functional link neural networks

Nonlinear subspace clustering based on a feed-forward neural network has been demonstrated to provide better clustering accuracy than some advanced subspace clustering algorithms. While this approach demonstrates impressive outcomes, it involves a balance between effectiveness and computational cost. In this study, we employ a functional link neural network to transform data samples into a nonlinear domain. Subsequently, we acquire a self-representation matrix through a learning mechanism that builds upon the mapped samples. As the functional link neural network is a single-layer neural network, our proposed method achieves high computational efficiency while ensuring desirable clustering performance. By incorporating the local similarity regularization to enhance the grouping effect, our proposed method further improves the quality of the clustering results. We name our method as Functional Link Neural Network Subspace Clustering (FLNNSC). Furthermore, we propose a convex combination subspace clustering scheme that combines a linear subspace clustering method with the functional link neural network subspace clustering approach. This combination method is named as Convex Combination Subspace Clustering (CCSC), which allows for a dynamic balance between linear and nonlinear representations. Extensive experiments conducted on four widely used datasets, including Extended Yale B, USPS, COIL20, and ORL, demonstrate that both FLNNSC and CCSC outperform several state-of-art subspace clustering methods in terms of clustering accuracy. Our affinity graph experiments reveal that FLNNSC exhibits clear block diagonal structures. We provide recommendations for hyperparameters in FLNNSC by performing a parameter sensitivity analysis, and empirically verify the convergence of FLNNSC. Additionally, we show that FLNNSC has a lower computational cost compared to two high-performing methods.

Dynamics of convolutional recurrent neural networks near their critical point

We examine the dynamical properties of a single-layer convolutional recurrent network with a smooth sigmoidal activation function, for small values of the inputs and when the convolution kernel is unitary, so all eigenvalues lie exactly at the unit circle. Such networks have a variety of hallmark properties: the outputs depend on the inputs via compressive nonlinearities such as cubic roots and both the and the depend sensitively on the inputs as power laws, both diverging as the input →0. The basic dynamical mechanism is that inputs to the network generate ongoing activity, which in turn controls how additional inputs or signals propagate spatially or attenuate in time. We present analytical solutions for the steady states when the network is forced with a single oscillation and when a background value creates a steady state of “ongoing activity” and derive the relationships shaping the value of the temporal decay and spatial propagation length as a function of this background value. Published by the American Physical Society 2024

Robustness Analysis of Multilayer Infrastructure Networks Based on Incomplete Information Stackelberg Game: Considering Cascading Failures

The growing importance of critical infrastructure systems (CIS) makes maintaining their normal operation against deliberate attacks such as terrorism a significant challenge. Combining game theory and complex network theory provides a framework for analyzing CIS robustness in adversarial scenarios. Most existing studies focus on single-layer networks, while CIS are better modeled as multilayer networks. Research on multilayer network games is limited, lacking methods for constructing incomplete information through link hiding and neglecting the impact of cascading failures. We propose a multilayer network Stackelberg game model with incomplete information considering cascading failures (MSGM-IICF). First, we describe the multilayer network model and define the multilayer node-weighted degree. Then, we present link hiding rules and a cascading failure model. Finally, we construct MSGM-IICF, providing methods for calculating payoff functions from the different perspectives of attackers and defenders. Experiments on synthetic and real-world networks demonstrate that link hiding improves network robustness without considering cascading failures. However, when cascading failures are considered, they become the primary factor determining network robustness. Dynamic capacity allocation enhances network robustness, while changes in dynamic costs make the network more vulnerable. The proposed method provides a new way of analyzing the robustness of diverse CIS, supporting resilient CIS design.

Diversifying Multi-Head Attention in the Transformer Model

Recent studies have shown that, due to redundancy, some heads of the Transformer model can be pruned without diminishing the efficiency of the model. In this paper, we propose a constrained optimization algorithm based on Hebbian learning, which trains specific layers in the Transformer architecture in order to enforce diversification between the different heads in the multi-head attention module. The diversification of the heads is achieved through a single-layer feed-forward neural network that is added to the Transformer architecture and is trained with the proposed algorithm. We utilize the algorithm in three different architectural variations of the baseline Transformer model. In addition to the diversification of the heads, the proposed methodology can be used to prune the heads that capture redundant information. Experiments on diverse NLP tasks, including machine translation, text summarization, question answering and large language modeling, show that our proposed approach consistently improves the performance of baseline Transformer models.

A single-layer dense convolutional reversible residual network for bearing fault diagnosis based on differential local adaptive

Abstract For the cross-domain fault diagnosis of industrial bearings under different working conditions and noise, most current domain adaptation methods in transfer learning only focus on either marginal distribution alignment or conditional distribution alignment. They fail to adequately combine discriminative and global distribution information. Furthermore, the majority of models have a very high parameter count and memory utilization, which makes it challenging to use them in real-world industrial situations. Therefore, a single-layer densely connected reversible residual network based on differential local adaptation is proposed. This network is more competitive in industrial applications than other fault diagnosis models since it not only uses less memory and has fewer parameters, but it also shows superior cross-domain fault diagnostic capacity in noisy situations. Additionally, to extract discriminative and global domain-invariant features, a domain adaptation module is created that takes into account local and global data distributions differently. Multiple transfer tasks and two distinct datasets are used to validate the model. Comparative tests reveal that the suggested model uses less memory and requires fewer parameters to attain good accuracy and transferability.

Backpropagation-based inference for spatial interpolation to estimate the blastability index in an open pit mine

The blastability index (BI) is a measure that indicates the resistance of rock to fragmentation when blasting. With novel technologies, miners are now able to collect and calculate BI at different depths while drilling. In this research, we propose an approach to estimate the BI at multiple depths for new areas using only spatial locations and observed BI measurements of previously drilled holes. Spatial interpolation techniques are investigated. This study introduces a novel treatment for Gaussian Processes (GPs) and Inverse Distance Weighting (IDW). Variography is leveraged to ensure an appropriate fit between the data and the spatial component. The parameters controlling anisotropy are constrained to intervals chosen to reflect the observed anisotropy. Gradient descent with back-propagation is used for optimization. The proposed approach improves the performance of GP and IDW at predicting BI. The similarities between the IDW variant proposed and a single-layer neural network are discussed.

Quantifying the reciprocal impacts of capital and logistics networks in the supply chains: A cyber–physical system approach

Capital flow and logistics are closely linked in the supply chain. Current research primarily assesses their independent resilience during disruptions, but this study reveals an iterative bidirectional impact. The interconnection of shareholding, mutual guarantees, and borrowing within the supply chain has created financial relationships that were previously nonexistent. However, the evolution of logistics has not aligned with these financial developments, leading to a growing heterogeneity between capital flow and logistics. While the heterogeneity between capital flow and logistics within supply chains is crucial, previous studies, especially those rooted in complex networks, often overlook its impact. Drawing inspiration from the cyber–physical system, this paper abstracts logistics and capital flow into respective networks. A model of mutual dependence quantifies reciprocal impacts between capital and logistics networks, exploring the consequences of heterogeneity on the supply chain. Simulation results are compared with traditional single-layer network models. Finally, an innovative edge removal method is proposed to enhance supply chain robustness. Our simulation results indicate that the influence of the capital network on the logistics network and their relationship to heterogeneity is nonlinear. The edge removal method enhances supply chain network robustness in a nonlinear manner, with optimal effectiveness achieved at p_l=0.2.

Investigating the Limits of Familiarity-Based Navigation.

Insect-inspired navigation strategies have the potential to unlock robotic navigation in power-constrained scenarios, as they can function effectively with limited computational resources. One such strategy, familiarity-based navigation, has successfully navigated a robot along routes of up to 60 m using a single-layer neural network trained with an Infomax learning rule. Given the small size of the network that effectively encodes the route, here we investigate the limits of this method, challenging it to navigate longer routes, investigating the relationship between performance, view acquisition rate and dimension, network size, and robustness to noise. Our goal is both to determine the parameters at which this method operates effectively and to explore the profile with which it fails, both to inform theories of insect navigation and to improve robotic deployments. We show that effective memorization of familiar views is possible for longer routes than previously attempted, but that this length decreases for reduced input view dimensions. We also show that the ideal view acquisition rate must be increased with route length for consistent performance. We further demonstrate that computational and memory savings may be made with equivalent performance by reducing the network size-an important consideration for applicability to small, lower-power robots-and investigate the profile of memory failure, demonstrating increased confusion across the route as it extends in length. In this extension to previous work, we also investigate the form taken by the network weights as training extends and the areas of the image on which visual familiarity-based navigation most relies. Additionally, we investigate the robustness of familiarity-based navigation to view variation caused by noise.

The self-similarity of the multilayer networks

Abstract Research on the self-similarity of multilayer networks is scarce, when compared to the extensive research conducted on the dynamics of these networks. In this paper, we use entropy to determine the edge weights in each sub-network, and apply the degree-degree distance to unify the weight values of connecting edges between different sub-networks, and unify the edges with different meanings in the multilayer network numerically. At this time, the multilayer network is compressed into a single-layer network, also known as the aggregated network. Furthermore, the self-similarity of the multilayer network is represented by analyzing the self-similarity of the aggregate network. The study of self-similarity was conducted on two classical fractal networks and a real-world multilayer network. The results show that multilayer networks exhibit more pronounced self-similarity, and the intensity of self-similarity in multilayer networks can vary with the connection mode of sub-networks.

Influence maximization based on discrete particle swarm optimization on multilayer network

Influence maximization (IM) aims to strategically select influential users to maximize information propagation in social networks. Most of the existing studies focus on IM in single-layer networks. However, we have observed that individuals often engage in multiple social platforms to fulfill various social needs. To make better use of this observation, we consider an extended problem of how to maximize influence spread in multilayer networks. The Multilayer Influence Maximization (MLIM) problem is different from the IM problem because information propagation behaves differently in multilayer networks compared to single-layer networks: users influenced on one layer may trigger the propagation of information on another layer. Our work successfully models the information propagation process as a Multilayer Independent Cascade model in multilayer networks. Based on the characteristics of this model, we introduce an approximation function called Multilayer Expected Diffusion Value (MLEDV) for it. However, the NP-hardness of the MLIM problem has posed significant challenges to our work. To tackle the issue, we devise a novel algorithm based on Discrete Particle Swarm Optimization. Our algorithm consists of two stages: 1) the candidate node selection, where we devise a novel centrality metric called Random connectivity Centrality to select candidate nodes, which assesses the importance of nodes from a connectivity perspective. 2)the seed selection, where we employ a discrete particle swarm algorithm to find seed nodes from the candidate nodes. We use MLEDV as a fitness function to measure the spreading power of candidate solutions in our algorithm. Additionally, we introduce a Neighborhood Optimization strategy to increase the convergence of the algorithm. We conduct experiments on four real-world networks and two self-built networks and demonstrate that our algorithms are effective for the MLIM problem.

Multi-Objective Optimization based decision-making process using Dombi Logarithmic aggregation operator in CNN environment and its application to optimally select suitable greenhouse site for tomato crops

In this research article, we have defined Dombi Logarithmic aggregation operational laws in the framework of Cylindrical Neutrosophic Numbers (CNN) and utilized these laws to establish a new aggregation operator namely Cylindrical Neutrosophic Dombi Weighted Logarithmic Aggregation Operator(C N DW L A ). The said aggregation operational laws & aggregation operator have been used to present a new and novel decision making process where Full Consistency Method (FUCOM) and Multi-Objective Optimization(MOO) have been inte- grated and embedded fruitfully. Here the objective functions were formulated using the concept of a single-layer neural network and the FUCOM method to assess criterion weights. We have resolved the most favorable Pareto optimal solution derived from Multi-Objective Optimization by employing simulation and the Technique for Or- der of Preference by Similarity to Ideal Solution (TOPSIS) approach. Finally, we have applied our proposed decision making method along with MARCOS and MOORA techniques to determine the best greenhouse site for cultivating tomato crops. An exhaustive sensitivity and comparative analysis has been conducted to assess the robustness and stability of our Multi Criteria Group Decision Making (MCGDM) techniques. It is observed that our proposed method has some advantages in cylindrical neutrosophic environment and it produces stable robust results with the variation of underlying model parameters.

LSTM Recurrent Neural Network Based Method for Identification of PDC Drill Bit Operating Conditions and Its Application

Abstract PDC drill bit underground operating conditions identification is one of of the difficulties during the drilling operation. It is of great significant for the drilling improvement and accurate construction formation to accurately identify the complex operating conditions during drilling the operation such as PDC bit wearing and stick-slip vibration, etc. In this paper, data of relationship between torque, rate of penetration (ROP) signal and weight on bit (WOB), rev were obtained through lab simulation experiment, after which, a relationship database of torque, ROP signal and WOB, rev was established, a single-layer LSTM recurrent neural network (LSTM-RNN) was constructed, the number of neurons, batch size and optimizer type of LSTM layer were optimized. With BPTT algorithm, LSTM model was trained and tested on the training set containing 6000 groups of data, and achieved the goal of identifying the bit operation conditions against the input of WOB, torque, rev and ROP signal parameters. Based on this model, the PDC bit underground operating conditions identification software was compiled, and was connected to the logging unit to realize real-time identification of bit underground operating conditions, and field test application was successfully carried out in Mahu 1 Well Region. The research showed that the convergence speed was fast and the time cost was at the lowest when the number of neurons of LSTM model was 50, and that the optimized Adam optimizer model could meet the requirements of convergence speed and prediction accuracy with the lowest loss value. The classification accuracy of this model on the test set reached 94% and the accuracy of the prediction method was also verified by field test application. To sum up, the method of LSTM-RNN based PDC bit underground operating conditions real-time identification can better meet the requirements of accurate identification of PDC bit underground operating conditions in the practical drilling, and has guiding significance for the judgement of PDC bit operating conditions on site and formulation of scientific construction decisions.

A novel method for identifying key nodes in multi-layer networks based on dynamic influence range and community importance

Identifying key nodes in multi-layer networks is a hot research topic in complex network science and has broad application prospects, such as in mining enterprises that significantly affecting multi-layer industrial chains. Unlike single-layer networks, nodes in multi-layer networks exhibit heterogeneity due to varying connections and locations. There are also correlations between different network layers, which is particularly evident in industrial chains where companies operate across multiple layers of production, supply and distribution. It is necessary to consider the impact of these layers on the global performance of key node identification. In addition, due to changes in connections, the community structure of each network layer should be different, reflecting the dynamic nature of industrial collaborations and partnerships. However, existing research lacks the model that addresses the above problems. Therefore, this paper proposes a key node identification method based on Dynamic Influence Range and Community Importance (DIRCI), using both local and global information of the multi-layer network simultaneously. DIRCI determines the importance of nodes through three centrality measures: dynamic influence range-based centrality, network layer centrality and community-based centrality. Dynamic influence range-based centrality models node heterogeneity by combining the influence range of nodes and their neighbors with lower computational costs. Network layer centrality captures the corresponding importance for different network layers. Community-based centrality comprehensively considers the importance of community, the importance of each node within the community and between different communities. Experimental results for nineteen multi-layer networks show that DIRCI achieves better performance of key node identification than the latest algorithms.

INVESTIGATION OF DEEP LEARNING MODELS BASED ON SINGLE-LAYER SimpleRNN, LSTM AND GRU NETWORKS FOR RECOGNIZING SOUNDS OF UAV DISTANCES

In recent years, the potential risks posed by easily moving objects have highlighted the need for intelligent surveillance systems in protected areas, primarily to ensure the safety of human lives. Among the most common of these objects are unmanned aerial vehicles (UAVs). Recent advances in deep learning techniques for recognizing audio signals have made these techniques effective in identifying moving or aerial objects, especially those powered by engines. And the growing deployment of UAVs has made their rapid recognition in various suspicious or unauthorized circumstances critical. Detecting suspicious drone flights, especially in restricted areas, remains a significant research challenge. It is vital to perform the task of determining their distance in order to quickly detect drones approaching people in such protected areas. Therefore, this paper aims to study the research question of recognizing UAV audio data from different distances. That is, recognizing drone audio at different distances was experimentally studied using Simple RNN, LSTM and GRU based deep learning models. The main objective of this study is based on finding one of the capable types of recurrent network for the task of recognizing UAV audio data at different distances. During the experimental study, the recognition abilities of Single-layer Simple RNN, LSTM and GRU recurrent network types were studied from two basic directions: with recognition accuracy curves and classification reports. As a result, LSTM and GRU based models showed high recognition ability for these types of audio signals. It was noted that UAVs can reliably predict distances greater than 10 meters based on the proposed deep learning architecture.

Adaptive dynamic programming–based fault-tolerant control of multi-UAV formation system

With the increasing utilization of unmanned aerial vehicle (UAV) swarms in civilian and military field applications, it is crucial to eliminate the potential impacts of UAV faults on task completion and efficiency. To address this issue, the present paper investigates a fault-tolerant control scheme based on the adaptive dynamic programming (ADP) method and fault observer. The whole closed-loop system is composed of a position loop and an attitude loop. An ADP-based position-tracking controller is established with an improved cost function that can simultaneously represent actuator faults, control inputs, and tracking errors. In addition, a single-layer critic neural network is constructed to estimate the cost function and approximate control inputs. In the process of adjusting the weights of the neural network, a unique adjustment method combining policy gradient and prioritized experience replay is adopted to improve the data usage efficiency and speed up the weight convergence. Furthermore, a sliding mode–based attitude controller is utilized for the inner-loop attitude subsystem. The significant feature of this scheme is that it has the ability to self-learn and self-adapt. A Lyapunov stability analysis is performed to verify whether the signals are uniformly ultimately bounded. Finally, simulation experiments are conducted to demonstrate the effectiveness of the proposed scheme.

Identification of important nodes in multi-layer hypergraphs based on fuzzy gravity model and node centrality distribution characteristics

Hyperedge is a common structure that represents high-order interactions between nodes in complex networks, and multi-layer networks provide more diverse node interactions than single-layer networks. Therefore, multi-layer hypergraphs can more clearly represent the relationships between nodes. However, there are few studies on identifying important nodes in this framework. This paper proposes a method called HCT to fill this gap. It consists of three parts, namely: Hypergraph Fuzzy Gravity Model (HFGM), Layer Weight Calculation Method based on Node Centrality Distribution Characteristics (CDLW) and Technique for Order Preference by Similarity to Ideal Solution (TOPSIS). HCT progressively analyzes the importance of a node from three levels: local (the node itself), semi-local (the hyperedges and community to which the node belongs), and global (the layer to which the node belongs). By combining these three results, the global centrality of a node in the entire network can be calculated. Simulation experiments demonstrate that the important nodes identified by HCT exhibit stronger contagion capabilities in nine networks compared to nine combinatorial methods and removing these nodes will seriously damage the connectivity and robustness of the network. The centrality of each node calculated by HCT is also consistent with its actual importance.

Pattern formation in reaction-diffusion information propagation model on multiplex simplicial complexes

Turing patterns for explaining spatial distribution in nature mostly focus on continuous media and existing networks, but only few attempts exist to study them on systems with high-order interactions. Considering that high-order interactions have a particularly significant impact on rumor propagation, this study establishes a generalized reaction-diffusion rumor propagation model based on a multiplex network, where simplicial complexes are employed to describe high-order structures. It aims to provide the spatial distribution patterns of the population participating in rumor propagation and identify the structural factors that affect such patterns. We theoretically provide the necessary conditions for Turing instability in single-layer and multiplex networks by considering high-order interactions. In the numerical simulation, we demonstrate that the Turing pattern could be controlled by adjusting the diffusion coefficient, high-order structure intensity, and average degree of the network. The results indicate that: (i) in a single-layer network, the Turing pattern only exists when high-order interactions appear, and the difference in diffusion rate plays a decisive role, (ii) in a multiplex network, the Turing pattern can still be observed under the same diffusion rates, which are affected by the difference in higher-order intensity between the two layers, and (iii) in existing networks, the average degree of the network has an important impact on Turing pattern. These findings contribute toward comprehending the impact of network structure on pattern formation, particularly the high-order interactions on the Turing pattern.

Digitalization of Analysis of a Concrete Block Layer Using Machine Learning as a Sustainable Approach

The concrete block pavement (CBP) system has a surface layer consisting of concrete block pavers and joint sand over a bedding sand layer. The non-homogeneous nature of the surface course of CBP, along with different laying patterns and shapes of block pavers, makes the analysis of CBP cumbersome. In this study, the surface course of CBP was modeled based on the slab action of the block pavers and joint sand, which are connected together in full contact. Four different laying patterns, including herringbone, stretcher, parquet, and square, were modeled using a finite element model. The elastic moduli of the block pavers varied from 2500 MPa to 45,000 MPa, with thicknesses ranging from 60 mm to 120 mm. As a result, modeling of CBP based on slab action can be considered a realistic strategy. In addition, a dataset was created based on quantitative inputs, e.g., elastic modulus and thickness of the block pavers, and qualitative input, i.e., block laying patterns. The approaches of machine learning adopted were support vector regression, Gaussian process regression, single-layer and deep artificial neural networks, and least squares boosting to implement prediction approach based on input and output. The analyses of statistical accuracy of all five machine learning methods showed high accuracy; however, the Gaussian process and deep artificial neural network methods resulted in the most accurate outputs and are recommended for further studies. Based on the machine learning models, digitalization is achieved through the development of simple, user-friendly software for electronic devices in order to perform a preliminary analysis of different laying patterns of CBP. Such a platform may result in less laboratory work and boosts the level of sustainability in concrete block pavement technology.

Short-term vital parameter forecasting in the intensive care unit: A benchmark study leveraging data from patients after cardiothoracic surgery.

Patients in an Intensive Care Unit (ICU) are closely and continuously monitored, and many machine learning (ML) solutions have been proposed to predict specific outcomes like death, bleeding, or organ failure. Forecasting of vital parameters is a more general approach to ML-based patient monitoring, but the literature on its feasibility and robust benchmarks of achievable accuracy are scarce. We implemented five univariate statistical models (the naïve model, the Theta method, exponential smoothing, the autoregressive integrated moving average model, and an autoregressive single-layer neural network), two univariate neural networks (N-BEATS and N-HiTS), and two multivariate neural networks designed for sequential data (a recurrent neural network with gated recurrent unit, GRU, and a Transformer network) to produce forecasts for six vital parameters recorded at five-minute intervals during intensive care monitoring. Vital parameters were the diastolic, systolic, and mean arterial blood pressure, central venous pressure, peripheral oxygen saturation (measured by non-invasive pulse oximetry) and heart rate, and forecasts were made for 5 through 120 minutes into the future. Patients used in this study recovered from cardiothoracic surgery in an ICU. The patient cohort used for model development (n = 22,348) and internal testing (n = 2,483) originated from a heart center in Germany, while a patient sub-set from the eICU collaborative research database, an American multicenter ICU cohort, was used for external testing (n = 7,477). The GRU was the predominant method in this study. Uni- and multivariate neural network models proved to be superior to univariate statistical models across vital parameters and forecast horizons, and their advantage steadily became more pronounced for increasing forecast horizons. With this study, we established an extensive set of benchmarks for forecast performance in the ICU. Our findings suggest that supplying physicians with short-term forecasts of vital parameters in the ICU is feasible, and that multivariate neural networks are most suited for the task due to their ability to learn patterns across thousands of patients.

The impact of indirect structure on functional connectivity in schizophrenia using a multiplex brain network

It is known that abnormal functional connectivity (FC) in schizophrenia (SZ) is closely related to structural connectivity (SC). We speculate that indirect SC also have an impact on FC in SZ patients. Conventional single-layer network has limitations for studying the relationship between indirect SC and FC. Thus, this study constructed a multiplex network based on structural connectivity and functional connectivity (SC-FC). The SC-FC bandwidth and SC-FC cost are used to analyze the impact of indirect SC on FC. Moreover, this paper proposed mediation ability, mediation cost, mediated strength and mediated cost to quantify the effects of mediator nodes and mediated nodes on indirect SC. The results show that SZ patients exhibit lower SC-FC bandwidth and SC-FC cost compared to healthy controls (HC), which could be caused by the limbic and subcortical network (LSN), default mode network (DMN) and visual network (VN). The mediator and mediated nodes in indirect SC of SZ patients also showed diminished effects. These findings suggest that functional communication ability and cost in SZ patients are influenced by indirect SC. This study provides new perspectives for understanding the relationship between indirect SC and FC, and provides strong evidence for interpreting the physiological mechanisms of SZ patients.