Nonlinear Interdependence Research Articles

Augmented Convolutional LSTMs for Generation of High-Resolution Climate Change Projections

Projection of changes in extreme indices of climate variables such as temperature and precipitation are critical to assess the potential impacts of climate change on human-made and natural systems, including critical infrastructures and ecosystems. While impact assessment and adaptation planning rely on high-resolution projections (typically in the order of a few kilometers), state-of-the-art Earth System Models (ESMs) are available at spatial resolutions of few hundreds of kilometers. Current solutions to obtain high-resolution projections of ESMs include downscaling approaches that consider the information at a coarse-scale to make predictions at local scales. Complex and non-linear interdependence among local climate variables (e.g., temperature and precipitation) and large-scale predictors (e.g., pressure fields) motivate the use of neural network-based super-resolution architectures. In this work, we present auxiliary variables informed spatio-temporal neural architecture for statistical downscaling. The current study performs daily downscaling of precipitation variable from an ESM output at 1.15 degrees (115 km) to 1/4 degrees (25 km) over one of the most climatically diversified countries, India. We showcase significant improvement gain against two popular state-of-the-art baselines with a better ability to predict statistics of extreme events. To facilitate reproducible research, we make available all the codes, processed datasets, and trained models in the public domain.

Non-Linear Interdependencies between International Stock Markets: The Polish and Spanish Case

This research analyzes non-linear interdependencies between the Polish (WIG20) and the Spanish (IBEX 35) stock market returns with some other relevant international stock market returns, such as the German (DAX-30), the British (FTSE-100), the American (S&P 500) and the Chinese (SSE Composite) stock markets. In addition, this research focuses on the impact of the stage of the economy on these interdependencies, in concrete, on the influence of the 2008 Global Financial Crisis. To that end, we use a nonlinear autoregressive distributed lag (NARDL) approach in the sample period between January 1998 to December 2018. Our results show positive interdependencies between the Polish and the Spanish stock markets with the international reference stock markets analyzed in this research, as well as significant long-run relations between most of the stock markets. Furthermore, the Polish and the Spanish stock market returns may similarly react to positive and negative changes in international stock market returns, evidencing strong short-run asymmetry. In addition, both countries show great persistence in response to both positive and negative changes in stock market returns in the other mayor international markets. Finally, the NARDL model proposed in this research would show good explanatory power, mainly to changes in the international stock market returns, except for the Chinese market.

A novel decentralized detection framework for quality-related faults in manufacturing industrial processes

Quality-related fault detection is the key subject under process monitoring and fault diagnosis (PM-FD) framework, which is an effective mean to guarantee safety production and obtain reliable product quality, and thus, has recently become active areas of process control fields. In this paper, a novel decentralized detection method for quality-related faults is proposed for manufacturing industrial processes, which will help field engineers to purposefully and real-time observe the operation state of the production process. Specifically, the main innovations are: 1) the linear and nonlinear dynamic interdependencies between process and quality variables are revealed and quantified by the minimal redundancy maximal relevance (mRMR) algorithm, and the most representative variables are selected in each subprocess; 2) a new mixed kernel function based dynamic mixed kernel principal component analysis (DMKPCA) model is designed for enhancing the generalization ability and maintaining the data characteristics of original sample space; 3) unified monitoring statistics are constructed in the subprocess level, and Bayesian fusion is implemented for forming monitoring decisions from the plant-wide level. Finally, a case study on an actual hot rolling mill process (HSMP) is finally given to validate the effectiveness of the proposed scheme, and several competitive methods are applied to carry out the quality-related fault detection process.

EHR application portfolio and hospital performance: Effects across hospitals with varying administrative scale and clinical complexity

Electronic Health Record (EHR) has the potential to transform the work required to create and deliver healthcare services. This has triggered large-scale adoption across hospitals. However, whether all hospitals obtain a similar effect from their EHR implementations remains an important question because there are significant differences differences in characteristics of hospitals adopting these systems. To examine differences in effects across hospitals, we examine whether the impacts of EHR applications are contingent on work domains, by assessing performance effects across hospitals with varying administrative scale and clinical complexity. Because EHRs constitute a suite of applications with different functionalities, examining the effects of different sets of applications is challenging due to nonlinear interdependencies between applications. Therefore, we use an archetype approach, identifying synergistic EHR (EHRSYN) archetype as an ideal portfolio—a conceptual anchor for a hospital’s EHR portfolio. We test contingencies by combining this technology archetype with the work domains—administrative scale and clinical complexity. We test our hypotheses using empirical data from 137 hospitals in California, hypothesizing the differences in effects of EHRs on financial, operational, and clinical performances across hospitals with different administrative scales (size) and clinical complexities (case-mix) of work. While hospitals gain the most by implementing a portfolio close to the EHRSYN archetype, our nuanced models reveal that the benefits of such portfolios increase for large hospitals and are greater for hospitals treating less complex cases. These findings underscore how variations in applications used and work domains demarcate boundaries related to the performance effects of EHRs. We present a detailed discussion of the theoretical contributions and practical implications of our findings.

Quantification of nonlinear interdependence in complex systems dynamics: simulations and applications

In our work we studied the nonlinear interdependence metric quantifying the mutual dynamics of two stochastic data series. This metric is based on the calculation of the Euclidean distances between points belonging to the trajectories of these series in the state-space. Using surrogate data as an example, the sensitivity of the metric to the autocorrelation properties of the studied data series, as well as to the amplitude and phase randomization, are investigated. We also considered the application of this metric to the analysis of backscatter signals in sea surface monitoring. We suggest that the nonlinear interdependence metric may be useful as a complementary indicator for the sea wave structure quantification and modeling.

Studying the Incubation of a New Product Market Through Realized and Alternative Histories

In this paper, we contribute to the literature on new market emergence by implementing a “history-friendly” simulation of the incubation period encompassing the decision processes that took place within General Motors (GM), Chrysler, and Ford during the design and development of the first U.S. minivan. Our work offers a “middle ground” alternative methodology for unpacking the “black box” of market incubation processes. Our middle ground approach is useful for exploring the complex interdependencies among four general mechanisms that shape market incubation: environmental shocks that open up new technological possibilities, firm-level capabilities that allow firms to differentially take advantage of these possibilities, various incentives (e.g., product cannibalization, rivalry) that influence a firm’s willingness to exploit new possibilities, and managerial beliefs about the viability of these possibilities. Complex nonlinear interdependencies exist among these mechanisms, and historical contingencies affect the way in which they interact. We identify important historical contingencies within and across GM, Ford, and Chrysler in their precommercialization managerial decisions prior to Chrysler’s introduction of the Voyager and Caravan in 1983. We use the historical details of actual market incubation to calibrate the simulation and develop plausible alternative (both near and hypothetical) histories of that incubation. Understanding why Chrysler, Ford, and General Motors made their respective product commercialization choices not only pertains to automotive history, but can also illuminate the complexities inherent in market incubation processes.

Multiphysics for nuclear energy applications using a cohesive computational framework

With the recent development of advanced numerical algorithms, software design, and low-cost high-performance computer hardware, reliance on coupled multiphysics to predict the behavior of complex physical systems is beginning to become standard practice. This is especially true in nuclear energy applications where strong nonlinear interdependencies exist between reactor physics, radiation transport, multi-scale nuclear fuels performance, thermal fluids, etc. Resolving these nonlinear dependencies requires choices in multiphysics software approaches. Two main multiphysics modeling and simulation approaches have emerged. The first is based upon “code coupling” where disparate physics codes of different software design, code languages, and spatial and temporal integration schemes are coupled together with relatively complex data passing interfaces. The second multiphysics software approach is to employ a “cohesive” framework where all physics applications are developed with a common software design, i.e., data structures, syntax, input format, integrated spatial and temporal discretization schemes, etc. Here we present the Multiphysics Object-Oriented Simulation Environment (MOOSE) development and runtime framework and describe the framework’s cohesive modeling and simulation multiphysics approach. Then, a “cohesive-like” extension of the MOOSE framework is presented where MOOSE-based physics software applications are efficiently coupled to non-MOOSE (external) physics codes to form multiphysics applications using MOOSE’s unique interface capabilities. Finally, several examples of MOOSE’s cohesive and cohesive-like multiphysics applications will be demonstrated. These multiphysics demonstrations will incorporate both MOOSE-based applications and external codes, including Nek5000, RELAP-7, TRACE, BISON, and Pronghorn.

Band Selection of Hyperspectral Images Using Attention-Based Autoencoders

Band selection is an effective method to reduce redundancy in a hyperspectral image (HSI) without compromising the original contents. Popular band selection methods usually use strong assumptions, such as linear or nonlinear assumptions with simple predefined Kernel functions, to model the correlations between bands. However, this kind of strong assumption may not valid in the real environment due to the complex interactions between bands. In this letter, we treat hyperspectral band selection as a spectral reconstruction task. By assuming that an HSI can be sparsely reconstructed from a few informative bands, we propose an attention-based autoencoder to model the underlying nonlinear interdependencies between bands. The proposed model consists of two parts: an attention module and an autoencoder. The attention module is used to produce the attention mask which selects the most informative bands for every pixel. The autoencoder uses these informative bands to reconstruct the raw HSI. The final band selection is conducted via clustering column vectors of the attention mask and exploring the most representative band for each cluster. Different from most of the existing band selection methods, the proposed method directly learns global nonlinear correlations between bands without strong assumptions. The proposed model is easy to implement and all the parameters can be jointly optimized using the stochastic gradient descend algorithm. Experiments on three open public data sets show that the proposed method offers the promising results.

BS-Nets: An End-to-End Framework for Band Selection of Hyperspectral Image

Hyperspectral image (HSI) consists of hundreds of continuous narrow bands with high spectral correlation, which would lead to the so-called Hughes phenomenon and the high computational cost in processing. Band selection has been proven effective in avoiding such problems by removing the redundant bands. However, many of existing band selection methods separately estimate the significance for every single band and cannot fully consider the nonlinear and global interaction between spectral bands. In this paper, by assuming that a complete HSI can be reconstructed from its few informative bands, we propose a general band selection framework, Band Selection Network (termed as BS-Net). The framework consists of a band attention module (BAM), which aims to explicitly model the nonlinear inter-dependencies between spectral bands, and a reconstruction network (RecNet), which is used to restore the original HSI cube from the learned informative bands, resulting in a flexible architecture. The resulting framework is end-to-end trainable, making it easier to train from scratch and to combine with existing networks. We implement two BS-Nets respectively using fully connected networks (BS-Net-FC) and convolutional neural networks (BS-Net-Conv), and compare the results with many existing band selection approaches for three real hyperspectral images, demonstrating that the proposed BS-Nets can accurately select informative band subset with less redundancy and achieve significantly better classification performance with an acceptable time cost.

Promoting Sustainability Culture among Citizens: A System Dynamic Analysis of the Role of Schools

This paper concerns the issue of how to promote sustainable behaviours among citizens and consumers, that is increasingly viewed as crucial to tackle global social and environmental problems. More in detail, it investigates the potential role of schools at all levels, from kinder garden to university, in their relation to (mainly) local stakeholders. The study aims at shedding light on potential diffusion dynamics of sustainable behaviour among individuals when schools, universities and other stakeholders are involved. To do so, we develop a simple system dynamics model, that is an approach that is well suited to model non-linear dynamics, interdependencies, and time lags typical of human decision processes and behaviour. The model proposed is a first preliminary attempt to highlight the system effects that could be triggered by active sustainability projects in schools. The variables included in the model and the data are taken from an action-research project ‘Sustainability Labs’ that is being carried out in a set of schools in the Milan (Italy) area. Results suggest that taking advantage of the school-university-local stakeholder triad is able to reach many people and in quite an engaging way that is likely to have interesting results in terms of behavioural change. &nbsp

COPULA-BASED STATISTICAL MODELLING OF SYNOPTIC-SCALE CLIMATE INDICES FOR QUANTIFYING AND MANAGING AGRICULTURAL RISKS IN AUSTRALIA

COPULA-BASED STATISTICAL MODELLING OF SYNOPTIC-SCALE CLIMATE INDICES FOR QUANTIFYING AND MANAGING AGRICULTURAL RISKS IN AUSTRALIA

Detecting and quantifying causal associations in large nonlinear time series datasets.

Identifying causal relationships and quantifying their strength from observational time series data are key problems in disciplines dealing with complex dynamical systems such as the Earth system or the human body. Data-driven causal inference in such systems is challenging since datasets are often high dimensional and nonlinear with limited sample sizes. Here, we introduce a novel method that flexibly combines linear or nonlinear conditional independence tests with a causal discovery algorithm to estimate causal networks from large-scale time series datasets. We validate the method on time series of well-understood physical mechanisms in the climate system and the human heart and using large-scale synthetic datasets mimicking the typical properties of real-world data. The experiments demonstrate that our method outperforms state-of-the-art techniques in detection power, which opens up entirely new possibilities to discover and quantify causal networks from time series across a range of research fields.

Using Information Flow for Whole System Understanding From Component Dynamics

AbstractComplex systems that exhibit emergent behaviors arise as a result of nonlinear interdependencies among multiple components. Characterizing how such whole system dynamics are sustained through multivariate interaction remains an open question. In this study, we propose an information flow‐based framework to investigate how the present state of any component arises as a result of the past interactions among interdependent variables, which is termed as causal history. Using a partitioning time lag, we divide this into immediate and distant causal history components and then characterize the information flow‐based interactions within these as self‐ and cross‐feedbacks. Such a partition allows us to characterize the information flow from the two feedbacks in both histories by using partial information decomposition as unique, synergistic, or redundant interactions. We employ this casual history analysis approach to investigate the information flows in a short‐memory coupled logistic model and a long‐memory observed stream chemistry dynamics. While the dynamics of the short‐memory system are mainly maintained by its recent historical states, the current state of each stream solute is sustained by self‐feedback‐dominated recent dynamics and cross‐dependency‐dominated earlier dynamics. The analysis suggests that the observed 1/fsignature of each solute is a result of the interactions with other variables in the stream. Based on high‐density data streams, the approach developed here for investigating multivariate evolutionary dynamics provides an effective way to understand how components of dynamical system interact to create emergent whole system behavioral patterns such as long‐memory dependency.

Evaluation of synchronization measures for capturing the lagged synchronization between EEG channels: A cognitive task recognition approach

During cognitive, perceptual and sensory tasks, connectivity profile changes across different regions of the brain. Variations of such connectivity patterns between different cognitive tasks can be evaluated using pairwise synchronization measures applied to electrophysiological signals, such as electroencephalography (EEG). However, connectivity-based task recognition approaches achieving viable recognition performance have been lacking from the literature. By using several synchronization measures, we identify time lags between channel pairs during different cognitive tasks.We employed mutual information, cross correntropy, cross correlation, phase locking value, cosine similarity and nonlinear interdependence measures. In the training phase, for each type of cognitive task, we identify the time lags that maximize the average synchronization between channel pairs. These lags are used to calculate pairwise synchronization values with which we construct the train and test feature vectors for recognition of the cognitive task carried out using Fisher's linear discriminant (FLD) analysis.We tested our framework in a motor imagery activity recognition scenario on PhysioNet Motor Movement/Imagery and BCI Competition-III Ⅳa datasets. For PhysioNet dataset, average performance results ranging between % 51 and % 61 across 20 subjects. For BCI Competition-Ⅲ dataset, we achieve an average recognition performance of % 76 which is above the minimum reliable communication rate (% 70).We achieved an average accuracy over the minimum reliable communication rate on the BCI Competition-Ⅲ dataset. Performance levels were lower on the PhysioNet dataset. These results indicate that a viable task recognition system is achievable using pairwise synchronization measures evaluated at the proper task specific lags.

A “Thermodynamic” Model of Central Commands Coming to the Muscles during Upper Limb Movements

A model of central commands (CCs) coming to the muscles during limb movements is proposed; it is based on experimental studies of the nonlinear dynamics of muscle contraction and of the parafrontal two-joint arm movements in humans. It is known that such CCs possess powerful hysteresis properties; levels of muscle activation rise within parts of the movement trajectories, where the muscles are actively shortened and overcome the action of an external force, but decrease during muscle stretching by the above force. Basing on an analogy with the classic thermodynamics, a “thermodynamic” model (T-model) is proposed, which allows one to predict the nonlinear functional interdependence of three basic parameters defining the muscle state, namely E (intensity of efferent activity), F (muscle force), and L (muscle length). The T-model includes the time-dependent integration of the equations defining the interrelations between infinitesimal changes of the above parameters, which are assumed to be the exact differentials. By inducing only one additional parameter, the hysteresis weight, the T-model can satisfactorily explain patterns of EMG records observed in real movements of the arm.

Maximum Entropy Analysis of Flow Networks: Theoretical Foundation and Applications.

The concept of a “flow network”—a set of nodes and links which carries one or more flows—unites many different disciplines, including pipe flow, fluid flow, electrical, chemical reaction, ecological, epidemiological, neurological, communications, transportation, financial, economic and human social networks. This Feature Paper presents a generalized maximum entropy framework to infer the state of a flow network, including its flow rates and other properties, in probabilistic form. In this method, the network uncertainty is represented by a joint probability function over its unknowns, subject to all that is known. This gives a relative entropy function which is maximized, subject to the constraints, to determine the most probable or most representative state of the network. The constraints can include “observable” constraints on various parameters, “physical” constraints such as conservation laws and frictional properties, and “graphical” constraints arising from uncertainty in the network structure itself. Since the method is probabilistic, it enables the prediction of network properties when there is insufficient information to obtain a deterministic solution. The derived framework can incorporate nonlinear constraints or nonlinear interdependencies between variables, at the cost of requiring numerical solution. The theoretical foundations of the method are first presented, followed by its application to a variety of flow networks.

EEG functional connectivity metrics wPLI and wSMI account for distinct types of brain functional interactions

The weighted Phase Lag Index (wPLI) and the weighted Symbolic Mutual Information (wSMI) represent two robust and widely used methods for MEG/EEG functional connectivity estimation. Interestingly, both methods have been shown to detect relative alterations of brain functional connectivity in conditions associated with changes in the level of consciousness, such as following severe brain injury or under anaesthesia. Despite these promising findings, it was unclear whether wPLI and wSMI may account for distinct or similar types of functional interactions. Using simulated high-density (hd-)EEG data, we demonstrate that, while wPLI has high sensitivity for couplings presenting a mixture of linear and nonlinear interdependencies, only wSMI can detect purely nonlinear interaction dynamics. Moreover, we evaluated the potential impact of these differences on real experimental data by computing wPLI and wSMI connectivity in hd-EEG recordings of 12 healthy adults during wakefulness and deep (N3-)sleep, characterised by different levels of consciousness. In line with the simulation-based findings, this analysis revealed that both methods have different sensitivity for changes in brain connectivity across the two vigilance states. Our results indicate that the conjoint use of wPLI and wSMI may represent a powerful tool to study the functional bases of consciousness in physiological and pathological conditions.

Temporal pattern attention for multivariate time series forecasting

Forecasting of multivariate time series data, for instance the prediction of electricity consumption, solar power production, and polyphonic piano pieces, has numerous valuable applications. However, complex and non-linear interdependencies between time steps and series complicate this task. To obtain accurate prediction, it is crucial to model long-term dependency in time series data, which can be achieved by recurrent neural networks (RNNs) with an attention mechanism. The typical attention mechanism reviews the information at each previous time step and selects relevant information to help generate the outputs; however, it fails to capture temporal patterns across multiple time steps. In this paper, we propose using a set of filters to extract time-invariant temporal patterns, similar to transforming time series data into its “frequency domain”. Then we propose a novel attention mechanism to select relevant time series, and use its frequency domain information for multivariate forecasting. We apply the proposed model on several real-world tasks and achieve state-of-the-art performance in almost all of cases. Our source code is available at https://github.com/gantheory/TPA-LSTM .

Alterations in EEG connectivity in healthy young adults provide an indicator of sleep depth.

Current sleep analyses have used electroencephalography (EEG) to establish sleep intensity through linear and nonlinear measures. Slow wave activity (SWA) and entropy are the most commonly used markers of sleep depth. The purpose of this study is to evaluate changes in brain EEG connectivity during sleep in healthy subjects and compare them with SWA and entropy. Four different connectivity metrics: coherence (MSC), synchronization likelihood (SL), cross mutual information function (CMIF), and phase locking value (PLV), were computed focusing on their correlation with sleep depth. These measures provide different information and perspectives about functional connectivity. All connectivity measures revealed to have functional changes between the different sleep stages. The averaged CMIF seemed to be a more robust connectivity metric to measure sleep depth (correlations of 0.78 and 0.84 with SWA and entropy, respectively), translating greater linear and nonlinear interdependences between brain regions especially during slow wave sleep. Potential changes of brain connectivity were also assessed throughout the night. Connectivity measures indicated a reduction of functional connectivity in N2 as sleep progresses. The validation of connectivity indexes is necessary because they can reveal the interaction between different brain regions in physiological and pathological conditions and help understand the different functions of deep sleep in humans.

A Call for Systems Epidemiology to Tackle the Complexity of Schistosomiasis, Its Control, and Its Elimination.

Ever since the first known written report of schistosomiasis in the mid-19th century, researchers have aimed to increase knowledge of the parasites, their hosts, and the mechanisms contributing to infection and disease. This knowledge generation has been paramount for the development of improved intervention strategies. Yet, despite a broad knowledge base of direct risk factors for schistosomiasis, there remains a paucity of information related to more complex, interconnected, and often hidden drivers of transmission that hamper intervention successes and sustainability. Such complex, multidirectional, non-linear, and synergistic interdependencies are best understood by looking at the integrated system as a whole. A research approach able to address this complexity and find previously neglected causal mechanisms for transmission, which include a wide variety of influencing factors, is needed. Systems epidemiology, as a holistic research approach, can integrate knowledge from classical epidemiology, with that of biology, ecology, social sciences, and other disciplines, and link this with informal, tacit knowledge from experts and affected populations. It can help to uncover wider-reaching but difficult-to-identify processes that directly or indirectly influence exposure, infection, transmission, and disease development, as well as how these interrelate and impact one another. Drawing on systems epidemiology to address persisting disease hotspots, failed intervention programmes, and systematically neglected population groups in mass drug administration programmes and research studies, can help overcome barriers in the progress towards schistosomiasis elimination. Generating a comprehensive view of the schistosomiasis system as a whole should thus be a priority research agenda towards the strategic goal of morbidity control and transmission elimination.