Temporal Entropy Research Articles

Overview of the Low Complexity Enhancement Video Coding (LCEVC) Standard

The Low Complexity Enhancement Video Coding (LCEVC) specification is a recent standard approved by the ISO/IEC JTC 1/SC 29/WG04 (MPEG) Video Coding. The main goal of LCEVC is to provide a standalone toolset for the enhancement of any other existing codec. It works on top of other coding schemes, resulting in a multi-layer video coding technology, but unlike existing scalable video codecs, adds enhancement layers completely independent from the base video. The LCEVC technology takes as input the decoded video at lower resolution and adds up to two enhancement sub-layers of residuals encoded with specialized low-complexity coding tools, such as simple temporal prediction, frequency transform, quantization, and entropy encoding. This paper provides an overview of the main features of the LCEVC standard: high compression efficiency, low complexity, minimized requirements of memory and processing power.

Capture and Prediction of Rainfall-Induced Landslide Warning Signals Using an Attention-Based Temporal Convolutional Neural Network and Entropy Weight Methods.

The capture and prediction of rainfall-induced landslide warning signals is the premise for the implementation of landslide warning measures. An attention-fusion entropy weight method (En-Attn) for capturing warning features is proposed. An attention-based temporal convolutional neural network (ATCN) is used to predict the warning signals. Specifically, the sensor data are analyzed using Pearson correlation analysis after obtaining data from the sensors on rainfall, moisture content, displacement, and soil stress. The comprehensive evaluation score is obtained offline using multiple entropy weight methods. Then, the attention mechanism is used to weight and sum different entropy values to obtain the final landslide hazard degree (LHD). The LHD realizes the warning signal capture of the sensor data. The prediction process adopts a model built by ATCN and uses a sliding window for online dynamic prediction. The input is the landslide sensor data at the last moment, and the output is the LHD at the future moment. The effectiveness of the method is verified by two datasets obtained from the rainfall-induced landslide simulation experiment.

LTL cross entropy optimisation for quadcopter task orchestration

ABSTRACT This paper presents a task orchestration framework for multi-agent systems utilising linear temporal logic (LTL) and cross entropy optimisation, a stochastic optimisation technique used for rare-event sampling. We define task orchestration as a combination of task decomposition, allocation and planning for a quadcopter or team of quadcopters given a high-level specification. Specifically, we consider tasks that are complex and consist of environment constraints, system constraints, or both, that must be satisfied. We first approach motion planning for the single agent case where transition systems for the environment allow tasks to be developed as linear temporal logic (LTL) specifications. Trajectories are then generated via motion primitives for a single quadcopter and optimised via cross entropy to ensure optimal satisfaction of a cost function. We extend this work to the multi-agent case where a team of homogeneous quadcopters are considered to satisfy an LTL specification. In order to provide faster computations and initial cost-agnostic sampling, we formulate the online version of multi-agent task allocation via cross entropy for tasks specified in LTL specifications. The results of this framework are verified in simulation and experimentally with a team of quadcopters.

Detecting the critical states during disease development based on temporal network flow entropy.

Complex diseases progression can be generally divided into three states, which are normal state, predisease/critical state and disease state. The sudden deterioration of diseases can be viewed as a bifurcation or a critical transition. Therefore, hunting for the tipping point or critical state is of great importance to prevent the disease deterioration. However, it is still a challenging task to detect the critical states of complex diseases with high-dimensional data, especially based on an individual. In this study, we develop a new method based on network fluctuation of molecules, temporal network flow entropy (TNFE) or temporal differential network flow entropy, to detect the critical states of complex diseases on the basis of each individual. By applying this method to a simulated dataset and six real diseases, including respiratory viral infections and tumors with four time-course and two stage-course high-dimensional omics datasets, the critical states before deterioration were detected and their dynamic network biomarkers were identified successfully. The results on the simulated dataset indicate that the TNFE method is robust under different noise strengths, and is also superior to the existing methods on detecting the critical states. Moreover, the analysis on the real datasets demonstrated the effectiveness of TNFE for providing early-warning signals on various diseases. In addition, we also predicted disease deterioration risk and identified drug targets for cancers based on stage-wise data.

Temporal Variation Measure Analysis: An Improved Second‐Order Difference Plot

In this study, an improved second‐order difference plot is proposed to analyze the variability of heart rate variability. Although the variation of the physiological status of the cardiovascular system can be shown graphically by the second‐order difference plot, the descriptive ability of existing indicators for this plot is insufficient. As a result, the physiological information contained in the second‐order difference plot cannot be extracted adequately. Addressing the problem, the temporal variation measure analysis is presented to describe distribution patterns of scatter points in the second‐order difference plot quantitatively and extract the acceleration information for a variation of heart rate variability. Experiment results demonstrate the effectiveness of the temporal variation measure analysis. As a quantitative indicator, the temporal variation entropy is properly designed and successfully applied in the recognition and classification of the physiological statuses of the heart.

Limiter-based entropy stabilization of semi-discrete and fully discrete schemes for nonlinear hyperbolic problems

The algebraic flux correction (AFC) schemes presented in this work constrain a standard continuous finite element discretization of a nonlinear hyperbolic problem to satisfy relevant maximum principles and entropy stability conditions. The desired properties are enforced by applying a limiter to antidiffusive fluxes that represent the difference between the high-order baseline scheme and a property-preserving approximation of Lax–Friedrichs type. In the first step of the limiting procedure, the given target fluxes are adjusted in a way that guarantees preservation of local and/or global bounds. In the second step, additional limiting is performed, if necessary, to ensure the validity of fully discrete and/or semi-discrete entropy inequalities. The limiter-based entropy fixes considered in this work are applicable to finite element discretizations of scalar hyperbolic equations and systems alike. The underlying inequality constraints are formulated using Tadmor’s entropy stability theory.The proposed limiters impose entropy-conservative or entropy-dissipative bounds on the rate of entropy production by antidiffusive fluxes and Runge–Kutta (RK) time discretizations. Two versions of the fully discrete entropy fix are developed for this purpose. The first one incorporates temporal entropy production into the flux constraints, which makes them more restrictive and dependent on the time step. The second algorithm interprets the final stage of a high-order AFC-RK method as a constrained antidiffusive correction of an implicit low-order scheme (algebraic Lax–Friedrichs in space + backward Euler in time). In this case, iterative flux correction is required, but the inequality constraints are less restrictive and limiting can be performed using algorithms developed for the semi-discrete problem. To motivate the use of limiter-based entropy fixes, we prove a finite element version of the Lax–Wendroff theorem and perform numerical studies for standard test problems. In our numerical experiments, entropy-dissipative schemes converge to correct weak solutions of scalar conservation laws, of the Euler equations, and of the shallow water equations.

Multi-Index Ecoacoustics Analysis for Terrestrial Soundscapes: A New Semi-Automated Approach Using Time-Series Motif Discovery and Random Forest Classification

High rates of biodiversity loss caused by human-induced changes in the environment require new methods for large scale fauna monitoring and data analysis. While ecoacoustic monitoring is increasingly being used and shows promise, analysis and interpretation of the big data produced remains a challenge. Computer-generated acoustic indices potentially provide a biologically meaningful summary of sound, however, temporal autocorrelation, difficulties in statistical analysis of multi-index data and lack of consistency or transferability in different terrestrial environments have hindered the application of those indices in different contexts. To address these issues we investigate the use of time-series motif discovery and random forest classification of multi-indices through two case studies. We use a semi-automated workflow combining time-series motif discovery and random forest classification of multi-index (acoustic complexity, temporal entropy, and events per second) data to categorize sounds in unfiltered recordings according to the main source of sound present (birds, insects, geophony). Our approach showed more than 70% accuracy in label assignment in both datasets. The categories assigned were broad, but we believe this is a great improvement on traditional single index analysis of environmental recordings as we can now give ecological meaning to recordings in a semi-automated way that does not require expert knowledge and manual validation is only necessary for a small subset of the data. Furthermore, temporal autocorrelation, which is largely ignored by researchers, has been effectively eliminated through the time-series motif discovery technique applied here for the first time to ecoacoustic data. We expect that our approach will greatly assist researchers in the future as it will allow large datasets to be rapidly processed and labeled, enabling the screening of recordings for undesired sounds, such as wind, or target biophony (insects and birds) for biodiversity monitoring or bioacoustics research.

Influence functional of many-body systems: Temporal entanglement and matrix-product state representation

Feynman–Vernon influence functional (IF) was originally introduced to describe the effect of a quantum environment on the dynamics of an open quantum system. We apply the IF approach to describe quantum many-body dynamics in isolated spin systems, viewing the system as an environment for its local subsystems. While the IF can be computed exactly only in certain many-body models, it generally satisfies a self-consistency equation, provided the system, or an ensemble of systems, are translationally invariant. We view the IF as a fictitious wavefunction in the temporal domain, and approximate it using matrix-product states (MPS). This approach is efficient provided the temporal entanglement of the IF is sufficiently low. We illustrate the broad applicability of the IF approach by analyzing several models that exhibit a range of dynamical behaviors, from thermalizing to many-body localized. In particular, we study the non-equilibrium dynamics in the quantum Ising model in both Floquet and Hamiltonian settings. We find that temporal entanglement entropy may be significantly lower compared to the spatial entanglement and analyze the IF in the continuous-time limit. We simulate the thermodynamic-limit evolution of local observables in various regimes, including the relaxation of impurities embedded in an infinite-temperature chain, and the long-lived oscillatory dynamics of the magnetization associated with the confinement of excitations. Furthermore, by incorporating disorder-averaging into the formalism, we analyze discrete time-crystalline response using the IF of a bond-disordered kicked Ising chain. In this case, we find that the temporal entanglement entropy scales logarithmically with evolution time. The IF approach offers a new lens on many-body non-equilibrium phenomena, both in ergodic and non-ergodic regimes, connecting the theory of open quantum systems to quantum statistical physics.

A machine learning-based approach for classifying tourists and locals using geotagged photos: the case of Tokyo

In tourism-dependent cities, investigating the spatiotemporal distribution and dynamics of tourist flows is crucial for better urban planning in both steady and perturbed states. In recent years, researchers have started relying more on photo-based, geotagged social data, which offer insights about tourists, popular hotspots, and mobility patterns. However, distinguishing between tourists and locals from this data is problematic since residence information is often not provided. While previous studies rely on heuristic (e.g., period of stay) and probabilistic (Shannon entropy) approaches, this paper proposes a method for classifying tourists and residents based on machine learning (ML) algorithms and considering parameters that could explain the variability between the two (e.g., weather, mobility, and photo content). This approach was applied to Flickr users’ geotagged photos taken in Tokyo’s 23 special wards from July 2008 to December 2019. The results show that stacked ensemble (SE) models are superior to models based on five supervised-learning algorithms, including gradient boosting machine (GBM), generalized linear model (GLM), distributed random forest (DRF), deep learning (DL), and extremely randomized trees (XRT). Temporal entropy (TEN), mobility on workdays, and frequent visits to amusement venues and crowded places influenced how users were classified. While temporal distribution showed similar monthly/hourly patterns, spatial distribution varied. The proposed approach might pave the way for scholars to carry out future tourism research on different topics and subsequently support policymakers in the decision-making process, specifically in urban settings.

Publisher’s Note

Feynman-Vernon influence functional (IF) was originally introduced to describe the effect of a quantum environment on the dynamics of an open quantum system. We apply the IF approach to describe quantum many-body dynamics in isolated spin systems, viewing the system as an environment for its local subsystems. While the IF can be computed exactly only in certain many-body models, it generally satisfies a self-consistency equation, provided the system, or an ensemble of systems, are translationally invariant. We view the IF as a fictitious wavefunction in the temporal domain, and approximate it using matrix-product states (MPS). This approach is efficient provided the temporal entanglement of the IF is sufficiently low. We illustrate the versatility of the IF approach by analyzing several models that exhibit a range of dynamical behaviors, from thermalizing to many-body localized. In particular, we study the non-equilibrium dynamics in the quantum Ising model in both Floquet and Hamiltonian settings. We find that temporal entanglement entropy may be significantly lower compared to the spatial entanglement and analyze the IF in the continuous-time limit. We simulate the thermodynamic-limit evolution of local observables in various regimes, including the relaxation of impurities embedded in an infinite-temperature chain, and the long-lived oscillatory dynamics of the magnetization associated with the confinement of excitations. By incorporating disorder-averaging into the formalism, we analyze discrete time-crystalline response using the IF method. In this case, we find that the temporal entanglement entropy scales logarithmically with evolution time. The IF approach offers a new lens on many-body non-equilibrium phenomena, both in ergodic and non-ergodic regimes, connecting the theory of open quantum systems theory to quantum statistical physics.

Novel Approach for EEG Signal Analysis in a Multifractal Paradigm of Motions. Epileptic and Eclamptic Seizures as Scale Transitions

Two different operational procedures are proposed for evaluating and predicting the onset of epileptic and eclamptic seizures. The first procedure analyzes the electrical activity of the brain (EEG signals) using nonlinear dynamic methods (the time variations of the standard deviation, the variance, the skewness and the kurtosis; the evolution in time of the spatial–temporal entropy; the variations of the Lyapunov coefficients, etc.). The second operational procedure reconstructs any type of EEG signal through harmonic mappings from the usual space to the hyperbolic one using the time homographic invariance of a multifractal-type Schrödinger equation in the framework of the scale relativity theory (i.e., in a multifractal paradigm of motions). More precisely, the explicit differential descriptions of the brain activity in the form of 2 × 2 matrices with real elements disclose, through the in-phase coherences at various scale resolutions (i.e., as scale transitions), the multitude of brain neuronal dynamics, especially sequences of epileptic and eclamptic seizures. These two operational procedures are not mutually exclusive, but rather become complementary, offering valuable information concerning epileptic and eclamptic seizures. In such context, the prediction of epileptic and eclamptic seizures becomes fundamental for patients not responding to medical treatment and also presenting an increased rate of seizure recurrence.

About the robustness of 1d cellular automata revising their temporal entropy

Complimentary to existing description methods of one-dimensional cellular automata dynamics the present study proposes a new ‘micro-historicizing’ approach inspired by Wolfram’s concept of temporal entropy and Kitano’s concept of robustness derived from biology: introducing ‘temporal sub-attractors’ the cell activity of specific automata settings can be typified in a novel way as kind of underlying phase space similar to a conditional heat map representing the activity of individual cells. The robustness of such a sub-phase space can be derived from two morphological trajectories: a robust sub-phase space can be expressed graphically as a sequence of temporal sub-attractor loops of rebalancing activity curves. Whereas a bifurcation of the sub-phase space with continuously diverging curves indicates a non-robust automaton configuration. Conceptually, the temporal sub-attractors bring a biological stress component into the physically inspired cellular automata models, which might not only be of help for modelling in biology, but also in material science and engineering science.

The eco-evolutionary significance of rainfall constancy for facultative CAM photosynthesis.

A flexible use of the crassulacean acid metabolism (CAM) has been hypothesised to represent an intermediate stage along a C3 to full CAM evolutionary continuum, when relative contributions of C3 vs CAM metabolism are co-determined by evolutionary history and prevailing environmental constraints. However, evidence for such eco-evolutionary interdependencies is lacking. We studied these interdependencies for the leaf-succulent genus Drosanthemum (Aizoaceae, Southern African Succulent Karoo) by testing for relationships between leaf δ13 C diagnostic for CAM dependence (i.e. contribution of C3 and CAM to net carbon gain), and climatic variables related to temperature and precipitation and their temporal variation. We further quantified the effects of shared phylogenetic ancestry on CAM dependence and its relation to climate. CAM dependence is predicted by rainfall and its temporal variation, with high predictive power of rainfall constancy (temporal entropy). The predictive power of rainfall seasonality and temperature-related variables was negligible. Evolutionary history of the tested clades significantly affected the relationship between rainfall constancy and CAM dependence. We argue that higher CAM dependence might provide an adaptive advantage in increasingly unpredictable rainfall environments when the anatomic exaptation (succulence) is already present. These observations might shed light on the evolution of full CAM.

Estimating Stable Measured Values and Detecting Anomalies in Groundwater Geochemistry Time Series Data Across the Athabasca Oil Sands Area, Canada

Regional groundwater monitoring in the Athabasca region of Alberta, Canada, provides information on groundwater quality and geochemical changes over time, including data useful for evaluating potential impacts of industrial activity such as oil sands mining and in situ operations. Data collected from over 5000 wells from the 1950s to 2014, including 161 wells from government’s monitoring network, were used to develop and apply bootstrap techniques for the detection of changes in groundwater geochemistry over time and at specific points in time. Increasing temporal anomalies were identified in Cl, TDS, B, and naphthenic acids in the McMurray formation across 2003 and 2008, while decreasing anomalies were found for SO4. Temporal variance for 15 indicators was quantified for a smooth bootstrap approach to arrive at stable values representative of the most recent samples taken from wells in the study area. Stable values revealed sampling bias in the Devonian, Grand Rapids, Empress, Channel Beverly, and Muriel Lake formations suggesting expansion of sampling may be necessary. Although temporal anomalies were found in the McMurray formation, sampling bias was not identified. The entropy and relative magnitude of time series were evaluated to identify candidate wells for continued observations, which consist of wells with low measurements and low entropy that are near active industry lease boundaries. Temporal anomalies, stable values, and entropy were combined into type-well information to provide plots for visual inspection and interpretation. Stable values are useful for regional mapping, for detecting future changes and trends, and for identifying areas of interest warranting further investigation.

Research on the Coupling Coordination Relationships between Urban Function Mixing Degree and Urbanization Development Level Based on Information Entropy.

With the rapid development of urbanization, the blind expansion of urban space has led to a series of social problems. In this process, the degree of urban function mixing affects the urbanization development level, making it particularly important to study the degree of coupling coordination between the two aspects. In this paper, taking Beijing as an example, we use urban point of interest (POI) data and taxi GPS trajectory data to calculate the urban POIs’ spatial entropy and taxis’ temporal entropy, based on the information entropy. We use the POIs’ spatial entropy and taxis’ temporal entropy to measure the urban function mixing degree. Also, the model of coupling coordination degree is used to measure the degree of coupling coordination between the urban function mixing degree and the urbanization development level. The results indicate the following: First, the POIs’ spatial entropy and taxis’ temporal entropy have significant regional imbalances. On the whole, both show a declining pattern when moving from the central urban area to the outer suburbs. The urban function mixing degree and urbanization development level are also higher in the central urban area than in the outer suburbs. Second, the coupling coordination among the urbanization development level, POIs’ spatial entropy, and taxis’ temporal entropy is distributed unevenly across various regions, which means that the three types of coupling coordination are in balanced development in the central urban area, but in unbalanced development in the outer suburbs. Third, from the perspective of spatial correlation characteristics, the higher is the degree of spatial agglomeration, the higher are the urban function mixing degree and urbanization development level, and the higher is the coupling coordination degree among the urbanization development level, POIs’ spatial entropy, and taxis’ temporal entropy. Therefore, relevant departments should plan the construction of urban functional areas reasonably, according to the degree of coupling coordination between the urban function mixing degree and the urbanization development level in different regions, so as to realize the healthy and sustainable development of a city.

A Novel Multivariate-Multiscale Approach for Computing EEG Spectral and Temporal Complexity for Human Emotion Recognition

This work proposes a novel multivariate-multiscale approach for computing the spectral and temporal entropies from the multichannel electroencephalogram (EEG) signal. This facilitates the recognition of three human emotions: positive, neutral, and negative. The proposed approach is based on the application of the Fourier-Bessel series expansion based empirical wavelet transform (FBSE-EWT). We have extended the existing FBSE-EWT method for multichannel signals and derived FBSE-EWT based multivariate Hilbert marginal spectrum (MHMS) for computing spectral Shannon and K-nearest neighbor (K-NN) entropies. The multivariate FBSE-EWT decomposes the multichannel EEG signals into narrow band subband signals. The multiscaling operation adapted in the spectral domain is based on the selection of successive joint instantaneous amplitude and frequency functions of the subband signals. On the other hand, the time domain multiscale K-NN entropy is computed from the cumulatively added multidimensional subband signals. The extracted spectral and temporal entropy features are smoothed and fed to the sparse autoencoder based random forest (ARF) classifier architecture for emotion classification. The proposed approach is tested using multichannel EEG signals available in a public database (SJTU emotion EEG dataset (SEED)). The bivariate EEG signals from different channel pairs with distinct spatial locations over the scalp are considered as input to our proposed system. The obtained overall classification accuracy of 94.4% reveals that the proposed approach is useful in classifying human emotions. The method is also validated using DREAMER emotion EEG public database. The method outperforms the existing state-of-the-art methods evaluated in these databases.

Unified Analog PUF and TRNG Based on Current-Steering DAC and VCO

This work presents a unified weak physical unclonable function (PUF) and a true random number generator (TRNG) based on the current-steering digital-to-analog converter (DAC) and ring voltage-controlled oscillator (VCO). Entropy source for the weak PUF is the mismatch between NMOS and PMOS transistors in a cascode current DAC as well as the mismatch between VCO quantizers, while entropy source for the TRNG is thermal noise in the DAC and VCO and clock jitter. Instead of using spatial entropy sources for the PUF, i.e., multiple unit PUF elements, the proposed architecture utilizes temporal entropy source by capturing the output of unit PUF element over multiple cycles, which reduces area significantly. A unified PUF/TRNG prototype is fabricated in 65-nm CMOS and consumes 0.36 pJ/bit at a throughput of 100 Mb/s. The PUF has a measured intra-HD of 0.0906 and inter-HD of 0.4859, while the raw TRNG bitstream has an entropy of 0.9991 and passes all the NIST statistical randomness tests.

Estimating spatiotemporal focus of documents using entropy with PMI

Many text documents are spatiotemporal in nature, i.e. contents of a document can be mapped to a specific time period or location. For example, a news article about the French Revolution can be mapped to year 1789 as time and France as place. Identifying this time period and location associated with the document can be useful for various downstream applications such as document reasoning or spatiotemporal information retrieval. In this paper, temporal entropy with pointwise mutual information (PMI) is proposed to estimate the temporal focus of a document. PMI is used to measure the association of words with time expressions. Moreover, a word's temporal entropy is considered as a weight to its association with a time point and a single time point with the highest overall score is chosen as the focus time of a document. The proposed method is generic in the sense that it can also be applied for spatial focus estimation of documents. In the case of spatial entropy with PMI, PMI is used to calculate the association between words and place entities. The effectiveness of our proposed methods for spatiotemporal focus estimation is evaluated on diverse datasets of text documents. The experimental evaluation confirms the superiority of our proposed temporal and spatial focus estimation methods.

Spatial and Temporal Entropies in the Spanish Football League: A Network Science Perspective.

We quantified the spatial and temporal entropy related to football teams and their players by means of a pass-based interaction. First, we calculated the spatial entropy associated to the positions of all passes made by a football team during a match, obtaining a spatial entropy ranking of Spanish teams during the 2017/2018 season. Second, we investigated how the player’s average location in the field is related to the amount of entropy of his passes. Next, we constructed the temporal passing networks of each team and computed the deviation of their network parameters along the match. For each network parameter, we obtained the permutation entropy and the statistical complexity of its temporal fluctuations. Finally, we investigated how the permutation entropy (and statistical complexity) of the network parameters was related to the total number of passes made by a football team. Our results show that (i) spatial entropy changes according to the position of players in the field, and (ii) the organization of passing networks change during a match and its evolution can be captured measuring the permutation entropy and statistical complexity of the network parameters, allowing to identify what parameters evolve more randomly.

Examination of a Shannon entropy-like figure of merit for controlled release of drugs and other molecules

We propose a figure of merit, based on temporal entropy, to evaluate the degree to which a controlled drug release system is able to adhere to its target release pattern. This criterion can be used, for example, to design multilayered drug release systems whose release flux optimally tracks the target pattern. Such multilayered systems are produced by sequential polymerization, sequential spraying, 3D printing, and other additive manufacturing techniques. After calculating this figure of merit for certain canonical controlled release mechanisms, with zero-order release as the target pattern, we optimize the pattern of initial drug concentrations in a multilayer system with respect to the figure of merit.