Second-order Information Research Articles

Distributionally Robust Two-Stage Energy Management for Hybrid Energy Powered Cellular Networks

Energy harvesting has been recognized as a good paradigm to decrease traditional grid energy expenditure, which caters for 5G visions on the green evolution of cellular networks. However, the harvested energy at each base station is random and intermittent, which imposes a great challenge to reliably satisfy the time-varying users’ wireless traffic requirements. What's worse, the perfect probability distributions of harvested energy and wireless traffic are hard to obtain in practical systems, making it difficult to apply the conventional optimization approach to cope with the uncertainties in the energy management problems. Inspired by this fact, we propose a distributionally robust two-stage stochastic optimization framework to minimize the expected total energy cost of mobile network operators (MNOs) in the finite time horizon. To circumvent the computational difficulty of this stochastic optimization problem, we define a novel ambiguity set to capture the uncertainties of the harvested energy and wireless traffic based on their first- and second-order statistics information. By employing the distributionally robust optimization (DRO) methodology, we equivalently transform the original two-stage stochastic programming problem into a computationally tractable second-order cone programming problem. Then, a distributionally robust two-stage energy management algorithm (DRTEMA) is proposed, which has good performance of low complexity and only requires partial information of the stochastic parameters. Rigorous computational complexity analysis and extensive numerical simulations are conducted to show the advantages of the proposed algorithm.

A Discussion on Variational Analysis in Derivative-Free Optimization

Variational Analysis studies mathematical objects under small variations. With regards to optimization, these objects are typified by representations of first-order or second-order information (gradients, subgradients, Hessians, etc). On the other hand, Derivative-Free Optimization studies algorithms for continuous optimization that do not use first-order information. As such, researchers might conclude that Variational Analysis plays a limited role in Derivative-Free Optimization research. In this paper we argue the contrary by showing that many successful DFO algorithms rely heavily on tools and results from Variational Analysis.

Video person re-identification with global statistic pooling and self-attention distillation

Most existing methods for video person re-identification apply spatial-temporal global average or attention pooling to aggregate frame-level feature and get video-level feature. The obtained video-level feature models only the first-order statistics of the appearance feature from holistic video, resulting in limited representation capability of the feature network. In this paper, we propose a novel Global Statistic Pooling network (GSPnet) which takes full advantage of the second-order information for enhancing modeling capability. Firstly, a novel global statistic pooling module is proposed to summarize both the first- and second-order statistics across frame-level feature, and then transfer them into a compact and robust video-level feature embedding. Secondly, a statistic-based attention block is incorporated into multiple stages of convolutional networks to fully explore the second-order representations from low- to high-level features. To enhance the representation learning ability and further boost re-identification (re-ID) performance, we also propose a multi-level self-attention distillation training scheme, which squeezes the knowledge learned in the deeper portion of the networks into the shallow ones. Extensive experimental results have demonstrated the effectiveness and superiority of our approach on four popular video person re-ID datasets.

Fail-safe topology optimization of continuum structures with fundamental frequency constraints based on the ICM method

It is an important topic to improve the redundancy of optimized configuration to resist the local failure in topology optimization of continuum structures. Such a fail-safe topology optimization problem has been solved effectively in the field of statics. In this paper, the fail-safe topology optimization problem is extended to the field of frequency topology optimization. Based on the independent continuous mapping (ICM) method, the model of fail-safe topology optimization is established with the objective of minimal weight integrating with the discrete condition of topological variables and the constraint of the fundamental frequency. The fail-safe optimization model established above is substituted by a sequence of subproblems in the form of the quadratic program with exact second-order information and solved efficiently by the dual sequence quadratic programming (DSQP) algorithm. The numerical result reveals that the optimized fail-safe structure has more complex configuration and preserved materials than the structure obtained from the traditional frequency topology optimization, which means that the optimized fail-safe structure has higher redundancy. Moreover, the optimized fail-safe structure guarantees that the natural frequency meets the constraint of fundamental frequency when the local failure occurs, which can avoid the structural frequency to be sensitive to local failure. The fail-safe optimization topology model is proved effective and feasible by four numerical examples.

Representation and reconstruction of covariance operators in linear inverse problems

We introduce a framework for the reconstruction and representation of functions in a setting where these objects cannot be directly observed, but only indirect and noisy measurements are available, namely an inverse problem setting. The proposed methodology can be applied either to the analysis of indirectly observed functional images or to the associated covariance operators, representing second-order information, and thus lying on a non-Euclidean space. To deal with the ill-posedness of the inverse problem, we exploit the spatial structure of the sample data by introducing a flexible regularizing term embedded in the model. Thanks to its efficiency, the proposed model is applied to MEG data, leading to a novel approach to the investigation of functional connectivity.

Nucleosome movement analysis based on second-order information entropy and density functional theory

Dynamics of +1 and -1 nucleosomes near TSS of yeast chromosome 2 were analyzed by using second-order information entropy and density functional theory method. Second-order information entropy can measure the interaction intensity between nucleosome sequences and nucleosome histones based on the intensity of base association. In addition, density functional theory method can be used to obtain the global interaction intensity between nucleosome sequences and nucleosome histones based on energy state size and active or non-active state of binucleoside pairs. Our results showed asymmetry of interaction intensity on both sides of the nucleosome central site, and that +1 nucleosomes tend to move toward the 5'-end and -1 nucleosomes tend to move toward the 3'-end. Under the dynamic balance of nucleosome movement, in roder to shut down gene transcription, +1 and -1 nucleosomes will cover TSS. If the dynamic balance is destroyed, +1 and -1 nucleosomes stay away from each other to expose TSS to restart gene transcription. The movement trend of +1 and -1 nucleosomes coincides with the biological mechanism of gene transcription and non-transcription, and the nucleosome sequences contain the dynamic information of nucleosome movement, which provides effective technical support for the study of gene transcription regulation mechanism.

Assimilating Second-Order Information for Building Non-Negative Latent Factor Analysis-Based Recommenders

A non-negative latent factor analysis (NLFA)-based recommender can make precise recommendations by correctly representing the non-negative characteristic of industrial data. It commonly relies on a nonconvex and bilinear optimization process, where the effects of first-order solvers maybe significantly reduced. Higher order solvers like a Newton-type method are expected to make a breakthrough; however, its computation efficiency and scalability are greatly limited due to the numerous parameters involved in a Hessian matrix. To address this issue, this article proposes an approach for assimilating second-order information for building NLFA-based recommenders. The key idea is an inner second-order solver that employs a Hessian-free method for avoiding the highly expensive manipulations of a Hessian matrix. Empirical studies on eight data cases emerging from real industrial applications indicate that the proposed approach outperforms state-of-the-art models in prediction accuracy with affordable computational burden.

Combinatorial Learning of Robust Deep Graph Matching: An Embedding Based Approach.

Graph matching aims to establish node correspondence between two graphs, which has been a fundamental problem for its NP-hard nature. One practical consideration is the effective modeling of the affinity function in the presence of noise, such that the mathematically optimal matching result is also physically meaningful. This paper resorts to deep neural networks to learn the node and edge feature, as well as the affinity model for graph matching in an end-to-end fashion. The learning is supervised by combinatorial permutation loss over nodes. Specifically, the parameters belong to convolutional neural networks for image feature extraction, graph neural networks for node embedding that convert the structural (beyond second-order) information into node-wise features that leads to a linear assignment problem, as well as the affinity kernel between two graphs. Our approach enjoys flexibility in that the permutation loss is agnostic to the number of nodes, and the embedding model is shared among nodes such that the network can deal with varying numbers of nodes for both training and inference. Moreover, our network is class-agnostic. Experimental results on extensive benchmarks show its state-of-the-art performance. It bears some generalization capability across categories and datasets, and is capable for robust matching against outliers.

Double-Mode Energy Management for Multi-Energy System via Distributed Dynamic Event-Triggered Newton-Raphson Algorithm

The islanded and network-connected modes are expected to be modeled into a unified form as well as in a distributed fashion for multi-energy system. In this way, the adaptability and flexibility of multi-energy system can be enhanced. To this aim, this paper establishes a double-mode energy management model for the multi-energy system. It is formed by many energy bodies. With such a model, each participant is able to adaptively respond to the change of mode switching. Furthermore, a novel distributed dynamic event-triggered Newton-Raphson algorithm is proposed to solve the double-mode energy management problem in a fully distributed fashion. In this method, the idea of Newton descent along with the dynamic event-triggered communication strategy are introduced and embedded in the execution of the proposed algorithm. With this effort, each participant can adequately utilize the second-order information to speed up convergence. The optimality is not affected. Meanwhile, the proposed algorithm can be implemented with asynchronous communication and without needing special initialization conditions. It exhibits better flexibility and adaptability especially when the system modes are changed. In addition, the continuous-time algorithm is executed with discrete-time communication driven by the proposed dynamic event-triggered mechanism.It results in reduced communication interaction and avoids needing continuous-time information transmission. It is also proved that each participant can asymptotically converge to the global optimal point. Finally, simulation results show the effectiveness of the proposed model and illustrate the faster convergence feature of the proposed algorithm.

Primal-dual fixed point algorithm based on adapted metric method for solving convex minimization problem with application

Optimization problems involving the sum of three convex functions have received much attention in recent years, where one is differentiable with Lipschitz continuous gradient, one is composed of a linear operator and the other is proximity friendly. The primal-dual fixed point algorithm is a simple and effective algorithm for such problems. To exploit the second-order derivatives information of the objective function, we propose a primal-dual fixed point algorithm with an adapted metric method. The proposed algorithm is derived from the idea of establishing a generally fixed point formulation for the solution of the considered problem. Under mild conditions on the iterative parameters, we prove the convergence of the proposed algorithm. Further, we establish the ergodic convergence rate in the sense of primal-dual gap and also derive the linear convergence rate with additional conditions. Numerical experiments on image deblurring problems show that the proposed algorithm outperforms other state-of-the-art primal-dual algorithms in terms of the number of iterations.

LGSLRR: Towards fusing discriminative ordinal local and global structured low-rank representation for image recognition

Structural information extraction has been a focal technique in many classification applications, such as image recognition and biometrics. However, it remains a challenge to simultaneously utilize local and global structural information in a classification model. In addition, in terms of the local information, the existing methods mainly seek to extract or preserve the first-order structure while ignoring the useful ordinal structural information for classification. To this end, this paper presents a discriminative ordinal local and global structured low-rank representation (LGSLRR) model that jointly preserves the local ordinal structure and global structure for image recognition. A discriminative block-diagonal low-rank representation is employed to obtain global information while the first-order and second-order local information is preserved by a joint graph based manifold embedding with two different Laplacian matrices. Some extensive comparison experiments on ten public image datasets are performed, and the results demonstrate the effectiveness and significant performance of the proposed method over some state-of-the-art methods.

Hyperspectral Image Classification Using Mixed Convolutions and Covariance Pooling

Recently, convolution neural network (CNN)-based hyperspectral image (HSI) classification has enjoyed high popularity due to its appealing performance. However, using 2-D or 3-D convolution in a standalone mode may be suboptimal in real applications. On the one hand, the 2-D convolution overlooks the spectral information in extracting feature maps. On the other hand, the 3-D convolution suffers from heavy computation in practice and seems to perform poorly in scenarios having analogous textures along with consecutive spectral bands. To solve these problems, we propose a mixed CNN with covariance pooling for HSI classification. Specifically, our network architecture starts with spectral-spatial 3-D convolutions that followed by a spatial 2-D convolution. Through this mixture operation, we fuse the feature maps generated by 3-D convolutions along the spectral bands for providing complementary information and reducing the dimension of channels. In addition, the covariance pooling technique is adopted to fully extract the second-order information from spectral-spatial feature maps. Motivated by the channel-wise attention mechanism, we further propose two principal component analysis (PCA)-involved strategies, channel-wise shift and channel-wise weighting, to highlight the importance of different spectral bands and recalibrate channel-wise feature response, which can effectively improve the classification accuracy and stability, especially in the case of limited sample size. To verify the effectiveness of the proposed model, we conduct classification experiments on three well-known HSI data sets, Indian Pines, University of Pavia, and Salinas Scene. The experimental results show that our proposal, although with less parameters, achieves better accuracy than other state-of-the-art methods.

Landscape factors driving the spread of the invasive grass,Hymenachne amplexicaulis, among wetlands in a Florida subtropical grazing land

AbstractWetlands embedded in agroecosystems provide vital ecosystem services (i.e., freeze protection, water retention, nutrient cycling, biodiversity support). However, they are particularly susceptible to invasion by nonnative species. West Indian marsh grass [Hymenachne amplexicaulis(Rudge) Nees] is a major wetland invader in Florida. Despite the documented consequences ofH. amplexicaulisinvasions, the landscape factors influencing the spread of this species are poorly understood. In this study, we asked whether landscape factors associated with wetland isolation, connectivity, and land management influence the presence ofH. amplexicaulisamong wetlands embedded in pastures. We recorded the presence or absence ofH.amplexicaulisin 158 seasonal wetlands embedded in different pasture types (semi-natural vs. intensively managed). Wetland area, isolation from neighboring wetlands, isolation from the nearest source ditch, and connectivity were determined using a geographic information system (GIS). We related landscape factors toH. amplexicaulisusing generalized linear models and model selection based on the second-order Akaike information criterion.Hymenachne amplexicauliswas first detected at the study site in the early 2000s. By 2018, we observed this species in 66% of the surveyed wetlands. The likelihood of observingH. amplexicauliswas higher in wetlands embedded in semi-natural pastures and higher in less isolated wetlands, especially when connected to a ditch. These results indicate thatH. amplexicaulisspreads both overland (during seasonal flooding) and via the ditch network. Future work is needed to understand whether seeds or stolons are the primary invasion propagule and whether the species forms a persistent seed bank that could slow down restoration efforts. Additionally, further research is required to understand the ecological impact of this highly invasive plant in Florida wetlands.

Fast gradient methods with alignment for symmetric linear systems without using Cauchy step

The performance of gradient methods has been considerably improved by the introduction of delayed parameters. Recently, the revealing of second-order information has given rise to the Cauchy-based methods with alignment, which are generally considered as the state of the art of gradient methods. This paper investigates the spectral properties of minimal gradient and asymptotically optimal steps, and then suggests three fast methods with alignment without using the Cauchy step. The convergence results are provided, and numerical experiments show that the new methods provide competitive alternatives to the classical Cauchy-based methods. In particular, alignment gradient methods present advantages over the Krylov subspace methods in some situations, which makes them attractive in practice.

Hybrid reconstruction method for multispectral bioluminescence tomography with log-sum regularization.

Bioluminescence tomography (BLT) has important applications in the in vivo visualization of a pathological process for preclinical studies. However, the reconstruction of BLT is severely ill-posed. To recover the bioluminescence source stably and efficiently, we use a log-sum regularization term in the objective function and utilize a hybrid optimization algorithm for solving the nonconvex regularized problems (HONOR). The hybrid optimization scheme of HONOR merges second-order information and first-order information to reconstruction by choosing either the quasi-Newton (QN) or gradient descent step at each iteration. The QN step uses the limited-memory Broyden-Fletcher-Goldfarb-Shanno algorithm (L-BFGS) to acquire second-order information. Simulations and in vivo experiments based on multispectral measurements demonstrated the remarkable performance of the proposed hybrid method in the sparse reconstruction of BLT.

Adaptive total variation and second-order total variation-based model for low-rank tensor completion

Recently, low-rank regularization has achieved great success in tensor completion. However, only considering the global low-rankness is not sufficient, especially for a low sampling rate (SR). Total variation (TV) is introduced into low-rank tensor completion (LRTC) problem to promote the local smoothness by incorporating the first-order derivatives information. However, TV usually leads to undesirable staircase effects. To alleviate these staircase effects, we suggest a first- and second-order TV-based parallel matrix factorization model for LRTC problem, which integrates the local smoothness and global low-rankness by simultaneously exploiting the first- and second-order derivatives information. To solve the proposed model, an efficient proximal alternating optimization (PAO)-based algorithm is developed with theoretical guarantee. Moreover, we suggest a regularization parameter selection strategy to automatically update two regularization parameters, which is able to take advantage of the best properties of each of the two regularization terms. Extensive experiments on different tensor data show the superiority of the proposed method over other methods, particularly for extremely low SRs.

Comparison of dual based optimization methods for distributed trajectory optimization of coupled semi-batch processes

The physical and virtual connectivity of systems via flows of energy, material, information, etc., steadily increases. This paper deals with systems of sub-systems that are connected by networks of shared resources that have to be balanced. For the optimal operation of the overall system, the couplings between the sub-systems must be taken into account, and the overall optimum will usually deviate from the local optima of the sub-systems. However, for reasons, such as problem size, confidentiality, resilience to breakdowns, or generally when dealing with autonomous systems, monolithic optimization is often infeasible. In this contribution, iterative distributed optimization methods based on dual decomposition where the values of the objective functions of the different sub-systems do not have to be shared are investigated. We consider connected dynamic systems that share resources. This situation arises for continuous processes in transient conditions between different steady states and in inherently discontinuous processes, such as batch production processes. This problem is challenging since small changes during the iterations towards the satisfaction of the overarching constraints can lead to significant changes in the arc structures of the optimal solutions for the sub-systems. Moreover, meeting endpoint constraints at free final times complicates the problem. We propose a solution strategy for coupled semi-batch processes and compare different numerical approaches, the sub-gradient method, ADMM, and ALADIN, and show that convexification of the sub-systems around feasible points increases the speed of convergence while using second-order information does not necessarily do so. Since sharing of resources has an influence on whether trajectory dependent terminal constraints can be satisfied, we propose a heuristic for the computation of free final times of the sub-systems that allows the dynamic sub-processes to meet the constraints. For the example of several semi-batch reactors which are coupled via a bound on the total feed flow rate, we demonstrate that the distributed methods converge to (local) optima and highlight the strengths and the weaknesses of the different distributed optimization methods.

Visualizing the dynamic change of Ocular Response Analyzer waveform using Variational Autoencoder in association with the peripapillary retinal arteries angle

The aim of the current study is to identify possible new Ocular Response Analyzer (ORA) waveform parameters related to changes of retinal structure/deformation, as measured by the peripapillary retinal arteries angle (PRAA), using a generative deep learning method of variational autoencoder (VAE). Fifty-four eyes of 52 subjects were enrolled. The PRAA was calculated from fundus photographs and was used to train a VAE model. By analyzing the ORA waveform reconstructed (noise filtered) using VAE, a novel ORA waveform parameter (Monot1-2), was introduced, representing the change in monotonicity between the first and second applanation peak of the waveform. The variables mostly related to the PRAA were identified from a set of 41 variables including age, axial length (AL), keratometry, ORA corneal hysteresis, ORA corneal resistant factor, 35 well established ORA waveform parameters, and Monot1-2, using a model selection method based on the second-order bias-corrected Akaike information criterion. The optimal model for PRAA was the AL and six ORA waveform parameters, including Monot1-2. This optimal model was significantly better than the model without Monot1-2 (p = 0.0031, ANOVA). The current study suggested the value of a generative deep learning approach in discovering new useful parameters that may have clinical relevance.

High-Density Surface EMG Denoising Using Independent Vector Analysis.

High-density surface electromyography (HD-sEMG) can provide rich temporal and spatial information about muscle activation. However, HD-sEMG signals are often contaminated by power line interference (PLI) and white Gaussian noise (WGN). In the literature, independent component analysis (ICA) and canonical correlation analysis (CCA), as two popular used blind source separation techniques, are widely used for noise removal from HD-sEMG signals. In this paper, a novel method to remove PLI and WGN was proposed based on independent vector analysis (IVA). Taking advantage of both ICA and CCA, this method exploits the higher order and second-order statistical information simultaneously. Our proposed method was applied to both simulated and experimental EMG data for performance evaluation, which was at least 37.50% better than ICA and CCA methods in terms of relative root mean squared error and 28.84% better than ICA and CCA methods according to signal to noise ratio. The results demonstrated that our proposed method performed significantly better than either ICA or CCA. Specifically, the mean signal to noise ratio increased considerably. Our proposed method is a promising tool for denoising HD-sEMG signals while leading to a minimal distortion.

Projective Quadratic Regression for Online Learning

This paper considers online convex optimization (OCO) problems - the paramount framework for online learning algorithm design. The loss function of learning task in OCO setting is based on streaming data so that OCO is a powerful tool to model large scale applications such as online recommender systems. Meanwhile, real-world data are usually of extreme high-dimensional due to modern feature engineering techniques so that the quadratic regression is impractical. Factorization Machine as well as its variants are efficient models for capturing feature interactions with low-rank matrix model but they can't fulfill the OCO setting due to their non-convexity. In this paper, We propose a projective quadratic regression (PQR) model. First, it can capture the import second-order feature information. Second, it is a convex model, so the requirements of OCO are fulfilled and the global optimal solution can be achieved. Moreover, existing modern online optimization methods such as Online Gradient Descent (OGD) or Follow-The-Regularized-Leader (FTRL) can be applied directly. In addition, by choosing a proper hyper-parameter, we show that it has the same order of space and time complexity as the linear model and thus can handle high-dimensional data. Experimental results demonstrate the performance of the proposed PQR model in terms of accuracy and efficiency by comparing with the state-of-the-art methods.