Cyclic Coordinate Descent Algorithm Research Articles

Blind non-linear spectral unmixing with spatial coherence for hyper and multispectral images

Multi and hyperspectral images have become invaluable sources of information, revolutionizing various fields such as remote sensing, environmental monitoring, agriculture and medicine. In this expansive domain, the multi-linear mixing model (MMM) is a versatile tool to analyze spatial and spectral domains by effectively bridging the gap between linear and non-linear interactions of light and matter. This paper introduces an upgraded methodology that integrates the versatility of MMM in non-linear spectral unmixing, while leveraging spatial coherence (SC) enhancement through total variation theory to mitigate noise effects in the abundance maps. Referred to as non-linear extended blind end-member and abundance extraction with SC (NEBEAE-SC), the proposed methodology relies on constrained quadratic optimization, cyclic coordinate descent algorithm, and the split Bregman formulation. The validation of NEBEAE-SC involved rigorous testing on various hyperspectral datasets, including a synthetic image, remote sensing scenarios, and two biomedical applications. Specifically, our biomedical applications are focused on classification tasks, the first addressing hyperspectral images of in-vivo brain tissue, and the second involving multispectral images of ex-vivo human placenta. Our results demonstrate an improvement in the abundance estimation by NEBEAE-SC compared to similar algorithms in the state-of-the-art by offering a robust tool for non-linear spectral unmixing in diverse application domains.

Regularized Poisson regressions predict regional innovation output

AbstractRegional innovation output is influenced by many factors such as macroeconomic environments, residents consumption, fixed asset investment, foreign trade, fiscal revenue and expenditure, education, and research and development (R&D) input. Correctly predicting regional innovation output is an important subject in the economic field. In this paper, we propose four regularized Poisson regressions to forecast regional innovation output for 31 provinces in China. Firstly, we screen out 20 important factors and combine with four penalties: ridge penalty , least absolute shrinkage and selection operator penalty (LASSO), smoothly clipped absolute deviation penalty (SCAD), and minimax concave penalty (MCP) to construct four regularized Poisson regressions. Secondly, we introduce the cyclic coordinate descent (CCD) algorithm and the training set to complete variable selection and obtain the least squares weighted iterative estimators and make model selection by introducing three criterions to compare goodness of fit to different models. Finally, we apply the testing set and the learned regressions to exhibit the prediction performances and found that SCAD/MCP regularized Poisson regression predicts better than /LASSO regularized Poisson regression in terms of root mean square error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). In particular, MCP regularized Poisson regression outperforms the other three regularized Poisson regressions in predicting the number of granted patents in the three regions.

Mapping the energy landscape of PROTAC-mediated protein-protein interactions

A principal challenge in computational modeling of macromolecules is the vast conformational space that arises out of large numbers of atomic degrees of freedom. Recently, growing interest in building predictive models of complexes mediated by Proteolysis Targeting Chimeras (PROTACs) has led to the application of state-of-the-art computational techniques to tackle this problem. However, repurposing existing tools to carry out protein-protein docking and linker conformer generation independently results in extensive sampling of structures incompatible with PROTAC-mediated complex formation. Here we show that it is possible to restrict the search to the space of protein-protein conformations that can be bridged by a PROTAC molecule with a given linker composition by using a cyclic coordinate descent algorithm to position PROTACs into complex-bound configurations. We use this methodology to construct potential energy and solvation energy landscapes of PROTAC-mediated interactions. Our results suggest that desolvation of amino acids at interfaces could play a dominant role in PROTAC-mediated complex formation.

Nonlinear extended blind end-member and abundance extraction for hyperspectral images

Hyperspectral images had become an essential tool in different application frameworks, such as mineral exploration, food inspection, and medical assessment, among others. However, the interpretability of these images involves an initial processing stage to model the optical interaction and analyze the spectral information. In this work, we study nonlinear unmixing of hyperspectral images by a multilinear mixture model (MMM). In this sense, we propose a nonlinear version of the extended blind end-member and abundance extraction (NEBEAE) method for blind unmixing, i.e. estimation of the end-members, their abundances, and the nonlinear interaction levels. In the problem formulation, we include a normalization step in the hyperspectral measurements for the end-members and abundances to improve robustness. The blind unmixing process can be separated into three estimation subproblems for each component in the model, which are solved by a cyclic coordinate descent algorithm and quadratic constrained optimizations. Each problem is mathematically formulated and derived to construct the general nonlinear iterative unmixing technique. We evaluated our proposal with synthetic and experimental datasets from the remote sensing literature (Cuprite, Urban and Pavia University scene datasets) and a biomedical hyperspectral imaging application. In the validation stage, we compared NEBEAE with three state-of-the-art methods to show its advantages in terms of precision and computational time.

Discrete Semantics-Guided Asymmetric Hashing for Large-Scale Multimedia Retrieval

Cross-modal hashing technology is a key technology for real-time retrieval of large-scale multimedia data in real-world applications. Although the existing cross-modal hashing methods have achieved impressive accomplishment, there are still some limitations: (1) some cross-modal hashing methods do not make full consider the rich semantic information and noise information in labels, resulting in a large semantic gap, and (2) some cross-modal hashing methods adopt the relaxation-based or discrete cyclic coordinate descent algorithm to solve the discrete constraint problem, resulting in a large quantization error or time consumption. Therefore, in order to solve these limitations, in this paper, we propose a novel method, named Discrete Semantics-Guided Asymmetric Hashing (DSAH). Specifically, our proposed DSAH leverages both label information and similarity matrix to enhance the semantic information of the learned hash codes, and the ℓ2,1 norm is used to increase the sparsity of matrix to solve the problem of the inevitable noise and subjective factors in labels. Meanwhile, an asymmetric hash learning scheme is proposed to efficiently perform hash learning. In addition, a discrete optimization algorithm is proposed to fast solve the hash code directly and discretely. During the optimization process, the hash code learning and the hash function learning interact, i.e., the learned hash codes can guide the learning process of the hash function and the hash function can also guide the hash code generation simultaneously. Extensive experiments performed on two benchmark datasets highlight the superiority of DSAH over several state-of-the-art methods.

Second-Order Disjoint Factor Analysis

Hierarchical models are often considered to measure latent concepts defining nested sets of manifest variables. Therefore, by supposing a hierarchical relationship among manifest variables, the general latent concept can be represented by a tree structure where each internal node represents a specific order of abstraction for the latent concept measured. In this paper, we propose a new latent factor model called second-order disjoint factor analysis in order to model an unknown hierarchical structure of the manifest variables with two orders. This is a second-order factor analysis, which—respect to the second-order confirmatory factor analysis—is exploratory, nested and estimated simultaneously by maximum likelihood method. Each subset of manifest variables is modeled to be internally consistent and reliable, that is, manifest variables related to a factor measure “consistently” a unique theoretical construct. This feature implies that manifest variables are positively correlated with the related factor and, therefore, the associated factor loadings are constrained to be nonnegative. A cyclic block coordinate descent algorithm is proposed to maximize the likelihood. We present a simulation study that investigates the ability to get reliable factors. Furthermore, the new model is applied to identify the underlying factors of well-being showing the characteristics of the new methodology. A final discussion completes the paper.

Finite-Alphabet Symbol-Level Multiuser Precoding for Massive MU-MIMO Downlink

We propose a finite-alphabet symbol-level precoding technique for massive multiuser multiple-input multiple-output (MU-MIMO) downlink systems which are equipped with finite-resolution digital-to-analog converters (DACs) of any precision. Using the idea of constructive interference (CI), we adopt a max-min fair design criterion which aims to maximize the minimum instantaneous received signal-to-noise ratio (SNR) among the user equipments (UEs) while ensuring a CI constraint for each UE under the restriction that the output of the precoder is a vector with finite-alphabet discrete elements. Due to this latter constraint, the design problem is an NP-hard quadratic program with discrete variables, and hence, is difficult to solve. In this paper, we tackle this difficulty by reformulating the problem in several steps into an equivalent continuous-domain biconvex form, including equivalent representations for discrete and binary constraints. Our final biconvex reformulation is obtained via an exact penalty approach and can efficiently be solved using a standard cyclic block coordinate descent algorithm. We evaluate the performance of the proposed finite-alphabet precoding design for DACs with different resolutions, where it is shown that employing low-resolution DACs can lead to higher power efficiencies. In particular, we focus on a setup with one-bit DACs and show through simulation results that compared to the existing schemes, the proposed design can achieve SNR gains of up to 2 dB. We further provide analytic and numerical analyses of complexity and show that our proposed algorithm is computationally efficient as it typically needs only a few tens of iterations to converge.

Research on virtual human running autonomous devices based on cyclic-coordinate descent algorithm

The traditional virtual human running research method completes the motion analysis under the ideal state, which lead to cannot obtain the accurate kinematic stability and the coincidence rate of the trajectory curve of the gravity center. To address these problems, the virtual human running research method based on cyclic-coordinate descent (CCD) algorithm is proposed in this paper. The skeleton driver is constructed based on CCD algorithm and the virtual human skeleton model is built. The running state of virtual human is analyzed through multi-joint movement, and the constraint equation of virtual human running is constructed to realize the data simulation of the running movement of virtual human. By using the skeleton model and constraint equation, the kinematics of virtual human is solved. The gravity center trajectory is calculated by using matrix transpose algorithm. Combining with the virtual human upper body movement and the parameters extraction of specified path running, the research of virtual human running based on CCD algorithm is realized. Experimental result show that the proposed virtual human running method improves the kinematic stability by 45.81% compared with the traditional method, and the coincidence rate of the gravity center trajectory is increased by 63.42%.

Exploration of virtual human running based on CCD algorithm

SummaryTo make a more thorough understanding of human movement, the research of virtual human running based on the CCD (Cyclic‐Coordinate Descent) algorithm was focused on. First of all, the CCD algorithm, the rotation range of the human joint, the running cycle, the key moment of running movement, and the decomposition of running movement were introduced in detail. Then, the inverse kinematics based on the CCD algorithm was solved, the inverse kinematics in the running was solved, and the motion simulation of the upper body and the specified path were carried out. Finally, the simulation results were analyzed. The results showed that the parameters such as the percentage of the support period, the vacant height, the running level, the step frequency, and the frame rate were adjusted by the interactive mode. In a word, it makes the running posture generated diversified and more characteristic.

Sparse low-rank separated representation models for learning from data.

We consider the problem of learning a multivariate function from a set of scattered observations using a sparse low-rank separated representation (SSR) model. The model structure considered here is promising for high-dimensional learning problems; however, existing training algorithms based on alternating least-squares (ALS) are known to have convergence difficulties, particularly when the rank of the model is greater than 1. In the present work, we supplement the model structure with sparsity constraints to ensure the well posedness of the approximation problem. We propose two fast training algorithms to estimate the model parameters: (i) a cyclic coordinate descent algorithm and (ii) a block coordinate descent (BCD) algorithm. While the first algorithm is not provably convergent owing to the non-convexity of the optimization problem, the BCD algorithm guarantees convergence to a Nash equilibrium point. The computational cost of the proposed algorithms is shown to scale linearly with respect to all of the parameters in contrast to methods based on ALS. Numerical studies on synthetic and real-world regression datasets indicate that the proposed SSR model structure holds significant potential for machine learning problems.

Bayesian variable selection for parametric survival model with applications to cancer omics data

BackgroundModeling thousands of markers simultaneously has been of great interest in testing association between genetic biomarkers and disease or disease-related quantitative traits. Recently, an expectation-maximization (EM) approach to Bayesian variable selection (EMVS) facilitating the Bayesian computation was developed for continuous or binary outcome using a fast EM algorithm. However, it is not suitable to the analyses of time-to-event outcome in many public databases such as The Cancer Genome Atlas (TCGA).ResultsWe extended the EMVS to high-dimensional parametric survival regression framework (SurvEMVS). A variant of cyclic coordinate descent (CCD) algorithm was used for efficient iteration in M-step, and the extended Bayesian information criteria (EBIC) was employed to make choice on hyperparameter tuning. We evaluated the performance of SurvEMVS using numeric simulations and illustrated the effectiveness on two real datasets. The results of numerical simulations and two real data analyses show the well performance of SurvEMVS in aspects of accuracy and computation. Some potential markers associated with survival of lung or stomach cancer were identified.ConclusionsThese results suggest that our model is effective and can cope with high-dimensional omics data.

Partial randomness hashing applied to remote sensing object classification

Recently, object classification of remote sensing images has attracted more and more research interests due to the development of satellite and aerial vehicle technologies. Hashing learning is an efficient method to handle the huge amount of the remote sensing data. We proposed a hashing learning method named partial randomness supervised discrete hashing (PRSDH), which combines data-dependent methods and data-independent methods. It jointly learns a discrete binary codes generation and partial random constraint optimization model. By random projection, the computation complexity is reduced effectively. With the weight matrix derived from the training data, the semantic similarity between the data can be well preserved while generating the hashing codes. For the discrete constraint problem, this paper adopts the discrete cyclic coordinate descent algorithm to optimize the codes bit by bit. The experimental results show that PRSDH outperforms other comparative methods and demonstrate that PRSDH has good adaptability to the characteristic of remote sensing object.

The spike-and-slab lasso Cox model for survival prediction and associated genes detection.

Large-scale molecular profiling data have offered extraordinary opportunities to improve survival prediction of cancers and other diseases and to detect disease associated genes. However, there are considerable challenges in analyzing large-scale molecular data. We propose new Bayesian hierarchical Cox proportional hazards models, called the spike-and-slab lasso Cox, for predicting survival outcomes and detecting associated genes. We also develop an efficient algorithm to fit the proposed models by incorporating Expectation-Maximization steps into the extremely fast cyclic coordinate descent algorithm. The performance of the proposed method is assessed via extensive simulations and compared with the lasso Cox regression. We demonstrate the proposed procedure on two cancer datasets with censored survival outcomes and thousands of molecular features. Our analyses suggest that the proposed procedure can generate powerful prognostic models for predicting cancer survival and can detect associated genes. The methods have been implemented in a freely available R package BhGLM ( http://www.ssg.uab.edu/bhglm/ ). nyi@uab.edu. Supplementary data are available at Bioinformatics online.

The Spike-and-Slab Lasso Generalized Linear Models for Prediction and Associated Genes Detection.

Large-scale "omics" data have been increasingly used as an important resource for prognostic prediction of diseases and detection of associated genes. However, there are considerable challenges in analyzing high-dimensional molecular data, including the large number of potential molecular predictors, limited number of samples, and small effect of each predictor. We propose new Bayesian hierarchical generalized linear models, called spike-and-slab lasso GLMs, for prognostic prediction and detection of associated genes using large-scale molecular data. The proposed model employs a spike-and-slab mixture double-exponential prior for coefficients that can induce weak shrinkage on large coefficients, and strong shrinkage on irrelevant coefficients. We have developed a fast and stable algorithm to fit large-scale hierarchal GLMs by incorporating expectation-maximization (EM) steps into the fast cyclic coordinate descent algorithm. The proposed approach integrates nice features of two popular methods, i.e., penalized lasso and Bayesian spike-and-slab variable selection. The performance of the proposed method is assessed via extensive simulation studies. The results show that the proposed approach can provide not only more accurate estimates of the parameters, but also better prediction. We demonstrate the proposed procedure on two cancer data sets: a well-known breast cancer data set consisting of 295 tumors, and expression data of 4919 genes; and the ovarian cancer data set from TCGA with 362 tumors, and expression data of 5336 genes. Our analyses show that the proposed procedure can generate powerful models for predicting outcomes and detecting associated genes. The methods have been implemented in a freely available R package BhGLM (http://www.ssg.uab.edu/bhglm/).

Network-Regularized Sparse Logistic Regression Models for Clinical Risk Prediction and Biomarker Discovery.

Molecular profiling data (e.g., gene expression) has been used for clinical risk prediction and biomarker discovery. However, it is necessary to integrate other prior knowledge like biological pathways or gene interaction networks to improve the predictive ability and biological interpretability of biomarkers. Here, we first introduce a general regularized Logistic Regression (LR) framework with regularized term , which can reduce to different penalties, including Lasso, elastic net, and network-regularized terms with different . This framework can be easily solved in a unified manner by a cyclic coordinate descent algorithm which can avoid inverse matrix operation and accelerate the computing speed. However, if those estimated and have opposite signs, then the traditional network-regularized penalty may not perform well. To address it, we introduce a novel network-regularized sparse LR model with a new penalty to consider the difference between the absolute values of the coefficients. We develop two efficient algorithms to solve it. Finally, we test our methods and compare them with the related ones using simulated and real data to show their efficiency.

Modeling Protein Loop Structure by Cyclic Coordinate Descent-based Approach.

In both predicting protein loop structure and flexible areas in a docking interface, modeling local loop conformation plays a vital role. We address the challenge with a topological manipulation method based on the cyclic coordinate descent (CCD) algorithm. The protein loop conformation sampler (LMbCCD2) presented here allows us to move some key points of the loop toward some specific positions. We test the method's performance on sets of loops, and the results show that LMbCCD2 accurately controls the loop's topology. At last we use LMbCCD2 on flexible protein-peptide docking, and it shows that LMbCCD2 can improve the accuracy of flexible peptide folding and the full docking result.

Improving protein-ligand docking with flexible interfacial water molecules using SWRosettaLigand.

Computational protein-ligand docking is of great importance in drug discovery and design. Conformational changes greatly affect the results of protein-ligand docking, especially when water molecules take part in mediating protein ligand interactions or when large conformational changes are observed in the receptor backbone interface. We have developed an improved protocol, SWRosettaLigand, based on the RosettaLigand protocol. This approach incorporates the flexibility of interfacial water molecules and modeling of the interface of the receptor into the original RosettaLigand. In a coarse sampling step, SWRosettaLigand pre-optimizes the initial position of the water molecules, docks the ligand to the receptor with explicit water molecules, and minimizes the predicted structure with water molecules. The receptor backbone interface is treated as a loop and perturbed and refined by kinematic closure, or cyclic coordinate descent algorithm, with the presence of the ligand. In two cross-docking test sets, it was identified that for 8 out of 14, and 16 out of 22, test instances, the top-ranked structures by SWRosettaLigand achieved better accuracy than other protocols.

Economic Study of Problems of Depletion of Several Interrelated Non-renewable Resources

In this paper we generalize the classic problem of the economic study of the extraction of non-renewable resources. The most notable generalizations presented here are the presence of constraints on the depletion rate, multiple resources and variable prices and costs over time. To solve the problem, we first use the theory of optimal control combined with a modification of the classic shooting method and an algorithm inspired by the cyclic coordinate descent algorithm. Numerous examples are presented to illustrate the possibilities the method offers.

Humanoid Robot Arm Kinematics Modeling and Motion Planning

This paper establishes the kinematics model of humanoid robot arm, the arm forward kinematics equations were built and solved, based on the advantages of CCD (Cyclic Coordinate Descent) and BFS (Broyden-Fletcher-Shanno) algorithm to solve the inverse kinematics of humanoid robot arm. In the joint space, using cubic polynomial and quintic polynomial interpolation method respectively for each joint angle interpolation. Cubic polynomial trajectory planning can meet the point-to-point movement in general, but can not guarantee the continuity of acceleration of each point; Quintic polynomial trajectory planning can ensure that each point is continuous of the joint angle, angular velocity, and angular acceleration, so this polynomial method can meet the movement of the humanoid robot arm.

Dglars: AnRPackage to Estimate Sparse Generalized Linear Models

dglars is a publicly available R package that implements the method proposed in Augugliaro, Mineo, and Wit (2013), developed to study the sparse structure of a generalized linear model. This method, called dgLARS, is based on a differential geometrical extension of the least angle regression method proposed in Efron, Hastie, Johnstone, and Tibshirani (2004). The core of the dglars package consists of two algorithms implemented in Fortran 90 to efficiently compute the solution curve: a predictor-corrector algorithm, proposed in Augugliaro et al. (2013), and a cyclic coordinate descent algorithm, proposed in Augugliaro, Mineo, and Wit (2012). The latter algorithm, as shown here, is significantly faster than the predictor-corrector algorithm. For comparison purposes, we have implemented both algorithms.