Joint Objective Function Research Articles

Preference Aware Dual Contrastive Learning for Item Cold-Start Recommendation

Existing cold-start recommendation methods often adopt item-level alignment strategies to align the content feature and the collaborative feature of warm items for model training, however, cold items in the test stage have no historical interactions with users to obtain the collaborative feature. These existing models ignore the aforementioned condition of cold items in the training stage, resulting in the performance limitation. In this paper, we propose a preference aware dual contrastive learning based recommendation model (PAD-CLRec), where the user preference is explored to take into account the condition of cold items for feature alignment. Here, the user preference is obtained by aggregating a group of collaborative feature of the warm items in the user's purchase records. Then, a group-level alignment between the user preference and the item's content feature can be realized via a proposed preference aware contrastive function for enhancing cold-item recommendation. In addition, a joint objective function is introduced to achieve a better trade-off between the recommendation performance of warm items and cold items from both item-level and group-level perspectives, yielding better overall recommendation performance. Extensive experiments are conducted to demonstrate the effectiveness of the proposed method, and the results show the superiority of our method, as compared with the state-of-the-arts.

Joint inversion method of multipoint ambient noise horizontal-to-vertical spectral ratio for 3-D velocity structure of local site and its application

SUMMARY The diffusion field theory has been widely used to interpret ambient noise wave fields. Based on this theory, 1-D subsurface velocity structure inversion method is developed. However, few studies have referred to the noise horizontal to vertical (NHV) spectral ratio inversion of 3-D subsurface velocity structures, and almost no effective 3-D NHV inversion tools have been developed. To develop a useful tool for obtaining 3-D soil layer velocity structures, we combined the NHV forward calculation formula derived from diffusion field theory with the guided Monte Carlo algorithm and then extended the single-point NHV inversion to multipoint joint inversion through a joint objective function. Subsequently, a new 3-D soil layer velocity structure inversion method was proposed. Subsequently, a synthetic 2-D case was used to verify the proposed method. Finally, the proposed method was applied to the Xiangtang Array in Tangshan, China, to identify the 3-D velocity structures of the site based on noise observations. The results show that the proposed multipoint joint 3-D inversion method is effective for identifying 3-D underground velocity structures.

Dynamic Security Assessment Framework for Steel Casting Workshops in Smart Factory

Security assessment (SA) system is crucial to ensure the production safety of a smart factory with rapid development of artificial intelligence. In this article, we propose a novel SA framework. Different from the conventional static monitoring systems based on traditional sensing technologies, the proposed framework can automatically detect objects via visual sensing. We use a skeleton-based graph convolutional network to generate action vocabulary for the intermediate representations of action-to-action cooccurrence relations. These representations are encoded into the sequential interaction models to form the interaction representations. Integrating the states of molten steel levels as the reference labels, the sequential representations are fed into a recurrent neural network model with multilayer gated recurrent units (GRUs) to capture the key interactions leading to the accidents, in which an attention mechanism is used to reweight the actions and eliminate the invalid interactions. The predicted labels and the hidden states of the scenes are passing among multilayer GRUs. Finally, we optimize the global output to dynamically assess the security by calculating a joint objective function with a regularized cross-entropy loss. On the self-collected dataset from our partner Iron and Steel company and on-line video clips, the proposed framework performs better than the existing SA schemes.

Deep document clustering via adaptive hybrid representation learning

Document clustering plays an important role in text mining and machine learning. Recently, various studies have indicated that a hybrid representation contributes to document clustering tasks. Adaptively adjusting representation learning to make each representation of documents mutually reinforced remains an open problem. In this study, we propose a deep document clustering model, DCAHR, to improve the document clustering performance by adaptively learning a hybrid representation. Specifically, an adaptive representation enhanced network (AREN) is designed using a composite Gated Linear Units (GLU) mechanism to adaptively capture the consistent information of semantic and structure representations, and utilize it to adaptively adjust each representation learning for studying the corresponding enhanced representations. A hybrid representation was then obtained by normalizing the two enhanced representations. Furthermore, an adaptive joint objective function was developed to learn document partitioning and supervise the entire model by adaptively learning the weights of each objective function. Experimental results on nine commonly used real-world datasets demonstrated the outstanding performance of the proposed DCAHR model.

Optimizing an Algorithm Designed for Sparse-Frequency Waveforms for Use in Airborne Radars

Low-frequency bands are an important way to realize stealth target detection for airborne radars. However, in a complex electromagnetic environment; when low-frequency airborne radar operates over land, it will inevitably encounter a lot of unintentional communication and intentional interference, while effective suppression of interference can not be achieved only through the adaptive processing of the receiver. To solve this problem, this paper proposes optimizing an algorithm designed for sparse-frequency waveforms for use in airborne radars. The algorithm establishes a joint objective function based on the criteria of minimizing waveform energy in the spectrum stopband and minimizing the integrated sidelobe level of specified range cells. The waveform is optimized by a cyclic iterative algorithm based on the Fast Fourier Transform (FFT) operation. It can ensure the frequency domain stopband constraint to realize the effective suppression of main-lobe interference while forming lower-range sidelobes at specified range cells to improve the ability to detect dim targets. Theoretical analysis and simulation results have shown that the algorithm has good anti-interference performance.

Reconstructing Three-dimensional geological structures by the Multiple-point statistics method coupled with a deep neural network: A case study of a metro station in Guangzhou, China

A credible large-scale three-dimensional (3D) geological model constructed from a small amount of local high-density data provides a vital data infrastructure for the design and construction of metro lines. The multiple-point statistics (MPS) method has successfully generated 3D geological models in many fields. However, the global cognition of the spatial relations between geological objects is difficult to reconstruct using a local optimization strategy with moving templates in the MPS simulation. This study presented an MPS simulation approach for constructing 3D geological structures with two-dimensional (2D) cross-sections, where the MPS process was coupled with a fully connected deep neural network (FCDNN) simulation. The core idea of the proposed method was that the global features of each geological object were generated from the trained FCDNN, the input parameter of which was the subsurface depth. Then, the entire model was constructed on the initial model using the following iterative MPS process: the semantic relationship of geological objects was extracted from 2D cross-sections and used as a constraint in the modeling process, which ensured the order of the stratigraphic sequence and rationality of the geological structure in the final model. The joint objective function of the proposed method was composed of the loss function of the FCDNN and conventional MPS objective function constrained by the spatial semantics of geological objects. The concrete example in the Jiangtai Road Metro Station of Guangzhou Line 11, China illustrated that the details of the complex geological structures could be obtained accurately by the presented method. The global features from the trained FCDNN provided the basic geometry for the final model, which overcame the limitations of the conventional MPS method.

Semi-supervised multivariate time series anomaly detection for wind turbines using generator SCADA data

The maintenance cost and unplanned downtime caused by faults are an important part of the operation cost of wind turbines. Supervisory control and data acquisition (SCADA) data is a multivariate time series (MTS) for monitoring the status of wind turbines, in which anomaly patterns may indicate potential faults. The existing anomaly detection methods can neither extract and process pattern information in MTS stably, nor make reasonable use of a small amount of valuable labeled data. In this paper, we propose an end-to-end semi-supervised anomaly detection model including reconstruction model, prediction model and auxiliary discriminator, with a joint objective function. Combining reconstruction model and prediction model, the unsupervised model can effectively extract the inter-variable correlation and temporal dependence of MTS data. Further, using the semi-supervised auxiliary discriminator based on adversarial training, the proposed model can integrate expert knowledge to incrementally upgrade performance from unsupervised to supervised level. Our evaluation experiments are conducted on a public server dataset and a real-world wind turbine SCADA dataset. The results show that the F1-score of unsupervised model can exceed the several state-of-the-art baseline methods by 3.86% and 2.89%, and the F1-score can be increased to 98.60% and 98.30% after using the auxiliary discriminator.© 2016 Published by Elsevier Ltd. Selection and/or peer-review under responsibility of Global Science and Technology Forum Pte Ltd

Joint towed streamer and ocean-bottom-seismometer data multi-parameter full waveform inversion in acoustic-elastic coupled media

Short-offset towed streamer data, and sparse ocean-bottom seismometer (OBS) data are not conducive to applying multi-parameter full waveform inversion (FWI) in production. It is challenging to reconstruct deep velocity in the former, and the latter suffers from severe acquisition footprints. We developed a joint acoustic-elastic coupled full waveform inversion (J-AEFWI) method, in which towed streamer data and ocean-bottom seismometer data were used jointly to build P-wave and S-wave velocity models. A new joint objective function was established using the least-squares theory, and the joint acoustic-elastic coupled full waveform inversion method on the acoustic-elastic coupled equation was derived. The method can inject the residuals of the towed streamer data and the ocean-bottom seismometer data in time backward propagating to update P-wave and S-wave velocity models. The synthetic experimental results show that joint acoustic-elastic coupled full waveform inversion obtains more accurate results than when using these two types of data alone. Compared to using the towed streamer or ocean-bottom seismometer data alone, the joint acoustic-elastic coupled full waveform inversion method leads to better illumination of the deep background velocities and suppression of acquisition footprints. The results of joint acoustic-elastic coupled full waveform inversion were slightly better than those of the cascaded full waveform inversion strategy. To further demonstrate the benefit of the proposed method, we applied it to the field data, and better results are obtained as expected.

A nonparametric test for cooperation in discrete games

Summary We propose a nonparametric test for cooperative behaviour among players in discrete, static games. Assuming that certain exchangeability conditions hold if we match observable characteristics across all players, we obtain testable implications for cooperative (coalitional) behaviour, which we define as occurring when players maximise an unknown joint objective function that is symmetric conditional on the matching. Cooperation implies the existence of a class of outcomes $\mathcal {Y}$ such that, conditional on the matching, the probability of observing an outcome $y\in \mathcal {Y}$ must be equal to the probability of observing any permutation of y. We present a nonparametric econometric test and we characterise its asymptotic properties. We apply our test to expansion/entry decisions of Lowe’s and Home Depot in the contiguous United States and we find that, while outcomes are consistent with noncooperative behaviour in larger markets, we fail to reject cooperation in smaller markets.

Maximizing the canopy openness for replanted Korean pine seedlings under secondary forests in northeast China

Accelerating the recovery and successional processes of secondary forests is an important task for sustainable forest management around the world. However, previous studies have mainly focused on the establishment and survival of seedlings, whereas the differences of light requirements in different growth stages of understory seedlings have been overlooked for a long time. Therefore, the present study developed a tree-level harvest decision tool that can account for adjustments in the light environment of the understory and optimize the stand structure of the overstory concurrently. A set of spatial (canopy openness, species mingling, and uniform angle) and nonspatial (the number of species and diameter classes, as well as the q-value of diameter distribution) parameters were employed to formulate the joint objective function. Then, three thinning scenarios were generated by adjusting the priority of the replanted Pinus koraiensis and their companion species. The developed tool and assumed thinning scenarios were then tested using four 100 m × 100 m mapped secondary forests in northeast China, where approximately 500 seedlings per hectare of P. koraiensis were replanted under the canopy in the 1980s. The results indicated that the tree-level harvest decision tool improved the joint objective function values by approximately 3.6–6.9 times for different thinning scenarios on average. Hereinto, the light conditions of P. koraiensis were improved by about 32 %, meanwhile the stand structures of the overstory were also increased by about 11 % of species mingling and about 1 % of the horizontal distribution. Thus, the developed tree-level harvest decision tool was effective and can be applied to manage secondary forests.

Efficient Learning of Interpretable Classification Rules

Machine learning has become omnipresent with applications in various safety-critical domains such as medical, law, and transportation. In these domains, high-stake decisions provided by machine learning necessitate researchers to design interpretable models, where the prediction is understandable to a human. In interpretable machine learning, rule-based classifiers are particularly effective in representing the decision boundary through a set of rules comprising input features. Examples of such classifiers include decision trees, decision lists, and decision sets. The interpretability of rule-based classifiers is in general related to the size of the rules, where smaller rules are considered more interpretable. To learn such a classifier, the brute-force direct approach is to consider an optimization problem that tries to learn the smallest classification rule that has close to maximum accuracy. This optimization problem is computationally intractable due to its combinatorial nature and thus, the problem is not scalable in large datasets. To this end, in this paper we study the triangular relationship among the accuracy, interpretability, and scalability of learning rule-based classifiers. The contribution of this paper is an interpretable learning framework IMLI, that is based on maximum satisfiability (MaxSAT) for synthesizing classification rules expressible in proposition logic. IMLI considers a joint objective function to optimize the accuracy and the interpretability of classification rules and learns an optimal rule by solving an appropriately designed MaxSAT query. Despite the progress of MaxSAT solving in the last decade, the straightforward MaxSAT-based solution cannot scale to practical classification datasets containing thousands to millions of samples. Therefore, we incorporate an efficient incremental learning technique inside the MaxSAT formulation by integrating mini-batch learning and iterative rule-learning. The resulting framework learns a classifier by iteratively covering the training data, wherein in each iteration, it solves a sequence of smaller MaxSAT queries corresponding to each mini-batch. In our experiments, IMLI achieves the best balance among prediction accuracy, interpretability, and scalability. For instance, IMLI attains a competitive prediction accuracy and interpretability w.r.t. existing interpretable classifiers and demonstrates impressive scalability on large datasets where both interpretable and non-interpretable classifiers fail. As an application, we deploy IMLI in learning popular interpretable classifiers such as decision lists and decision sets. The source code is available at https://github.com/meelgroup/mlic.

Towards Generating Adversarial Examples on Combined Systems of Automatic Speaker Verification and Spoofing Countermeasure

The security of unprotected automatic speaker verification (ASV) system is vulnerable to a variety of spoofing attacks where an attacker (adversary) disguises him/herself as a specific targeted user. It is a common practice to use spoofing countermeasure (CM) to improve the security of ASV systems so as to avoid illegal access. However, recent studies have shown that both ASV and CM systems are vulnerable to adversarial attacks. Previous researches mainly focus on adversarial attacks on a single ASV or CM system. But in practical scenarios, ASVs are typically deployed in conjunction with CM. In this paper, we investigate attacking the tandem system of ASV and CM with adversarial examples. The joint objective function is designed to restrict the generating process of adversarial examples. The joint gradient of the ASV and CM system is derived to generate adversarial examples. Fast Gradient Sign Method (FSGM) and Projected Gradient Descent (PGD) are utilized to study the vulnerability of tandem verification systems against white-box adversarial attacks. Through our attack, audio samples whose original labels are spoof or nontarget can be successfully accepted by the tandem system. Experimental results on the ASVSpoof2019 dataset show that the tandem system is vulnerable to our proposed attack.

The algorithm of stereo vision and shape from shading based on endoscope imaging

With medical endoscopic equipment development, minimally invasive surgery (MIS) has gradually become an essential technical means in daily medical practice. In recent years, minimally invasive surgery has been widely used because of its small incision and quick recovery. However, at the same time, minimally invasive surgery has put forward higher requirements for the operator. A 3D reconstruction framework combined with stereo vision and Shape from Shading (SFS) was proposed to improve endoscopic imaging accuracy and reduce the difficulty of minimally invasive surgery. This paper constructs a joint objective function based on the improved SFS and the classical stereo matching method. The optimization algorithm of the depth map under the joint objective function is given. Finally, the experimental verification of the joint reconstruction algorithm is carried out. The joint reconstruction framework's effectiveness is verified by qualitative and quantitative comparison and analysis based on the silica-gel-heart model and real-heart image datasets. The experimental results show that the joint reconstruction framework can restore the heart surface shape as a whole and retain the local details. Compared with the classical stereo vision and SFS methods, the proposed joint reconstruction method has better robustness and reconstruction accuracy.

Localization of Data Injection Attacks on Distributed M-Estimation

This paper examines data injection attacks on distributed statistical estimation. We consider a dynamically changing distributed network consisting of N agents exchanging information over time. The N agents share the common goal of minimizing a joint objective function, which is the average of the private objective functions in a distributed manner. The private objective function is a realization of an objective function known to all the agents, but uses private data known to the agent alone. The agents’ data are independent and identically distributed. We have previously proposed a novel data injection attack on the Distributed Projected Gradient (DPG) algorithm which is performed locally by malicious nodes in the network that steer the network’s final state to a state of their choice. The proposed attack cannot be detected using previously described techniques. We propose a new detection and localization scheme, performed in a single instance unlike other methods that require the algorithm to run for many instances to acquire statistics over time. This detection and localization scheme is performed by each agent and is purely local, and does not involve decisions made by other agents. Whenever an agent suspects another agent to be an attacker, it will block its data, and maintain convergence to the true optimal state. We provide exponential bounds for the probability of false alarm and probability of attacker detection and localization. Simulations show that when all the attackers are detected and isolated by each agent, the network will recover and converge to the true optimal state.

Alzheimer's Disease Classification Accuracy is Improved by MRI Harmonization based on Attention-Guided Generative Adversarial Networks.

Alzheimer's disease (AD) accounts for 60% of dementia cases worldwide; patients with the disease typically suffer from irreversible memory loss and progressive decline in multiple cognitive domains. With brain imaging techniques such as magnetic resonance imaging (MRI), microscopic brain changes are detectable even before abnormal memory loss is detected clinically. Patterns of brain atrophy can be measured using MRI, which gives us an opportunity to facilitate AD detection using image classification techniques. Even so, MRI scanning protocols and scanners differ across studies. The resulting differences in image contrast and signal to noise make it important to train and test classification models on multiple datasets, and to handle shifts in image characteristics across protocols (also known as domain transfer or domain adaptation). Here, we examined whether adversarial domain adaptation can boost the performance of a Convolutional Neural Network (CNN) model designed to classify AD. To test this, we used an Attention-Guided Generative Adversarial Network (GAN) to harmonize images from three publicly available brain MRI datasets - ADNI, AIBL and OASIS - adjusting for scanner-dependent effects. Our AG-GAN optimized a joint objective function that included attention loss, pixel loss, cycle-consistency loss and adversarial loss; the model was trained bidirectionally in an end-to-end fashion. For AD classification, we adapted the popular 2D AlexNet CNN to handle 3D images. Classification based on harmonized MR images significantly outperformed classification based on the three datasets in non-harmonized form, motivating further work on image harmonization using adversarial techniques.

Optimal Operational Profiles in an Electrodialysis Unit for Ion Recovery

Electrodialysis is an efficient process for the cleaning of industrial water effluent streams from toxic ionic substances with the simultaneous recovery of valuable ions for reuse. Energy consumption and waste recovery are the two key goals of the process. A dynamic optimisation program aims to determine the optimal operating conditions in terms of applied voltage and effluent flow rate in a batch operating scheme. A model is developed that accounts for the ion transfer through the ion selective membranes and the dynamics of the system. A combination of three objective functions targeting the minimisation of the overall batch process time, the minimisation of the electrical energy consumption required for the ion transfer, and the maximisation of the overall degree of separation is investigated. The manipulated variables applied voltage and feed stream flow rate are considered as piecewise constant during each time interval, spanning the duration of the entire batch. The dynamic optimisation problem is solved through standard non-linear programming techniques which calculate the optimal batch duration and condition profiles for the system. A multi-objective analysis is presented for various combinations of weight values for the joint objective function through the development of the Pareto optimal front. The current approach has been implemented in the removal and recovery of sulfuric anions from an aqueous solution and resulted in the achievement of a high degree of separation in a shorter period at a much lower energy consumption.

Joint Inversion of Receiver Functions and Apparent Incidence Angles for Sparse Seismic Data

AbstractThe estimation of crustal structure and thickness is essential in understanding the formation and evolution of terrestrial planets. Initial planetary missions with seismic instrumentation on board face the additional challenge of dealing with seismic activity levels that are only poorly constrained a priori. For example, the lack of plate tectonics on Mars leads to low seismicity, which could, in turn, hinder the application of many terrestrial data analysis techniques. Here we propose using a joint inversion of receiver functions and apparent incidence angles, which contain information on absolute S‐wave velocities of the subsurface. Since receiver function inversions suffer from a velocity depth trade‐off, we in addition exploit a simple relation that defines apparent S‐wave velocity as a function of observed apparent P‐wave incidence angles to constrain the parameter space. We then use the Neighborhood Algorithm for the inversion of a suitable joint objective function. The resulting ensemble of models is then used to derive uncertainty estimates for each model parameter. In preparation for the analysis of data from the InSight mission, we show the application of our proposed method on Mars synthetics and sparse terrestrial data sets from different geological settings using both single and multiple events. We use information‐theoretic statistical tests as model selection criteria and discuss their relevance and implications in a seismological framework.

A generative-discriminative framework that integrates imaging, genetic, and diagnosis into coupled low dimensional space

We propose a novel optimization framework that integrates imaging and genetics data for simultaneous biomarker identification and disease classification. The generative component of our model uses a dictionary learning framework to project the imaging and genetic data into a shared low dimensional space. We have coupled both the data modalities by tying the linear projection coefficients to the same latent space. The discriminative component of our model uses logistic regression on the projection vectors for disease diagnosis. This prediction task implicitly guides our framework to find interpretable biomarkers that are substantially different between a healthy and disease population. We exploit the interconnectedness of different brain regions by incorporating a graph regularization penalty into the joint objective function. We also use a group sparsity penalty to find a representative set of genetic basis vectors that span a low dimensional space where subjects are easily separable between patients and controls. We have evaluated our model on a population study of schizophrenia that includes two task fMRI paradigms and single nucleotide polymorphism (SNP) data. Using ten-fold cross validation, we compare our generative-discriminative framework with canonical correlation analysis (CCA) of imaging and genetics data, parallel independent component analysis (pICA) of imaging and genetics data, random forest (RF) classification, and a linear support vector machine (SVM). We also quantify the reproducibility of the imaging and genetics biomarkers via subsampling. Our framework achieves higher class prediction accuracy and identifies robust biomarkers. Moreover, the implicated brain regions and genetic variants underlie the well documented deficits in schizophrenia.

TransLoc: A Heterogeneous Knowledge Transfer Framework for Fingerprint-Based Indoor Localization

Transfer learning algorithms (TLAs) are often used to solve the distribution discrepancy issue in fingerprint-based indoor localization. However, existing TLAs cannot react well to real time changes in the environmental dynamics of the target space due to three remarkable shortcomings: a) redundant knowledge in source domain may lead to “negative transfer”; b) the required target domain samples to calculate the distributions are unrealistically feasible for real-time positioning; c) they cannot transfer knowledge efficiently across domains with heterogeneous feature spaces. In this paper, we propose TransLoc, a heterogeneous knowledge transfer framework for fingerprint-based indoor localization, which can perform knowledge transfer efficiently even with only one sample in the target domain. Specifically, we first refine the source domain according to the target domain by removing redundant knowledge in the source domain. Then, we derive a cross-domain mapping, which transfers the specific knowledge of one domain to another domain, to construct a homogeneous feature space. In this new feature space, the transfer weights are computed for training a classifier for target location prediction. To further train the framework efficiently, we combine the mapping and weights learning into a joint objective function and solve it by a three-step iterative optimization algorithm. Extensive simulation and real-world experimental results verify that TransLoc not only significantly outperforms state-of-the-art methods but is also very robust to changing environment.

Generalized Embedding Regression: A Framework for Supervised Feature Extraction.

Sparse discriminative projection learning has attracted much attention due to its good performance in recognition tasks. In this article, a framework called generalized embedding regression (GER) is proposed, which can simultaneously perform low-dimensional embedding and sparse projection learning in a joint objective function with a generalized orthogonal constraint. Moreover, the label information is integrated into the model to preserve the global structure of data, and a rank constraint is imposed on the regression matrix to explore the underlying correlation structure of classes. Theoretical analysis shows that GER can obtain the same or approximate solution as some related methods with special settings. By utilizing this framework as a general platform, we design a novel supervised feature extraction approach called jointly sparse embedding regression (JSER). In JSER, we construct an intrinsic graph to characterize the intraclass similarity and a penalty graph to indicate the interclass separability. Then, the penalty graph Laplacian is used as the constraint matrix in the generalized orthogonal constraint to deal with interclass marginal points. Moreover, the L2,1 -norm is imposed on the regression terms for robustness to outliers and data's variations and the regularization term for jointly sparse projection learning, leading to interesting semantic interpretability. An effective iterative algorithm is elaborately designed to solve the optimization problem of JSER. Theoretically, we prove that the subproblem of JSER is essentially an unbalanced Procrustes problem and can be solved iteratively. The convergence of the designed algorithm is also proved. Experimental results on six well-known data sets indicate the competitive performance and latent properties of JSER.