Poor Model Performance Research Articles

Unsupervised simple Siamese representation learning for blind super-resolution

Deep convolutional neural networks have made unprecedented achievements in image super-resolution (SR) and dominated the field due to their remarkable performance. When the degradation pattern of the test images is inconsistent with the training images, it leads to poor model performance. For example, the degradation could happen after a dimensional stretching. In this case, the most common method is to take blurry, noise, and low-resolution (LR) images and reconstructs SR images by degradation estimation. However, the SR results for this method are highly dependent on the estimation accuracy. To overcome the difficulty with the degradation estimation, this paper designs a degradation representation attention network (DRAN) for image SR. In which, we propose the use of a simple Siamese representation learning to extract the degradation information from various LR images. Specifically, DRAN distinguishes degradation information instead of performing degradation estimation, which can greatly reduce the difficulty. In other words, DRAN can avoid pixel-level operations, transform degradation computation problems into degradation classification problems and flexibly process LR images through degradation representation learning. Finally, DRAN also introduces a channel attention mechanism to enhance the performance of SR. Experimental results show that the proposed scheme can distinguish different degradation modes and obtain accurate degradation information. Meanwhile, experiments on synthetic and real images show that the DRAN achieves remarkable performance on blind SR tasks with good visual effects.

Energy-Saving Applications Based on Train Mass Online Learning Using Time-Varying Train Model

The current train model of the train control system is unable to accurately reflect the influence of nonlinear running resistance, line conditions, the mutative train mass value, and external environment changes on the model in train dynamics, resulting in a defect of poor train model performance. The train basic model and additional resistances are discussed in this paper, a novel neural network online learning method of the time-varying dynamic train model is proposed, combined with the characteristics of rail transit lines, and a neural network learning algorithm is designed by categories and steps. This method can identify the train mass value that changes continuously with passengers during running. The energy savings resulting from using the actual varying train mass in the train control system are calculated. The results show that, when compared to the traditional model’s invariant approximate empirical parameters, the time-varying parameter model can follow changes in the train and line environment and obtain quantitative expressions of curve resistance and tunnel resistance with speed. The time-varying train model was used to conduct engineering tests on the Beijing Capital Airport Line; the online learning deviation of train mass was controlled within a margin of 3.08%, and at the same time, energy consumption decreased by 6.13%.

Human Mobility Support for Personalized Data Offloading

WiFi Access Points (APs) can be used to offload data or computation tasks while users are commuting. However, due to APs&#x2019; limited coverage, offloading performance is heavily impacted by the users&#x2019; mobility. This work proposes to leverage human mobility to inform offloading tasks, taking a data based approach leveraging granular mobility datasets from two cities: Porto and Beijing. We define Offloading Regions (ORs) as areas where a commuter&#x2019;s mobility would enable offloading, and propose an unsupervised learning methodology to extract ORs from mobility traces. Then, we characterise and analyse ORs according to offloading opportunity metrics such as type, availability, total time to offload, and offloading delay. Results show that in 50&#x0025; of the trips, users spend more than 48&#x0025; of the travel time inside ORs extracted according to the proposed methodology. The ability to predict the next ORs would benefit offloading orchestration. Offloading mobility predictability, although crucial, proves to be challenging, expressed by the poor predictive performance of well-known models (<inline-formula> <tex-math notation="LaTeX">$\approx $ </tex-math></inline-formula> 37&#x0025; acc. for the best predictor). We show that mobility regularity properties improve predictive performance up to <inline-formula> <tex-math notation="LaTeX">$\approx $ </tex-math></inline-formula> 35&#x0025;. Finally, we look into the impact of further OR extraction and prediction parameters. We show that the exploration phase length does not impact the discovery of low relevance ORs, and that both filtering low relevance OR and predicting multiple ORs increase predictability. By characterising the trade-off between mobility predictability and offloading opportunities in transit, we highlighting the need for offloading systems to adopt hybrid strategies, i.e., mixing opportunistic and predictive strategies. The conclusions and findings on offloading mobility properties are likely to generalise for varied urban scenarios given the high degree of similarity between the results obtained for the two different and independently collected mobility datasets.

Application of machine learning to evaluating and remediating models for energy and environmental engineering

Machine learning (ML) algorithms have been increasingly successful in their applications to solve energy and environmental engineering problems. ML algorithms have the advantage of being able to solve highly nonlinear issues effectively. Furthermore, considering the limited sample size of data collected in energy and environmental engineering, obtaining a ML model with reasonable accuracy is simple. Unfortunately, the vast majority of the current applications of ML algorithms lack effective screening of dominant factors and comprehensive model validation, which weakens the predictive ability of the models. The present study takes the minimum miscible pressure (MMP) of CO2 - oil systems as an example. It establishes a systematic and robust predictive model to address this issue. Based on 147 sets of slim tube tests, the predictive models of the MMPs are investigated by application of eight ML algorithms. The paper concludes that most of the published ML models in the field of energy and environmental engineering prediction are not reliable. Furthermore, it addresses the main reasons for the poor performance of some predictive models built by ML and provides guidelines on how to make such models robust. To the best of our knowledge, this is the first study to point out the defects of current ML modeling methods and propose countermeasures for their application in energy and environmental engineering problems.

Coal free-swelling index modeling by an ordinal-based soft computing approach

ABSTRACT Coking coal is the most important reductant agent in the steel-making industry and approximately does not have any substitution. Thus, it is globally still on the list of critical raw materials. The free swelling index (FSI) as a qualitative index is used to assess and rank the coking-ability of coal samples. Although the accurate classification of FSI values is one of the most crucial issues in the coke-making industry, it has become one of the most challenging data-driven problems due to the specific difficulties that lie in coal data characteristics. In this study, by analyzing a robust dataset with over 3600 records from U.S. Geological Survey Coal Quality, an insight was obtained into the FSI classification problem factors that have led to the poor performance of machine learning models. By settling the class imbalance at the data level and applying an efficient model training strategy based on the ordinal classification at the algorithmic level, a novel structure for FSI modeling has been proposed. This study also demonstrates direct relation between the model accuracy and the impact of nonlinear feature selection using Mutual Information instead of traditional linear methods.

Development and internal validation of a multivariable risk prediction model for severe rebound pain after foot and ankle surgery involving single-shot popliteal sciatic nerve block

BackgroundRebound pain occurs after up to 50% of ambulatory surgeries involving regional anaesthesia. To assist with risk stratification, we developed a model to predict severe rebound pain after foot and ankle surgery involving single-shot popliteal sciatic nerve block. MethodsAfter ethics approval, we performed a single-centre retrospective cohort study. Patients undergoing lower limb surgery with popliteal sciatic nerve block from January 2016 to November 2019 were included. Exclusion criteria were uncontrolled pain in the PACU, use of a perineural catheter, or loss to follow-up. We developed and internally validated a multivariable logistic regression model for severe rebound pain, defined as transition from well-controlled pain in the PACU (numerical rating scale [NRS] 3 or less) to severe pain (NRS ≥7) within 48 h. A priori predictors were age, sex, surgery type, planned admission, local anaesthetic type, dexamethasone use, and intraoperative anaesthesia type. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC), Nagelkerke's R2, scaled Brier score, and calibration slope. ResultsThe cohort included 1365 patients (mean [standard deviation] age: 50 [16] yr). The primary outcome was abstracted in 1311 (96%) patients, with severe rebound pain in 652 (50%). Internal validation revealed poor model performance, with AUROC 0.632 (95% confidence interval [CI]: 0.602–0.661; bootstrap optimisation 0.021), Nagelkerke's R2 0.063, and scaled Brier score 0.047. Calibration slope was 0.832 (95% CI: 0.623–1.041). ConclusionsWe show that a multivariable risk prediction model developed using routinely collected clinical data had poor predictive performance for severe rebound pain after foot and ankle surgery. Prospective studies involving other patient-related predictors are needed. Clinical trial registrationNCT05018104.

Does poor methodological quality of prediction modeling studies translate to poor model performance? An illustration in traumatic brain injury

BackgroundPrediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation.MethodsWe systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering.ResultsWe included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR − 4% to 21%]) and negative in high RoB models (dAUC − 18%, [IQR − 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (− 32% (95% CI: − 48 to − 15) and unclear RoB models (− 13% (95% CI: − 16 to − 10)) compared to that seen in low RoB models.ConclusionLower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies.

Machine Learning Methods Applied to the Global Modeling of Event-Driven Pitch Angle Diffusion Coefficients During High Speed Streams

Whistler-mode waves in the inner magnetosphere cause electron precipitation in the atmosphere through the physical process of pitch-angle diffusion. The computation of pitch-angle diffusion relies on quasi-linear theory and becomes time-consuming as soon as it is performed at high temporal resolution from satellite measurements of ambient wave and plasma properties. Such an effort is nevertheless required to capture accurately the variability and complexity of atmospheric electron precipitation, which are involved in various Earth’s ionosphere-magnetosphere coupled problems. In this work, we build a global machine-learning model of event-driven pitch-angle diffusion coefficients for storm conditions based on the data of a variety of storms observed by the NASA Van Allen Probes. We first proceed step-by-step by testing 8 nonparametric machine learning methods. With them, we derive machine learning based models of event-driven diffusion coefficients for the storm of March 2013 associated with high-speed streams. We define 3 diagnostics that allow highlighting of the properties of the selected model and selection of the best method. Three methods are retained for their accuracy/efficiency: spline interpolation, the radial basis method, and neural networks (DNN), the latter being selected for the second step of the study. We then use event-driven diffusion coefficients computed from 32 high-speed stream storms in order to build for the first time a statistical event-driven diffusion coefficient that is embedded within the retained DNN model. We achieve a global mean event-driven model in which we introduce a two-parameter dependence, with both the Kp-index and time kept as in an epoch analysis following the storm evolution. The DNN model does not entail any issue to reproduce quite perfectly its target, i.e., averaged diffusion coefficients, with rare exception in the Landau resonance region. The DNN mean model is then used to analyze how mean diffusion coefficients behave compared with individual ones. We find a poor performance of any mean models compared with individual events, with mean diffusion coefficients computing the general trend at best, due to their large variability. The DNN-based model allows simple and fast data exploration of pitch-angle diffusion among its multiple variables. We finally discuss how to conduct uncertainty quantification of Fokker-Planck simulations of storm conditions for space weather nowcasting and forecasting.

Achieving Robust and Transferable Performance for Conservation‐Based Models of Dynamical Physical Systems

AbstractBecause physics‐based models of dynamical systems are constrained to obey conservation laws, they must typically be fed long sequences of temporally consecutive (TC) data during model calibration and evaluation. When memory time scales are long (as in many physical systems), this requirement makes it difficult to ensure distributional similarity when partitioning the data into independent, TC, calibration and evaluation subsets. The consequence can be poor and/or uncertain model performance when applied to new situations. To address this issue, we propose a novel strategy for achieving robust and transferable model performance. Instead of partitioning the data into TC calibration and evaluation periods, the model is run in continuous simulation mode for the entire period, and specific time steps are assigned (via a deterministic data‐allocation approach) for use in computing the calibration and evaluation metrics. Generative adversarial testing shows that this approach results in consistent calibration and evaluation data subset distributions. When tested using three conceptual rainfall‐runoff models applied to 163 catchments representing a wide range of hydro‐climatic conditions, the proposed “distributionally consistent (DC)” strategy consistently resulted in better overall performance than achieved using the traditional “TC” strategy. Testing on independent data periods confirmed superior robustness and transferability of the DC‐calibrated models, particularly under conditions of larger runoff skewness. Because the approach is generally applicable to physics‐based models of dynamical systems, it has the potential to significantly improve the confidence associated with prediction and uncertainty estimates generated using such models.

Partial domain adaption based prediction calibration methodology for fault detection and diagnosis of chillers under variable operational condition scenarios

Data-driven fault detection and diagnosis (FDD) for building energy system has drawn much attention from both industry and academy due to their ability to learn complex relationships from big data. Most existing FDD studies only focus on the model diagnosis accuracy where the operational conditions of test dataset are similar with training dataset. However, it is rare for online operational data to be similar with offline training data. To better simulate the online environment in the aspect of operational conditions, a novel online data simulation method, which is named as variable operational conditions test, is proposed in this study. Furthermore, in order to improve the poor diagnosis performance of FDD models under variable operational condition scenarios, an unsupervised partial domain adaption algorithm is employed to calibrate the predictions of the FDD model within a short period of time. A comprehensive data experiment was conduct based on ASHARE RP-1043. The experimental results indicate the prediction calibration method yields decent performance improvement compared with three traditional data-driven-based FDD models, the highest and overall improvements are 55.07% and 14.5%, respectively. The research outcomes provide practical guidelines for the online application of FDD models when the training data cannot cover the whole sample space.

External Evaluation of a Single Center Precision Tacrolimus Dosing Model for Pediatric Heart Transplant

<h3>Purpose</h3> Extensive variability in tacrolimus pharmacokinetics (PK) influences attainment of therapeutic immunosuppression following pediatric heart transplant. Population PK (popPK) model-based personalized dosing of tacrolimus can reduce the need for tacrolimus titration and improve long-term pediatric heart transplant outcomes. The objective of this study was to externally evaluate a previously described single-center popPK model for personalized tacrolimus therapy. <h3>Methods</h3> A multicenter, retrospective analysis utilized demographics, tacrolimus dosing and concentration data, and laboratory values collected from pediatric inpatients during the first 6 weeks post-transplant, or until discharge, from three sites. PopPK analysis used the modeling software NONMEM v7.2. Observed concentration data were compared to data simulated with our single-center popPK model, using median prediction error (MPE, a measure of bias, <15% acceptable) and median absolute prediction error (MAPE, a measure of accuracy, <30% acceptable). <h3>Results</h3> A total of 285 children: 115 female, 196 white, 32 black, were included. Median age and creatinine clearance were 9.1 years and 101 mL/min/1.73m², respectively. Fluconazole use was rare, occurring in only 6 subjects. When utilizing all data the MPE and MAPE (interquartile range) were acceptable, at -3% (-25%, 10%) and 18% (5%, 49%), respectively, and not different by institution. However, when allowed only 3 concentrations (i.e. the minimum number to adequately predict future concentrations in our prior single-center study) the model poorly predicted future concentrations at all sites (MPE=31%; MAPE=47%). MPE and MAPE varied by site, ranging from 7 to 61% and 30 to 63%, respectively. Including site as a covariate in the popPK model improved predictions (MPE=21%; MAPE=37%) and decreased variability in predictive ability between sites (MPE: 11 to 34%; MAPE: 32 to 45%). <h3>Conclusion</h3> Our single-center popPK model did not adequately predict tacrolimus concentrations from external institutions. Study site appears to drive some of the poor model performance. Future directions include identifying study site differences that influence model performance, as well as identifying additional patient-specific factors affecting tacrolimus dosing requirements.

Generalizability of Cardiovascular Disease Clinical Prediction Models: 158 Independent External Validations of 104 Unique Models.

Background:While clinical prediction models (CPMs) are used increasingly commonly to guide patient care, the performance and clinical utility of these CPMs in new patient cohorts is poorly understood.Methods:We performed 158 external validations of 104 unique CPMs across 3 domains of cardiovascular disease (primary prevention, acute coronary syndrome, and heart failure). Validations were performed in publicly available clinical trial cohorts and model performance was assessed using measures of discrimination, calibration, and net benefit. To explore potential reasons for poor model performance, CPM-clinical trial cohort pairs were stratified based on relatedness, a domain-specific set of characteristics to qualitatively grade the similarity of derivation and validation patient populations. We also examined the model-based C-statistic to assess whether changes in discrimination were because of differences in case-mix between the derivation and validation samples. The impact of model updating on model performance was also assessed.Results:Discrimination decreased significantly between model derivation (0.76 [interquartile range 0.73–0.78]) and validation (0.64 [interquartile range 0.60–0.67], P<0.001), but approximately half of this decrease was because of narrower case-mix in the validation samples. CPMs had better discrimination when tested in related compared with distantly related trial cohorts. Calibration slope was also significantly higher in related trial cohorts (0.77 [interquartile range, 0.59–0.90]) than distantly related cohorts (0.59 [interquartile range 0.43–0.73], P=0.001). When considering the full range of possible decision thresholds between half and twice the outcome incidence, 91% of models had a risk of harm (net benefit below default strategy) at some threshold; this risk could be reduced substantially via updating model intercept, calibration slope, or complete re-estimation.Conclusions:There are significant decreases in model performance when applying cardiovascular disease CPMs to new patient populations, resulting in substantial risk of harm. Model updating can mitigate these risks. Care should be taken when using CPMs to guide clinical decision-making.

ATDNNS: An adaptive time–frequency decomposition neural network-based system for tropical cyclone wave height real-time forecasting

Accurate prediction of anomalous wave heights caused by tropical cyclones (TC) is extremely significant for coastal safety and offshore operations. Previous studies have barely considered TC wave height data which are characterized by short, nonlinear and non-stationary data, in this paper, an adaptive time–frequency decomposition method and a mixture of deep learning methods are used to develop a real-time TC wave height forecasting system (ATDNNS). Specifically, based on wave height data obtained from distributed sensors offshore China, the system uses ensemble empirical mode decomposition (EEMD) to decompose wave height data into multiple subseries containing local features of the original signal at different time scales, and these smoothed subseries are used as inputs to a long short-term memory network (LSTM) to reconstruct all outputs to provide accurate TC wave height forecasts. This study involves 28 TC events in the northwest Pacific Ocean with data collected from 14 coastal buoys over 9 years. The experimental results show that ATDNNS significantly outperforms the baseline model and previous studies. By comparing the results with the operational results of the state-of-the-art operational numerical models, it is shown that the system has the potential to fill the gap of poor initial forecast performance of numerical models, and we further check the effectiveness of ATDNNS for wave heights caused by the recent super typhoon Kong-Ray, achieving excellent results.

High resolution species distribution and abundance models cannot predict separate shrub datasets in adjacent Arctic fjords

AbstractAimImproving species distribution models (SDMs) and species abundance models (SAMs) of woody shrubs is critical for predicting biodiversity changes in the Arctic, which is experiencing especially high warming rates. Yet, it remains relatively unexplored if SDMs and SAMs can explain local scale patterns. We aim to identify predictor differences for the distribution versus abundance of two widespread Arctic shrub species with high resolution models and to compare validation approaches to assess the models’ predictive abilities.LocationNuup Kangerlua (NK) and Kangerluarsunnguaq (K), two adjacent fjords in Southwest Greenland.MethodsWe conducted two separate field surveys in either fjord to construct high resolution (~90 m) SDMs and SAMs forBetula nanaandSalix glauca, analysing the predictive influences of local scale climate, topography and soil moisture indicators. We then alternatively trained and validated models in either NK or K fjord and compared these results with the common split‐sample validation approach. Finally, we assessed if including local scale biotic predictors improves SAM performance.ResultsTemperature extremes and precipitation best predicted the distributions of both species, whereas insolation and soil moisture indicators best predicted abundances. Compared to split‐sample validation, both SDM and SAM performance was substantially reduced with separate survey validation. Regardless of validation approach, models performed poor to moderately well, and including local scale biotic parameters improved SAM performance.Main conclusionsSubstantial differences in model performance between validation approaches highlight the usefulness of using a separate survey for validating model predictive performance. We discuss various factors that might have caused poor model performance, such as not capturing all relevant predictors or enough local scale heterogeneity in predictor or response variables. We emphasise the need to include predictors relevant at the spatial scale of study, such as local scale biotic interactions, for improved predictions at high spatial resolution.

Statistical Analysis Dow Jones Stock Index—Cumulative Return Gap and Finite Difference Method

This study was motivated by the poor performance of the current models used in stock return forecasting and aimed to improve the accuracy of the existing models in forecasting future stock returns. The current literature largely assumes that the residual term used in the existing model is white noise and, as such, has no valuable information. We exploit the valuable information contained in the residuals of the models in the context of cumulative return and construct a new cumulative return gap (CRG) model to overcome the weaknesses of the traditional cumulative abnormal returns (CAR) and buy-and-hold abnormal returns (BHAR) models. To deal with the residual items of the prediction model and improving the prediction accuracy, we also lead the finite difference (FD) method into the autoregressive (AR) model and autoregressive distributed lag (ARDL) model. The empirical results of the study show that the cumulative return (CR) model is better than the simple return model for stock return prediction. We found that the CRG model can improve prediction accuracy, the term of the residuals from the autoregressive analysis is very important in stock return prediction, and the FD model can improve prediction accuracy.

Multiexpert Adversarial Regularization for Robust and Data-Efficient Deep Supervised Learning

Deep neural networks (DNNs) can achieve high accuracy when there is abundant training data that has the same distribution as the test data. In practical applications, data deficiency is often a concern. For classification tasks, the lack of enough labeled images in the training set often results in overfitting. Another issue is the mismatch between the training and the test domains, which results in poor model performance. This calls for the need to have robust and data efficient deep learning models. In this work, we propose a deep learning approach called Multi-Expert Adversarial Regularization learning (MEAR) with limited computational overhead to improve the generalization and robustness of deep supervised learning models. The MEAR framework appends multiple classifier heads (experts) to the feature extractor of the legacy model. MEAR aims to learn the feature extractor in an adversarial fashion by leveraging complementary information from the individual experts as well as the ensemble of the experts to be more robust for an unseen test domain. We train state-of-the-art networks with MEAR for two important computer vision tasks, image classification and semantic segmentation. We compare MEAR to a variety of baselines on multiple benchmarks. We show that MEAR is competitive with other methods and more successful at learning robust features.

Determination of Survival of Gastric Cancer Patients With Distant Lymph Node Metastasis Using Prealbumin Level and Prothrombin Time: Contour Plots Based on Random Survival Forest Algorithm on High-Dimensionality Clinical and Laboratory Datasets

PurposeThis study aimed to identify prognostic factors for patients with distant lymph node-involved gastric cancer (GC) using a machine learning algorithm, a method that offers considerable advantages and new prospects for high-dimensional biomedical data exploration.Materials and MethodsThis study employed 79 features of clinical pathology, laboratory tests, and therapeutic details from 289 GC patients whose distant lymphadenopathy was presented as the first episode of recurrence or metastasis. Outcomes were measured as any-cause death events and survival months after distant lymph node metastasis. A prediction model was built based on possible outcome predictors using a random survival forest algorithm and confirmed by 5×5 nested cross-validation. The effects of single variables were interpreted using partial dependence plots. A contour plot was used to visually represent survival prediction based on 2 predictive features.ResultsThe median survival time of patients with GC with distant nodal metastasis was 9.2 months. The optimal model incorporated the prealbumin level and the prothrombin time (PT), and yielded a prediction error of 0.353. The inclusion of other variables resulted in poorer model performance. Patients with higher serum prealbumin levels or shorter PTs had a significantly better prognosis. The predicted one-year survival rate was stratified and illustrated as a contour plot based on the combined effect the prealbumin level and the PT.ConclusionsMachine learning is useful for identifying the important determinants of cancer survival using high-dimensional datasets. The prealbumin level and the PT on distant lymph node metastasis are the 2 most crucial factors in predicting the subsequent survival time of advanced GC.Trial RegistrationChiCTR Identifier: ChiCTR1800019978

Deep Learning Using Synthetic Seismic Data by Fourier Domain Adaptation in Seismic Structure Interpretation

Deep learning is a data-driven technique that demands network models trained using big datasets. In seismic structure interpretation, it is very difficult, time-consuming, and relatively economic costs to prepare training datasets by directly annotating real seismic data. Seismic convolution method is an efficient way to synthesize seismic data, which can easily and quickly generate large amount of training datasets. However, there are large differences in feature space between synthetic seismic data and real seismic data. That results in the poor performance of network models trained using synthetic training datasets on real seismic data. In this paper, we propose to use Fourier domain adaptation (FDA) to achieve domain transfer. First, amplitude spectrum of synthetic seismic data is replaced with those of real seismic data to make feature space mapping. Then synthetic seismic data is used for transfer training of network models to improve its performance on real seismic data. The experimental results demonstrate that the FDA performs the domain transfer of synthetic seismic data to real seismic data, which improves the generalization of network models trained based on synthetic seismic data. Meanwhile, the FDA is a promising method for deep learning model transfer training in seismic structure interpretation.

A Cross-Project Defect Prediction Model Based on Deep Learning With Self-Attention

Cross-project defect prediction technique is a hot topic in the field of software defect research because of the huge difference in data distribution between source project and target project. The previous defect prediction technique used manually defined parameters to extract the features of the project, which were used by classifiers to construct a defect prediction model. However, traditional features lack sufficient semantic information that exists between source codes, resulting in poor performance of the prediction models. So, driven by the idea that sufficient semantic information helps to construct more accurate prediction models, we propose a cross-project defect prediction framework named BSLDP, which implements semantic extraction of source code files through a bidirectional long and short-term memory network with self-attention mechanism. In particular, we use the proposed semantic extractor named ASL to extract source code semantics based on source code files, and then we employ the proposed classification algorithm fed the semantic information of source project and target project, namely BSL, to build a prediction model. Furthermore, we propose an equal meshing mechanism that ASL generates semantic information on small fragments by dividing the numerical token vector to further improve the performance of the proposed model. We evaluated the performance of the proposed model on a publicly available PROMISE dataset. Compared with the four state-of-the-art methods, the experimental results indicate that on average BSLDP improves the performance of cross-project defect prediction in terms of F1 by 14.2%, 34.6%, 32.2% and 23.6%, respectively.

DCCL: Dual-channel hybrid neural network combined with self-attention for text classification.

Text classification is a fundamental task in natural language processing. The Chinese text classification task suffers from sparse text features, ambiguity in word segmentation, and poor performance of classification models. A text classification model is proposed based on the self-attention mechanism combined with CNN and LSTM. The proposed model uses word vectors as input to a dual-channel neural network structure, using multiple CNNs to extract the N-Gram information of different word windows and enrich the local feature representation through the concatenation operation, the BiLSTM is used to extract the semantic association information of the context to obtain the high-level feature representation at the sentence level. The output of BiLSTM is feature weighted with self-attention to reduce the influence of noisy features. The outputs of the dual channels are concatenated and fed into the softmax layer for classification. The results of the multiple comparison experiments showed that the DCCL model obtained 90.07% and 96.26% F1-score on the Sougou and THUNews datasets, respectively. Compared to the baseline model, the improvement was 3.24% and 2.19%, respectively. The proposed DCCL model can alleviate the problem of CNN losing word order information and the gradient of BiLSTM when processing text sequences, effectively integrate local and global text features, and highlight key information. The classification performance of the DCCL model is excellent and suitable for text classification tasks.