Combination Of Data Mining Techniques Research Articles

Predicting insurance claims through a variety of data mining techniques: facing lots of missing values and moderate class-imbalanced levels

Abstract This paper copes with a real-world classification problem related to the management of claims received in an insurance company. The way to obtain the classifier is not easy due to the high amount of missing values as well as the inherent imbalanced scenario within class labels. Once the data partition has been done, the training set is submitted to an intensive double grid search in order to obtain the most promising type of missing value imputation approach and then a step ahead is done using the best method and it starts the next round of data mining strategies which now falls into data rebalancing umbrella. Again, a grid search from an undersampling and oversampling family with different settings is done taking into account only seen data. The training data obtained after the first grid search are now submitted to the second step according the second grid search in order to get the ready training set for the further classifier training. The main objective of the work is to find the best combination of data mining techniques that suits the data set with a pipeline containing two types of data preparation methods coming from different families. As an outcome, first the problem of the presence of missing values has been addressed and then the data rebalancing techniques has been applied. The study focuses on obtaining classifiers based on Bayesian and lazy approaches as well as decision trees, evaluated on metrics such as the area under the ROC curve (AUC), Cohen’s kappa, Accuracy and the F-measure, among others. The imputation by the mean the mode is preferable to the Expectation Maximization Imputation in the scenario faced in this paper taking into account that the amount of missing values is higher than a forty percent for many features.

Underlying mechanisms of gold nanoalloys stabilization.

Gold nanoclusters have attracted significant attention due to their unique physical-chemical properties, which can be tuned by alloying with elements such as Cu, Pd, Ag, and Pt to design materials for various applications. Although Au-nanoalloys have promising applications, our atomistic understanding of the descriptors that drive their stability is far from satisfactory. To address this problem, we considered 55-atom model nanoalloys that have been synthesized by experimental techniques. Here, we combined data mining techniques for creating a large sample of representative configurations, density functional theory for performing total energy optimizations, and Spearman correlation analyses to identify the most important descriptors. Among our results, we have identified trends in core-shell formation in the AuCu and AuPd systems and an onion-like design in the AuAg system, characterized by the aggregation of gold atoms on nanocluster surfaces. These features are explained by Au's surface energy, packing efficiency, and charge transfer mechanisms, which are enhanced by the alloys' preference for adopting the structure of the alloying metal rather than the low-symmetry one presented by Au55. These generalizations provide insights into the interplay between electronic and structural properties in gold nanoalloys, contributing to the understanding of their stabilization mechanisms and potential applications in various fields.

Enhancing loan fraud detection process in the banking sector using data mining techniques

<span>Ongoing loan fraud is a source of concern for financial institutions, as it has a direct financial impact and also scares off customers. This pattern, which can be traced to the development of modern technology, the introduction of novel ideas, and the quickening pace of international connections, makes the detection of fraud an expensive endeavour. This article proposes a novel framework for enhancing the fraud detection of loan banking using data mining algorithms. The framework extracts a number of predictive analysis techniques for identifying loan fraud. Several methods employing a wide range of pipeline architectures have been tried in order to select the optimal champion model. Autotuning has also been used to find the best possible setting for the model’s hyperparameters. The results of the evaluation show that autoencoder with gradient boosting outperformed the other classification algorithms with an accuracy of 98.62%. The proposed framework has the potential to significantly improve the fraud detection process of loan banking, which can ultimately lead to better faster fraud detects rates by combining data mining techniques with dimensionality reduction strategies in the feature space.</span>

Exploring social responsibility, social marketing and happiness using artificial intelligence, automated text analysis and correspondence analysis

PurposeThis work aims to examine the communication on Twitter of the most responsible companies in Spain to identify the topics covered on corporate social responsibility (CSR) from the perspective of happiness and social marketing. In addition, the profiles of the messages that show an association with the impact of the messages have been identified.Design/methodology/approachAn empirical analysis of the Twitter posts of Spain's ten most responsible companies has been carried out. The methodology of this work combines data mining techniques, sentiment analysis and content analysis, both from a quantitative and qualitative approach.FindingsThe results show that most brand tweets do not deal with CSR-related topics. The topics they address the most are those related to sports and the weather. From the perspective of social marketing, conversational-type tweets are the most published and have achieved the most significant reaction from the public. In addition, four messages' profiles have been identified based on the company and the emotional connotation associated with the impact, giving rise to more outstanding promotion of social causes.Originality/valueOur main contribution to this work has been to value positive communication and social marketing to promote better CSR on Twitter. In this sense, it has been verified that there is a relationship between the public's reaction, the affective connotation and the company that issues the messages.

Research on energy saving prediction of building clusters based on data mining algorithm

In order to solve the problem of excessive burden of electricity and energy consumption in urban landscape buildings clusters, the study combined data mining algorithms to establish a prediction model for energy-saving renovation of urban landscape building clusters. Firstly, the energy demand and energy consumption of the urban landscape buildings complex were analysed, a mathematical model was established to predict the energy consumption of the building complex. Then, the prediction model of energy-saving retrofitting of building clusters was constructed by combining data mining techniques. The experimental results show that the change trend of total energy consumption is different under different single influencing factors of energy consumption. Among them, the lighting power density factor has the greatest influence on energy consumption, and its annual energy consumption change rate can reach about 0.35. Applying the prediction model to the energy consumption prediction of 15 urban single buildings, it was found that the total energy consumption of the buildings before the retrofit was much higher than that after the retrofit, and the energy-saving rate of the whole observed sample building group was as high as 18.5%, meanwhile, the highest energy-saving rate of the single buildings reached 30.1%.

Business Process Complexity Measurement: A Systematic Literature Review

Business process complexity is an important factor that affects process quality and has a significant impact on the maintenance, optimization, and execution efficiency of processes. To review the application progress and trends of process complexity measurement in business process management, this study uses the systematic literature review (SLR) method to qualitatively and quantitatively analyze the theoretical background, rationality, effectiveness, and comprehensiveness of 92 process complexity metrics. The findings showed that the measurement of process complexity primarily encompasses four dimensions: activity complexity, control-flow complexity, data-flow complexity, and resource complexity. However, most metrics consider only one or two aspects of complexity in the process and rely mainly on empirical validation, thus lacking theoretical validation support. Currently, the most popular and widely used complexity metric is Control Flow Complexity (CFC). Process complexity metrics mainly focus on measuring the complexity of process models, and the majority of them focus on activity complexity or control-flow complexity. The future research trend is to combine data mining techniques with process log data to analyze process complexity.

Segmentation of Potential Fraud Taxpayers and Characterization in Personal Income Tax Using Data Mining Techniques

This paper proposes an analytical framework that combines dimension reduction and data mining techniques to obtain a sample segmentation according to potential fraud probability. In this regard, the purpose of this study is twofold. Firstly, it attempts to determine tax benefits that are more likely to be used by potential fraud taxpayers by means of investigating the Personal Income Tax structure. Secondly, it aims at characterizing through socioeconomic variables the segment profiles of potential fraud taxpayer to offer an audit selection strategy for improving tax compliance and improve tax design. An application to the annual Spanish Personal Income Tax sample designed by the Institute for Fiscal Studies is provided. Results obtained confirm that the combination of data mining techniques proposed offers valuable information to contribute to the study of tax fraud.

HOUSEHOLD ELECTRICITY LOAD FORECASTING TOWARD DEMAND RESPONSE PROGRAM USING DATA MINING TECHNIQUES IN A TRADITIONAL POWER GRID

At present, the continuous increase of household electricity demand is strategic and crucial in electricity demand management. Household electricity consumers can play an important role in this issue. The rationalization of electricity consumption might be achieved by using an efficient Demand Response (DR) program. In this paper a new methodology is suggested using a combination of data mining techniques namely K-means clustering, K-Nearest Neighbors (K-NN) classification and ARIMA for electricity load forecasting using consumers' electricity prepaid bills data set of an ordinary electricity grid with prepaid electricity meters. As a result of applying this methodology, various DR programs are recommended as an attempt to assist the management of electricity system to manage the electricity demand issues from demand-side in an efficient and effective manner, which can be put into practice. A case study has been carried out in Tulkarm District, Palestine. The performance of applying the suggested methodology is measured, and the results are considered very well.Keywords: Demand Response (DR); K-means Clustering; K-Nearest Neighbor classification (K-NN); ARIMA model; Prepaid electricity metersJEL Classifications: Q4, Q41, Q47, Q49DOI: https://doi.org/10.32479/ijeep.11192

Customization of Diet May Promote Exercise and Improve Mental Wellbeing in Mature Adults: The Role of Exercise as a Mediator.

Diet, dietary practices and exercise are modifiable risk factors for individuals living with mental distress. However, these relationships are intricate and multilayered in such a way that individual factors may influence mental health differently when combined within a pattern. Additionally, two important factors that need to be considered are gender and level of brain maturity. Therefore, it is essential to assess these modifiable risk factors based on gender and age group. The purpose of the study was to explore the combined and individual relationships between food groups, dietary practices and exercise to appreciate their association with mental distress in mature men and women. Adults 30 years and older were invited to complete the food–mood questionnaire. The anonymous questionnaire link was circulated on several social media platforms. A multi-analyses approach was used. A combination of data mining techniques, namely, a mediation regression analysis, the K-means clustering and principal component analysis as well as Spearman’s rank–order correlation were used to explore these research questions. The results suggest that women’s mental health has a higher association with dietary factors than men. Mental distress and exercise frequency were associated with different dietary and lifestyle patterns, which support the concept of customizing diet and lifestyle factors to improve mental wellbeing.

Detection and Classification of Malicious Flows in Software-Defined Networks Using Data Mining Techniques.

The increasing availability of mobile devices and applications, the progress in virtualisation technologies, and advances in the development of cloud-based distributed data centres have significantly stimulated the growing interest in the use of software-defined networks (SDNs) for both wired and wireless applications. Standards-based software abstraction between the network control plane and the underlying data forwarding plane, including both physical and virtual devices, provides an opportunity to significantly increase network security. In this paper, to secure SDNs against intruders’ actions, we propose a comprehensive system that exploits the advantages of SDNs’ native features and implements data mining to detect and classify malicious flows in the SDN data plane. The architecture of the system and its mechanisms are described, with an emphasis on flow rule generation and flow classification. The concept was verified in the SDN testbed environment that reflects typical SDN flows. The experiments confirmed that the system can be successfully implemented in SDNs to mitigate threats caused by different malicious activities of intruders. The results show that our combination of data mining techniques provides better detection and classification of malicious flows than other solutions.

Portfolio Optimization Using an Alternative Approach. Towards Data Mining Techniques Way

In the light of the current budget constraints, the investors face a challenge when building their stock portfolios that should lead to a minimized risk for an expected level of return. Mathematical tools have become essential for portfolio theory formulation in the last decades. In this article, our main objective is to illustrate the utility of some data mining tools and techniques, with a focus on principal components analysis and cluster analysis. The case study reveals a comparative empirical results analysis of the classical Markowitz portfolio optimization model and a combined data mining techniques model. The results show how useful data mining techniques can be for the finance area, with positive implications for investors’ strategy design and implementation. Our study reveals that stocks selection requires the use of modern techniques that take in account the multidimensional perspective of investment decision. Henceforth, we propose that a debate should be launched concerning the design of stock markets design, which generally focus on simple design oriented to the stocks liquidity. In order to help investors, those indices should combine multiple dimensions of stocks definition, as the return, the risk and the liquidity of a stock are at least of the same importance from an investment decision perspective.

Approaches to text mining for analyzing treatment plan of quit smoking with free-text medical records: A PRISMA-compliant meta-analysis.

Background:Smoking is a complex behavior associated with multiple factors such as personality, environment, genetics, and emotions. Text data are a rich source of information. However, pure text data requires substantial human resources and time to extract and apply the knowledge, resulting in many details not being discovered and used. This study proposes a novel approach that explores a text mining flow to capture the behavior of smokers quitting tobacco from their free-text medical records. More importantly, the paper examines the impact of these changes on smokers. The goal is to help smokers quit smoking. The study population included adult patients that were >20 years old of age who consulted the medical center's smoking cessation outpatient clinic from January to December 2016. A total of 246 patients visited the clinic in the study period. After excluding incomplete medical records or lost follow up, there were 141 patients included in the final analysis. There are 141 valid data points for patients who only treated once and patients with empty medical records. Two independent review authors will make the study selection based on the study eligibility criteria. Our participants are from all the patients that were involved in this study and the staff of Division of Family Medicine, National Taiwan University Hospital. Interventions and study appraisal are not required.Methods:The paper develops an algorithm for analyzing smoking cessation treatment plans documented in free-text medical records. The approach involves the development of an information extraction flow that uses a combination of data mining techniques, including text mining. It can use not only to help others quit smoking but also for other medical records with similar data elements. The Apriori associations of our algorithm from the text mining revealed several important clinical implications for physicians during smoking cessation. For example, an apparent association between nicotine replacement therapy (NRT) and other medications such as Inderal, Rivotril, Dogmatyl, and Solaxin. Inderal and Rivotril use in patients with anxiety disorders as anxiolytics frequently.Results:Finally, we find that the rules associating with NRT combination with blood tests may imply that the use of NRT combination therapy in smokers with chronic illness may result in lower abstinence. Further large-scale surveys comparing varenicline or bupropion with NRT combination in smokers with a chronic disease are warranted. The Apriori algorithm suffers from some weaknesses despite being transparent and straightforward. The main limitation is the costly wasting of time to hold a vast number of candidates sets with frequent itemsets, low minimum support, or large itemsets.Conclusion:In the paper, the most visible areas for the therapeutic application of text mining are the integration and transfer of advances made in basic sciences, as well as a better understanding of the processes involved in smoking cessation. Text mining may also be useful for supporting decision-making processes associated with smoking cessation. Systematic review registration number is not registered.

Blurred Shots: Investigating the Information Crisis Around Vaccination in Italy

This article aims at exploring a case of information crisis in Italy through the lens of vaccination-related topics. Such a controversial issue, dividing public opinion and political agendas, has received diverse information coverage and public policies over time in the Italian context, whose situation appears quite unique compared with other countries because of a strong media spectacularization and politicization of the topic. In particular, approval of the “Lorenzin Decree,” increasing the number of mandatory vaccinations from 4 to 10, generated a nationwide debate that divided public opinion and political parties, triggering a complex informative crisis and fostering the perception of a social emergency on social media. This resulted in negative stress on lay publics and on the public health system. The study adopted an interdisciplinary framework, including political science, public relations, and health communication studies, as well as a mixed-method approach, combining data mining techniques related to news media coverage and social media engagement, with in-depth interviews to key experts, selected among researchers, journalists, and communication managers. The article investigates reasons for the information crisis and identifies possible solutions and interventions to improve the effectiveness of public health communication and mitigate the social consequences of misinformation around vaccination.

Semi supervised data mining model for the prognosis of pre-diabetic conditions in type 2 Diabetes Mellitus.

Diabetic Mellitus is the leading disease in the world irrespective of age and geographical location. It is estimated that 43% of the overall population is affected by the disease. The reasons for the disease include inappropriate diet lifestyle with allied symptoms like obesity. Therefore, the prognosis and diagnosis of the disease are important for adequate combat and care. The prognosis related known symptoms of the disease include incontinence (inability to control urination) and frequent fatigue. Moreover, early prediction of the disease plays an important role in the prognosis of other associated conditions such as heart failure leading to chronic illness. Hence, it is of interest to describe a data mining based prediction model using known features (derived from epidemiological data collected from the public hospital using routine tests) to help in the prognosis of the disease. We used data pre-processing techniques for handling missing values and dimensionality reduction models to improve data quality. The Minimum Description Length principle (MDL) model for discretization (replacing a continuum with a finite set of points) is used to reduce high-level dimensionality of the dataset, which enabled to categorize the dataset into small groups in ordered intervals. Thus, we describe a semi-supervised learning technique (identifies promising attributes using clustering and classification methods) by combining data mining techniques for reasonable accuracy having adequate sensitivity and specificity for further discussion, cross-validation, revaluation, and application. Early prediction of the disease with improved accuracy by analysing specificity ranges in blood pressure and glucose levels will be useful to combat Diabetes Mellitus.

CROWD4EMS: A CROWDSOURCING PLATFORM FOR GATHERING AND GEOLOCATING SOCIAL MEDIA CONTENT IN DISASTER RESPONSE

Abstract. Increase in access to mobile phone devices and social media networks has changed the way people report and respond to disasters. Community-driven initiatives such as Stand By Task Force (SBTF) or GISCorps have shown great potential by crowdsourcing the acquisition, analysis, and geolocation of social media data for disaster responders. These initiatives face two main challenges: (1) most of social media content such as photos and videos are not geolocated, thus preventing the information to be used by emergency responders, and (2) they lack tools to manage volunteers contributions and aggregate them in order to ensure high quality and reliable results. This paper illustrates the use of a crowdsourcing platform that combines automatic methods for gathering information from social media and crowdsourcing techniques, in order to manage and aggregate volunteers contributions. High precision geolocation is achieved by combining data mining techniques for estimating the location of photos and videos from social media, and crowdsourcing for the validation and/or improvement of the estimated location. The evaluation of the proposed approach is carried out using data related to the Amatrice Earthquake in 2016, coming from Flickr, Twitter and Youtube. A common data set is analyzed and geolocated by both the volunteers using the proposed platform and a group of experts. Data quality and data reliability is assessed by comparing volunteers versus experts results. Final results are shown in a web map service providing a global view of the information social media provided about the Amatrice Earthquake event.

Mid-long term oil spill forecast based on logistic regression modelling of met-ocean forcings

Past major oil spill disasters, such as the Prestige or the Deepwater Horizon accidents, have shown that spilled oil may drift across the ocean for months before being controlled or reaching the coast. However, existing oil spill modelling systems can only provide short-term trajectory simulations, being limited by the typical met-ocean forecast time coverage. In this paper, we propose a methodology for mid-long term (1–6 months) probabilistic predictions of oil spill trajectories, based on a combination of data mining techniques, statistical pattern modelling and probabilistic Lagrangian simulations. Its main features are logistic regression modelling of wind and current patterns and a probabilistic trajectory map simulation. The proposed technique is applied to simulate the trajectory of drifting buoys deployed during the Prestige accident in the Bay of Biscay. The benefits of the proposed methodology with respect to existing oil spill statistical simulation techniques are analysed.

Quasar lenses and pairs in the VST-ATLAS and Gaia

We report on discovery results from a quasar lens search in the ATLAS public footprint, extending quasar lens searches to a regime without $u-$band or fiber-spectroscopic information, using a combination of data mining techniques on multi-band catalog magnitudes and image-cutout modelling. Spectroscopic follow-up campaigns, conducted at the 2.6m Nordic Optical Telescope (La Palma) and 3.6m New Technology Telescope (La Silla) in 2016, yielded seven pairs of quasars exhibiting the same lines at the same redshift and monotonic flux-ratios with wavelength (hereafter NIQs, Nearly Identical Quasar pairs). The quasar redshifts range between $\approx1.2$ and $\approx 2.7;$ contaminants are typically pairs of bright blue stars, quasar-star alignments along the line of sight, and narrow-line galaxies at $0.3<z<0.7.$ Magellan data of A0140-1152 (01$^h$40$^m$03.0$^s$-11$^d$52$^m$19.0$^s$, $z_{s}=1.807$) confirm it as a lens with deflector at $z_{l}=0.277$ and Einstein radius $\theta_{\rm E}=(0.73\pm0.02)^\ase$. We show the use of spatial resolution from the Gaia mission to select lenses and list additional systems from a WISE-Gaia-ATLAS search, yielding three additional lenses (02$^h$35$^m$27.4$^s$-24$^d$33$^m$13.2$^s$, 02$^h$59$^m$33.$^s$-23$^d$38$^m$01.8$^s$, 01$^h$46$^m$32.9$^s$-11$^d$33$^m$39.0$^s$). The overall sample consists of 11 lenses/NIQs, plus three lenses known before 2016, over the ATLAS-DR3 footprint ($\approx3500$~deg$^2$). Finally, we discuss future prospects for objective classification of pair/NIQ/contaminant spectra.

Maintenance scheduling using data mining techniques and time series models

Condition-based maintenance (CBM) should be derived carefully to reduce maintenance costs along with useless maintenance shifts and to predict ideal time to do the maintenance. In this paper, a new method is proposed by the combination of data mining techniques and time series models to schedule maintenance activities. Considering a real database which contains failures and values of factors degrading the pump in the time of failure, a clustering algorithm is used to categorize failures based on the similarity in types of maintenance activities. Then, rules are extracted for characterizing the clusters and presenting a range for each factor by applying a proper association rule algorithm. Subsequently, time series models are applied to predict the time period that a factor may meet its rule’s range. Thus, a novel method is presented for a relative comparison between rules and predicted factor’s values and a prognostic scheduling is designed with respect to the effects of previous maintenance activities. The results of numerical experiments reveal that the proposed method can effectively determine when and which maintenance activities should be performed.

A Novel Continuous Blood Pressure Estimation Approach Based on Data Mining Techniques.

Continuous blood pressure (BP) estimation using pulse transit time (PTT) is a promising method for unobtrusive BP measurement. However, the accuracy of this approach must be improved for it to be viable for a wide range of applications. This study proposes a novel continuous BP estimation approach that combines data mining techniques with a traditional mechanism-driven model. First, 14 features derived from simultaneous electrocardiogram and photoplethysmogram signals were extracted for beat-to-beat BP estimation. A genetic algorithm-based feature selection method was then used to select BP indicators for each subject. Multivariate linear regression and support vector regression were employed to develop the BP model. The accuracy and robustness of the proposed approach were validated for static, dynamic, and follow-up performance. Experimental results based on 73 subjects showed that the proposed approach exhibited excellent accuracy in static BP estimation, with a correlation coefficient and mean error of 0.852 and -0.001±3.102 mmHg for systolic BP, and 0.790 and -0.004±2.199 mmHg for diastolic BP. Similar performance was observed for dynamic BP estimation. The robustness results indicated that the estimation accuracy was lower by a certain degree one day after model construction but was relatively stable from one day to six months after construction. The proposed approach is superior to the state-of-the-art PTT-based model for an approximately 2-mmHg reduction in the standard derivation at different time intervals, thus providing potentially novel insights for cuffless BP estimation.

Application of data mining techniques and data analysis methods to measure cancer morbidity and mortality data in a regional cancer registry: The case of the island of Crete, Greece

Background and ObjectiveMicro or macro-level mapping of cancer statistics is a challenging task that requires long-term planning, prospective studies and continuous monitoring of all cancer cases. The objective of the current study is to present how cancer registry data could be processed using data mining techniques in order to improve the statistical analysis outcomes. MethodsData were collected from the Cancer Registry of Crete in Greece (counties of Rethymno and Lasithi) for the period 1998–2004. Data collection was performed on paper forms and manually transcribed to a single data file, thus introducing errors and noise (e.g. missing and erroneous values, duplicate entries etc.). Data were pre-processed and prepared for analysis using data mining tools and algorithms. Feature selection was applied to evaluate the contribution of each collected feature in predicting patients’ survival. Several classifiers were trained and evaluated for their ability to predict survival of patients. Finally, statistical analysis of cancer morbidity and mortality rates in the two regions was performed in order to validate the initial findings. ResultsSeveral critical points in the process of data collection, preprocessing and analysis of cancer data were derived from the results, while a road-map for future population data studies was developed. In addition, increased morbidity rates were observed in the counties of Crete (Age Standardized Morbidity/Incidence Rates ASIR= 396.45 ± 2.89 and 274.77 ±2.48 for men and women, respectively) compared to European and world averages (ASIR= 281.6 and 207.3 for men and women in Europe and 203.8 and 165.1 in world level). Significant variation in cancer types between sexes and age groups (the ratio between deaths and reported cases for young patients, less than 34 years old, is at 0.055 when the respective ratio for patients over 75 years old is 0.366) was also observed. ConclusionsThis study introduced a methodology for preprocessing and analyzing cancer data, using a combination of data mining techniques that could be a useful tool for other researchers and further enhancement of the cancer registries.