Text Mining Algorithms Research Articles

Text Mining in Electronic Medical Records Enables Quick and Efficient Identification of Pregnancy Cases Occurring After Breast Cancer.

To apply text mining (TM) technology on electronic medical records (EMRs) of patients with breast cancer (BC) to retrieve the occurrence of a pregnancy after BC diagnosis and compare its performance to manual curation. The training cohort (Cohort A) comprised 344 patients with BC age ≤ 40 years old treated at Institut Curie between 2005 and 2007. Manual curation consisted in manually reviewing each EMR to retrieve pregnancies. TM consisted of first applying a keyword filter ("accouch*" or "enceinte," French terms for "deliver*" and "pregnant," respectively) to select a subset of EMRs, and, second, checking manually EMRs to confirm the pregnancy. Then, we applied our TM algorithm on an independent cohort of patients with BC treated between 2008 and 2012 (Cohort B). In Cohort A, 36 pregnancies were identified among 344 patients (10.5%; 2,829 person-years of EMR). Thirty were identified by manual review versus 35 by TM. TM resulted in a lower percentage of manual checking (26.7% v 100%, respectively) and substantial time gains (time to identify a pregnancy: 13 minutes for TM v 244 minutes for manual curation, respectively). Presence of any of the two TM filters showed excellent sensitivity (97%) and negative predictive value (100%). In Cohort B, 67 pregnancies were identified among 1,226 patients (5.5%; 7,349 person-years of EMR). Similarly, for Cohort B, TM spared 904 (73.7%) EMRs from manual review and quickly generated a cohort of 67 pregnancies after BC. Incidence rate of pregnancy after BC was 0.01 pregnancy per person-year of EMR in both cohorts. TM is highly efficient to quickly identify rare events and is a promising tool to improve rapidity, efficiency, and costs of medical research.

Comparative between optimization feature selection by using classifiers algorithms on spam email

<span lang="EN-US">Spam mail has become a rising phenomenon in a world that has recently witnessed high growth in the volume of emails. This indicates the need to develop an effective spam filter. At the present time, Classification algorithms for text mining are used for the classification of emails. This paper provides a description and evaluation of the effectiveness of three popular classifiers using optimization feature selections, such as Genetic algorithm, Harmony search, practical swarm optimization, and simulating annealing. The research focuses on a comparison of the effect of classifiers using K-nearest Neighbor (KNN), Naïve Bayesian (NB), and Support Vector Machine (SVM) on spam classifiers (without using feature selection) also enhances the reliability of feature selection by proposing optimization feature selection to reduce number of features that are not important.</span>

Improved Text Mining Algorithm for Fault Detection using Combined D-Matrix

Systematic diagnostic version of Fault dependency (D-matrix) mostly use for setup the fault method records and its contributing courting on the classified system-degree. It includes dependencies and association between recognizable failure approaches and signs and symptoms related to a machine. Proposed system in this paper describes an relations of domain primarily based textual content repository for construction and renovate combined data dependency matrix through mining lacks of the tuple exact unstructured text ,cumulative during the analysis incidents. Here paradigm is combined D matrix and then fault analysis through textual content mining using advance data preprocessing technique approach to pick out dependencies. Using real-existence statistics accumulated and validated in proposed method.

NoteSum: An integrated note summarization system by using text mining algorithms

This study implemented an integrated system of Note Summarization (NoteSum) that merged with multi-users’ notes and searched for relevant information on the Internet and, slides, and textbooks to create a summary for students to learn effectively. The integrated system's framework consists of four different modules: Topic Identification Module, Supporting Material Finding Module, Content Mapping Module and Learning Material Integrating Module. Five experiments were conducted; these resulted in the following findings. First, translating notes with the assistance of topic terms could enhance translation quality. Second, when mapping contents, NoteSum performed better in a discussion-based course rather than in a technical course. Third, the Jensen-Shannon (JS) Divergence was used to assess the generated summary that performed better for the discussion-based course. Fourth, the three attributes—presence of topic terms, number of non-topic words, and ratio of the words with important parts of speech—had different effects on different subjects. Finally, we compared NoteSum with other existing summarization systems. The results indicated that the NoteSum-generated summary was closer to students’ original notes and thus resulted in better performance in readability, informativeness, and completeness. All the results confirm that our proposed NoteSum is an effective note summarization system for student learning.

A hybrid structured deep neural network with Word2Vec for construction accident causes classification

According to the latest fatal work injury rates reported by the Bureau of Labors Statistics, construction sites remain the most hazardous workplaces. In the construction sector, fatality investigation summary reports are available for past accidents and by investigating such reports, valuable insights can be gained. In this study, text mining algorithms are explored for automatic construction accident causes classification. To be more specific, Word2Vec skip-gram model is utilized to learn word embedding from a domain-specific corpus and a hybrid structured deep neural network is proposed by incorporating the learned word embedding for accident reports classification. Dataset from Occupational Safety and Health Administration (OSHA) is employed in the experiment to evaluate the performance of the proposed approach. Besides, five baseline models: support vector machine (SVM), linear regression (LR), K-nearest neighbor (KNN), decision tree (DT), Naive Bayes (NB) are employed to compare with the proposed approach. Experiment results show that the proposed model achieves the highest average weighted F1 score among all models considered in this study. The result also proves the effectiveness of applying Word2Vec skip-gram algorithm for semantic information augmentation. As a result, robustness of the model is improved when classifying cases of low support values.

Mapping research in assisted reproduction worldwide.

What are the current research trends in human assisted reproduction around the world? An analysis of 26,000+ scientific publications (articles, letters and reviews) produced worldwide between 2005 and 2016. The corpus of publications indexed in PubMed was obtained by combining the Medical Subject Heading (MeSH) terms: 'Reproductive techniques', 'Reproductive medicine', 'Reproductive health', 'Fertility', 'Infertility' and 'Germ cells'. An analysis was then carried out using text mining algorithms to obtain the main topics of interest. A total of 44 main topics were identified, which were then further grouped into 11 categories: 'Laboratory techniques', 'Male factor', 'Quality of ART, ethics and law', 'Female factor', 'Public health and infectious diseases', 'Basic research and genetics', 'Pregnancy complications and risks', 'General - infertility & ART', 'Psychosocial aspects', 'Cancer' and 'Research methodology'. The USA was the leading country in terms of number of publications, followed by the UK, China and France. Research content in high-income countries is fairly homogeneous across categories and it is dominated by 'Laboratory techniques' in Western-Southern Europe, and by 'Quality of ART, ethics and law' in North America, Australia and New Zealand. 'Laboratory techniques' is also the most abundant category on a yearly basis. This study identifies the current hot topics on human assisted reproduction worldwide and their temporal trends for 2005-2016. This provides an innovative picture of the current research that could help explore the areas where further research is needed.

Sentiment Analysis for Product Recommendation System Using Enhanced Stochastic Learning Algorithm

E-Commerce has been known as a rapidly growing commercial enterprise, and even though on line purchasing has no longer accompanied those identical boom patterns within the beyond, it's miles now being diagnosed for its capability. Sentiment evaluation is one of the current research subjects in the subject of textual content mining. Opinions and sentiments mining from natural language are very difficult task. Sentiment analysis is the best solution. This gives important information for decision making in various domains. Various sentiment detection methods are available which affect the quality of result. In this project we are finding the sentiments of people related to the services of E-shopping websites. The sentiments include reviews, ratings and emoticons. The main goal is to recommend the products to users which are posted in E-shopping website and analyzing which one is the best. For this we use hybrid learning algorithm which analyze various feedbacks related to the services. Text mining algorithm is used to find scores of each word. Then sentiments are classified as negative, positive and neutral. It has been observed that the pre-processing of the data is greatly affecting the quality of detected sentiments. Finally analysis takes place based on classification. To find out fake review in the website can be analyzed. This device will discover fake critiques made via posting fake remarks about a product via figuring out the MAC deal with in conjunction with assessment posting styles. User will login to the device using his consumer identification and password and could view various merchandise and will give assessment approximately the product. To discover the evaluation is fake or authentic, system will find out the MAC address of the consumer if the machine observes fake assessment send by way of the identical MAC Address many a times it'll inform the admin to do away with that overview from the device. This gadget uses information mining technique. This machine allows the user to find out accurate overview of the product.

Text mining assisted review of the literature on Li-O2 batteries

The high theoretical capacity of Li-O2 batteries attracts a lot of attention and this field has expanded significantly in the last two decades. In a more general way, the large number of articles being published daily makes it difficult for researchers to keep track of the progress in science. Here we develop a text mining program in an attempt to facilitate the process of reviewing the literature published in a scientific field and apply it to Li-O2 batteries. We analyze over 1800 articles and use the text mining program to extract reported discharge capacities, for the first time, which allows us to show the clear progress made in recent years. In this paper, we focus on three main challenges of Li-O2 batteries, namely the stability-cyclability, the low practical capacity and the rate capability. Indeed, according to our text mining program, articles dealing with these issues represent 86% of the literature published in the field. For each topic, we provide a bibliometric analysis of the literature before focusing on a few key articles which allow us to get insights into the physics and chemistry of such systems. We believe that text mining can help readers find breakthrough papers in a field (e.g. by identifying papers reporting much higher performances) and follow the developments made at the state of the art (e.g. by showing trends in the numbers of papers published—a decline in a given topic probably being the sign of limitations). With the progress of text mining algorithms in the future, the process of reviewing a scientific field is likely to become more and more automated, making it easier for researchers to get the ‘big picture’ in an unfamiliar scientific field.

Sentiment Analysis for Product Recommendation System Using Hybrid Algorithm

E-Commerce has been known as a rapidly growing commercial enterprise, and even though on line purchasing has no longer accompanied those identical boom patterns within the beyond, it's miles now being diagnosed for its capability. Sentiment evaluation is one of the current research subjects in the subject of textual content mining. Opinions and sentiments mining from natural language are very difficult task. Sentiment analysis is the best solution. This gives important information for decision making in various domains. Various sentiment detection methods are available which affect the quality of result. In this paper, finding the sentiments of people related to the services of E-shopping websites. The sentiments include reviews, ratings and emoticons. The main goal is to recommend the products to users which are posted in E-shopping website and analyzing which one is the best and use hybrid learning algorithm which analyze various feedbacks related to the services. Text mining algorithm is used to find scores of each word. Then sentiments are classified as negative, positive and neutral. It has been observed that the pre-processing of the data is greatly affecting the quality of detected sentiments. Finally analysis takes place based on classification. To find out fake review in the website can be analyzed. This device will discover fake critiques made via posting fake remarks about a product via figuring out the MAC deal with in conjunction with assessment posting styles. User will login to the device using his consumer identification and password and could view various merchandise and will give assessment approximately the product. To discover the evaluation is fake or authentic, system will find out the MAC address of the consumer if the machine observes fake assessment send by way of the identical MAC Address many a times it'll inform the admin to do away with that overview from the device. This gadget uses information mining technique. This machine allows the user to find out accurate overview of the product.

A Novel Hybrid Genetic-Whale Optimization Model for Ontology Learning from Arabic Text

Ontologies are used to model knowledge in several domains of interest, such as the biomedical domain. Conceptualization is the basic task for ontology building. Concepts are identified, and then they are linked through their semantic relationships. Recently, ontologies have constituted a crucial part of modern semantic webs because they can convert a web of documents into a web of things. Although ontology learning generally occupies a large space in computer science, Arabic ontology learning, in particular, is underdeveloped due to the Arabic language’s nature as well as the profundity required in this domain. The previously published research on Arabic ontology learning from text falls into three categories: developing manually hand-crafted rules, using ordinary supervised/unsupervised machine learning algorithms, or a hybrid of these two approaches. The model proposed in this work contributes to Arabic ontology learning in two ways. First, a text mining algorithm is proposed for extracting concepts and their semantic relations from text documents. The algorithm calculates the concept frequency weights using the term frequency weights. Then, it calculates the weights of concept similarity using the information of the ontology structure, involving (1) the concept’s path distance, (2) the concept’s distribution layer, and (3) the mutual parent concept’s distribution layer. Then, feature mapping is performed by assigning the concepts’ similarities to the concept features. Second, a hybrid genetic-whale optimization algorithm was proposed to optimize ontology learning from Arabic text. The operator of the G-WOA is a hybrid operator integrating GA’s mutation, crossover, and selection processes with the WOA’s processes (encircling prey, attacking of bubble-net, and searching for prey) to fulfill the balance between both exploitation and exploration, and to find the solutions that exhibit the highest fitness. For evaluating the performance of the ontology learning approach, extensive comparisons are conducted using different Arabic corpora and bio-inspired optimization algorithms. Furthermore, two publicly available non-Arabic corpora are used to compare the efficiency of the proposed approach with those of other languages. The results reveal that the proposed genetic-whale optimization algorithm outperforms the other compared algorithms across all the Arabic corpora in terms of precision, recall, and F-score measures. Moreover, the proposed approach outperforms the state-of-the-art methods of ontology learning from Arabic and non-Arabic texts in terms of these three measures.

Identifying the Risk Culture of Banks Using Machine Learning

Although risk culture is a key element in the risk management process, identifying the risk culture of a firm can be challenging due to the abstract concept of culture. We introduce text mining and unsupervised machine learning algorithms to define the risk culture for U.S. bank holding companies and examine the relation between risk culture and performance. Applying principal component analysis on textually extracted features from 10-K filings identifies uncertainty, litigious and constraining sentiments among risk culture features to be significant in defining risk culture of banks. Cluster analysis of these features proposes three distinct risk culture clusters which we label as good, fair and poor. Consistent with regulatory expectations, sound risk culture in banks is characterized by high profitability ratios, bank stability, lower default risk and good governance.

Bug report severity level prediction in open source software: A survey and research opportunities

Context: The severity level attribute of a bug report is considered one of the most critical variables for planning evolution and maintenance in Free/Libre Open Source Software. This variable measures the impact the bug has on the successful execution of the software system and how soon a bug needs to be addressed by the development team. Both business and academic community have made an extensive investigation towards the proposal methods to automate the bug report severity prediction. Objective: This paper aims to provide a comprehensive mapping study review of recent research efforts on automatically bug report severity prediction. To the best of our knowledge, this is the first review to categorize quantitatively more than ten aspects of the experiments reported in several papers on bug report severity prediction. Method: The mapping study review was performed by searching four electronic databases. Studies published until December 2017 were considered. The initial resulting comprised of 54 papers. From this set, a total of 18 papers were selected. After performing snowballing, more nine papers were selected. Results: From the mapping study, we identified 27 studies addressing bug report severity prediction on Free/Libre Open Source Software. The gathered data confirm the relevance of this topic, reflects the scientific maturity of the research area, as well as, identify gaps, which can motivate new research initiatives. Conclusion: The message drawn from this review is that unstructured text features along with traditional machine learning algorithms and text mining methods have been playing a central role in the most proposed methods in literature to predict bug severity level. This scenario suggests that there is room for improving prediction results using state-of-the-art machine learning and text mining algorithms and techniques.

Investigation of Trends and Analysis of Hidden New Patterns in Prominent News Agencies of Iran Using Data Mining and Text Mining Algorithms

Investigation of Trends and Analysis of Hidden New Patterns in Prominent News Agencies of Iran Using Data Mining and Text Mining Algorithms

A Learning Ecosystem for Linemen Training based on Big Data Components and Learning Analytics

In this paper we present our ongoing work on developing a Learning Ecosystem for Training Linemen in Maintenance Maneuvers using the Experience API standard, Big Data components, and Learning Analytics. The paper describes the architecture of the ecosystem, elaborates on collecting learning experiences and emotional states, and applies analytics for the exploitation of both, legacy and new data. In the former, we exploit legacy e-Learning data for building a Domain model using Text Mining and unsupervised clustering algorithms.

Parental stress in the relationship with the child and personality traits that parents shape in their children

ABSTRACT The aim of the research was to determine what relations occur between the stress experienced by parents in the relationship with their children and the personality traits that they shape in them. Personal characteristics that parents shape in their children have been classified into metatraits described in the Circumplex of Personality model. The study was carried out on a sample of 319 parents of preschool children. Two data mining methods were used: text mining algorithms and cluster analysis carried out by data mining algorithms. Parents experiencing higher stress more often shaped Self-restraint (Delta Plus) personality features. They tried to ensure that their children did not develop Sensation-Seeking (Delta Minus) features associated with impulsivity and dominance over others. Parents experiencing lower parental stress developed Stability (Alpha Plus) qualities related to social adaptation in their children and tried to ensure that they did not develop the antisocial features of Disinhibition (Alpha Minus).

History, Evolution and Future of Big Data and Analytics: A Bibliometric Analysis of Its Relationship to Performance in Organizations

AbstractBig data and analytics (BDA) are gaining momentum, particularly in the practitioner world. Research linking BDA to improved organizational performance seems scarce and widely dispersed though, with the majority focused on specific domains and/or macro‐level relationships. In order to synthesize past research and advance knowledge of the potential organizational value of BDA, the authors obtained a data set of 327 primary studies and 1252 secondary cited papers. This paper reviews this body of research, using three bibliometric methods. First, it elucidates its intellectual foundations via co‐citation analysis. Second, it visualizes the historical evolution of BDA and performance research and its substreams through algorithmic historiography. Third, it provides insights into the field's potential evolution via bibliographic coupling. The results reveal that the academic attention for the BDA–performance link has been increasing rapidly. The study uncovered ten research clusters that form the field's foundation. While research seems to have evolved following two main, isolated streams, the past decade has witnessed more cross‐disciplinary collaborations. Moreover, the study identified several research topics undergoing focused development, including financial and customer risk management, text mining and evolutionary algorithms. The review concludes with a discussion of the implications for different functional management domains and the gaps for both research and practice.

Informing Policy with Text Mining: Technological Change and Social Challenges

The fast development and adoption of ICT technologies and digital services has major social implications. Policy-makers often struggle to design appropriate regulations, defend the rights of citizens and ensure competition. The aim of this work is to present various methodologies that support stakeholders with identification of emerging technologies and related social challenges based on the text mining of news articles. The analysis demonstrates that while each text mining algorithm provides insightful results, their combination yields more detailed and robust overview of media discussions. The results present early signals and trends, the relationships between technologies and social challenges, and changing attitudes towards selected tech issues.

Data science: opportunities to transform education

The article concerns the issue of data science tools implementation, including the text mining and natural language processing algorithms for increasing the value of high education for development modern and technologically flexible society. Data science is the field of study that involves tools, algorithms, and knowledge of math and statistics to discover knowledge from the raw data. Data science is developing fast and penetrating all spheres of life. More people understand the importance of the science of data and the need for implementation in everyday life. Data science is used in business for business analytics and production, in sales for offerings and, for sales forecasting, in marketing for customizing customers, and recommendations on purchasing, digital marketing, in banking and insurance for risk assessment, fraud detection, scoring, and in medicine for disease forecasting, process automation and patient health monitoring, in tourism in the field of price analysis, flight safety, opinion mining etc. However, data science applications in education have been relatively limited, and many opportunities for advancing the fields still unexplored.

Heracles: A framework for developing and evaluating text mining algorithms

Many of today’s businesses are driven by data, and while traditionally only quantitative data is considered, the role of textual data in our digital world is rapidly increasing. Text mining allows to extract and aggregate numerical data from textual documents, which in turn can be used to improve key decision processes. In this paper, we propose Heracles, a framework for developing and evaluating text mining algorithms, with a broad range of applications in industry. In contrast to other frameworks, Heracles supports both the development and evaluation stages of text mining algorithms. A practical use case shows the versatility and ease-of-use of the proposed framework in the domain of aspect-based sentiment analysis.

A text-mining approach to obtain detailed treatment information from free-text fields in population-based cancer registries: A study of non-small cell lung cancer in California.

BackgroundPopulation-based cancer registries have treatment information for all patients making them an excellent resource for population-level monitoring. However, specific treatment details, such as drug names, are contained in a free-text format that is difficult to process and summarize. We assessed the accuracy and efficiency of a text-mining algorithm to identify systemic treatments for lung cancer from free-text fields in the California Cancer Registry.MethodsThe algorithm used Perl regular expressions in SAS 9.4 to search for treatments in 24,845 free-text records associated with 17,310 patients in California diagnosed with stage IV non-small cell lung cancer between 2012 and 2014. Our algorithm categorized treatments into six groups that align with National Comprehensive Cancer Network guidelines. We compared results to a manual review (gold standard) of the same records.ResultsPercent agreement ranged from 91.1% to 99.4%. Ranges for other measures were 0.71–0.92 (Kappa), 74.3%-97.3% (sensitivity), 92.4%-99.8% (specificity), 60.4%-96.4% (positive predictive value), and 92.9%-99.9% (negative predictive value). The text-mining algorithm used one-sixth of the time required for manual review.ConclusionSAS-based text mining of free-text data can accurately detect systemic treatments administered to patients and save considerable time compared to manual review, maximizing the utility of the extant information in population-based cancer registries for comparative effectiveness research.