Unstructured Big Data Research Articles

벡터자기회귀모형을 이용한 온라인 뉴스기사와 아파트 매매가격의 동태적 관계 연구: 비정형 빅데이터를 활용한 감성분석 기법의 적용

벡터자기회귀모형을 이용한 온라인 뉴스기사와 아파트 매매가격의 동태적 관계 연구: 비정형 빅데이터를 활용한 감성분석 기법의 적용

A big data approach to examining social bots on Twitter

PurposeSocial bots are prevalent on social media. Malicious bots can severely distort the true voices of customers. This paper aims to examine social bots in the context of big data of user-generated content. In particular, the author investigates the scope of information distortion for 24 brands across seven industries. Furthermore, the author studies the mechanisms that make social bots viral. Last, approaches to detecting and preventing malicious bots are recommended.Design/methodology/approachA Twitter data set of 29 million tweets was collected. Latent Dirichlet allocation and word cloud were used to visualize unstructured big data of textual content. Sentiment analysis was used to automatically classify 29 million tweets. A fixed-effects model was run on the final panel data.FindingsThe findings demonstrate that social bots significantly distort brand-related information across all industries and among all brands under study. Moreover, Twitter social bots are significantly more effective at spreading word of mouth. In addition, social bots use volumes and emotions as major effective mechanisms to influence and manipulate the spread of information about brands. Finally, the bot detection approaches are effective at identifying bots.Research limitations/implicationsAs brand companies use social networks to monitor brand reputation and engage customers, it is critical for them to distinguish true consumer opinions from fake ones which are artificially created by social bots.Originality/valueThis is the first big data examination of social bots in the context of brand-related user-generated content.

Towards privacy preserving unstructured big data publishing

Towards privacy preserving unstructured big data publishing

How Large Are Your G-Values? Try Gosset’s Guinnessometrics When a Little “p” Is Not Enough

A crisis of validity has emerged from three related crises of science, that is, the crises of statistical significance and complete randomization, of replication, and of reproducibility. Guinnessometrics takes commonplace assumptions and methods of statistical science and stands them on their head, from little p-values to unstructured Big Data. Guinnessometrics focuses instead on the substantive significance which emerges from a small series of independent and economical yet balanced and repeated experiments. Originally developed and market-tested by William S. Gosset aka “Student” in his job as Head Experimental Brewer at the Guinness Brewery in Dublin, Gosset’s economic and common sense approach to statistical inference and scientific method has been unwisely neglected. In many areas of science and life, the 10 principles of Guinnessometrics or G-values outlined here can help. Other things equal, the larger the G-values, the better the science and judgment. By now a colleague, neighbor, or YouTube junkie has probably shown you one of those wacky psychology experiments in a video involving a gorilla, and testing the limits of human cognition. In one video, a person wearing a gorilla suit suddenly appears on the scene among humans, who are themselves engaged in some ordinary, mundane activity such as passing a basketball. The funny thing is, prankster researchers have discovered, when observers are asked to think about the mundane activity (such as by counting the number of observed passes of a basketball), the unexpected gorilla is frequently unseen (for discussion see Kahneman 2011). The gorilla is invisible. People don’t see it.

Factors Influencing Customers' Satisfaction and Dissatisfaction with Hotels: A Text-Mining Approach

User-generated review of hotels plays an important role in the e-commerce and big data era. The digital and big data era has created novel sources of information that can be used by scholars for knowledge creation, business intelligence, and bringing meaning into unstructured big data. In addition to build a big picture on sources of satisfaction and dissatisfaction, this study aims to develop results that are more practical for hoteliers. A text-mining approach was applied. Reviews of 10 hotels were collected from TripAdvisor.com for Mazandaran province in Iran. The findings of the research show that "location" appearing 99 times, "room" 53 times, and "staff" and "restaurant" 40 times are the most influential factors that determine positive reviews (customer satisfaction). At the same time, the main determinants of customer dissatisfaction were the words "restaurant," "Wi-Fi," and again "room." Given these results, it can be said that customer satisfaction and dissatisfaction can coexist, as literature shows. In this respect, the factors (e.g., room & restaurant) that make customers satisfied have potential to also make them dissatisfied, if they are not provided and properly delivered. Managerial implications are also discussed.

아파트 매매가격과 부동산 온라인 뉴스의 교차상관관계와 인과관계 분석 : 온라인 뉴스 기사의 비정형 빅데이터를 활용한 감성분석 기법의 적용

아파트 매매가격과 부동산 온라인 뉴스의 교차상관관계와 인과관계 분석 : 온라인 뉴스 기사의 비정형 빅데이터를 활용한 감성분석 기법의 적용

A secure technique for unstructured big data using clustering method

Analyzing the multi-dimensional data in faster way is an important and basic aspect in any clustering mechanism. At the same time the clustering mechanism should provide a security structure to protect data loss by preventing distinguish security attacks and transmission errors. Also, to control the data size, security overheads and data loss due to data and time overheads during clustering of unstructured and uncertain big data, a data compression system is also needed. Among the various researches, the current literature fails to suggest any integrated technique to solve these issues in a combinatorial way. Henceforth, this research provides a new compact clustering technique which prevents data loss by including SDES encryption technique to protect security attacks, a new error control scheme to address any number of transmission errors and Huffman compression to control the data size and extra security overheads. Apart from that, the faster execution of proposed integrated technique reduces the time overhead. The experimental result shows, it offers higher data integrity by producing lower percentage of Information Loss, higher SNR and compression ratio. Furthermore, the capacity to produce higher Throughputs and low Cyclomatic Complexity shows its time efficiencies.

Do Big Unstructured Biodiversity Data Mean More Knowledge?

Conserving species biodiversity demands decisive and effective action. Effective action requires an understanding of species population dynamics. Therefore, robust measures which track temporal changes in species populations are needed. This need, however, must be balanced against the scale at which population change is being assessed. Advances in citizen science and remote sensing technology have heralded an era of unstructured for biodiversity conservation. However, the value of big unstructured data for assessing changes in species populations, and effectively guiding conservation management has not been rigorously assessed. This can be achieved only by benchmarking big unstructured data against high-quality structured datasets, and ensuring the latter are not lost through an over-emphasis on data. Here, we illustrate the current trend to disproportionately prioritize data quantity over data quality and highlight the discrepancy in global availability between both data types. We propose a research agenda to test whether this trend will result in a net decrease of useful knowledge for biodiversity conservation. We exemplify this by examining the availability of big unstructured data vs. standardized data using data from global repositories on birds as an example. We share experiences from the data collation exercise needed to develop the Australian Threatened Species Index. We argue there is an urgent need to validate and enhance the utility of big unstructured data by: (1) maintaining existing well-designed, standardized long-termspecies population studies; (2) strengthening data quality control, management, and curation of any type of dataset; and (3) developing purpose-specific rankings to assess data quality.

Limitations of information extraction methods and techniques for heterogeneous unstructured big data

During the recent era of big data, a huge volume of unstructured data are being produced in various forms of audio, video, images, text, and animation. Effective use of these unstructured big data is a laborious and tedious task. Information extraction (IE) systems help to extract useful information from this large variety of unstructured data. Several techniques and methods have been presented for IE from unstructured data. However, numerous studies conducted on IE from a variety of unstructured data are limited to single data types such as text, image, audio, or video. This article reviews the existing IE techniques along with its subtasks, limitations, and challenges for the variety of unstructured data highlighting the impact of unstructured big data on IE techniques. To the best of our knowledge, there is no comprehensive study conducted to investigate the limitations of existing IE techniques for the variety of unstructured big data. The objective of the structured review presented in this article is twofold. First, it presents the overview of IE techniques from a variety of unstructured data such as text, image, audio, and video at one platform. Second, it investigates the limitations of these existing IE techniques due to the heterogeneity, dimensionality, and volume of unstructured big data. The review finds that advanced techniques for IE, particularly for multifaceted unstructured big data sets, are the utmost requirement of the organizations to manage big data and derive strategic information. Further, potential solutions are also presented to improve the unstructured big data IE systems for future research. These solutions will help to increase the efficiency and effectiveness of the data analytics process in terms of context-aware analytics systems, data-driven decision-making, and knowledge management.

Multimodal Emotion and Sentiment Modeling From Unstructured Big Data: Challenges, Architecture, & Techniques

The exponential growth of multimodal content in today's competitive business environment leads to a huge volume of unstructured data. Unstructured big data has no particular format or structure and can be in any form, such as text, audio, images, and video. In this paper, we address the challenges of emotion and sentiment modeling due to unstructured big data with different modalities. We first include an up-to-date review on emotion and sentiment modeling including the state-of-the-art techniques. We then propose a new architecture of multimodal emotion and sentiment modeling for big data. The proposed architecture consists of five essential modules: data collection module, multimodal data aggregation module, multimodal data feature extraction module, fusion and decision module, and application module. Novel feature extraction techniques called the divide-and-conquer principal component analysis (Div-ConPCA) and the divide-and-conquer linear discriminant analysis (Div-ConLDA) are proposed for the multimodal data feature extraction module in the architecture. The experiments on a multicore machine architecture are performed to validate the performance of the proposed techniques.

WearableDL: Wearable Internet-of-Things and Deep Learning for Big Data Analytics—Concept, Literature, and Future

This work introduces Wearable deep learning (WearableDL) that is a unifying conceptual architecture inspired by the human nervous system, offering the convergence of deep learning (DL), Internet-of-things (IoT), and wearable technologies (WT) as follows: (1) the brain, the core of the central nervous system, represents deep learning for cloud computing and big data processing. (2) The spinal cord (a part of CNS connected to the brain) represents Internet-of-things for fog computing and big data flow/transfer. (3) Peripheral sensory and motor nerves (components of the peripheral nervous system (PNS)) represent wearable technologies as edge devices for big data collection. In recent times, wearable IoT devices have enabled the streaming of big data from smart wearables (e.g., smartphones, smartwatches, smart clothings, and personalized gadgets) to the cloud servers. Now, the ultimate challenges are (1) how to analyze the collected wearable big data without any background information and also without any labels representing the underlying activity; and (2) how to recognize the spatial/temporal patterns in this unstructured big data for helping end-users in decision making process, e.g., medical diagnosis, rehabilitation efficiency, and/or sports performance. Deep learning (DL) has recently gained popularity due to its ability to (1) scale to the big data size (scalability); (2) learn the feature engineering by itself (no manual feature extraction or hand-crafted features) in an end-to-end fashion; and (3) offer accuracy or precision in learning raw unlabeled/labeled (unsupervised/supervised) data. In order to understand the current state-of-the-art, we systematically reviewed over 100 similar and recently published scientific works on the development of DL approaches for wearable and person-centered technologies. The review supports and strengthens the proposed bioinspired architecture of WearableDL. This article eventually develops an outlook and provides insightful suggestions for WearableDL and its application in the field of big data analytics.

Keywords extraction in Chinese–Vietnamese bilingual news based on hypergraph

The keywords extraction of bilingual news events in China and Vietnam has a very important role in understanding bilingual news events. It can quickly locate and briefly compare the news of the same events reported by the two countries. Chinese–Vietnam news texts are typically unstructured big data. How to extract the keywords that characterize the news in these unstructured data is the difficult problem of unstructured big data analysis. Bilingual documents are difficult to understand because bilingual Chinese and Vietnamese are not in the same language space. However, the hypergraph of the hypergraph model can better express the multiple relations of the vocabulary association and the entity association for bilingual news. Therefore, a method based on hypergraph for bilingual news keywords extraction is proposed. In this method, bilingual news words are extracted to construct a bilingual word set, and the words are taken as vertices. Chinese–Vietnamese sentences and bilingual words with the same semantic meaning as different types of hyperedges and the bilingual word frequency are used as the attribute to construct a bilingual news item word hypergraph model. Then, the directional diffusion algorithm in the wireless sensor network is used to iteratively calculate the weights of the vertices so as to realize the extraction of keywords in the Chinese–Vietnam bilingual news. The experimental results show that the proposed hypergraph method is better than the single-document extraction method, which can better obtain the keywords of the bilingual unstructured text data.

Application of Text Mining Techniques in Railway Safety Supervision System

Railway safety is the most important in railway transportation. It is the cornerstone of all railway work and involves a wide range of departments. In the age of unstructured big data, it is an urgent problem to solve unstructured data, which contains a lot of information value by using advanced technology. This paper introduces the application of text analysis in railway safety supervision system based on massive data of accident and fault analysis report, analysis the importance of text mining techniques for railway safety system, the key technology of text analysis. Finally, based on the Railway Security Big Data Application Platform, we mainly introduce two classic functions of full-text retrieval and text classification based on text analysis technology on this platform. The application of the platform proves that text analysis plays a key role in railway safety supervision system

A Big Data Layered Architecture and Functional Units for the Multimedia Internet of Things

The escalating growth of multimedia content in Internet of Things (IoT) applications leads to a huge volume of unstructured data being generated. Unstructured Big data has no particular format or structure and can be in any form such as text, audio, images, and video. Furthermore, current IoT systems cannot successfully realize the notion of having ubiquitous connectivity of everything if they are not capable to include ‘multimedia things’. In this paper, we address two issues by proposing a new architecture for the Multimedia Internet of Things (MIoT) with Big multimodal computation layer. We first introduce MIoT as a novel paradigm in which smart heterogeneous multimedia things can interact and cooperate with one another and with other things connected to the Internet to facilitate multimedia-based services and applications that are globally available to the users. The MIoT architecture consists of six layers. The computation layer is specially designed for Big multimodal analytics. This layer has four important functional units: Data Centralized Unit, Multimodal Data Aggregation Unit, Multimodal Data Divide & Conquer Computation Unit, and Fusion & Decision Making Unit. A novel and highly scalable technique called the Divide & Conquer Principal Component Analysis (DC-PCA) for feature extraction in the divide and conquer mechanism is proposed to be used together with the Divide & Conquer Linear Discriminant Analysis (DC-LDA) for multimodal Big data analytics. Experiments are conducted to confirm the good performance of these techniques in the functional units of the Divide & Conquer computational mechanisms. The final section of the paper gives application on a camera sensing IoT platform and real-world data analytics on multicore architecture implementations.

Challenges and Issues in Unstructured Big Data: A Systematic Literature Review

In past years, data mining had been a standard approach to extract hidden knowledge or pattern known as Knowledge Discovery in Database (KDD). However, KDD approach is unable to handle a terabytes or petabytes of ’Big Data’. The presence of Big Data term arises due to the growth of data volume, velocity, variety, veracity, and value. The objectives of this study are to explore the various data types available in Big Data and current challenges in Big Data. A systematic literature review is conducted to fulfill the research objectives. Several research questions arise in this study for example what type of data available in Big Data as well as what are the major challenges and issues emerged from previous studies. The outcomes of this review identified three data types in Big Data which are unstructured, semi-structured and structured data. Besides, there are also three greatest challenges in Big Data which are data, processing and management challenge. The result also reveals that process challenge is the most critical challenge in Big Data. This extensive systematic literature review can contribute knowledge to the Big Data area as well as for future research.

Text mining geo-visualization of patent documents on geo-spatial big-data industry

This study attempts to establish prototype-leveled patent fusion data based on collecting structured and unstructured geo-spatial big data (GSBD) patent information, to distinguish GSBD technical ecosystems into their spatial and non-spatial aspects, and to propose a method to analyze visualizations in a multi-dimensional way. Spatially, we visualize the patent citation data among applicants for a patent at local and national levels, and implement a visualization analysis of the competitive relations for the locational traits of applicants for patent and technology innovation by comparing technology dependence and technology impacts in GSBD technology. Non-spatially, we analyzed the trend of time series of GSBD technology innovation activities based on Industry Classification and technology keywords. We establish the related networks among industry classification, IPC patent classification and technology keywords and implement a visualization analysis of convergence structure in element technologies through graph network analysis and Venn diagram analysis. We extracted issues related with the establishment of patent fusion data and interpretation of visualization analysis through the examination of research methodology and analysis results and discussed future research tasks to solve these problems.

Big Data Econometrics: Now Casting and Early Estimates

This paper aims at providing a primer on the use of big data in macroeconomic nowcasting and early estimation. We discuss: (i) a typology of big data characteristics relevant for macroeconomic nowcasting and early estimates, (ii) methods for features extraction from unstructured big data to usable time series, (iii) econometric methods that could be used for nowcasting with big data, (iv) some empirical nowcasting results for key target variables for four EU countries, and (v) ways to evaluate nowcasts and ash estimates. We conclude by providing a set of recommendations to assess the pros and cons of the use of big data in a speci c empirical nowcasting context.

An Investigation Into the Efficacy of Deep Learning Tools for Big Data Analysis in Health Care

This article describes how machine learning (ML) algorithms are very useful for analysis of data and finding some meaningful information out of them, which could be used in various other applications. In the last few years, an explosive growth has been seen in the dimension and structure of data. There are several difficulties faced by conventional ML algorithms while dealing with such highly voluminous and unstructured big data. The modern ML tools are designed and used to deal with all sorts of complexities of data. Deep learning (DL) is one of the modern ML tools which are commonly used to find the hidden structure and cohesion among these large data sets by giving proper training in parallel platforms with intelligent optimization techniques to further analyze and interpret the data for future prediction and classification. This article focuses on the use of DL tools and software which are used in past couple of years in various areas and especially in the area of healthcare applications.

Performance Evaluation of Unstructured PBRA for Bigdata with Cassandra and MongoDB in Cloud

In this article, performance evaluation of web collection data in data stores, such as NoSQL-Cassandra and MongoDB is presented, yielding scalability of applications. In addition to scalability, security of NoSQL databases remains highly unproved. It is noteworthy that existing works in the area of cloud with NoSQL focus on either scalability or security but not both aspects. Also, security, if provided, is at minor interface levels. In this article, the PBRA system is designed to deal with highly unstructured big data emerging from the twitter social networking service, which is new of its kind to strengthen the bigdata security. PBRA is Passphrase Based REST API model where the REST API methods are integrated with the user generated passphrase in addition to the private key for a set of records of user desirable number before storing into the Cassandra and MongoDB databases. Results are presented to illustrate the same for nearly 1 million records and the efficiency of Cassandra over MongoDB is observed. It is observed from the results that though the time taken to load and retrieve bulk data records is higher than dealing with cipher text, Cassandra performs better than MongoDB with the proposed security model.

A Technique for Classifying Unstructured Big Data Files

A Technique for Classifying Unstructured Big Data Files