Unstructured Data Format Research Articles

Enhancing applications of surveillance through multimedia data mining

Over recent years, multimedia data has become a cornerstone for insightful data analysis, yielding vital information crucial for informed decision-making processes. This diverse data format encompasses audio, video, images, and text, offering a wealth of valuable knowledge. Advancements in multimedia acquisition, storage, and processing technologies have significantly enhanced analytical capabilities, overcoming challenges posed by semi-structured and unstructured data formats. Various entities including corporations, governmental bodies, and academic institutions are keenly interested in harnessing insights from the vast reservoirs of multimedia data generated across diverse sources. Consequently, researchers have delved into data mining methodologies, uncovering effective strategies for extracting insights from multimedia datasets. This study aims to probe the conceptual and practical dimensions of multimedia data mining within surveillance contexts, elucidating its transformative impact on diverse sectors by facilitating efficient data collection, analysis, and dissemination processes. Moreover, it underscores the significance of incorporating relevant cryptography methods to bolster the system’s integrity and completeness.

Crop Information Retrieval Framework Based on LDW-Ontology and SNM-BERT Techniques

Currently, on the Internet, the information about agriculture is augmenting extremely; thus, searching for precise, relevant data of various details is highly complicated. To deal with particular difficulties like lower relevancy rate, false detection of retrieval resources, poor similarity rate, unstructured data format, multivariate data, irrelevant spelling, and higher computation time, an intelligent Information Retrieval (IR) system is required. An IR Framework centered on Levenshtein Distance Weight-centric Ontology (LDW-Ontology) and Sutskever Nesterov Momentum-centred Bidirectional Encoder Representation from Transformer (SNM-BERT) methodologies is presented here to overcome the complications as mentioned earlier. Firstly, the data is pre-processed, transmuting the unstructured data into a structured format, thus mitigating the error probabilities. Then, the LDW-Crop Ontology construction is done regarding the structured data. In the methodology presented, significance, frequency,and the suggestion of word in mind are considered to build Crop ontology. In the MongoDB database, the data being constructed are amassed. Then, by utilizing SNM-BERT, the data is trained for IR regarding clustered input produced by Inter Quartile Pruning Range-centred Hierarchical Divisive Clustering (IQPR-HDC) model. The LDW is computed for the provided user query; subsequently, the similarity evaluation outcomes are obtained from the database. The experiential evaluation displays that when analogized with the prevailing methodologies, a better accuracy of 94 % for simple queries and 92% for complex queries is achieved. Along with retrieval rate with lower computation time is achieved by the proposed methodology.

How do you say it matters? A multimodal analytics framework for product return prediction in live streaming e-commerce

As a new shopping paradigm, live streaming e-commerce has undergone rapid development in recent years. However, a higher return rate than that of traditional e-commerce not only affects the consumption experience of users but also is detrimental to the development of the entire industry. Previous studies on product returns have mainly focused on traditional e-commerce, with few studies considering signals in unstructured data formats. The main contribution of our research is the design of a multimodal analytics framework to predict product returns in live streaming e-commerce settings. Multimodal features (e.g., visual, textual, and acoustic) and their correlations extracted from different data sources (product descriptions and live streaming records) are considered in our prediction model. In addition, to better study the impact of anchor speech during live streaming, we propose an acoustic feature extraction model called “OpenTransformer,” which extracts acoustic features of anchor audio and outperforms baselines. We have conducted experiments based on a real-world dataset collected from Taobao live streaming. The experimental results demonstrate the effectiveness of using multimodal signals from products and anchors for product return prediction. The managerial implications of our research are that live streaming platforms and potential consumers can prejudge which sold products will have high return rates in a timely fashion, and based on these predictions, merchants and anchors can better design their product descriptions and speech behaviors. This study thus supports more effective development of live streaming e-commerce markets.

Leveraging deep learning algorithms for synthetic data generation to design and analyze biological networks

The use of synthetic data is gaining an increasingly prominent role in data and machine learning workflows to build better models and conduct analyses with greater statistical inference. In the domains of healthcare and biomedical research, synthetic data may be seen in structured and unstructured formats. Concomitant with the adoption of synthetic data, a sub-discipline of machine learning known as deep learning has taken the world by storm. At a larger scale, deep learning methods tend to outperform traditional methods in regression and classification tasks. These techniques are also used in generative modeling and are thus prime candidates for generating synthetic data in both structured and unstructured formats. Here, we emphasize the generation of synthetic data in healthcare and biomedical research using deep learning methods for unstructured data formats such as text and images. Deep learning methods leverage the neural network algorithm, and in the context of generative modeling, several neural network architectures can create new synthetic data for a problem at hand including, but not limited to, recurrent neural networks (RNNs), variational autoencoders (VAEs), and generative adversarial networks (GANs). To better understand these methods, we will look at specific case studies such as generating realistic clinical notes of a patient, the generation of synthetic DNA sequences, as well as to enrich experimental data collected during the study of heterotypic cultures of cancer cells.

Forecast Aggregated Supply Curves in Power Markets Based On LSTM Model

One of the key steps for optimal bidding in power markets is to estimate the rivals’ bidding behaviors. However, for most participants, it would be difficult to directly forecast the rivals’ individual bids due to the information privacy and volatile characteristics of individual bidding behaviors. From another point of view, the aggregation of individual bids, denoted as aggregated supply curve (ASC), might be helpful to offset the uncertainties of individual bidding behaviors and can be used as reference for optimal bidding. In fact, the real ASC data contains bidding information from thousands of participants, which would be formulated with high dimensionality and unstructured formats, not applicable for general forecasting methods. Thus, a novel data-driven ASC forecasting framework based on long-short term memory (LSTM) model and corresponding data processing techniques is proposed in this paper. In detail, A paradigmatic data integration method is proposed to fix the unstructured data formats. A feature extraction method is developed to simplify the high dimensionality of ASC. Then, a LSTM model is customized to forecast ASCs. At last, real data from Midcontinent Independent System Operator market in the U.S. are utilized to demonstrate the forecasting performance of the proposed framework.

Impacts of Urban Floods on Road Connectivity - A Review and Systematic Bibliometric Analysis

Urban floods are paralyzing surface transportation and inflicting heavy economic losses. Climate-induced increase in frequency and intensity of rainfalls and excessive urbanization makes urban centers even more vulnerable to floods. It is necessary to quantify all dimensions of losses caused to road connectivity to improve flood mitigation policy. There is a need to consolidate the existing body of peer-reviewed contemporary literature on flood inundation modeling and its impacts on road connectivity. This will improve the awareness of policymakers and researchers and help in science-based decision making. Articles archived in the Web of Science database having the keywords floods and road in their title published between 1977 and 2020 were analyzed using the blibliometrix library of R. Analysis shows that the flood inundation and flood extent modeling has evolved from the conventional hydrological models to the near real-time crowd-sourced modeling methods. Applications of geographical information systems and advanced remote sensing methods have been growing in identifying road network vulnerabilities. We observed a gap in harmonized data availability, due to the unstructured data formats at several scales, which hinders a generalized approach for flood risk modeling studies for urban planning. Concentrated efforts have to be made to fill the gaps in data availability and research methodologies, especially using crowd-sourced data. Further, efforts have to be made to increase awareness, early warning systems, and alternate transport networks, to make the cities less vulnerable to floods.

An Investigative Testing of Structured and Unstructured Data Formats in Big Data Application Using Apache Spark

An Investigative Testing of Structured and Unstructured Data Formats in Big Data Application Using Apache Spark

Sentimental classification analysis of polarity multi-view textual data using data mining techniques

The data and information available in most community environments is complex in nature. Sentimental data resources may possibly consist of textual data collected from multiple information sources with different representations and usually handled by different analytical models. These types of data resource characteristics can form multi-view polarity textual data. However, knowledge creation from this type of sentimental textual data requires considerable analytical efforts and capabilities. In particular, data mining practices can provide exceptional results in handling textual data formats. Besides, in the case of the textual data exists as multi-view or unstructured data formats, the hybrid and integrated analysis efforts of text data mining algorithms are vital to get helpful results. The objective of this research is to enhance the knowledge discovery from sentimental multi-view textual data which can be considered as unstructured data format to classify the polarity information documents in the form of two different categories or types of useful information. A proposed framework with integrated data mining algorithms has been discussed in this paper, which is achieved through the application of X-means algorithm for clustering and HotSpot algorithm of association rules. The analysis results have shown improved accuracies of classifying the sentimental multi-view textual data into two categories through the application of the proposed framework on online polarity user-reviews dataset upon a given topics.

Data Encryption and Decryption Techniques for a High Secure Dataset using Artificial Intelligence

The science of extracting patterns, trends, and actionable data analysis detail of large data sets. The growing existence of data in different county’s servers with structured, semi-structured, and unstructured data formats, such as the data. The demands of these are not met by conventional IT infrastructure, a modern landscape of "Data Analysis." For these reasons, several companies are turning to as a possible solution to this unmet commercial business, Hadoop (open-source projects). The amount of data collected by organizations, especially unstructured data, as businesses burst, Hadoop is increasingly emerging as one of the primary alternatives to store and execute operations on that data. The secondary question of data analysis is defense, the rapid increase in internet use, the dramatic shift in acceptance of people who use social media apps that allow users to generate content freely and intensify the already enormous amount of the site. In today's firms, there are a few stuff to bear in mind when starting innovation ventures for big data and analytics. In the business environment, the need for secure data analytics tools is mandatory. In the previous paper, they implemented a high profile dataset using the encryption technique. Using only the encryption method, cannot secure data very highly. There is a chance of knowing the original data to the third party. To reduce the above issues, the paper introduces a new technology called “Artificial intelligence". Using this new technology, paper can achieve more security for data sets. Using both encryption and decryption models in artificial intelligence can solve the drawback in an existing paper. This will provide the data with either a significant degree of authentication analyzed to ever be. The provision of data analytics is pursued with attribute-based restricts Data extraction allows enabled. This model will work better than the present model. In both security and sensitive economic restructuring, data analytical tools.

A study on 3D/BIM-based on-site performance measurement system for building construction

ABSTRACT Performance measurement has become conventional in construction management. However, it is non-trivial to establish an effective system to capture, evaluate, and feedback the on-site data. Whilst 3D object–based technology prevails in the industry, project managers remain accustomed to text-based system. Unstructured data format requires a tremendous amount of time and effort when analyzing a project in detail. To overcome this issue, an extensive literature review in conjunction with industry feedback has been conducted. Based on this preliminary investigation, the authors have proposed an on-site performance measurement system that interlinks 3D object with a spreadsheet. The system enables project managers to recognize the performance and/or productivity of a project in real time and helps find timely remedies in updating the original plan. One of the key findings includes that the identified eight on-site productivity factors should be embedded into a 3D model in effectively measuring the performance for a particular project. The new system also provides a useful tool in managing the productivity of a project. The main contribution of this study is the proposition and verification of a novel methodology that effectively manages on-site performance information by relating 3D objects with productivity factors in building construction.

Heart Failure Prediction Models using Big Data Techniques

Big Data technologies have a great potential in transforming healthcare, as they have revolutionized other industries. In addition to reducing the cost, they could save millions of lives and improve patient outcomes. Heart Failure (HF) is the leading death cause disease, both nationally and internally. The Social and individual burden of this disease can be reduced by its early detection. However, the signs and symptoms of HF in the early stages are not clear, so it is relatively difficult to prevent or predict it. The main objective of this research is to propose a model to predict patients with HF using a multi-structure dataset integrated from various resources. The underpinning of our proposed model relies on studying the current analytical techniques that support heart failure prediction, and then build an integrated model based on Big Data technologies using WEKA analytics tool. To achieve this, we extracted different important factors of heart failure from King Saud Medical City (KSUMC) system, Saudi Arabia, which are available in structured, semi-structured and unstructured format. Unfortunately, a lot of information is buried in unstructured data format. We applied some pre-processing techniques to enhance the parameters and integrate different data sources in Hadoop Distributed File System (HDFS) using distributed-WEKA-spark package. Then, we applied data-mining algorithms to discover patterns in the dataset to predict heart risks and causes. Finally, the analyzed report is stored and distributed to get the insight needed from the prediction. Our proposed model achieved an accuracy and Area under the Curve (AUC) of 93.75% and 94.3%, respectively.

WBS-based dynamic multi-dimensional BIM database for total construction as-built documentation

Abstract As-built documentation is valuable to assist decision-making in project control, and building operation and maintenance applications. Given the importance of as-built documentation, the quickly maturing technology of building information modeling (BIM), an object-oriented information-integration platform, has led to the emergence of concurrent as-built documentation during the construction phase. However, in the current practice, integrating construction records into BIM remains a challenge due to their heterogeneous and unstructured data formats. This paper presents a work breakdown structure (WBS)-based database design to enable the creation of a dynamic multi-dimensional BIM database to incorporate construction records in a timely manner. In this study, the construction records were defined as additional dimension, and a WBS code-based automated linking mechanism between construction project tasks and BIM objects was created to generate the multi-dimensional BIM database. The automatic linking mechanism and the WBS-based dynamic multi-dimensional BIM database were tested and verified through the implementation of a real construction project schedule and a BIM model. The test results illustrate that the newly created method is successful in incorporating construction records collected during the construction phase to generate a dynamic BIM database for generating complete as-built documentation.

Big Data, Predictive Analytics, and Quality Improvement in Kidney Transplantation: A Proof of Concept.

We sought proof of concept of a Big Data Solution incorporating longitudinal structured and unstructured patient-level data from electronic health records (EHR) to predict graft loss (GL) and mortality. For a quality improvement initiative, GL and mortality prediction models were constructed using baseline and follow-up data (0-90 days posttransplant; structured and unstructured for 1-year models; data up to 1 year for 3-year models) on adult solitary kidney transplant recipients transplanted during 2007-2015 as follows: Model 1: United Network for Organ Sharing (UNOS) data; Model 2: UNOS & Transplant Database (Tx Database) data; Model 3: UNOS, Tx Database & EHR comorbidity data; and Model 4: UNOS, Tx Database, EHR data, Posttransplant trajectory data, and unstructured data. A 10% 3-year GL rate was observed among 891 patients (2007-2015). Layering of data sources improved model performance; Model 1: area under the curve (AUC), 0.66; (95% confidence interval [CI]: 0.60, 0.72); Model 2: AUC, 0.68; (95% CI: 0.61-0.74); Model 3: AUC, 0.72; (95% CI: 0.66-077); Model 4: AUC, 0.84, (95 % CI: 0.79-0.89). One-year GL (AUC, 0.87; Model 4) and 3-year mortality (AUC, 0.84; Model 4) models performed similarly. A Big Data approach significantly adds efficacy to GL and mortality prediction models and is EHR deployable to optimize outcomes.

Conociendo Big Data

<p align="justify">Teniendo en cuenta la importancia que ha adquirido el término Big Data, la presente investigación buscó estudiar y analizar de manera exhaustiva el estado del arte del Big Data; además, y como segundo objetivo, analizó las características, las herramientas, las tecnologías, los modelos y los estándares relacionados con Big Data, y por último buscó identificar las características más relevantes en la gestión de Big Data, para que con ello se pueda conocer todo lo concerniente al tema central de la investigación.</p><p align="justify">La metodología utilizada incluyó revisar el estado del arte de Big Data y enseñar su situación actual; conocer las tecnologías de Big Data; presentar algunas de las bases de datos NoSQL, que son las que permiten procesar datos con formatos no estructurados, y mostrar los modelos de datos y las tecnologías de análisis de ellos, para terminar con algunos beneficios de Big Data.</p><p align="justify">El diseño metodológico usado para la investigación fue no experimental, pues no se manipulan variables, y de tipo exploratorio, debido a que con esta investigación se empieza a conocer el ambiente del Big Data.</p>

Multimodal Image Retrieval from Construction Databases and Model-Based Systems

In the modern and dynamic construction environment it is important to access information in a fast and efficient manner in order to improve the decision making processes for construction managers. This capability is, in most cases, straightforward with today’s technologies for data types with an inherent structure that resides primarily on established database structures like estimating and scheduling software. However, previous research has demonstrated that a significant percentage of construction data is stored in semi-structured or unstructured data formats (text, images, etc.) and that manually locating and identifying such data is a very hard and time-consuming task. This paper focuses on construction site image data and presents a novel image retrieval model that interfaces with established construction data management structures. This model is designed to retrieve images from related objects in project models or construction databases using location, date, and material information (extracted from the image content with pattern recognition techniques).

Development and verification of a coupled DSMC–NS scheme using unstructured mesh

An efficient and accurate parallel coupled DSMC–NS method using three-dimensional unstructured grid topology is proposed and verified for the simulation of high-speed gas flows involving continuum and rarefied regimes. A domain overlapping strategy, taking advantage of unstructured data format, with Dirichlet–Dirichlet type boundary conditions based on two breakdown parameters is used iteratively to determine the choice of solvers in the spatial domain. The selected breakdown parameters for this study include: (1) a local maximum Knudsen number defined as the ratio of the local mean free path and local characteristic length based on property gradient and (2) a thermal non-equilibrium indicator defined as the ratio of the difference between translational and rotational temperatures to the translational temperature. A supersonic flow (M∞=4) over a quasi-2-D 25° wedge is employed as the first step in verifying the present coupled method. The results of simulation using the coupled method are in excellent agreement with those of the pure DSMC method, which is taken as the benchmark solution. Effects of the size of overlapping regions and the choice of breakdown parameters on the convergence history are discussed. Results show that the proposed iteratively coupled method predicts the results more accurately as compared to the “one-shot” coupled method, which has been often used in practice. Further, a realistic 3-D nitrogen flow, which two near-continuum parallel orifice jets underexpand into a near-vacuum environment, is simulated using the present coupled method to demonstrate its capability. Finally, developments in extending the present method are also outlined in this paper.

Web Mining Through The Online Analyst

While the amount of data available to us through the Web and the intranets is increasing, our capacity of reading and analyzing this information remains constant. Search engines instead of reducing the problem augment it making more and more documents quickly available to us. Web mining is a new research area that tries to solve the information overload problem by exploiting recent advances in different fields of technology. Documents and web pages are a source of knowledge in an unstructured data format that can be decodified, analyzed and turned into actionable intelligence thanks to online analysis by text mining. In this paper an example of a real and operative application to competitive intelligence (CI) is given: the Online Analyst. The final objective of this application is to give to end users an intelligent agent to read and quickly analyze huge volumes of documents retrieved online, especially from the web. Its typical users are intelligence analysts, in military as in business or politics field. The most exciting conclusion is that the shown approach, as also the prototype, here shown in a CI application case, may be already easily used in all e-commerce applications (currently under development worldwide), that take into account the necessity of working with web and /or text data.