Data Extraction Technology Research Articles

Analysis of Unstructured Document Data Extraction Technology

Analysis of Unstructured Document Data Extraction Technology

A Study on Intelligent Document Processing using AWS

This article mainly approaches the topic of Intelligent Document Processing from the viewpoint of the users of cloud computing platforms and the end-users. The market for commercial OCR, document categorization and data extraction technologies are briefly reviewed to the extent that it is publicly available. The necessity for an effective and efficient retrieval of the stored information is increased by digital repositories. In this study, we suggest that the phases of document layout analysis, document picture categorization and understanding on digital documents should intensively apply intelligent techniques. Specifically, the Intelligent Document Processing can be used instead of Artificial Intelligence, Deep Learning and RPA to retrieve the data and convert to understandable form.

Design and Implementation of Wastewater Pollutant Data Analysis and Visualization System for Vessels

To monitor the discharge of water pollutants from vessels, a monitoring system for the discharge of water pollutants from vessels has been established on the Yangtze River. The system generates tens of thousands of ship wastewater pollutant data every day. The analysis and visualization of wastewater pollutant discharge data can help the monitoring department to grasp the discharge situation and improve the accuracy of supervision. This system extracts the data to the data warehouse in real-time based on the data extraction technology, improves the query efficiency based on the three-level storage architecture, makes real-time pollutant data statistics based on the stream computing technology, and realizes the visual display of the data based on the front-end and back end-separation technology. The system has been applied in the maritime supervision department to provide a data basis for vessel wastewater pollutant supervision and decision-making.

Study on data mining method of network security situation perception based on cloud computing

Abstract In recent years, the network has become more complex, and the attacker’s ability to attack is gradually increasing. How to properly understand the network security situation and improve network security has become a very important issue. In order to study the method of extracting information about the security situation of the network based on cloud computing, we recommend the technology of knowledge of the network security situation based on the data extraction technology. It converts each received cyber security event into a standard format that can be defined as multiple brochures, creating a general framework for the cyber security situation. According to the large nature of network security situation data, the Hadoop platform is used to extract aggregation rules, and perform model extraction, pattern analysis, and learning on a network security event dataset to complete network security situation rule mining, and establish a framework for assessing the state of network security. According to the results of the federal rule extraction, the level of network node security risk is obtained in combination with signal reliability, signal severity, resource impact, node protection level, and signal recovery factor. A simulation test is performed to obtain the intrusion index according to the source address of the network security alarm. Through the relevant experiments and analysis of the results, the attack characteristics obtained in this study were obtained after manually reducing the network security event in the 295 h window. The results show that after the security event is canceled, the corresponding window attack index decreases to 0, indicating that this method can effectively implement a network security situation awareness. The proposed technique allows you to accurately sense changes in network security conditions.

Anti-monopoly supervision model of platform economy based on big data and sentiment.

With the advent of the cloud computing era, big data technology has also developed rapidly. Due to the huge volume, variety, fast processing speed and low value density of big data, traditional data storage, extraction, transformation and analysis technologies are not suitable, so new solutions for big data application technologies are needed. However, with the development of economic theory and the practice of market economy, some links in the industrial chain of natural monopoly industries already have a certain degree of competitiveness. In this context, the article conducts a research on the anti-monopoly supervision mode of platform economy based on big data and sentiment analysis. This paper introduces the main idea of MapReduce, the current software implementation specifies a Map function that maps a set of key-value pairs into a new set of key-value pairs. It specifies a concurrent Reduce function that guarantees that each of all mapped key-value pairs share the same set of keys. establishes a vector space model, and basically realizes the extraction of text emotional elements. It introduces the theoretical controversy of antitrust regulation of predatory pricing behavior of third-party payment platforms, and conducted model experiments. The experimental results show that the throughput of 40 test users in 1 h of test is determined by two factors, QPS and the number of concurrent, where QPS = 40/(60*60) transactions/second. The time for each test user to log in to the system is 10 min, and the average response time is 10*60 s, then the number of concurrency = QPS*average response time = 40/(60*60)*10*60 = 6.66. This paper has successfully completed the research on the anti-monopoly supervision model of platform economy based on big data and sentiment analysis.

Research on the Key Technology of Web Data Extraction and Mining Based on the Probability Distribution

In the development of internet technology innovation, the quantity of big data information continues to rise, showing the characteristics of dynamic, heterogeneous, massive, and so on. How to explore valuable and potential knowledge from the network system is the main topic of research and scholars. Based on the understanding of web data extraction and web mining technology, this paper makes a systematic study of web text mining, proposes the most commonly used hidden Markov model in probability distribution, and constructs the corresponding text mining method. The hidden Markov model is a statistical model, which is mainly used to represent a Markov process with unknown parameters. The difficulty in practical application lies in how to define the hidden parameters of the process from observable parameters and then use these parameters to conduct in-depth research. Finally, the practical results show that the hidden Markov model can not only obtain the information of different regions but also deeply analyze the data set, which proves the feasibility of this research technology.

RFID Data Analysis and Evaluation Based on Big Data and Data Clustering.

The era people live in is the era of big data, and massive data carry a large amount of information. This study aims to analyze RFID data based on big data and clustering algorithms. In this study, a RFID data extraction technology based on joint Kalman filter fusion is proposed. In the system, the proposed data extraction technology can effectively read RFID tags. The data are recorded, and the KM-KL clustering algorithm is proposed for RFID data, which combines the advantages of the K-means algorithm. The improved KM-KL clustering algorithm can effectively analyze and evaluate RFID data. The experimental results of this study prove that the recognition error rate of the RFID data extraction technology based on the joint Kalman filter fusion is only 2.7%. The improved KM-KL clustering algorithm also has better performance than the traditional algorithm.

Travel preference of bicycle-sharing users: A multi-granularity sequential pattern mining approach

Public bicycles are an indispensable part of green public transportation and are also a convenient and economical manner for the general public. In operation management, it is very important and imperative to understand the user demand and pattern of the public bicycle system. This paper took the public bicycle system in Hohhot as the research object, collected nearly 4 years of operating data, and studied the travel preferences of users in the public bicycle system in view of multiple granularities. Specifically, the data of car rental users at three time-granularities were obtained through data extraction technology. Finally, frequent pattern mining was performed on car rental data based on different time granularities and mapped to the user’s riding preference, and then the riding modes of different car rental users founded on different time granularities were determined. Finally, this article gave different management opinions based on the different riding preferences of public bicycle users in Hohhot.

Analysis of the Coordination Relationship between the Green Principle of Civil Law and Environmental Law in Environmental Pollution and Ecological Destruction.

Under the dual influence of environmental pollution and ecological damage, the green principles of civil law and environmental law can be better coordinated and developed. In the past, environmental pollution and ecological damage in the author's country were very serious. Hence, they designed and experimented with data extraction technology and environmental big data sets. A distribution model investigates it. Experiments show the following: (1) The system is developed from big data and can fully reflect the ecological environment of all parts of the country. Finally, it is concluded that the author's country's environmental pollution and ecological damage are very serious. (2) According to the experimental data of the figures and tables, it is concluded that the cooperation and coordination relationship between the green principles of civil law and environmental law not only has a protective effect on the ecological environment but also increases the economy of society and people. With the continuous updating and coordination of environmental laws and green prototypes, the author's country's environmental pollutants are significantly declining. The author believes that in the future, the country's ecological environment will become better, and the green principles of civil law will be more closely related to environmental law. The country's environmental pollution and people's lives are becoming more closely related. To better solve environmental problems, we have introduced the green principles of civil law and environmental law. It makes people's quality of life and the economy better. The main significance of the study is to use the method with the least damage to the environment to obtain the maximum economic benefit to achieve long-term sustainable development.

Privacy data access control of internet of things based on user attributes

Aiming at the problems of poor access security and long access time overhead of traditional privacy data access control methods, a privacy data access control of internet of things (IoT) based on user attributes is designed. Use data extraction technology to collect all data in the IoT, set privacy data constraints, complete the privacy data collection of the IoT, and fuse similar privacy data through similarity calculation. Set the structure of privacy data access in the IoT, and calculate the key weight of privacy data. The trust degree between private data is analysed with the help of user attributes, the confirmed data is sorted according to the trust degree, and encrypted to complete the access control of private data. Experimental results show that the maximum access security coefficient of this method is about 0.93. When the data is 10,000, the access time overhead is 0.20 s.

Web News Data Extraction Technology Based on Text Keywords

In order to shorten the time for users to query news on the Internet, this paper studies and designs a network news data extraction technology, which can obtain the main news information through the extraction of news text keywords. Firstly, the TF-IDF keyword extraction algorithm, TextRank keyword extraction algorithm, and LDA keyword extraction algorithm are analyzed to understand the keyword extraction process, and the TF-IDF algorithm is optimized by Zipf’s law. By introducing the idea of model fusion, five schemes based on waterfall fusion and parallel combination fusion are designed, and the effects of the five schemes are verified by experiments. It is found that the designed extraction technology has a good effect on network news data extraction. News keyword extraction has a great application prospect, which can provide the basis for the research fields of news key phrases, news abstracts, and so on.

Research on Grid Data Model Fusion Based on Big Data Extraction Technology

The task of the electric power Internet of Things is to break through the various obstacles that form an open and shared energy ecology, which needs to be broken through by building an integrated energy system. Besides, the theoretical and practical application of cloud computing-based power big data task scheduling and extraction methods can help power companies to form the effect of resource sharing. Therefore, based on dynamic task priority, a scheduling algorithm is proposed in the paper to dynamically calculate the priority of all tasks, and effectively use idle periods to make tasks execute as quickly as possible to shorten the execution time, which is used for low-latency tasks such as power systems. After experimental analysis and comparison tests with other methods, it proves that the data extraction technology studied in the paper has the best effect and can be used to obtain better data sharing and access results in power companies.

Observe the temporal evolution of deep tunnel's 3D deformation by 3D laser scanning in the Jinchuan No. 2 Mine

The three-dimensional (3D) deformation of underground tunnel is meaningful for assessing its mechanical stability, especially for the tunnel with large deformation disaster. In this study, a technology for measuring the 3D deformation of tunnel is presented based on a 3D laser scanner and processing algorithm for point cloud data. This technology includes splicing technology for multi piece point clouds in the same time, filtering technology for 3D point cloud data at individual times, alignment technology for point cloud data from different times, and data extraction technology for typical cross-section deformation of the tunnel and calculates the 3D deformation of the tunnel at different times. This technology was applied to monitor the temporal evolution of the 3D deformation of a deep tunnel in the Jinchuan No. 2 Mine, where the large deformation of the surrounding rock caused serious failure of the supporting structures. The resulting characterization of the dimensional and temporal deformation process of the surrounding rock in the Jinchuan underground tunnel included the 2D deformation distribution characteristics of the cross section, the spatial deformation distribution of the tunnel’s 3D surface, and the 4D spatiotemporal evolution process of the tunnel’s deformation. The observed results provide key information for predicting deformational performance and determining the reasonable support opportunity for a tunnel undergoing large deformation problem.

How did the data extraction business model come to dominate? Changes in the web use ecosystem before mobiles surpassed personal computers

It is widely believed that the spread of data extraction technologies on the Internet has led to the erosion of traditional professional content providers and the transformation of the online media ecosystem. To investigate this shift in media ecology, we conduct relational analyses of actual user behavior, departing from existing research that primarily focuses on business institutions and designs of technology. We assess the prevalence of the data extraction business model by grouping websites along two architectural traits that afford data extraction – user content generation and curation – and analyzing how some website architectures get privileged in the web use ecosystem. Since data extraction is relational, we advocate a network measure to capture shared usage in addition to individual popularity of websites. Our analyses of world’s 850 most popular websites in 2009, 2011, and 2013 reveal that data extraction fostered a two-tier hierarchical web use ecosystem, marked by interdependence between professional content producers and data extractors. Our study thereby shows that the dynamics in play are more complicated than what is captured by explanations centered on either capabilities of platform giants or the decline of traditional journalism and media organizations.

Effects of Alternative Treatments on Behavioral Outcomes In Patients with Mild Cognitive Impairment, Alzheimer’S Disease and Dementia: A Comprehensive Summary of Evidence and Meta-analysis

PURPOSE: Mild cognitive impairment, Alzheimer’s disease (AD) and dementia are associated with onset behavioral disturbances such as depression, psychosis, and anxiety. These ailments affect overall outcomes, decrease quality of life, and are primarily treated with antidepressants. However, there are additional therapies available to alleviate and mitigate these onset disturbances. Based on published clinical data, this comprehensive review and meta-analysis evaluates the effects of alternative therapies and treatments on behavioral outcomes in patients with mild cognitive impairment (MCI), mild or moderate AD, and dementia. METHODS: We applied novel clinical data extraction and aggregation technologies developed by MedAware Systems, Inc. It is a patent-pending process where two scientists, blinded to each other, extract data from the same study. Intelligent software compares each data field for matches (or mismatches). A senior scientist reconciles data mismatches. Where pre- and post-treatment outcomes are available, a standardized mean difference is calculated as the MedAware Standardized Index of treatment effect (MSI-E). This methodology was used to examine differences in efficacy and behavioral outcomes among the alternative treatments (supplements, therapy and counseling, exercise, education and training, and rehabilitation) to identify efficacious treatments for behavioral outcomes. RESULTS: A total of 89 studies reporting anxiety and depression outcomes in patients with MCI, AD or dementia were located. There were no significant differences in MSI-E among treatments. Cognitive and behavioral showed the most efficacy (MSI-E = 0.023 ± 0.002). Among alternative treatment categories, there were no significant differences, with rehabilitation showing the most efficacy (MSI-E = +0.254 ± 0.063), and exercise showing the least (MSI-E = -0.105 ± 0.013). CONCLUSIONS: Using the MedAware Systems, Inc. literature database and meta-analytic methodology, we found that rehabilitation therapy may decrease anxiety and depression, while exercise has almost no effect. Overall results for alternative therapies were not significant. Because the number of studies within each therapy is small, additional research on alternative therapies is warranted. Supported by MedAware Systems, Inc.

Construction of on-Campus 3D Model based on GIS Technology and OpenGL

This paper mainly uses professional measurement technology and national standard to carry out actual measurement on the building complex, green vegetation, intra-college roads and other irregular building groups on the campus and realizes data collection and accurate analysis of data. The AutoCAD software is used to draw the map, and the visualization of the campus 3D model is realized through OpenGL. The feasibility of data extraction and related technology is verified.

AGE AND GENDER RELATED VARIATIONS IN HUMAN EEG SIGNALS

This research is aimed at analysing EEG data streams from FP1 and F7 channels that reflect activities of the human brain with respect to age and gender. It is mainly focused on finding significant variations in brainwaves detected at frontal parietal lobes of children and adults in both genders. EEG data gathering, data extraction, data cleaning and data filtering technologies were used to observe variations in brainwave patterns among children and adults especially in regions of the brain associated with logical thinking and linguistics. We used cross correlation functionality to obtain similarity between subject datasets in the sample space. Results indicate increased active brain signal patterns near the frontal parietal lobe of young male subjects than in female subjects.

DIFFERENTIAL EFFECTS OF PHARMACOLOGIC AND NON-PHARMACOLOGIC TREATMENTS OF ALZHEIMER’S DISEASE: A COMPREHENSIVE SUMMARY OF EVIDENCE AND META-ANALYSIS

The objective of any Alzheimer's disease (AD) treatment is to achieve clinically meaningful symptom reductions without compromising the quality of life (QOL) in patients. Despite the promise of emerging therapies, existing pharmacological treatments have dominated the AD treatment landscape. The acceptance of these treatment norms, especially in comparison to non-pharmacological treatments has not been adequately examined. We performed an exhaustive, quantitative review of their comparative effects, based on all published clinical data, which has not been methodologically feasible until now. This investigation applies novel clinical data extraction and aggregation technologies developed by MedAware Systems, Inc. It is a patent-pending process where two scientists, blinded to each other, extract data from the same study. Intelligent software compares each data field for matches (or mismatches). Where there is a mismatch, a senior scientist reconciles the data mismatch. This methodology is used to thoroughly profile the differences in efficacy and QOL/ADL outcomes among leading pharmacologic and non-pharmacologic treatments to identify the complete domain of AD literature, to extract information on intervention, efficacy, and cohort-level evidence available, and to elucidate the great variation in therapy effect. A total of 795 clinical studies reporting AD interventions and outcome data were located. Of those, 280 and 84 publications were identified that investigated drug and non-pharmacologic treatments, respectively, and were included in the analysis. Overall, no significant reduction in cognitive decline or improvement in QOL/ADL scores was demonstrated with either pharmacologic (MSI-E = 0.17±7.41) or non-pharmacologic (MSI-E = -0.49±2.33) treatments. While acetylcholinesterase inhibitors showed the most overall cognitive benefit (MSI-E = 0.56±7.05), the class was also associated with significant reduction in ADL/QOL (MSI-E = -1.13±11.92) and the greatest variability in effect. Within non-pharmacologic treatments, integrative therapies showed the poorest efficacy (MSI-E = -1.77±3.73) but suggested conserved ADL/QOL (MSI-E = 0.01±2.76). The MedAware Systems, Inc. literature database and meta-analytic methodology readily profiles the diversity, poor efficacy, and the corresponding QOL effect of leading drug and non-pharmacologic AD treatments. The roles of publication-, treatment-, and patient-level factors in reported safety and efficacy profiles warrant further investigation .

Distributed Wireless Measurement System for Transient Pressure with Data Extraction Technology

Transient pressure measurement system applied in an explosion field is usually characterized by non-real-time measurement due to the conflict between quick signal acquisition and limited communication bandwidth. The correct signal acquisition is launched by triggering, but the false triggering often leads to measurement failure due to the fact that the operating status of system cannot be real-time detected in an explosion condition. A distributed wireless measurement system with data extraction technology is proposed in this paper to improve the reliability of transient pressure measurement. A long-time signal acquisition is achieved by mass data storage to eliminate the trigger mode. On the other hand, an effective data segment is extracted by signal identification, which improves the efficiency of data transmission and data processing. A measurement system based on data extraction technology was designed and implemented in an explosion field, and the measurement node was characterized by sampling rate of 1 Msps, data storage of 128 Gb, effective data segment of 512 kW, and Wi-Fi wireless network coverage.

Big Data analytics for transaction banking: The differentiator

In the wake of increased regulatory compliance and intense competition, transaction banks today have been emphasising differentiating themselves by improving customer experience and offering relationship-based value-added services. Traditionally, transaction banks have stored large pools of data sourced from financial transactions, exchange rate feeds and customer information. Burdened by ageing IT applications, fragmented infrastructures and archaic data extraction technologies, banks have not been able to derive value from these data pools. But this trend is likely to change with the emergence of Big Data analytics technologies. Given the advantages such technologies offer, we believe Big Data analytics will play an important role in shaping the future of transaction banking, especially in enriching customer experience and offering new revenue streams. This paper presents an awareness of how transaction banks can leverage the benefits of Big Data analytics and apply these within their organisational boundaries. We have focused on a number of analytics tools available in the market that may be of interest for transaction banks. We have also highlighted some of the challenges transaction banks might face when implementing a Big Data analytics framework and have presented our point of view on how best to manage them. Finally, drawing experiences from the transaction banking industry, we have highlighted some business cases/use cases of value.