Problem Of Data Inconsistency Research Articles

A Study on the Man-Hour Prediction in Structural Steel Fabrication

Longitudinal cutting is the most common process in steel structure manufacturing, and the man-hours of the process provide an important basis for enterprises to generate production schedules. However, currently, the man-hours in factories are mainly estimated by experts, and the accuracy of this method is relatively low. In this study, we propose a system that predicts man-hours with history data in the manufacturing process and that can be applied in practical structural steel fabrication. The system addresses the data inconsistency problem by one-hot encoding and data normalization techniques, Pearson correlation coefficient for feature selection, and the Random Forest Regression (RFR) for prediction. Compared with the other three Machine-Learning (ML) algorithms, the Random Forest algorithm has the best performance. The results demonstrate that the proposed system outperforms the conventional approach and has better forecast accuracy so it is suitable for man-hours prediction.

Учет мелиорированных земель в Российской Федерации

Purpose: the review of the current state of legal aspects and features of the reclaimed land (RL) registering. Discussion. Information on the availability, areas and RL use is currently provided to various departments. The number of bodies exceeds a dozen. Different interpretations of the requirements for input data and technical errors during input lead to inconsistencies in the data of various departments. A significant number of changes and innovations in regulatory and legal documentation indicate the imperfection of the RL registration process and constant attempts to improve it. Vast areas of reclamation systems and their division in forms of ownership complicate the registration of the available RL. The confidentiality of information obtained during the RL registration does not allow for a detailed analysis of the effectiveness of the reclamation systems functioning, even within the same department. As a result, it is impossible to create an objective picture of the reclamation system functioning and the reclamation complex as a whole. The solution to the problem of data inconsistency, including when taking into account RL, as well as quick and accurate determination of the RL location and GTS, is seen in the creation of a geographic information system (GIS) within a single digital portal. Certain elements of GIS technologies have already applied today, but the main advantage of the system being developed should be the integration of information from different sources to create a reference source of information. The development of such a system has been ongoing since 2008; currently it is a Unified Federal Information System on Agricultural Lands (UFIS AL). Conclusions. A review of the current state of legal aspects and features of the RL registering allowed coming to the conclusion that filling the UFIS AL with accurate and reliable information will allow using it for creating services to solve various problems, including the development of digital services, for example, facilitating the treaty-making process for the provision of water supply and (or) drainage services.

Improving Code Completion by Solving Data Inconsistencies in the Source Code with a Hierarchical Language Model

In the field of software engineering, applying language models to the token sequence of source code is the state-of-the-art approach to building a code recommendation system. When applying language models to source code, it is difficult for state-of-the-art language models to deal with the data inconsistency problem, which is caused by the free naming conventions of source code. It is common for user-defined variables or methods with similar semantics in source code, to have different names in different projects. This means that a model trained on one project may encounter many words the model has never seen before during another project. Those freely named variables or functions in the code will bring difficulties to the processes of training and prediction and cause a data inconsistency problem between projects. However, we discover that the syntax tree of source code has hierarchical structures. This code structure has strong regularity in different projects and can be used to combat data inconsistency. In this paper, we propose a novel Hierarchical Language Model (HLM) to improve the robustness of the state-of-the-art recurrent language model, in order to be able to deal with data inconsistency between training and testing. The newly proposed HLM takes the hierarchical structure of the code tree into consideration to predict code. The proposed HLM method generates the embedding for each sub-tree according to hierarchies and collects the embedding of each sub-tree in context, to predict the next piece of code. The experiments on inner-project and cross-project datasets indicate that the newly proposed HLM method performs better than the state-of-the-art recurrent language model in dealing with the data inconsistency between training and testing, and achieves an average improvement in prediction accuracy of 11.2%.

The sensitivity to metocean data on using data-driven methods for a Valemax vessel speed prediction

Accurate ship speed prediction is not only important for economic benefits and emission reduction, but also plays a vital role in making satisfactory routing plans. To investigate a speed prediction method of the largest bulk carriers ever constructed, we firstly compare several widely-used data-driven methods and then pick out the most appropriate model for the prediction task. Being as the input attributes of the data-driven models, the metocean data are collected from forecasting products and carefully cleaned up by data pre-processing steps. However, we have observed from literatures that the data for training are mostly collected from observational systems, such as the noon-reports, sensor measurements, on-board monitoring systems and reanalysis products, which may have certain differences with the data from forecasting systems. To figure out whether the data inconsistency problem will affect the model effectiveness on prediction, exhaustive experiments of a sensitivity study are carried out by utilizing observational data for training models while testing the prediction performance with forecasting data.

Comprehensive Knowledge Archive Network harvester improvement for efficient open‐data collection and management

With the recent increase in data disclosure, the Comprehensive Knowledge Archive Network (CKAN), which is an open-source data distribution platform, is drawing much attention. CKAN is used together with additional extensions, such as Datastore and Datapusher for data management and Harvest and DCAT for data collection. This study derives the problems of CKAN itself and Harvest Extension. First, CKAN causes two problems of data inconsistency and storage space waste for data deletion. Second, Harvest Extension causes three additional problems, namely source deletion that deletes only sources without deleting data themselves, job stop that cannot delete job during data collection, and service interruption that cannot provide service, even if data exist. Based on these observations, we propose herein an improved CKAN that provides a new deletion function solving data inconsistency and storage space waste problems. In addition, we present an improved Harvest Extension solving three problems of the legacy Harvest Extension. We verify the correctness and the usefulness of the improved CKAN and Harvest Extension functions through actual implementation and extensive experiments.

Comparing the Performance of Deep Learning Methods to Predict Companies’ Financial Failure

One of the most crucial problems in the field of business is financial forecasting. Many companies are interested in forecasting their incoming financial status in order to adapt to the current financial and business environment to avoid bankruptcy. In this work, due to the effectiveness of Deep Learning methods with respect to classification tasks, we compare the performance of three well-known Deep Learning methods (Long-Short Term Memory, Deep Belief Network and Multilayer Perceptron model of 6 layers) with three bagging ensemble classifiers (Random Forest, Support Vector Machine and K-Nearest Neighbor) and two boosting ensemble classifiers (Adaptive Boosting and Extreme Gradient Boosting) in companies' financial failure prediction. Because of the inherent nature of the problem addressed, three extremely imbalanced datasets of Spanish, Taiwanese and Polish companies' data have been considered in this study. Thus, five oversampling balancing techniques, two hybrid balancing techniques (oversampling-undersampling) and one clustering-based balancing technique have been applied to avoid data inconsistency problem. Considering the real financial data complexity level and type, the results show that the Multilayer Perceptron model of 6 layers, in conjunction with SMOTE-ENN balancing method, yielded the best performance according to the accuracy, recall and type II error metrics. In addition, Long-Short Term Memory and ensemble methods obtained also very good results, outperforming several classifiers used in previous studies with the same datasets.

A Dynamic Data Slice Approach to the Vulnerability Analysis of E-Commerce Systems

The e-commerce business process net (EBPN) is a novel formal model for describing an e-commerce system and its interactive parts, such as shoppers, merchants, and the third-party payment platforms. Vulnerability analysis has a great impact on the trustworthiness of EBPN, which is an issue stemming from data inconsistency problems. Data inconsistency problems affect the consistency of the EBPN transaction analysis. The underlying causes of inconsistent data are closely related to concurrent operations, such as control flow and data flow. However, most of the existing detection methods have difficulties characterizing the vulnerabilities and interactions of control and data flows. In this paper, we propose a new method based on the dynamic data slice (DDS) that considers both transaction consistency and data state consistency. First, by analyzing control flow characteristics of EBPN, we obtain the dynamic slice. This dynamic slice is based on all paths of the EBPN reachability graph. Second, we perform the data inconsistency analysis by considering both transaction consistency and data-state consistency. Based on these, we construct a DDS to characterize the behavioral logic and the data-dependence information. The DDS acquires the dynamic data firing sequence. Based on that sequence and a given data marking, we can construct the DDSs for several types of EBPNs. Constructing the DDS can be completed in polynomial time. The DDS is designed to characterize the behavioral logic and data-dependence information. Based on these, we design a method to judge the data constraints. This method satisfies the EBPN need for transaction consistency by considering both the control and data states. In addition, according to the data-dependence information, we can lock the vulnerable regions caused by abnormal trading data in the system. Finally, we give a method to compute the vulnerability level.

Blockchain-based data transmission control for Tactical Data Link

Tactical Data Link (TDL) is a communication system that utilizes a particular message format and a protocol to transmit data via wireless channels in an instant, automatic, and secure way. So far, TDL has shown its excellence in military applications. Current TDL adopts a distributed architecture to enhance anti-destruction capacity. However, It still faces a problem of data inconsistency and thus cannot well support cooperation across multiple militarily domains. To tackle this problem, we propose to leverage blockchain to build an automatic and adaptive data transmission control scheme for TDL. It achieves automatic data transmission and realizes information consistency among different TDL entities. Besides, applying smart contracts based on blockchain further enables adjusting data transmission policies automatically. Security analysis and experimental results based on simulations illustrate the effectiveness and efficiency of our proposed scheme.

Information Systems Development Plan For AJ Twenty-Four Seven

The manual system presents a lot of problems such as duplication of data, data inconsistency, and other problems that possible manifest during inventory. It also is a factor of being behind with those arising firms as so economic grows. This paper intended to improve transaction process, business process and routines by proposing Information Systems and sophisticated computer technologies. Transaction Processing System and Decision Support System is now widely recognized in the business realm as the key tool in improving business process and routines. TPS is commonly used by companies worldwide in different functional areas. There's a lot of types of transaction processes that are adapted before by firms until now that fuels the growth of the economy of different countries. To make it effective, it should be accompanied by a Decision Support System. DSS is composed of sophisticated software, models, analysis, forecasting and requires critical thinking and high-level strategy in management implementation. By implementing IS and computer technologies firms survive and succeed in a competitive business environment.

Efficient checking of functional dependencies for relations

Data consistency is very important in relational databases, because relational databases are adopted by most operational database and inconsistent data can generate inconsistent reports or search results. In order to avoid data inconsistency problem, the underlying relation variables should be at least in the third form. But, the relational variables may not be in the third normal form due to incorrect database design. In this paper an efficient method is suggested to prevent possible data inconsistency problem due to incorrect database design based on functional dependency checking on non-key attributes for stored data. The idea of the method is based on the principle of functional dependency on relational variables and sorting on non-key attribute values. An experiment was performed to show how we may detect possible functional dependencies in real database. A database called HR from Oracle is used for the experiment, where HR account has example relations designed by experts for practice. The experiment showed good results and found several possible functional dependencies between non-key attributes so that a database designer may use the found functional dependencies to improve the structure of the database.

Fast replica recovery and adaptive consistency preservation for edge cloud system

Edge cloud extends the power of cloud computing to the edge of the devices that are closest to the demands of big connection, low latency and large bandwidth. However, there are still many challenges due to the dynamic, heterogeneous and real-time bandwidth of the node in the edge cloud environment. This paper studies dynamic replica creation, fast replica recovery and adaptive consistency preservation. Before solving the problem of data inconsistency caused by frequent updates of replicas, we consider the strategy of fast replica recovery. Firstly, the DRC-DS is based on the regional structure, considering the number of replicas and the creation location of the data process and copying the data information with high access frequency and long average response time. Moreover, the FRR-LB is based on the heat ranking of the replica, the source node and the target node are determined according to the double-cycle lookup structure. Besides, the ACP-IMP is proposed to solve the problem of replica consistency in the environment of high failure rate of network and node. The more certainty replica nodes are elected as the leaders of edge nodes, and the message transmission of reconfirmation is reduced.

Novel trajectory data publishing method under differential privacy

The existing location-based services have collected a large amount of user trajectory data, and if these data are directly released without any processing, the user's personal privacy will be leaked. At present, differential privacy protection technology is favored by many scholars, but how to apply it reasonably to location-based services is also a challenge for us. Trajectory is spatiotemporal continuous, but most existing methods only consider the single location of moving objects at a certain time without considering the entire trajectory, which may destroy the spatiotemporal integrity of the trajectory. In this paper, we address this problem and firstly propose a Sequence R (SR)-tree structure that satisfies the differential privacy based on the R-tree, and we construct the SR-Tree by using the trajectory sequence instead of the minimum bounding rectangle of the R-tree. Then we put forward an attack model called non-location sensitive information attack, in order to resist this attack, we add noise into the location data and non-location sensitive data using differential privacy techniques. Finally, the Algorithm can be consistently dealt with the problem of data inconsistency after adding noise. Experimental results show that our Algorithm not only has high data availability, operational efficiency, but also has good scalability.

Seeing Isn’t Believing: QoE Evaluation for Privacy-Aware Users

More and more network media users concern about their privacy issues since they know that their network behaviors are being observed, and thus the observable users’ data are not reliable and sufficient in this case. How to evaluate the true quality of experience (QoE) of the privacy-aware users has become a significant technical challenge because of the most majority of existing data-driven QoE evaluation schemes based on the premise of the true and adequate users’ observations. To get over this dilemma, this paper proposes a systematic and robust QoE evaluation scheme with unreliable and insufficient observation data. Specifically, we first translate the subjective privacy-aware QoE evaluation problem into an objective rational user analysis procedure. Then, a semantics-based similarity measurement for multidimensional correlation analysis is constructed to classify the observable data. Subsequently, the highlight of this paper lies in proposing a class-level joint user classification and data cleaning strategy by frequently updating the training processes. Through elaborately designing an iterative framework, it can effectively resolve the data sparsity and inconsistency problems due to the user privacy-aware preferences. Importantly, we also introduce an efficient QoE model construction method for online implementation, and numerical results validate its efficiency for different kinds of privacy-aware users.

Scalable and dynamic replica consistency maintenance for edge-cloud system

With the increasing number of edge devices in the Internet of things (IoT), data storage in edge cloud environment better meet the real-time requirements of IoT. In order to improve the data reliability, reduce the overhead and the response latency, and make more reasonable use of storage resources in edge cloud environment, we conduct an in-depth study on the replica creation strategy and consistency maintenance strategy. In order to reduce the waste of storage resources, a dynamic replica creation strategy based on block popularity and node load (DRC-BN) is proposed. DRC-BN strategy considers the storage cost of edge cloud and the communication cost between central cloud and edge cloud when optimizing the number of replicas, and improves the data reliability. In order to solve the problem of data inconsistency caused by multi-user concurrency, a replica consistency maintenance strategy based on Fast Paxos algorithm (RC-FP) is proposed. RC-FP strategy considers the comprehensive performance of nodes and selects the node with better performance as the leader according to the comprehensive performance of nodes, and reduces the time of consistency maintenance. Experimental results show that the proposed strategies perform well in terms of the consistency maintenance time and the node throughput. At the same time, the proposed strategies also reduce the overall overhead.

Emergy-based comparative analysis of urban metabolic efficiency and sustainability in the case of big and data scarce medium-sized cities: A case study for Jing-Jin-Ji region (China)

Abstract Emergy-based Urban Sustainability Assessment Framework (EmUSAF) is an effective and widely advocated method to evaluate the general condition at city-scale. However, applying this framework to medium-sized cities has encountered obstacles owing to the data scarcity and inconsistency problem. One the one hand, a range of data such as detailed import-export goods data are only compiled at provincial level therefore for prefectural cities the lack of data is very common. On the other hand, the original environmental data provided by local government is usually not uniform and well-formatted, meaning that the data inventory disclosed by the environmental data compilation administration in different cities are not identical, leading to the difficulty of comparison. Meanwhile, for a given year, not all elements needed in the emergy synthesis inventory are available due to the difference compilation and disclosure timelines of different statistics reports. These facts hinder both city-specific research at local scale and regional research. To address this lingering issue, we developed a methodology for deriving substitute data from extant available data of upper-level administration division and for integrating them into the EUSAF. With a case study on Jing-Jin-Ji region, covering 11 medium-sized prefecture cities, we conducted an emergy analysis of the whole region and examined the methodology discussing its uncertainties and limitations. This study contributes to emergy synthesis in four aspects: (1) Provide a solution to the enduring data scarcity and inconsistency problem of emergy-based urban sustainability assessment; (2) Establish the deduction method which can deduce data at larger scale to smaller scale when lacking physical data at smaller scale instead of using monetary estimates, and also the accuracy of the deduced data can be identified by using uncertainty analysis; (3) Investigate the strategically essential Jing-Jin-Ji region as a case, applying the new approach to get the data for prefectural cities and carrying out an emergy analysis. We found that the indicators, reflecting the specific development traits of each city, are well confirmed by the status quo in the region. This fact proves that our new method framework is affordable and could be extended in the future study of medium-sized cities suffering from data scarcity. (4) Demonstrate the significant gap between using money value and physical amount to calculate emergy of imported goods, proving the importance of physical amount data in the Emergy-based comparative analysis of urban metabolic efficiency and sustainability.

Financial statements based bank risk aggregation

One of the major challenges involved in risk aggregation is the lack of risk data. Recently, researchers have found that mapping financial statements into risk types is a satisfactory way to resolve the problem of data shortage and inconsistency. Nevertheless, ignoring off-balance sheet (OBS) items has so far been regarded as the usual practice in risk aggregation, which may lead to deviations in conclusions. Hence, we improve the financial statements based risk aggregation framework by mapping OBS items into risk types. Based on 487 quarterly financial statements from all 16 listed Chinese commercial banks over the period 2007–2014, we empirically study whether the overall impact of OBS activities and the individual impact of each of the OBS risk types on total risk depend on bank size. Moreover, this research divides the sample into two subsets, during and after the subprime crisis, to find out how the subprime crisis affects risks of Chinese banks. Our empirical results show that although OBS credit risk is positively linked to total risk while OBS operational risk is negatively linked to total risk for both large and small banks, the overall impact of OBS activities on total risk depends on bank size. The overall OBS activities are positively related to the large bank’s total risk while they are negatively related to the small bank’s total risk. Besides, we also found that it is the increase of liquidity risk and market risk that leads to the larger total risk of Chinese banks during the subprime crisis.

A data-check based distributed storage model for storing hot temporary data

For the purpose of ensuring data security, traditional systems have widely used redundancy backup to store multiple copies of data. Multiple copies technology has high reliability, but also has the disadvantage of high redundancy storage and low space utilization. On the contrary, EC (Erasure Coding) technology has a high utilization rate of storage space, but the overhead of coding, decoding and data reconstruction is great. So, this paper demonstrates a data backup method based on XOR checksum being suitable for storing hot temporary data, which first splits the data into two parts and then performs the XOR operation of the two parts to generate another part of the data. Finally, the XOR checksum stores the three data parts into different nodes. The checksum not only ensures the security of data but also saves the storage space, thus improving the performance of reading and writing. This strategy achieves a mutual backup between the three nodes in order to ensure data security. Because there is only one copy of original data in the system, this model resolves the data inconsistency problem reasonably and simplifies the data version control existing in the redundancy backup model. Actual data test results show that, compared with the current mainstream Cassandra redundant backup model, the performance of a data backup model based on the XOR checksum proposed and implemented in this paper has been significantly improved: the reading performance improves by an average of 10%, and the writing performance improves by an average of 30%.

Approximation for vertex cover in $$\beta $$ β -conflict graphs

Conflict graph is a union of finite given sets of disjoint complete multipartite graphs. Vertex cover on this kind of graph is used first to model data inconsistency problems in database application. It is NP-complete if the number of given sets r is fixed, and can be approximated within \(2-\frac{1}{2^r}\) (Miao et al. in Proceedings of the 9th international conference on combinatorial optimization and applications, vol 9486. COCOA 2015, New York. Springer, New York, pp 395–408, 2015). This paper shows a better algorithm to improve the approximation for dense cases. If the ratio of vertex not belongs to any wheel complete multipartite graph is no more than \(\beta <1\), then our algorithm will provide a \((1+\beta +\frac{1-\beta }{k})\)-approximation, where k is a parameter related to degree distribution of wheel complete multipartite graph.

Fixing inconsistencies of fuzzy spatiotemporal XML data

Fuzzy spatiotemporal data models have been used to support spatial and temporal knowledge representation and reasoning in the presence of fuzziness. In the meantime, XML is expected to become the next generation standard language for exchanging data over the Internet, which will become a trend to represent fuzzy spatiotemporal data based on XML. However, fuzzy spatiotemporal XML documents may contain inconsistencies violating predefined spatial and temporal constraints, which cause the data inconsistency problems. Although those consistency problems in XML documents have been widely studied, their studies only take the general data into account, and the studies on consistencies of fuzzy spatiotemporal data are still open issues. In this paper we put forward solutions to the problems of inconsistencies in fuzzy spatiotemporal XML documents. We also analyze inconsistent states which are named discontinuity overlap or cycle of the temporal labels of some incoming edges. Then, we put forward the corresponding approaches to checking and fixing fuzzy spatiotemporal XML documents according to the inconsistent states. Finally, the experimental results show that our proposed algorithms can fix inconsistencies of fuzzy spatiotemporal XML documents significantly.

Organizing earth observation data inside a spatial data infrastructure

Scientists as well public institutions dealing with geospatial data often work with a large amount of heterogeneous data deriving from different sources. Without a well-defined, organized structure they face problems in finding and reusing existing data, and as consequence this may cause data inconsistency and storage problems. A catalog system based on the metadata of spatial data facilitates the management of large amount of data and offers service to retrieve, discover and exchange geographic data in an quick and easy fashion. Currently, most online catalogs are more focusing on the geographic data and there has been only few interests in catalogizing Earth observation data, in which in addition the acquisition information matters. This article presents an automatic metadata extraction approach that creates from different optical data deriving from various satellite missions of scientific interest (i.e. MODIS, LANDSAT, RapidEye, Suomi-NPP VIIRS, Sentinel-1A, Sentinel-2A) metadata information, based on an extended model of the standard ISO 19115. The XML schema ISO 19139-2 with the support of gridded and imagery information defined in ISO 19115-2 was examined, and based on the requirements of experts working in the research field of Earth observation the schema was extended. The XML schema ISO 19139-2 and its extension has been deployed as a new schema plugin in the spatial catalog Geonetwork Open Source in order to store all relevant metadata information about satellite data and the appropriate acquisition and processing information in an online catalog. A real-world scenario that is productively used in the EURAC research group institute for Applied Remote Sensing illustrates a workflow management for Earth observation data including data processing, metadata extraction, generation and distribution.