Data Processing Technology Research Articles

Hole Inpainting Algorithm for Half-Organized Point Cloud Obtained by Structured-Light Section System

The advances in sensors and data processing technologies enrich the types of 3D point clouds acquirement, empowering numerous extensive and novel applications such as 3D reconstruction in various scenarios. However, the hole defects affect the accuracy and fidelity of the acquired point clouds, hindering further development and application of 3D point clouds. Aiming at the hole defects in point clouds, a Bayesian hole inpainting algorithm for the half-organized point cloud is proposed, where the point cloud is obtained by a structured-light section system. The algorithm establishes a Bayesian probability model in the hole region, which adopts specific distributions of the point cloud to estimate the maximum likelihood parameter. Simulation and experimental results show that the proposed approach outperforms other competing algorithms significantly in repairing various types of holes, both in objective and subjective qualities. In addition, the proposed algorithm has better scalabilities in the cases of wrong topology definition and self-intersection confusion. This is the first algorithm specially designed for hole inpainting in half-organized point clouds, which maximizes the comprehensive consideration of local features and global optimization, supplemented by targeted prior knowledge of density, Riemannian manifold, and discrete attributes.

Indoor Point Cloud Segmentation Using a Modified Region Growing Algorithm and Accurate Normal Estimation

With the development of 3D sensors, 3D point cloud data can now be obtained conveniently, which has made the development of automatic point cloud data processing technology crucial. Region growing is a commonly used algorithm to segment point cloud, which greatly depends on the accuracy of point normals and requires the tuning of two thresholds; namely, the increment threshold of curvature (σ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sub> ) and normal angles (θ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sub> ). In this paper, we improve the region growing algorithm in two ways: Accurate normal estimation and strengthening the region growing criteria. For the first aspect, principal component analysis (PCA) is utilized to estimate the initial normals of the point cloud. Then, the points are divided into regular points (RP) and sharp feature points (SFP), according to their initial normals. A robust estimator-based PCA is then applied to refine the SFP normals. For the latter aspect, non-connective points are detected according to shared neighbor points, and non-coplanar points are determined by comparing the residual with a robust scale of it. In addition, σ <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sub> is set as the 95 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> percentile of curvature, allowing for easier parameter adjustment. Finally, the segmentation effect of the proposed method is evaluated through internal and external indices. The results indicate that the proposed method can accurately estimate the point normals within an acceptable time, and can obtain a better segmentation result than the classic PCA-based region growing algorithm and advanced DetMM-based methods.

The study of labor pendulum migration using big data processing technologies and the capabilities of the VKontakte social network

Official statistical data do not allow estimating the magnitude of labor pendulum migration in the context of regions and municipalities on a regular basis. The purpose of this research was to study the possibility of using open data of the VKontakte social network to identify and analyze labor pendulum migration on the example of Ekaterinburg and 12 satellite cities. Depersonalized information of users of the VKontakte social network and Rosstat data acted as empirical data. The research established a connection between the distance from the city of residence to Ekaterinburg and the proportion of citizens-users of the VKontakte network included in the process of pendulum labor migration. The analysis of incoming and outgoing flows of labor pendulum migration showed that despite significant differentiation by cities, in all cities the outgoing flow of labor migrants to Ekaterinburg exceeded the incoming one. The results of the research can be used in the analysis of the features, intensity and directions of pendulum migration for their consideration in the elaboration of strategic documents in the field of spatial development of territories. Further research by the authors will be aimed at improving the methodology for studying migration processes that occur between cities of millions and satellite cities, in order to identify agglomeration effects using big data technologies.

Data Mining Techniques and Applications

Abstract: In recent days internet is considered as the main supply for searching the information and collecting data . The extraction of the data from the web offers several query results. Machine-controlled tools are needed through queries from the amount of pages by using the internet to spot the connected info. Data mining method is taken into account an efficient method of extracting the relevant information from databases. This method is employed for the pattern identification. Data mining could be a method that finds helpful patterns from great amount of knowledge. The paper discusses few of the information mining techniques, algorithms and a few of the organizations that have adapted data processing technology to enhance their businesses and located glorious results

Government surveillance and facial recognition system in the context of modern technologies and security challenges

The challenges to privacy resulting from the use of modern and constantly improving data processing technologies pose a significant threat not only at the personal level, but also at the national level. The proliferation of biometric systems, and specifically facial recognition technologies in our everyday lives, has prompted the need to foster a public discussion regarding the associated societal and ethical concerns. Modern information and communication technologies allow organizations to process data on an extremely large scale and at an increasing speed. This thesis is reaffirmed by the exceptional economic strength of companies that process and store personal data on a worldwide scale (Alphabet, also known as Google or Meta, the company that owns Facebook and Instagram), or by the recent acknowledgment of this issue by several countries and the European Union. This paper highlights the state-of-the-art in data processing technology and centers on the numerous difficulties in handling data. It focuses on ethical, technical and legal concerns regarding facial recognition systems and also emphasizes the technologies available and their capacity for handling the larger volume and variety of personal data processing and Big Data Analytics.

ТЕХНОЛОГИЯ ОБРАБОТКИ БОЛЬШИХ ДАННЫХ (BIG DATA)

The description of big data, big data processing technologies (big data), Hadoop big data storage systems are considered

Research and Measurement Status and Data Processing Technology of Highway Asphalt Intelligent Testing Equipment

This paper introduces the current status of the asphalt viscosity testing equipment in the field of the road engineering, domestic and foreign study about the high viscosity standard that is used for the calibration of such equipment, as well as the developing technologies of the viscosity standard; it also predicts and gives the outlook about the development trend of the high viscosity standard in the future.

Integration of satellite data processing technologies of external providers into Roscosmos information system on example of the Satellite Center of the Far Eastern Branch of the Russian Academy of Sciences

On an example of a service for merged SST charts computing, the problems of integrating a sophisticated technology of external provider into the Roscosmos State Information System for the Operational Delivery of Earth Remote Sensing Data (SIS OD ERS) are considered. The main goal of the SIS OD ERS is the implementation of a simple and uniform access to satellite data of both Russian and worldwide archives, as well as to high-level processing products in a required thematic area. External provider gets access to satellite data archives and computing resources of SIS OD ERS. The key issue is minimizing the cost of processing data from both Roscosmos and external providers. The Multiple Access Center for Regional Satellite Monitoring of the Environment of the Far East Branch of the Russian Academy of Sciences deploy its services to SIS OD ERS. The options for arranging the exchange of data and services between a provider and SIS OD ERS are proposed. A REST web services are made, with its support in the Python environment. As a result of the integration, daily merged SST charts are built that preserve the structure of thermal fronts for any water area and with the accuracy that corresponds the international standards. The service is distributed. Both the resources and data of Roscosmos and the Satellite Center of the Far Eastern Branch of the Russian Academy of Sciences, and the data from the foreign information systems are used.

Using the Potential of Social Media in Profiling and Attributing Precarious Employment of Socio-Demographic Groups in Russia

In the past three decades have seen the rapid development of Internet and its wide distribution across countries and socio-demographic groups. The use of social media for researches may be really useful, as users of media platforms often post data or enter queries related with their current work or job search. The research presents the results of an analysis of data found in social media and open information resources related to precarious employment in Russia. There was used thesaurus technology in the processing of information which consists of scientific articles and public discussions. Automatic monitoring of news, posts and discussions of precarious employment issue was carried out. Identification of precarious employment actors were carried out using the Big Data processing technology. The gender, age and professional connection of precarious employment actors, as well as their geographical location in Russia, has been revealed. The author evaluates the scale of precarious employment in the context of Russian federal districts. There are factors, attributes and consequences of labor precarisation in Russia are identified in result of social media data analysis. Sociodemographic, professional and regional profile of a subject with high risks of precarious employment in Russia has been compiled. Based on the monitoring of references to precarious employment in social networks, blogs and public information resources, its manifestations were systematized in the context of a negative impact on the quality of working life, employment incomes and households’ standard of living.

Sistem Informasi Point of Sale Berbasis Web pada Toko Afridah Cake

The development of internet technology in Indonesia, especially in the use of computer technology for data processing, is also a concern in the business world. Through computerized data processing, all items that have been sold can be quickly identified, along with their advantages. Toko Afridah Cake is a business that sells various types of cakes and snacks in Tembilahan which is still with an offline transaction system. Therefore, it is necessary to build a point of sale information system at the Afridah Cake store so that it makes it easier to develop a business. Researchers use the SDLC (System Development Life Cycle) system development method and use pieces analysis. The system is designed using three levels of users, namely admin, cashier, and consumer. The system can be used to exist at Toko Afridah Cake because it has passed the blackbox testing stage without any errors. This web-based point of sale system makes it easier for store owners, especially to monitor the transaction process in the store without having to come to the store because the system is web-based so that it can be accessed anywhere while still being able to connect to the internet.

Vehicle Trajectory-Based Calibration Procedure for Microsimulation

In practice, traffic engineers and researchers calibrate microscopic driver behavior in microsimulation models using macroscopic inputs (e.g., aggregated traffic throughput) instead of microscopic inputs (e.g., inter-vehicle spacing and acceleration). This mismatch has naturally led to concerns that these models have not accurately captured the microscopic driver behaviors, despite the apparent goodness-of-fit in the macroscopic performance measures. There is renewed interest in trajectory-based calibration for microsimulation models given the recent improvements in data collection and data processing technologies. Toward this end, this project collected aerial vehicle trajectories using cameras attached to drones at three real-world sites (producing 800 ft trajectories) and a helicopter at one site (producing 1.2 mi trajectories). This study used these datasets to develop a new microsimulation model trajectory-based calibration method and calibrated four urban freeway models: I-270, I-15, I-75, and I-95. Next, this study calibrated models of the same four sites using macroscopic data (e.g., average segment speed and throughout). Finally, this study compared the two methods’ calibration accuracy to understand the benefits of the trajectory-based calibration method. The experimental results provided evidence that analysts should not trust traditional calibration methods to produce models with realistic vehicle trajectories. However, explicit integration of trajectories into the calibration process (i.e., hybrid calibration methods) can significantly improve the modeled trajectories’ realism. Calibrated model results were most impressive at I-75, which is the only site that collected data via helicopter, yielding significantly longer trajectories.

Learning Analytics and Higher Music Education: Perspectives and Challenges

Continually monitoring student learning, improving tutoring, predicting academic risks such as performance drops or dropouts, assessing more objectively or understanding the behavior of student groups are some of the tasks that have been beyond the reach of music teachers. The current technology of massive data processing (Big Data) and its analysis (Learning Analytics-LA) allows to achieve these goals with relative ease. The possibility of extracting individual behavior patterns facilitates attention to diversity, reduces school dropout and failure, and opens the possibility of implementing new educational strategies. The phenomenon of data-based education has led to different types of studies. This paper reflects on three trends or fundamental perspectives in the use of the collection of massive information applied to learning and teaching. We offer an overview of research and applications of Learning Analytics specifically in the field of music education, as well as a reflection on its possible practical uses in higher music education in conservatories. For this purpose, we discuss some practical examples of how this technological methodology could be incorporated into music and music education research, and its influence on possible new educational paradigms that lead to innovation on teaching-learning process through new technological resources.

Photovoltaic power combination prediction system based on improved multi‐objective optimization algorithm and nonlinear weighting strategy

AbstractThe large‐scale connection of photovoltaic (PV) power generation to the power grid introduces considerable challenges to the grid's stable operation. To ensure grid stability, the key is to improve the accuracy of PV power prediction. Currently, PV power prediction is primarily dependent on single‐model construction, which ignores the role of data processing technologies, thus resulting in inadequate prediction performance. Accordingly, in this study, a combined prediction system is developed by integrating advanced deep learning algorithms and data processing techniques to improve the accuracy of PV power prediction. In addition, a nonlinear weighting strategy based on an improved multi‐objective dragonfly optimization algorithm (IMODA) is proposed to determine the final prediction result. In the IMODA, cloud model generator, tent mapping, and Pareto solution selection strategy based on knee points are introduced to resolve the defects of the original algorithm. The performance of the proposed combined system was scientifically evaluated and analysed by considering the PV power datasets of four seasons in Belgium as an example. The mean absolute percentage errors of the proposed system on the four datasets are 4.0198, 4.7943, 4.3587, and 5.9286. These values indicate an improvement range of 5%–30% compared with the errors of other benchmark models.

Ratio Sum Versus Sum Ratio for Linear Discriminant Analysis.

Dimension reduction is a critical technology for high-dimensional data processing, where Linear Discriminant Analysis (LDA) and its variants are effective supervised methods. However, LDA prefers to feature with smaller variance, which causes feature with weak discriminative ability retained. In this paper, we propose a novel Ratio Sum for Linear Discriminant Analysis (RSLDA), which aims at maximizing discriminative ability of each feature in subspace. To be specific, it maximizes the sum of ratio of the between-class distance to the within-class distance in each dimension of subspace. Since the original RSLDA problem is difficult to obtain the closed solution, an equivalent problem is developed which can be solved by an alternative optimization algorithm. For solving the equivalent problem, it is transformed into two sub-problems, one of which can be solved directly, the other is changed into a convex optimization problem, where singular value decomposition is employed instead of matrix inversion. Consequently, performance of algorithm cannot be affected by the non-singularity of covariance matrix. Furthermore, Kernel RSLDA (KRSLDA) is presented to improve the robustness of RSLDA. Additionally, time complexity of RSLDA and KRSLDA are analyzed. Extensive experiments show that RSLDA and KRSLDA outperforms other comparison methods on toy datasets and multiple public datasets.

Design of intelligent window opening system based on multi-sensor fusion

The traditional electric window opener has the problems of low efficiency and single control mode. In view of the above shortcomings, an intelligent electric window opener control system based on multi-sensor fusion is designed. The system uses data acquisition, data processing, wireless communication, remote control, and other technologies to realize the real-time monitoring of environmental status through the fusion of temperature, light intensity, smoke, rain, and other sensors. When the monitoring data reaches the set value, the intelligent window opener will automatically open or close. The whole system is based on an STM32 single-chip microcomputer and uses the ESP8266 Internet of things module to realize the remote monitoring and control of the platform. In the aspect of window opening motor control, the proportional integral derivative (PID) is used to control the DC motor for the accurate window opening. The experimental results show that the overall operation of the system is stable, the response is rapid, the sensor accuracy is high, the system error is less than 5%, and the long-term operation shows good stability and reliability.

Building Artificial-Intelligence Digital Fire (AID-Fire) system: A real-scale demonstration

The identification of building fire evolution in real-time is of great significance for firefighting, evacuation, and rescue. This work proposed a novel framework of Artificial-Intelligence Digital Fire (AID-Fire) that can identify complex building fire information in real-time. The smart system consists of four main parts, Internet of Things sensor network (data collection and transfer), cloud server (data storage and management), AI Engine (data processing), and User Interface (fire information display). A large numerical database, containing 533 fire scenarios with varying fire size, positions, and number of fire sources, is established to train a Convolutional Long-Short Term Memory (Conv-LSTM) neural network. The proposed fire digital twin is demonstrated and validated in a full-scale fire test room (26 m 2 ). Results show that the AI engine successfully identify the fire information by learning the spatial-temporal features of the temperature data with a relative error of less than 15% and a delay time of less than 1 s. Moreover, detailed fire development and spread can be accurately displayed in the digital-twin interface. This proposed AID-Fire system can provide valuable support for smart firefighting practices, thus paving the way for a fire-resilient smart city. • A smart building digital twin was developed to identify fire scene in real-time. • Wireless IoT sensors and deep learning model drive the digital twin of building fire. • AI model identifies fire information in digital twin with an error <15% and time lag <1 s. • The digital fire system demonstrates a prototype of future smart firefighting system.

Integration of Computer Virtual Reality Technology to College Physical Education

The progress of the times has brought about a leap forward in people’s thinking. Under the rapid economic development environment, the physical field of young people has been unable to withstand the current teaching system, and many problems of poor physical fitness have emerged. In order to solve similar problems, to improve the physical quality of young people, it is bound to find a new teaching method different from the traditional physical education teaching in a special environment, The research on the integration of virtual reality technology and teaching has been rolling forward and never stopped in recent years. With the continuous upgrading of virtual reality technology, virtual reality devices that can be used have already joined the ranks of families. Let these virtual reality devices connect to the Internet through the WEB application settings, and then design according to different situations has become a reality. This research is an application development based on Web virtual reality technology, including network virtual reality technology, application modeling design and the use of Internet connection. With reference to this application, physical education teaching and classroom practice data, this research mainly introduces the current research situation of virtual reality technology and teaching, and establishes the model based on the integration of Web application design and virtual reality technology, Extract, including but not limited to the number of projects, activities and experiences, use the Internet of Things data upload and data processing technology to complete the data screening, obtain valuable data, and then use these data for the practical application value of virtual reality technology in physical education teaching. Complete the improvement. It is in line with the integration of modern virtual reality technology and physical education teaching to obtain data that can reflect the real situation, then conduct practical teaching, obtain reliable practical data, and evaluate teaching. It can be clearly seen from the research results that the combination of virtual reality technology and physical education teaching is of great significance to improve students’ interest in learning and enthusiasm for sports, but there are also some shortcomings, such as different teaching steps and goals will affect students’ enthusiasm for learning. Therefore, further improvement is needed to find a stable way to improve students’ enthusiasm for learning. The integration of virtual reality technology and physical education teaching is proposed.

개인정보 보호와 활용에 관한 연구

Since the 1970s, with the advancement of information and communication technology, the creation and processing of data has been activated, and we are enjoying abundance unimaginable before that. This revolutionary change has been made possible because of the development of data processing technology and the freedom to create, process, and transfer data. However, as the importance of personal information protection is emphasized, prior consent for the collection, processing and transfer of personal information is being strengthened. In addition, it is reaching a level that hinders free distribution itself beyond the purpose of safe distribution of data through personal information protection. In order for the information society to develop normally, the smooth distribution of personal information is necessary. In the information society, as information and communication technology develop, not only information itself has value, but also new value is created by information. Information has value only when combined with personal information. For example, personal information is essential when trading stocks, managing banks, and buying and selling goods online. However, failure to properly manage data can easily expose an individual's privacy, resulting in various side effects. From a balance between the development of the data industry and personal information protection, institutional supplementation is needed to reduce excessive prior consent to personal information collection and transfer to third parties, widen the scope of alias processing, and strengthen post-responsibility as an alternative.

Real-Time Big Data Processing and Analytics: Concepts, Technologies, and Domains

In the digital era, data is one of the most important assets since it conceals valuable information. Developers of data-intensive systems have new challenges at each level of streaming, storing, and processing large quantities of data in a variety of forms and speeds. Obtaining useful information at the proper time and place is also crucial. Since the value of information is inversely proportional to time, real-time data processing and analytics are receiving more attention. Due to the importance of real-time data processing and analytics, this study focuses on real-time data processing concepts and terminology, popular technologies used in real-time data processing and analytics, popular NoSQL storage technologies used in real-time data processing, and real-time data processing application areas. The purpose of this paper is to provide researchers of real-time analysis and developers of data-intensive systems with a comparative perspective on real-time data processing by highlighting the key characteristics of real-time data processing technologies, NoSQL storage technologies, their application domains, and selected examples from previous studies.

Application of FCM Clustering Algorithm in Digital Library Management System

Traditional library systems are gradually being replaced by digital libraries. Digital libraries are developing from simple database-based storage and retrieval to knowledge-based implementation. The fuzzy C-means (FCM) clustering algorithm is an example of data collection and data processing technology. It evaluates and draws conclusions based on mathematics, large data, and other technologies. In order to better improve the digital library management system, this paper applied FCM clustering algorithm to the digital library management system. Based on the in-depth study of the FCM clustering algorithm, this paper built a digital library management system. The clustering algorithm was used to cluster library borrowing records and reader information. It provided technical support and suggestions on library collection construction and book purchase and promoted book management to form a good spitting cycle. The experimental results extracted during the evaluation phase demonstrated that the overall error rate of the suggested FCM clustering algorithm for information clustering is 3.66%, which is better than the existing comparative models. This shows that applying the FCM clustering algorithm to a digital library management system has some practical significance.