Reduce Data Processing Research Articles

Electrode-Dependent and Tunable Sub-to-Super-Linear Responsivity in Mott Material-Enabled Near-Infrared Photodetectors for Advanced Near-Sensor Image Processing.

Brain-like intelligence is ushering humanity into an era of the Internet of Perceptions (IoP), where the vast amounts of data generated by numerous sensing nodes pose significant challenges to transmission bandwidth and computing hardware. A recently proposed near-sensor computing architecture offers an effective solution to reduce data processing delays and energy consumption. However, a pressing need remains for innovative hardware with multifunctional near-sensor image processing capabilities. In this work, Mott material (vanadium dioxide)-based photothermoelectric near-infrared photodetectors are developed that exhibit electrode-dependent and tunable super-linear photoresponse (exponent α > 33) with ultralow modulation bias. These devices demonstrate an opto-thermo-electro-coupled phase transition, resulting in a large photocurrent on/off ratio (>105), high responsivity (≈500 A W-1), and well detectivity (≈3.9× 1012 Jones), all while maintaining rapid response speeds (τr = 2 µs and τd = 5 µs) under the bias of 1V. This electrode-dependent super-linear response is found to arise from the electron doping effect determined by the polarity of the Seebeck coefficient. Furthermore, the work showcases intensity-selective near-sensor processing and night vision pattern reorganization, even with noisy inputs. This work paves the way for developing near-sensor devices with potential applications in medical image preprocessing, flexible electronics, and intelligent edge sensing.

Optimization model for vehicular network data queries in edge environments

As the Internet of Vehicles advances, the demand for timely data acquisition by vehicle users continues to escalate, albeit confronted with the challenge of excessive data retrieval latency. The emergence of edge computing provides technical support for the development of vehicular networks by caching data in advance to reduce data acquisition latency. Therefore, how to effectively cache and query data becomes a key issue in addressing the timeliness of data acquisition in vehicular networks. In this paper, we investigate an efficient query optimization model to minimize data acquisition latency. Firstly, based on the distribution of data query frequencies across different servers, we propose an edge collaborative caching strategy using a tabu search algorithm. This strategy prioritizes high-traffic data by finding two optimal storage nodes for each high-traffic data in descending order of data popularity, ensuring a backup within the collaborative domain for each data segment. This not only reduces data transmission latency between nodes during task execution but also prevents single-point failures. Secondly, we deploy cuckoo filters on edge nodes to enable rapid localization of cached data nodes when users query data, thus reducing data processing latency. Finally, simulation results demonstrate that the proposed query optimization model outperforms other schemes in terms of average data query latency.

Candidates resume analyzer using artificial intelligence

In the modern recruitment process, where the number of candidates and job vacancies constantly increases, automating resume analysis becomes critically important for efficient selection. This paper focuses on developing a system for automated resume analysis using artificial intelligence, specifically the OpenAI model (ChatGPT). The paper thoroughly examines the system's components, structure, and functional capabilities. The system consists of several key modules: the user interface ensures interaction with users, the Application Programming Interface Gateway facilitates data exchange between the interface and the server, and the authentication module controls system access, ensuring user security. The main part of the system's business logic is implemented on the server, which interacts with the natural language processing module, utilizing the ChatGPT model for processing resume text data. The use of natural language processing technologies allows the system to automatically extract and analyse key elements of resumes, such as education, work experience, skills, and certifications of candidates. This significantly speeds up the processing of large volumes of resumes, improving selection efficiency and accuracy. Data obtained during the analysis is stored in a PostgreSQL database, ensuring reliable storage and quick access for further processing. The paper also emphasizes the importance of integrating the system with other recruiting platforms and databases for more efficient management of the hiring process. Such integration allows for automatic synchronization of job vacancies and candidates, optimizing the search and selection process. Additionally, the paper highlights the key challenges associated with automating resume processing, such as the diversity of resume formats submitted by candidates, differences in data structure, and potential errors in automatically recognizing important information. Despite these challenges, the use of artificial intelligence and natural language processing methods significantly enhances the productivity and accuracy of recruitment processes. Thus, the developed automated resume analysis system based on Artificial Intelligence can become a significant tool in the modern candidate selection process. Its functionality not only enables quick and accurate resume analysis but also allows integration with other HR systems, simplifying the recruitment process, reducing data processing time, and improving selection accuracy.

A random feature mapping method based on the AdaBoost algorithm and results fusion for enhancing classification performance

The feature mapping method can improve data separability, enhance data representation ability, and reduce data processing complexity. However, on the one hand, the existing feature mapping methods have difficulty processing datasets of different dimensions and distributions adaptively, limiting the scope of application; on the other hand, a single feature mapping method has the problem of instability and poor generalization ability, weakening the classification ability of subsequent classifiers. This paper proposes a random feature mapping method based on the AdaBoost algorithm and results fusion to enhance classification performance. The method adopts horizontal expansion, fine-tuning weights through sparse autoencoders, and uses input-mapped features as feature nodes to generate multiple feature subsets for increasing stability. After training weak classifiers on each multiple feature subset, the weights of classifiers are adjusted adaptively by the Adaboost ensemble algorithm. Finally, the method fuses weak classifiers twice to enhance classification performance, which abandons the traditional voting method and uses the weighted probability selection method. Experiments on twenty classic datasets show that the proposed method can effectively mine essential features and enhance classification accuracy compared with original datasets. For instance, the Balance dataset has an average classification accuracy of more than 20% higher than the original dataset on the KNN classifier. The proposed method outperforms alternative feature mapping methods in terms of performance and efficiency on different classifiers in most cases.

Simulation-based data reduction and data processing for sheet metal forming in the hybrid twin framework

In sheet metal forming, the interaction between virtual models and the real world remains challenging. Process simulations can exhibit significant errors, and reliable measurements are often scarce during early production stages. This study presents a hybrid twin framework that systematically unifies computer-aided design, simulation, and measurement data in an adaptive manner. Central to this framework is a reverse engineering algorithm that reconstructs and transforms the geometry of deep-drawn components from optical scan data into B-spline surfaces. The algorithm demonstrated high precision, indicating its suitability for process control and geometric analysis. The hybrid twin framework integrates virtual data from simulations and real-world data, as evidenced by a sensor concept for inline surface measurement. The framework ensures robust and redundant measurement concepts by estimating complete geometries from a few systematically preselected measuring points. This adaptive approach permits continuous updates and extensions to the database, accommodating both sparse inline signals and offline inspection data. This framework provides a conceptual model for integrating direct feedback interactions between virtual and physical environments, thereby enhancing the precision of analytical and predictive models in sheet metal forming processes.

Identification of dominant tree species based on Resource-1 02D hyperspectral image data

Fine-grained classification of tree species by using high-spectral image data has garnered considerable attention from scholars. In this study, through field measurements from Maguan County, Wenshan Prefecture, Yunnan Province, China, high-spectral image data from the Chinese Resource-1 02D satellite were used as the data source. Various analyses were conducted on the original image’s spectral curve, the spectral curve after envelope removal, the spectral curve after first-order differential transformation, and the spectral curve after second-order differential transformation. A spectral angle mapping classification method was employed to classify and identify four dominant tree species in Maguan County, and the accuracy of the classification results was validated using a confusion matrix. Results indicate that the highest accuracy in tree species classification was achieved when first-order differential transformation and envelope removal were used for the spectral curve; the overall accuracy exceeded 95%, and the kappa value was approximately 0.95. The classification results for the spectral curve after second-order differential transformation were the lowest, with an overall accuracy of 81.69% and a kappa value of 0.76. This research demonstrates that applying first-order differential transformation or envelope removal in combination with spectral angle mapping classification considerably reduces data processing time and improves tree species classification accuracy.

IMPLEMENTATION OF RPA BOTS IN COLD SUPPLY CHAIN LOGISTICS

Due to the significant use of low-temperature logistics in the transportation of perishable goods, the demand for the cold chain has increased. To ensure delivery efficiency and reduce damage, logistics companies should monitor the status of deliveries over short time intervals. Tracking the delivery status is a time-consuming, resource-intensive, inefficient and repetitive process. Therefore, robotic Process Automation (RPA) applications have attracted the attention of practitioners in the cold chain logistics industry. By studying the workflow of cold chain logistics, this study helps to identify possible areas that require automation. As part of the case study, the performance of two automatic RPA robots used in a forwarding company to check the condition of cargo and temperature conditions was tested and evaluated. The results showed that the introduction of RPA into the workflow significantly reduces data processing time.

Illuminating the nanostructure of diffuse interfaces: Recent advances and future directions in reflectometry techniques

Diffuse soft matter interfaces take many forms, from end-tethered polymer brushes or adsorbed surfactants to self-assembled layers of lipids. These interfaces play crucial roles across a multitude of fields, including materials science, biophysics, and nanotechnology. Understanding the nanostructure and properties of these interfaces is fundamental for optimising their performance and designing novel functional materials. In recent years, reflectometry techniques, in particular neutron reflectometry, have emerged as powerful tools for elucidating the intricate nanostructure of soft matter interfaces with remarkable precision and depth. This review provides an overview of selected recent developments in reflectometry and their applications for illuminating the nanostructure of diffuse interfaces. We explore various principles and methods of neutron and X-ray reflectometry, as well as ellipsometry, and discuss advances in their experimental setups and data analysis approaches. Improvements to experimental neutron reflectometry methods have enabled greater time resolution in kinetic measurements and elucidation of diffuse structure under shear or confinement, while innovation in analysis protocols has significantly reduced data processing times, facilitated co-refinement of reflectometry data from multiple instruments and provided greater-than-ever confidence in proposed structural models. Furthermore, we highlight some significant research findings enabled by these techniques, revealing the organisation, dynamics, and interfacial phenomena at the nanoscale. We also discuss future directions and potential advancements in reflectometry techniques. By shedding light on the nanostructure of diffuse interfaces, reflectometry techniques enable the rational design and tailoring of interfaces with enhanced properties and functionalities.

DATA-DRIVEN TRANSFORMATION: OPTIMIZING ENTERPRISE FINANCIAL MANAGEMENT AND DECISION-MAKING WITH BIG DATA

This study explores the transformative impact of big data on enterprise financial management, highlighting significant improvements in decision-making efficiency, data quality, and risk management capabilities. Through a mixed-methods approach that includes meta-analysis, surveys, interviews, and case studies, the research reveals a substantial increase in decision-making efficiency, better integration of diverse data sources, and more accurate financial reporting. Practical benefits such as reduced data processing times and improved credit risk assessments are identified, alongside challenges like the skill gap in data science, cultural resistance, and technical difficulties in integrating new technologies with legacy systems. Addressing these challenges requires strategic leadership, continuous training, and investment in scalable infrastructure. Despite these hurdles, the long-term advantages of big data adoption, including enhanced financial reporting and resource allocation, underscore its value in driving organizational performance and competitiveness. This study provides empirical evidence and practical insights, offering a valuable foundation for organizations aiming to leverage big data in their financial management practices.

Research on proportional valve spool wear diagnostic method hybrid-driven by the valve port energy loss mechanism model and data

High performance proportional valves are commonly utilized for precise control of aircraft actuators to ensure flight safety. Fault diagnosis play a crucial role in maintaining equipment reliability. However, traditional diagnostic methods using vibration signals face challenges such as inability to directly measure failure points, fault characteristics being influenced by sensor position, and large data processing volume, limiting engineering application scope. This paper introduces a valve spool wear fault diagnostic method based on energy loss model and data hybrid drive, leveraging throttling loss characteristics. An energy loss failure mechanism model, incorporating differential pressure and flow rate of valve port, was established, and experimentally verified. This method effectively addresses the limitations of vibration sensors in diagnosing aviation hydraulic systems. By combining particle swarm optimization algorithm with deep extreme learning machine, the number of neurons, weight distribution, and data set ratio are rapidly optimized, reducing data processing complexity and enhancing diagnostic efficiency. Comparative tests demonstrate a significant increase in average diagnostic accuracy rate, with more than 8% improvement after integrating energy loss information and particle swarm optimization, reaching over 98%. The proposed method exhibits superior performance in average diagnostic accuracy rate and stability compared to other methods and can be served as a valuable reference for predicting faults in aviation hydraulic valves.

An Image Quality Evaluation and Masking Algorithm Based On Pretrained Deep Neural Networks

With the growing amount of astronomical data, there is an increasing need for automated data processing pipelines, which can extract scientific information from observation data without human interventions. A critical aspect of these pipelines is the image quality evaluation and masking algorithm, which evaluate image qualities based on various factors such as cloud coverage, sky brightness, scattering light from the optical system, point-spread-function size and shape, and read-out noise. Occasionally, the algorithm requires masking of areas severely affected by noise. However, the algorithm often necessitates significant human interventions, reducing data processing efficiency. In this study, we present a deep-learning-based image quality evaluation algorithm that uses an autoencoder to learn features of high quality astronomical images. The trained autoencoder enables automatic evaluation of image quality and masking of noise affected areas. We have evaluated the performance of our algorithm using two test cases: images with point spread functions of varying full width half magnitude, and images with complex backgrounds. In the first scenario, our algorithm could effectively identify variations of the point spread functions, which can provide valuable reference information for photometry. In the second scenario, our method could successfully mask regions affected by complex regions, which could significantly increase the photometry accuracy. Our algorithm can be employed to automatically evaluate image quality obtained by different sky surveying projects, further increasing the speed and robustness of data processing pipelines.

Diffractive deep neural networks: Theories, optimization, and applications

Optical neural networks (ONN) are experiencing a renaissance, driven by the transformative impact of artificial intelligence, as arithmetic pressures are progressively increasing the demand for optical computation. Diffractive deep neural networks (D2NN) are the important subclass of ONN, providing a novel architecture for computation with trained diffractive layers. Given that D2NN directly process light waves, they inherently parallelize multiple tasks and reduce data processing latency, positioning them as a promising technology for future optical computing applications. This paper begins with a brief review of the evolution of ONN and a concept of D2NN, followed by a detailed discussion of the theoretical foundations, model optimizations, and application scenarios of D2NN. Furthermore, by analyzing current application scenarios and technical limitations, this paper provides an evidence-based prediction of the future trajectory of D2NN and outlines a roadmap of research and development efforts to unlock its full potential.

Quality of Service Based on Scheduling in Cloud/Fog Environment

In the area of fog and cloud computing, efficient work scheduling is critical for optimising resource allocation and raising Quality of Service (QOS) levels. Research in this area has resulted in a variety of ways for dealing with this problem. One line of research focuses on lowering data latency and improving QOS in fog-cloud systems by precise job scheduling. This requires intelligently dividing jobs across fog and cloud resources to reduce data processing and delivery delays, ultimately improving the overall user experience. Furthermore, the investigation encompasses novel methodologies such as hybrid evolutionary algorithms and heuristic strategies. These approaches seek to create a compromise between competing objectives, such as cost and energy efficiency, while also achieving QOS criteria. Researchers aim to develop scheduling systems that optimise resource utilisation and reduce operating costs without sacrificing service quality by applying evolutionary concepts or combining heuristic methods. Furthermore, ongoing discussions focus on improving resource scheduling approaches to achieve optimal QOS results. This entails continual evaluation and adaption of scheduling algorithms to changing environmental conditions, workload fluctuations, and user requests. Furthermore, developing methodologies such as the CODA approach provide intriguing avenues for efficient job scheduling in fog computing settings, with the potential to revolutionise future resource allocation and QOS enhancement efforts.

FPGA Implementation of a Pipelined Kalman Filter for Object Tracking in Two Dimensions

In a location tracking system for a moving object, not only accuracy but also real-time processing are important factors. The Kalman filter, as a recursive function, stands out as one of the prominent algorithms for object tracking. It continuously compares measured data with predicted values based on the system characteristics in real time, and then corrects the error of the predicted values while considering the noise of both system and measured data. This paper focuses on designing a hardware-based Kalman filter for object tracking in two dimensions. Following an analysis of the Kalman filter algorithm, the blocks capable of parallel processing are identified and configured to be processed in parallel, effectively reducing data processing time. The clock speed is enhanced by using the pipeline technique. In addition, the time-sharing technique is applied to increase the utilization of hardware resources and reduce the area. Data was processed at 32-bit floating points to uphold accuracy comparable to software-implemented Kalman filters. The proposed Kalman filter architecture is designed using verilog HDL and then simulated in Synopsys VCS/Verdi. And the accuracy is verified by comparing results with a software-based Kalman filter designed using MATLAB. It was implemented using Zynq ZYNQ-7 ZC702 Programmable logic via Xilnix Vivado, and can operate at 33MHz. It takes a total of 44 clocks, or 1.32 us, to process one data. Therefore, it was confirmed that the designed Kalman filter hardware is suitable for real-time processing.

Study on tiered storage algorithm based on heat correlation of astronomical data

With the surge in astronomical data volume, modern astronomical research faces significant challenges in data storage, processing, and access. The I/O bottleneck issue in astronomical data processing is particularly prominent, limiting the efficiency of data processing. To address this issue, this paper proposes a tiered storage algorithm based on the access characteristics of astronomical data. The C4.5 decision tree algorithm is employed as the foundation to implement an astronomical data access correlation algorithm. Additionally, a data copy migration strategy is designed based on tiered storage technology to achieve efficient data access. Preprocessing tests were conducted on 418GB NSRT (Nanshan Radio Telescope) formaldehyde spectral line data, showcasing that tiered storage can potentially reduce data processing time by up to 38.15%. Similarly, utilizing 802.2 GB data from FAST (Five-hundred-meter Aperture Spherical radio Telescope) observations for pulsar search data processing tests, the tiered storage approach demonstrated a maximum reduction of 29.00% in data processing time. In concurrent testing of data processing workflows, the proposed astronomical data heat correlation algorithm in this paper achieved an average reduction of 17.78% in data processing time compared to centralized storage. Furthermore, in comparison to traditional heat algorithms, it reduced data processing time by 5.15%. The effectiveness of the proposed algorithm is positively correlated with the associativity between the algorithm and the processed data. The tiered storage algorithm based on the characteristics of astronomical data proposed in this paper is poised to provide algorithmic references for large-scale data processing in the field of astronomy in the future.

Improving time efficiency in big data through progressive sampling-based classification model

<span>The proposed system aims to overcome challenges posed by large databases, data imbalance, heterogeneity, and multidimensionality through progressive sampling as a novel classification model. It leverages sampling techniques to enhance processing performance and overcome memory restrictions. The random forest regressor feature importance technique with the gini significance method is employed to identify important characteristics, reducing the data’s features for classification. The system utilizes diverse classifiers such as random forest, ensemble learning, support vector machine (SVM), k-nearest neighbors’ algorithm (KNN), and logistic regression, allowing flexibility in handling different data types and achieving high accuracy in classification tasks. By iteratively applying progressive sampling to the dataset with the best features, the proposed technique aims to significantly improve performance compared to using the entire dataset. This approach focuses computational resources on the most informative subsets of data, reducing time complexity. Results show that the system can achieve over 85% accuracy even with only 5-10% of the original data size, providing accurate predictions while reducing data processing requirements. In conclusion, the proposed system combines progressive sampling, feature selection using random forest regressor feature importance (RFRFI-PS), and a range of classifiers to address challenges in large databases and improve classification accuracy. It demonstrates promising results in accuracy and time complexity reduction.</span>

Integrating Quantum Computing and NLP for Advanced Cyber Threat Detection

The exponential growth of digital data and the increasing sophistication of cyber threats demand more advanced methods for threat analysis. This paper explores the integration of quantum computing and natural language processing (NLP) to enhance cyber threat analysis. Traditional computing methods struggle to keep up with the scale and complexity of modern cyber threats, but quantum computing offers a promising avenue for accelerated data processing, while NLP provides sophisticated tools for interpreting and understanding human language, crucial for analysing threat intelligence. Our proposed framework leverages quantum algorithms for rapid anomaly detection and advanced NLP techniques for precise threat identification and analysis. The methodology includes data collection from diverse sources, pre-processing for normalization, quantum-assisted data processing using Grover's search and Quantum Approximate Optimization Algorithm (QAOA), NLP analysis with transformers and BERT-based models, and integration of findings to build comprehensive threat profiles. Experimental results demonstrate significant improvements: quantum algorithms reduced data processing time by up to 50%, NLP models achieved 92% accuracy in threat identification, and the false positive rate was reduced by 30%. These findings indicate a promising direction for next-generation cybersecurity solutions, enabling more proactive and efficient threat mitigation. Future work will focus on refining quantum algorithms, enhancing NLP models, and expanding the framework for real-time threat detection capabilities.

Advancing Video Search Capabilities: Integrating Feedforward Neural Networks for Efficient Fragment-Based Retrieval

In the context of rapidly increasing volumes of video data, the problem of their efficient search and analysis becomes more acute. This research aims to develop and test an innovative system to improve the speed and accuracy of video search, utilizing the capabilities of Deep Convolutional Neural Networks (DCNN) and Feedforward Neural Networks (FFNN). Within the methodology developed for this study, video data are processed through several sequential stages: from feature extraction to key frame identification and the formation of an abstract vector representation. Deep Convolutional Neural Networks are central to the system for image analysis and Feedforward Neural Networks for optimizing the search process. The main results of the study include an increase in video search efficiency by reducing data processing time and increasing the accuracy of identifying relevant fragments. The originality of the work lies in the integration of two types of neural networks for structured analysis of video data, which is a new step in the development of video search technologies. The practical significance of the research is expressed in the possibility of applying the developed system in various areas where fast and accurate video search is needed: from the media industry to security systems. The scope of further research includes adapting the system to specific types of video content and expanding the capabilities of artificial intelligence for a deeper understanding of video data.

Characterization of Active Riverbed Spatiotemporal Dynamics through the Definition of a Framework for Remote Sensing Procedures

The increasing availability and quality of remote sensing data are changing the methods used in fluvial geomorphology applications, allowing the observation of hydro-morpho-biodynamics processes and their spatial and temporal variations at broader and more refined scales. With the advent of cloud-based computing, it is nowadays possible to reduce data processing time and increase code sharing, facilitating the development of reproducible analyses at regional and global scales. The consolidation of Earth Observation mission data into a single repository such as Google Earth Engine (GEE) offers the opportunity to standardize various methods found in literature, in particular those related to the identification of key geomorphological parameters. This work investigates different computational techniques and timeframes (e.g., seasonal, annual) for the automatic detection of the active river channel and its multi-temporal aggregation, proposing a rational integration of remote sensing tools into river monitoring and management. In particular, we propose a quantitative analysis of different approaches to obtain a synthetic representative image of river corridors, where each pixel is computed as a percentile of the bands (or a combination of bands) of all available images in a given time span. Synthetic images have the advantage of limiting the variability of individual images, thus providing more robust results in terms of the classification of the main components of the riverine ecosystem (sediments, water, and riparian vegetation). We apply the analysis to a set of rivers with analogous bioclimatic conditions and different levels of anthropic pressure, using a combination of Landsat and Sentinel-2 data. The results show that synthetic images derived from multispectral indexes (such as NDVI and MDWI) are more accurate than synthetic images derived from single bands. In addition, different temporal reduction statistics affect the detection of the active channel, and we suggest using the 90th percentile instead of the median to improve the detection of vegetated areas. Individual representative images are then aggregated into multitemporal maps to define a systematic and easily replicable approach for extracting active river corridors and their inherent spatial and temporal dynamics. Finally, the proposed procedure has the potential to be easily implemented and automated as a tool to provide relevant data to river managers.

The use of blockchain technology in law enforcement

This article explores the possibilities of using blockchain technology in police work. Examples of the use of blockchain in various areas of police activity, such as personal data management, control of drug trafficking and other prohibited substances, traffic monitoring and the fight against cybercrime, are considered. The authors note that thanks to the storage of data in the blockchain, it becomes possible to increase the protection of the confidentiality of personal information, ensure transparency and efficiency of police work, as well as prevent fraud and corruption. The conclusion of the article emphasizes that the use of blockchain can improve the work of the police and ensure greater security of citizens. Distributed ledger technology, or blockchain as a service (BaaS) is indeed a relatively new product on the market that allows you to provide blockchain services for corporate clients. This solution allows you to use more reliable and secure methods of data processing and transaction management within the organization. All these economic effects can lead to a reduction in costs and an increase in the efficiency of the police. In general, the use of blockchain technology in the police can have a number of advantages, such as increasing transparency and accountability, reducing data processing time and combating data falsification. However, it is necessary to take into account some risks, such as the possibility of data privacy violations, as well as difficulties in integrating with existing systems and training personnel. In general, the introduction of blockchain technology into the police requires careful analysis and an approach that takes into account all aspects of the use of technology and its impact on the organization.