Data Analysis Algorithms Research Articles

Deep Learning-Based Analysis of Emotional and Content Relevance Between Bullet Screens and Subtitles as Movie Narrative Medium

This study borrows elements of film text in the field of film studies to develop the potential of data analysis using AI algorithms, while aiming to explore the emotional and content relevances between the two narrative mediums in film, bullet screens and subtitles, as well as the narrative effect variations that they bring about. The film sample Forever Young was analyzed by deep learning, data analysis, and knowledge discovery algorithms and tools. The BERT model was used to quantify the intensity of emotions and conduct fine-grained emotional classification of film characters. Emotion curves are plotted based on time series data. Thematic words are extracted by the LDA algorithm, and text similarity is computed using Word2Vec. The study found differences in emotional expression between bullet screens and subtitles, and they also exhibit strong emotional and content relevances. Yet, this correlation is not only influenced by temporal deviation factors but also by specific emotional types, different stages of story development, and the collective emotions of online views. The conclusion suggests that the effective collection of plot metronomes in film narratives helps reveal crucial plot nodes and emotional trends in films that cause obvious emotional changes, and placing them appropriately is fundamental for continuously attracting and retaining viewers’ attention. The interaction between bullet comments and subtitles is not only about emotional expression but also enhances the effectiveness of the film narrative. Overall, this study demonstrates the importance of database logic and technology in understanding and expressing film emotions and meanings, and providing new ideas and methods for enhancing film narrative effectiveness.

PLAN: Variance-Aware Private Mean Estimation

Differentially private mean estimation is an important building block in privacy-preserving algorithms for data analysis and machine learning. Though the trade-off between privacy and utility is well understood in the worst case, many datasets exhibit structure that could potentially be exploited to yield better algorithms. In this paper we present Private Limit Adopted Noise (PLAN), a family of differentially private algorithms for mean estimation in the setting where inputs are independently sampled from a distribution D over R^d, with coordinate-wise standard deviations σ in R^d. Similar to mean estimation under Mahalanobis distance, PLAN tailors the shape of the noise to the shape of the data, but unlike previous algorithms the privacy budget is spent non-uniformly over the coordinates. Under a concentration assumption on D, we show how to exploit skew in the vector σ, obtaining a (zero-concentrated) differentially private mean estimate with l_2 error proportional to ||σ||_1. Previous work has either not taken σ into account, or measured error in Mahalanobis distance --- in both cases resulting in l_2 error proportional to sqrt{d}||σ||_2, which can be up to a factor sqrt{d} larger. To verify the effectiveness of PLAN, we empirically evaluate accuracy on both synthetic and real-world data.

Implementation of an Artificial Intelligence Based Learning Management System for Adaptive Learning

In the world of modern education, the need for adaptive and personalized learning systems is becoming increasingly important to accommodate the unique learning needs of each individual. Implementing artificial intelligence (AI) in learning management systems offers great potential for optimizing adaptive learning experiences, where the system automatically adjusts learning content and approaches based on student profiles and responses. This research aims to develop and implement an artificial intelligence-based learning management system to personalize learning. The primary objective is to assess the effectiveness of the system in improving student engagement and learning outcomes and identify factors that influence its success. The method includes developing a system using machine learning algorithms for real-time analysis of student learning data. This research uses a mixed design that combines quantitative and qualitative data to analyze learning outcomes and understand user perceptions of the system. The research results show that this AI-based learning management system successfully increases student engagement and provides effective personalization of learning. Data shows improvements in grades and understanding of the material, especially in concepts students previously found difficult. Implementing an artificial intelligence-based learning management system has proven effective in supporting adaptive learning. This system improves learning outcomes and provides new insights into how AI technology can effectively integrate into educational contexts to support diverse and dynamic learning needs. The research also identified several challenges, including the need for more significant and varied training data and smoother integration with school curricula.

Fragment-Fusion Transformer: Deep Learning-Based Discretization Method for Continuous Single-Cell Raman Spectral Analysis.

Raman spectroscopy has become an important single-cell analysis tool for monitoring biochemical changes at the cellular level. However, Raman spectral data, typically presented as continuous data with high-dimensional characteristics, is distinct from discrete sequences, which limits the application of deep learning-based algorithms in data analysis due to the lack of discretization. Herein, a model called fragment-fusion transformer is proposed, which integrates the discrete fragmentation of continuous spectra based on their intrinsic characteristics with the extraction of intrafragment features and the fusion of interfragment features. The model integrates the intrinsic feature-based fragmentation of spectra with transformer, constructing the fragment transformer block for feature extraction within fragments. Interfragment information is combined through the pyramid design structure to improve the model's receptive field and fully exploit the spectral properties. During the pyramidal fusion process, the information gain of the final extracted features in the spectrum has been enhanced by a factor of 9.24 compared to the feature extraction stage within the fragment, and the information entropy has been enhanced by a factor of 13. The fragment-fusion transformer achieved a spectral recognition accuracy of 94.5%, which is 4% higher compared to the method without fragmentation and fusion processes on the test set of cell Raman spectroscopy identification experiments. In comparison to common spectral classification models such as KNN, SVM, logistic regression, and CNN, fragment-fusion transformer has achieved 4.4% higher accuracy than the best-performing CNN model. Fragment-fusion transformer method has the potential to serve as a general framework for discretization in the field of continuous spectral data analysis and as a research tool for analyzing the intrinsic information within spectra.

Caching or not: An online cost optimization algorithm for geodistributed data analysis in cloud environments

With the wide application of big data technology, a large number of data geographically stored in data centers across various regions are generated everyday, waiting to be analyzed by big data tasks. Examples of such data analysis tasks include weather prediction and intelligent healthcare applications. Clouds are being used by more and more enterprises due to their nearly infinite resources, ease of scaling, and other characteristics. Organizations often rent StaaS (Storage-as-a-Service) storage products offered by cloud providers, such as OSS (Object Storage Service), for massive data storage, while building a big data cluster in cloud environments to analyze the collected data from various regions. However, when the data to be analyzed are not located in the rented cluster, how to efficiently and economically process the distributed input data stored in clouds becomes an urgent problem to be solved. A simple approach is to only cache frequently accessed data rather than all data from other regions into the cluster to reduce total traffic costs. However, it is generally very hard to predict future data access curves. Thus, a rash caching decision may incur more costs. To address this problem, in this paper we propose an online algorithm for guiding cloud users to make cost-effective caching decisions properly, while not requiring any future information. We prove theoretically that the competitive ratio of our online algorithm is less than 2. Finally we verify the effectiveness of our proposed algorithm through extensive experiments based on the real price of Alibaba’s public IaaS cloud products using both real-world Yahoo S2 data and synthesized datasets.

Robust Quantification of Live-Cell Single-Molecule Tracking Data for Fluorophores with Different Photophysical Properties.

High-speed single-molecule tracking in live cells is becoming an increasingly popular method for quantifying the spatiotemporal behavior of proteins in vivo. The method provides a wealth of quantitative information, but users need to be aware of biases that can skew estimates of molecular mobilities. The range of suitable fluorophores for live-cell single-molecule imaging has grown substantially over the past few years, but it remains unclear to what extent differences in photophysical properties introduce biases. Here, we tested two fluorophores with entirely different photophysical properties, one that photoswitches frequently between bright and dark states (TMR) and one that shows exceptional photostability without photoswitching (JFX650). We used a fusion of the Escherichia coli DNA repair enzyme MutS to the HaloTag and optimized sample preparation and imaging conditions for both types of fluorophore. We then assessed the reliability of two common data analysis algorithms, mean-square displacement (MSD) analysis and Hidden Markov Modeling (HMM), to estimate the diffusion coefficients and fractions of MutS molecules in different states of motion. We introduce a simple approach that removes discrepancies in the data analyses and show that both algorithms yield consistent results, regardless of the fluorophore used. Nevertheless, each dye has its own strengths and weaknesses, with TMR being more suitable for sampling the diffusive behavior of many molecules, while JFX650 enables prolonged observation of only a few molecules per cell. These characterizations and recommendations should help to standardize measurements for increased reproducibility and comparability across studies.

APIR: Aggregating Universal Proteomics Database Search Algorithms for Peptide Identification with FDR Control.

Advances in mass spectrometry (MS) have enabled high-throughput analysis of proteomes in biological systems. The state-of-the-art MS data analysis relies on database search algorithms to quantify proteins by identifying peptide-spectrum matches (PSMs), which convert mass spectra to peptide sequences. Different database search algorithms use distinct search strategies and thus may identify unique PSMs. However, no existing approaches can aggregate all user-specified database search algorithms with a guaranteed increase in the number of identified peptides and a control on the false discovery rate (FDR). To fill in this gap, we proposed a statistical framework, Aggregation of Peptide Identification Results (APIR), that is universally compatible with all database search algorithms. Notably, under an FDR threshold, APIR is guaranteed to identify at least as many, if not more, peptides as individual database search algorithms do. Evaluation of APIR on a complex proteomics standard dataset showed that APIR outpowers individual database search algorithms and empirically controls the FDR. Real data studies showed that APIR can identify disease-related proteins and post-translational modifications missed by some individual database search algorithms. The APIR framework is easily extendable to aggregating discoveries made by multiple algorithms in other high-throughput biomedical data analysis, e.g., differential gene expression analysis on RNA sequencing data. The APIR R package is available at https://github.com/yiling0210/APIR.

Sarcopenia diagnosis in patients with colorectal cancer: a review of computed tomography-based assessments and emerging ways to enhance practicality.

Traditionally, cancer treatment has focused on the stages of the disease; however, recent studies have highlighted the importance of considering the overall health status of patients in the prognosis of cancer. Loss of skeletal muscle, known as sarcopenia, has been found to significantly affect outcomes in many different types of cancers, including colorectal cancer. In this review, we discuss the guidelines for diagnosing sarcopenia, with a specific focus on CT-based assessments. Many groups worldwide, including those in Europe and Asia, have introduced their own diagnostic guidelines for sarcopenia. Seemingly similar yet subtle discrepancies, particularly in the cutoff values used, limit the use of these guidelines in the general population, warranting a more universal guideline. Although CT-based measurements, such as skeletal muscle index and radiodensity, have shown promise in predicting outcomes, the lack of standardized values in these measurements hinders their universal adoption. To overcome these limitations, innovative approaches are being developed to assess changes in muscle mass trajectories and introduce new indices, such as skeletal and appendicular muscle gauges. Additionally, machine learning models have shown superior performance in predicting sarcopenic status, providing an alternative to CT-based diagnosis, particularly after surgery. CT has tremendous benefits and a significant role in visually as well as quantitatively retrieving information on patient body composition. In order to compensate for the limitation of standard cutoff value, 3-dimensional analysis of the CT, artificial intelligence-based body composition analysis, as well as machine learning algorithms for data interpretation and analysis have been proposed and are being utilized. In conclusion, despite the varying definitions of sarcopenia, CT-based measurements coupled with machine-learning models are promising for evaluating patients with cancer. Standardization efforts can improve diagnostic accuracy, reduce the reliance on CT examinations, and make sarcopenia assessments more accessible in clinical settings.

Anomaly‐detection‐based learning for real‐time data processing in non‐intrusive load monitoring

AbstractA power system can be regarded as a cyber‐physical system with physical power networks and a cyber system based on increasing engagement with information communication technologies for smart grid functionalities for more efficient operations and control. Non‐intrusive load monitoring (NILM), an emerging smart‐grid technology, can be used to better understand the electricity usage profile and composition of smart meters using advanced data analysis algorithms. Although NILM enables various smart grid services, wider applications of NILM require addressing the challenges regarding cyber security and data privacy risks. Anomaly detection in appliance data is one of the most effective measures against potential cyber intrusions from a data perspective. This study proposes a framework of anomaly detection‐based learning algorithms to identify the anomalous periods of electricity loading data, which may be a subject for potential cyber‐attacks. Comparison studies with the hidden Markov model are performed to validate the proposed approaches. The simulation results show that these anomaly detection‐based learning algorithms work well and can precisely determine anomalous loading periods. Moreover, these trained models perform well on the testing dataset without prior knowledge of the data, providing the possibility of the real‐time assessment of power‐ loading states. The proposed framework can also be used to develop protective measures to ensure secure system operation and user data privacy.

Unveiling Novel Structural Biomarkers for the Diagnosis of Glaucoma.

Glaucoma, a leading cause of irreversible blindness, poses a significant global health burden. Early detection is crucial for effective management and prevention of vision loss. This study presents a collection of novel structural biomarkers in glaucoma diagnosis. By employing advanced imaging techniques and data analysis algorithms, we now can recognize indicators of glaucomatous progression. Many research studies have revealed a correlation between the structural changes in the eye or brain, particularly in the optic nerve head and retinal nerve fiber layer, and the progression of glaucoma. These biomarkers demonstrate value in distinguishing glaucomatous eyes from healthy ones, even in the early stages of the disease. By facilitating timely detection and monitoring, they hold the potential to mitigate vision impairment and improve patient outcomes. This study marks an advancement in the field of glaucoma, offering a promising avenue for enhancing the diagnosis and possible management.

The Effect of Theory of Planned Behavior (TBP) and Creativity-Based Industry Perception on Digital Entrepreneurship: An Innovativeness as Mediator

This research delves into the factors influencing digital entrepreneurial intention using the Smart PLS algorithm for data analysis. The analysis results strongly support the proposed hypotheses, with indicator values above 0.6 indicating the reliability of variables in the context of digital entrepreneurial intention. The main findings affirm that innovation plays a key mediating role in connecting attitude, subjective norm, and self-efficacy to digital entrepreneurial intention. It is also found that excessive digital entrepreneurial self-efficacy can hinder innovation, and a positive industry perception does not always correlate with high levels of innovation. Creativity proves to be a key element, positively impacting digital entrepreneurial intention and innovation. Other factors, such as planned behavior theory and digital literacy, also contribute positively to digital entrepreneurial intention and innovation. This research provides in-depth insights into the complex interaction of key factors in shaping attitudes and digital entrepreneurial intention. This conclusion has significant implications for the development of a dynamic and innovative digital entrepreneurial ecosystem.

The Role of Adsorption Phenomena in ac Conductivity Measurements of Dielectric Nanoparticle Suspensions

The authors of numerous measurements of conductivity for suspensions of nanoparticles of various types carried out in the last decade came to the general conclusion that the well-known Maxwell’s theory is not applicable to quantitative explanation of the properties of such systems. In the present work, we demonstrate that the Maxwell’s theory can be still applicable even for such systems, but the specifics of the standard ac measurements have to be correctly taken into account. Namely, the dependence of the capacitance of “metal–electrolyte” cuvette boundaries on nanoparticle adsorption, which in its turn dramatically depends on nanoparticle concentration, has to be taken into account. The latter circumstance strongly (via the RC characteristic of the circuit) affects the results of conductivity measurements. We propose the new algorithm of the impedance measurement data analysis for the particles’ concentration dependence of conductivity σ(ϕ) (where ϕ is the volume fraction of nanoparticles in suspension) for the suspensions of the diamante nanoparticles in alcohol which demonstrates the adequate correspondence of the Maxwell’s theory to the available experimental findings.

Risk of Tree Fall on High-Traffic Roads: A Case Study of the S6 in Poland

Modern technologies, such as airborne laser scanning (ALS) and advanced data analysis algorithms, allow for the efficient and safe use of resources to protect infrastructure from potential threats. This publication presents a study to identify trees that may fall on highways. The study used free measurement data from airborne laser scanning and wind speed and direction data from the Institute of Meteorology and Water Management in Poland. Two methods were used to determine the crown tops of trees: PyCrown and OPALS. The effect of wind direction on potential hazards was then analyzed. The OPALS method showed the best performance in terms of detecting trees, with an accuracy of 74%. The analysis showed that the most common winds clustered between 260° and 290°. Potential threats, i.e., trees that could fall on the road, were selected. As a result of the analysis, OPALS detected between 140 and 577 trees, depending on the chosen strategy. The presented research shows that combining ALS technology with advanced algorithms and wind data can be an effective tool for identifying potential hazards associated with falling trees on highways.

Influence of the Development of Internet Big Data on College Students' Music Education

Under the background of globalisation and diversification, how to construct a music culture with national characteristics and develop music education with both diversity and localisation is a major issue that music education theory and research in China must face up to. This study begins with a theoretical overview and uses big data analysis to construct a framework for teaching localisation. Then the localisation methods of educational concepts and practical operations are sorted out, and the construction of the localised teaching model of Orff music pedagogy is introduced. Finally, this paper describes the big data analysis of the localised teaching model in terms of cluster trend analysis and determining the number of clusters analysis. The results show that the Chinese music pedagogy of Orff is realised on the basis of improving students' music knowledge and achieving their all-round development. The experimental data effectively improved the speed and accuracy of the big data analysis algorithm. This study is significant for the localised music education system in China.

Variable Neighborhood Search Based Human Learning Optimization Algorithm for Secure Data Analysis and Computing

Variable Neighborhood Search Based Human Learning Optimization Algorithm for Secure Data Analysis and Computing

Applying a two-step cluster algorithm in traffic accident data analysis

Cluster analysis is often employed as the initial stage in organizing heterogeneous data into homogeneous groups. Choosing an effective clustering approach and an ideal number of clusters in a traffic accident dataset might be complex and challenging. This study aims to evaluate the effectiveness of k-means and two-step cluster methods. Subsequently, the two-step cluster method and GIS are applied to analyze the traffic accident datasets from 2015 to 2017 in Hanoi, Vietnam. First, according to the Silhouette score, the two-step cluster method achieved a higher score of 0.563, while the k-means method scored 0.341. A higher Silhouette score indicates more well-defined clusters. Second, the research suggests combining the two-step cluster method with GIS for analyzing traffic accident datasets. The outcome identifies five typical types of accidents in Hanoi. In addition, the locations of various accident types were visually illustrated on a map, enabling traffic officials to recommend precise and urgent countermeasures. Importantly, the clustering results reveal that the two-step cluster method exhibits a significantly higher rate of homogeneous data in the clusters compared to the k-means method. This study demonstrates that the two-step cluster method is not only more effective than the k-means method in terms of clustering ability but also in data pre-processing. The study's results enable authorities to gain a more detailed understanding of typical traffic accident patterns in Hanoi. Besides, the employed methods could potentially be applied to other regions, providing an additional avenue for analysis

Designing a semi-automated decision-making system for selecting recipients of social services

The object of this research is the decision-making process in the context of selecting recipients of social services using a semi-automated expert system. The main focus is on improving the mechanisms of assessment and selection of candidates eligible for assistance, in order to ensure a more efficient and objective allocation of resources. The problem addressed in this study is the need to improve the accuracy and efficiency of decision-making processes in social services through the implementation of semi-automated systems. In particular, reducing subjective influence in selection processes, as well as reducing the time and resources required to process applications. The study shows that the introduction of a semi-automated system can significantly reduce the response time to applications, increase the accuracy of candidate selection and ensure greater transparency in the decision-making process. The system, based on data analysis algorithms and production rules, is able to adapt to changing conditions and requirements, providing solutions based on up-to-date information. The effectiveness of the semi-automated system is due to the use of modern technologies for processing large volumes of data and the use of complex mathematical models for the analysis of this data. The implementation of a modular system with individually adjustable parameters allows the system to accurately evaluate each case based on the expected criteria, ensuring a high level of adaptability and accuracy. The results of the research can be applied in practice in various social security institutions, where there is a need to automate the processes of selection and decision-making. Important conditions for the effective implementation of the system are the availability of sufficient technical support, a high level of qualification of the personnel engaged in putting the system into operation, as well as a clear understanding of the rules and procedures that regulate social protection. In addition, to guarantee the successful application of the system, it is necessary to ensure compliance with all regulatory and legislative requirements, especially regarding the protection of personal data.

LoRa Radio Frequency Fingerprinting with Residual of Variational Mode Decomposition and Hybrid Machine-Learning/Deep-Learning Optimization

Radio Frequency Fingerprinting (RFF) refers to the technique for identifying and classifying wireless devices on the basis of their physical characteristics, which appear in the digital signal transmitted in space. Small differences in the radio frequency front-end of the wireless devices are generated across the same wireless device model during the implementation and manufacturing process. These differences create small variations in the transmitted signal, even if the wireless device is still compliant with the wireless standard. By using data analysis and machine-learning algorithms, it is possible to classify different electronic devices on the basis of these variations. This technique has been well proven in the literature, but research is continuing to improve the classification performance, robustness to noise, and computing efficiency. Recently, Deep Learning (DL) has been applied to RFF with considerable success. In particular, the combination of time-frequency representations and Convolutional Neural Networks (CNN) has been particularly effective, but this comes at a great computational cost because of the size of the time-frequency representation and the computing time of CNN. This problem is particularly challenging for wireless standards, where the data to be analyzed is extensive (e.g., long preambles) as in the case of the LoRa (Long Range) wireless standard. This paper proposes a novel approach where two pre-processing steps are adopted to (1) improve the classification performance and (2) to decrease the computing time. The steps are based on the application of Variational Mode Decomposition (VMD) where (in opposition to the known literature) the residual of the VMD application is used instead of the extracted modes. The concept is to remove the modes, which are common among the LoRa devices, and keep with the residuals the unique intrinsic features, which are related to the fingerprints. Then, the spectrogram is applied to the residual component. Even after this step, the computing complexity of applying CNN to the spectrogram is high. This paper proposes a novel step where only segments of the spectrogram are used as input to CNN. The segments are selected using a machine-learning approach applied to the features extracted from the spectrogram using the Local Binary Pattern (LBP). The approach is applied to a recent LoRa radio frequency fingerprinting public data set, where it is shown to significantly outperform the baseline approach based on the full use of the spectrogram of the original signal in terms of both classification performance and computing complexity.

Anemia detection and classification from blood samples using data analysis and deep learning*

This study aims to examine the possibility and impact of utilizing data science on blood samples to rapidly and proactively identify underlying health issues. By utilizing effective algorithms, models will be constructed to address these problems and determine potential healthcare options based on geographical location. Once data is gathered, health officials will be notified of major diseases and individuals at risk or already affected. Authentic blood samples are used to ensure the credibility and validity of the proposed system. The data was collected during a volunteer-led hemoglobin blood test camp specifically for women residing in impoverished areas, resulting in a total of 551 samples. The effectiveness of this technique has been assessed through experimental results based on Hb, RDW%, MCV, RBC, and M-Index. The proposed data analysis and deep learning algorithm achieved average values of haemoglobin count 11.67 g/dL with a 1.33 standard deviation, RDW 14.59%, MCV 81.45, RBC 4.37 per microliter with a variance of 0.5, and M-Index 19.56. The experimental results achieved 97.60% accuracy, demonstrating the effectiveness of the proposed technique for classifying anemia and its subtypes. The experimental results indicate better overlap between the automated identification of anemia and manual detection by the experts.

Gravity profiles interpretation applying a metaheuristic particle optimization algorithm of mineralized bodies resembled by finite elements

The interpretation of gravity anomalies is crucial for identifying subsurface mineralized targets and understanding the density variations between the targets and the surrounding structures. To confirm the presence of ore and mineral targets, simple geometric bodies are often used. One of the commonly used global metaheuristic algorithms for gravity data analysis is the particle optimization algorithm. In this study, we employed this method to determine the parameters of buried bodies that resemble finite vertical cylinders by inferring gravity anomalies profiles (amplitude coefficient, depth to top, depth to bottom, origin, and length of the target representing the difference between two depths). The algorithm utilizes particle movement to identify the best way to reach the global or optimum solution. The algorithm's performance was evaluated on synthetic-examples with and without noise (5 % and 10 % levels) and also verified on a real dataset for mineral exploration from Canada. The results showed that the algorithm's stability and accuracy were not affected by the presence of noise and multi-models. Moreover, the field case results were consistent with the existing geological information, borehole data, and previously published outcomes.