Automated Data Analysis Research Articles

Digital image processing (DIP) applied to the analysis of the use and coverage of São Sebastião, federal district, Brazil

The article explores the application of Digital Image Processing (DIP) and Remote Sensing (RS) techniques in analyzing land use and land cover changes in the Administrative Region of São Sebastião, Federal District, Brazil. DIP plays a pivotal role in automated data analysis and interpretation, particularly in the context of RS, where it efficiently processes Earth's surface data collected by sensors on satellites, aircraft, and Unmanned Aerial Vehicles (UAVs). The study covers the period from 2000 to 2020, utilizing data from satellites such as Landsat to provide a continuous view of the region. The spatial and temporal analyses incorporate considerations of hypsometry, slope, and surface temperature, aiming to understand the dynamics of land use and its impact on the microclimate. The methodology involves a comprehensive literature review, acquisition of spatial data from government agencies, and the use of computational tools such as ArcGIS and ENVI. Classification of Landsat images employs false-color compositions and Maximum Likelihood Classification to identify urban areas, water bodies, vegetation, shrubland, and exposed soil. The analysis also includes generating land surface temperature maps. Results indicate significant urban expansion, particularly in the Southeast direction, with a concentration of urban activities. The study highlights a continuous growth trend over the years, emphasizing the importance of understanding and monitoring these transformations for sustainable planning. The article contributes to the field by providing valuable insights into the dynamics of urban growth, offering a clear spatial distribution of changes over time. The detailed methodology ensures the accuracy and reliability of the analysis, supporting informed decision-making in the context of environmental exploration and regional development planning.

REKP: Refined External Knowledge into Prompt-Tuning for Few-Shot Text Classification

Text classification is a machine learning technique employed to assign a given text to predefined categories, facilitating the automatic analysis and processing of textual data. However, an important problem is that the number of new text categories is growing faster than that of human annotation data, which makes many new categories of text data lack a lot of annotation data. As a result, the conventional deep neural network is forced to over-fit, which damages the application in the real world. As a solution to this problem, academics recommend addressing data scarcity through few-shot learning. One of the efficient methods is prompt-tuning, which transforms the input text into a mask prediction problem featuring [MASK]. By utilizing descriptors, the model maps output words to labels, enabling accurate prediction. Nevertheless, the previous prompt-based adaption approaches often relied on manually produced verbalizers or a single label to represent the entire label vocabulary, which makes the mapping granularity low, resulting in words not being accurately mapped to their label. To address these issues, we propose to enhance the verbalizer and construct the refined external knowledge into a prompt-tuning (REKP) model. We employ the external knowledge bases to increase the mapping space of tagged terms and design three refinement methods to remove noise data. We conduct comprehensive experiments on four benchmark datasets, namely AG’s News, Yahoo, IMDB, and Amazon. The results demonstrate that REKP can outperform the state-of-the-art baselines in terms of Micro-F1 on knowledge-enhanced text classification. In addition, we conduct an ablation study to ascertain the functionality of each module in our model, revealing that the refinement module significantly contributes to enhancing classification accuracy.

Process monitoring and automated data analysis in 3D printing

The Fused Layer Modeling (FLM) 3D printing process becomes more and more common in industrial applications. In an environment of rapid production and customer services, the printed parts need to be quality proved for a save and reliable usage. In order to achieve this without expensive post processing steps, the monitoring and analysis of the printing process is necessary. This work presents the results of feasibility studies for different low cost sensors to monitor the FLM printing process and the development of a data analysis system for evaluating the printing process. The process should be sufficiently monitored to avoid costly ensuing part testing.

End-to-end system for monitoring the state of rivers using a drone

Automatic data analysis systems in the Internet of Things are a key element. One such case is the use of drones to monitor rivers, which are quite often located around built-up areas. This is an important element for the analysis of urban areas that are exposed to various environmental challenges such as pollution and animal habitats. Data analysis allows the detection of anomalies in the state of rivers, reducing the risk of ecological disasters or even floods. Additionally, constant control of areas enables analysis of the impact of urbanization on a given area as well as environmental protection. In this paper, we propose an end-to-end system, where the user performs measurements with a drone and the result is a segmentation mask from the U-Net network, but improved by image processing algorithms. The system is based on performing segmentation with a neural network, imposing the obtained mask on the image that was previously subjected to edge detection. All pixels under the mask are analyzed by the clustering method in terms of belonging to a river or bank. In addition, when there are other measurements from the same area, they are used to compare and analyze changes. The proposed system architecture is based on the automation of activities due to the combination of various graphics processing methods. Moreover, the method allows for obtaining more accurate segmentation results than classic methods. The proposition was tested on data gathered near river areas in southern Poland to show the possibilities and effectiveness of the system. Proposed methodology reached 0.8524 of Dice coefficient using VGG16 as encoder.

Autonomous GIS: the next-generation AI-powered GIS

ABSTRACTLarge Language Models (LLMs), such as ChatGPT, demonstrate a strong understanding of human natural language and have been explored and applied in various fields, including reasoning, creative writing, code generation, translation, and information retrieval. By adopting LLM as the reasoning core, we introduce Autonomous GIS as an AI-powered geographic information system (GIS) that leverages the LLM's general abilities in natural language understanding, reasoning, and coding for addressing spatial problems with automatic spatial data collection, analysis, and visualization. We envision that autonomous GIS will need to achieve five autonomous goals: self-generating, self-organizing, self-verifying, self-executing, and self-growing. We developed a prototype system called LLM-Geo using the GPT-4 API, demonstrating what an autonomous GIS looks like and how it delivers expected results without human intervention using three case studies. For all case studies, LLM-Geo returned accurate results, including aggregated numbers, graphs, and maps.. Although still in its infancy and lacking several important modules such as logging and code testing, LLM-Geo demonstrates a potential path toward the next-generation AI-powered GIS. We advocate for the GIScience community to devote more efforts to the research and development of autonomous GIS, making spatial analysis easier, faster, and more accessible to a broader audience.

Software pipelines for RNA-Seq, ChIP-Seq and germline variant calling analyses in common workflow language (CWL).

Background: Automating data analysis pipelines is a key requirement to ensure reproducibility of results, especially when dealing with large volumes of data. Here we assembled automated pipelines for the analysis of High-throughput Sequencing (HTS) data originating from RNA-Seq, ChIP-Seq and Germline variant calling experiments. We implemented these workflows in Common workflow language (CWL) and evaluated their performance by: i) reproducing the results of two previously published studies on Chronic Lymphocytic Leukemia (CLL), and ii) analyzing whole genome sequencing data from four Genome in a Bottle Consortium (GIAB) samples, comparing the detected variants against their respective golden standard truth sets. Findings: We demonstrated that CWL-implemented workflows clearly achieved high accuracy in reproducing previously published results, discovering significant biomarkers and detecting germline SNP and small INDEL variants. Conclusion: CWL pipelines are characterized by reproducibility and reusability; combined with containerization, they provide the ability to overcome issues of software incompatibility and laborious configuration requirements. In addition, they are flexible and can be used immediately or adapted to the specific needs of an experiment or study. The CWL-based workflows developed in this study, along with version information for all software tools, are publicly available on GitHub (https://github.com/BiodataAnalysisGroup/CWL_HTS_pipelines) under the MIT License. They are suitable for the analysis of short-read (such as Illumina-based) data and constitute an open resource that can facilitate automation, reproducibility and cross-platform compatibility for standard bioinformatic analyses.

Materials characterization: Can artificial intelligence be used to address reproducibility challenges?

Material characterization techniques are widely used to characterize the physical and chemical properties of materials at the nanoscale and, thus, play central roles in material scientific discoveries. However, the large and complex datasets generated by these techniques often require significant human effort to interpret and extract meaningful physicochemical insights. Artificial intelligence (AI) techniques such as machine learning (ML) have the potential to improve the efficiency and accuracy of surface analysis by automating data analysis and interpretation. In this perspective paper, we review the current role of AI in surface analysis and discuss its future potential to accelerate discoveries in surface science, materials science, and interface science. We highlight several applications where AI has already been used to analyze surface analysis data, including the identification of crystal structures from XRD data, analysis of XPS spectra for surface composition, and the interpretation of TEM and SEM images for particle morphology and size. We also discuss the challenges and opportunities associated with the integration of AI into surface analysis workflows. These include the need for large and diverse datasets for training ML models, the importance of feature selection and representation, and the potential for ML to enable new insights and discoveries by identifying patterns and relationships in complex datasets. Most importantly, AI analyzed data must not just find the best mathematical description of the data, but it must find the most physical and chemically meaningful results. In addition, the need for reproducibility in scientific research has become increasingly important in recent years. The advancement of AI, including both conventional and the increasing popular deep learning, is showing promise in addressing those challenges by enabling the execution and verification of scientific progress. By training models on large experimental datasets and providing automated analysis and data interpretation, AI can help to ensure that scientific results are reproducible and reliable. Although integration of knowledge and AI models must be considered for the transparency and interpretability of models, the incorporation of AI into the data collection and processing workflow will significantly enhance the efficiency and accuracy of various surface analysis techniques and deepen our understanding at an accelerated pace.

Comparison of supervised algorithms for automated data analysis in existing buildings

Non-residential buildings are equipped with building automation systems (BAS). BAS provide a vast amount of building operation data. However, the analysis of this data is laborious and requires a scalable and automated process. The authors developed the BUDO schema for a standardized, object-oriented description of monitoring metadata. The translation process employs artificial intelligence to translate the properties of data streams (labels, time series) from BAS into the BUDO schema. The translated data is then analyzed using various applications to detect inefficiencies, such as incorrect placement of outdoor temperature sensors or a weekend shutdown that has not been properly implemented. The paper presents a process that automatically translates, structures and analyzes operational data to identify potential for energy savings. The OOM4ABDO project monitors 138 buildings to analyze and improve building operations, whose subset of data we are using here. In addition, we use an open source data set of about 70 buildings. We compare several feature extraction methods and classifiers on their ability to support each of the stages (label translation, time series translation, application). The algorithms used reach F1 scores of up to 98% for label translation, 90% for time series translation and 92% for applications.

The application of WeChat navigation assistant

The field of artificial intelligence and deep learning has made significant progress in recent years, with the development of innovative techniques and technologies that have the potential to improve various aspects of human life. However, the elderly population faces significant challenges when it comes to utilizing electronic devices, as the demand for convenient and easy-to-use products in their daily lives remains unmet. To address this challenge, this paper proposes the design of a software solution, consisting of four key components. Firstly, the software application is simulated, and the process of software operation when the elderly use WeChat is modeled. Secondly, automatic speech recognition technology is utilized to enhance the usability of WeChat for the elderly. Thirdly, a dataset is created for data collection, processing, and analysis, with the aim of constantly improving the software. Finally, evaluation methods such as formative and summative evaluations are utilized to assess and enhance the effectiveness of the software. The proposed software solution has the potential to significantly improve the quality of life of the elderly population by enabling them to better access and utilize electronic devices. Moreover, the incorporation of automatic speech recognition technology and data analysis for continuous improvement has the potential to contribute to the advancement of the field of artificial intelligence and deep learning. Further research should focus on refining the software solution to better cater to the specific needs of the elderly population.

Digital Patient Reported Outcome Measures Platform for Post-COVID-19 Condition and Other Long-Term Conditions: User-Centered Development and Technical Description.

Post-COVID-19 condition (PCC), colloquially known as long COVID, is a multisystem condition characterized by persistent symptoms beyond 4 weeks after the SARS-CoV-2 infection. More than 60 million people with PCC worldwide need prompt assessment, diagnosis, and monitoring, with many requiring specialist help from a multidisciplinary team of health care professionals (HCPs). Consequently, a scalable digital system is required for both people with PCC and HCPs to capture the breadth of symptoms and their impact on health, using patient-reported outcome measures (PROMs) and patient-reported experience measures (PREMs). We aim to develop and implement a novel PCC digital PROM (DPROM) platform for (1) securely collecting PROM and PREM data from people with PCC, (2) enabling users to monitor symptoms longitudinally and assess response to treatment, (3) generating reports for the electronic health records (EHRs), (4) providing summary reports on PCC services based on national requirements, and (5) facilitating the sharing of relevant data with authorized research teams to accelerate our understanding of this new condition and evaluate new strategies to manage PCC. We (1) undertook requirement analysis with people with PCC, HCPs, and researchers to identify the needs of the DPROM platform and determine its required functionalities; (2) designed and developed a clinically useful web portal for staff and a mobile app for patients, with a web-based alternative app to improve patient and staff choice, limit the risk of digital exclusion, and account for variability across services; (3) determined the PROMs and PREMs that PCC services would prefer to use on the platform; and (4) designed the summary report function that can be generated for each user for the EHR and for reporting to national health authorities. A DPROM platform to record PCC symptom profile, condition severity, functional disability, and quality of life, based on the C19-YRS (Yorkshire Rehabilitation Scale) and other PROMs and PREMs, was developed. Individual-level medical information and details on the COVID-19 illness can be captured systematically. The platform generates easy-to-understand scores, radar plots and line graphs for people with PCC to self-monitor their condition and for HCPs to assess the natural course of the condition and the response to interventions. Clinics can configure a suite of PROMs and PREMs based on their local and national service and commissioning requirements and support research studies which require large-scale data collection on PROMs. The DPROM platform enables automatic aggregate data analysis for services to undertake service evaluation and cost-effectiveness analysis. The DPROM platform generated summary report can be uploaded to the EHRs of people with PCC. A multifunctional DPROM platform to assess, grade, and monitor PCC has been developed. Future research will analyze the system's usability in specialist PCC clinical services and other long-term conditions.

AI in Auditing: A Comprehensive Review of Applications, Benefits and Challenges

The rapid evolution of artificial intelligence (AI) technologies has brought about transformative changes in various industries, and the field of auditing is no exception. This research paper presents a comprehensive review of the integration of AI in auditing practices, highlighting its applications, benefits, and associated challenges. Auditing, a critical process for ensuring the accuracy and reliability of financial information, has traditionally been a labor-intensive and time-consuming endeavor. The emergence of AI technologies, such as machine learning, natural language processing, and data analytics, has revolutionized the way audits are conducted. AI-powered auditing tools offer advanced capabilities for data analysis, pattern recognition, anomaly detection, and risk assessment. These capabilities enhance the effectiveness and efficiency of audits by allowing auditors to focus on high-risk areas and perform more in-depth analysis. The paper explores various applications of AI in auditing, including: Automated Data Analysis, Predictive Analytics, Fraud Detection, and Natural Language Processing (NLP), Continuous Monitoring. While AI brings significant benefits to the auditing process, its adoption also presents certain challenges like Data Quality and Integration, Interpretability, Ethical Considerations, Technical Expertise, Regulatory Frame Work. In conclusion, AI has the potential to revolutionize auditing practices by enhancing efficiency, accuracy, and risk assessment. However, successful integration requires addressing challenges related to data quality, transparency, ethics, skills, and regulations. As AI technologies continue to evolve, auditors and stakeholders must collaborate to harness the full potential of AI while maintaining the integrity and credibility of the auditing process. This paper serves as a comprehensive resource for auditors, researchers, and policymakers seeking to understand the current landscape and future directions of AI in auditing.

The Observation of Constitutionality on Police Authority in Using Data: Center on the Judgment“Automated Data Analysis”Made by German Federal Constitutional Court

The Observation of Constitutionality on Police Authority in Using Data: Center on the Judgment“Automated Data Analysis”Made by German Federal Constitutional Court

A robust pipeline for high-content, high-throughput immunophenotyping reveals age- and genetics-dependent changes in blood leukocytes

High-dimensional flow cytometry is the gold standard to study the human immune system in large cohorts. However, large sample sizes increase inter-experimental variation because of technical and experimental inaccuracies introduced by batch variability. Our high-throughput sample processing pipeline in combination with 28-color flow cytometry focuses on increased throughput (192 samples/experiment) and high reproducibility. We implemented quality control checkpoints to reduce technical and experimental variation. Finally, we integrated FlowSOM clustering to facilitate automated data analysis and demonstrate the reproducibility of our pipeline in a study with 3,357 samples. We reveal age-associated immune dynamics in 2,300 individuals, signified by decreasing T and B cell subsets with age. In addition, by combining genetic analyses, our approach revealed unique immune signatures associated with a single nucleotide polymorphism (SNP) that abrogates CD45 isoform splicing. In summary, we provide a versatile and reliable high-throughput, flow cytometry-based pipeline for immune discovery and exploration in large cohorts.

P‐IT‐6 | Development of an Efficient, Automated Data Analytics Platform by Integration of Relational Databases

P‐IT‐6 | Development of an Efficient, Automated Data Analytics Platform by Integration of Relational Databases

Research and Design of Automatic Test System for Ventilator Performance

Ventilator is an important medical instrument which can replace the function of autonomous ventilation artificially. Its safety and reliability are related to the health and even life safety of patients. With the publishing of the new national standard and international standard for ventilators, higher requirements are put forward for the detection and evaluation. This study mainly introduces an automatic test system for ventilator performance. The test system is based on PF-300 air-flow analyzer of Imtmedical and standard simulation lung. The automatic switch module of simulation lung is developed, and the automatic test system of ventilator is designed using the software development platform based on Python. It can not only automatically test all ventilation control parameters and monitoring parameters of the ventilator, but also realize automatic data recording, form reports and data analysis, and improve the efficiency and quality of inspection, detection and quality control.

Hyperparameter analysis of wide-kernel CNN architectures in industrial fault detection: an exploratory study

AbstractIndustrial fault detection has become more data-driven due to advancements in automated data analysis using deep learning. Such methods make it possible to extract useful features, e. g., from time series data retrieved from sensors, which is typically of complex nature. This allows for effective fault detection and prognostics that boost the efficiency and productivity of industrial equipment. This work explores the influence of a variety of architectural hyperparameters on the performance of one-dimensional convolutional neural networks (CNN). Using a multi-method approach, this paper focuses specifically on wide-kernel CNN models for industrial fault detection, that have proven to perform well for tasks such as classifying vibration signals retrieved from sensors. By varying hyperparameters such as the kernel size, stride and number of filters, an extensive hyperparameter space search was conducted; to identify optimal settings, we collected a total of 12,960 different combinations on three datasets into a model hyperparameter dataset, with their respective performance on the underlying fault detection task. Afterwards, this dataset was explored with follow-up analysis including statistical, feature, pattern and hyperparameter impact analysis. We find that although performance varies substantially depending on hyperparameter choices, there is no single simple strategy to optimise performance across the three datasets. However, an optimal setting in terms of performance can be found in the number of filters used in the later layers of the architecture for all datasets. Furthermore, hyperparameter importance differs across and within the datasets, and we found nonlinear relationships between hyperparameter settings and performance. Our analysis highlights key considerations when applying a wide-kernel CNN architecture to new data within the field of industrial fault detection. This supports practitioners who wish to apply and train state-of-the-art convolutional learning methods to apply to similar fault detection settings, e. g., vibration data arising from new combinations of sensors and/or machinery in the context of bearing faults.

Concept for a digital twin of railway bridges on the example of the new Filstal bridges

AbstractThe preservation of infrastructure is becoming increasingly important. The current reactive maintenance management only initiates maintenance measures after damage has occurred. Short‐term measures can lead to unforeseen restrictions in infrastructure availability and thus cause high social and economic damage.High‐quality measurement technology, automated data analysis, structured data management and collaborative work are just a few technological developments that form the cornerstones for a future predictive maintenance strategy. The digital twin, a digital representation of the real object, combines these developments and is a promising method for a sustainable preservation of infrastructure.Using the example of the Filstal bridges, a concept for a digital bridge twin is introduced. The digital twin combines measurement data, bridge information, inspection results as well as structural safety verifications and aggregates them to continuously display the current bridge condition and thus supports the maintenance of bridges. For intuitive use, all information is located and presented in a way that is consumable by the user.

Embracing Innovative Approaches in Data Science: Investigating Contemporary Trends in Data Collection, Analysis, and Visualization Methods

Today businesses are becoming more productive and their return on investment (ROI) is increasing with the development of new technologies like data science, artificial intelligence and data analytics. In today's trend organizations are dealing with big data and these data can drive the whole organization in many ways. The process of doing data analysis and extracting meaningful insight is known as data science. Most business organizations are taking data driven models to ease their work and for making intelligent business decisions. The life cycle of a data science involves so many steps like understanding the business, data collection, analysis and data modelling etc., and to achieve these steps various new technologies and methods are available. Firstly, the process of data collection has been significantly augmented by artificial intelligence, allowing businesses to gather vast amounts of structured and unstructured data efficiently. This rich pool of data serves as the foundation upon which strategic decisions are made. By leveraging advanced data collection methods, organizations gain invaluable insights into market trends, customer behaviour, and operational patterns, empowering them to make informed, data-driven decisions. Secondly, data analysis, a core element of data science, plays a pivotal role in extracting meaningful insights from the collected data. Through sophisticated analytical techniques, businesses can uncover hidden patterns, correlations, and trends within the data. This deep understanding of the data not only facilitates efficient problem-solving but also enables the identification of opportunities for innovation and growth. Informed by data analysis, businesses can optimize processes, identify cost-saving measures, and enhance overall operational efficiency. Lastly, data visualization techniques such as real-time visualization and augmented analytics empower organizations to transform complex data sets into easily understandable visual representations. Real-time visualization provides businesses with up-to-the-minute insights, enabling them to respond promptly to market changes and emerging trends. Augmented analytics, on the other hand, leverages machine learning algorithms to automate data analysis and present actionable insights in an intuitive manner, further accelerating the decision-making process. In this study the recent trends in data science like artificial intelligence for data collection, augmented analytics and predictive analysis for data analysis and data democratization & real time visualization techniques for data visualization are discussed in detail. This study also presents the tools, key challenges and applications of these recent methods in brief.

Student Competition (Technology Innovation) ID 1970388

Background The evaluation of hand function after spinal cord injury (SCI) is conducted in clinical settings, which may not accurately reflect hand function in the real world, thereby limiting the efficacy assessment of new treatments. Wearable cameras, also known as egocentric video, are a novel method to evaluate hand function in non-clinical environments. Nonetheless, manual processing of vast quantities of complex video data is difficult, highlighting the need for automated data analysis. The objective of this study was to automatically identify distinct hand postures in egocentric video using unsupervised machine learning. Methods Seventeen participants with cervical SCI recorded activities of daily living in a home simulation laboratory. A hand pose estimation algorithm was applied on detected hands to determine 2D joint locations, which were lifted to 3D coordinates. The resulting hand posture information was subjected to a number of clustering techniques. Hand grasps were manually labelled into four categories for evaluation purposes: power, precision, intermediate, and non-prehensile. Results K-Means clustering consistently exhibited the highest Silhouette score, which reflects the presence of discrete clusters in the data. When comparing with manual annotations, Spectral Clustering applied to a feature space consisting of 2D pose estimation with confidence scores yield the best performance as quantified by maximum match (0.48), Fowlkes-Mallows score (0.46), and normalized mutual information (0.22). Conclusions This is the first attempt to develop an unsupervised, data-driven hand taxonomy for individuals with SCI using wearable technology. The findings suggest that the method is capable of grouping similar hand grasps.

Navigating Regulatory Complexity: Leveraging AI/ML for Accurate Reporting

In today's dynamic regulatory landscape, compliance with reporting standards is a critical challenge for organizations across various industries. Navigating through the complexities of regulatory requirements demands innovative solutions. This paper explores the application of Artificial Intelligence (AI) and Machine Learning (ML) techniques to enhance the accuracy and efficiency of reporting processes. By leveraging AI/ML, organizations can automate data analysis, identify patterns, and ensure compliance with regulatory frameworks. This study investigates the potential benefits, challenges, and best practices associated with integrating AI/ML technologies into reporting systems. Through a comprehensive review of relevant literature and case studies, insights are provided to assist organizations in effectively harnessing AI/ML for navigating regulatory complexity and achieving precise reporting outcomes.