Deep Learning Solutions Research Articles

Segment and Support: a dual-purpose Deep Learning solution for Limited Angle Holographic Tomography

Segment and Support: a dual-purpose Deep Learning solution for Limited Angle Holographic Tomography

Deep Learning Solution for Quality Control in a Die Casting Process

Industry 4.0 aims for a digital transformation of manufacturing and production systems, producing what is known as smart factories, where information coming from Cyber-Physical Systems (core elements in Industry 4.0) will be used in all the manufacturing stages to improve productivity. Cyber-physical systems through their control and sensor systems, provide a global view of the process, and generate large amounts of data that can be used for instance to produce datadriven models of the processes. However, having data is not enough, we must be able to store, visualize and analyze them, and to integrate induced knowledge in the whole production process. In this work, we present a solution to automate the quality control process of manufactured parts through image analysis. In particular, we present a Deep Learning solution to detect defects in manufactured parts from thermographic images of a die casting machine at an aluminum foundry.

Decoding viewer emotions in video ads

Understanding and predicting viewers’ emotional responses to videos has emerged as a pivotal challenge due to its multifaceted applications in video indexing, summarization, personalized content recommendation, and effective advertisement design. A major roadblock in this domain has been the lack of expansive datasets with videos paired with viewer-reported emotional annotations. We address this challenge by employing a deep learning methodology trained on a dataset derived from the application of System1’s proprietary methodologies on over 30,000 real video advertisements, each annotated by an average of 75 viewers. This equates to over 2.3 million emotional annotations across eight distinct categories: anger, contempt, disgust, fear, happiness, sadness, surprise, and neutral, coupled with the temporal onset of these emotions. Leveraging 5-second video clips, our approach aims to capture pronounced emotional responses. Our convolutional neural network, which integrates both video and audio data, predicts salient 5-second emotional clips with an average balanced accuracy of 43.6%, and shows particularly high performance for detecting happiness (55.8%) and sadness (60.2%). When applied to full advertisements, our model achieves a strong average AUC of 75% in determining emotional undertones. To facilitate further research, our trained networks are freely available upon request for research purposes. This work not only overcomes previous data limitations but also provides an accurate deep learning solution for video emotion understanding.

A protocol for trustworthy EEG decoding with neural networks

Deep learning solutions have rapidly emerged for EEG decoding, achieving state-of-the-art performance on a variety of decoding tasks. Despite their high performance, existing solutions do not fully address the challenge posed by the introduction of many hyperparameters, defining data pre-processing, network architecture, network training, and data augmentation. Automatic hyperparameter search is rarely performed and limited to network-related hyperparameters. Moreover, pipelines are highly sensitive to performance fluctuations due to random initialization, hindering their reliability. Here, we design a comprehensive protocol for EEG decoding that explores the hyperparameters characterizing the entire pipeline and that includes multi-seed initialization for providing robust performance estimates. Our protocol is validated on 9 datasets about motor imagery, P300, SSVEP, including 204 participants and 26 recording sessions, and on different deep learning models. We accompany our protocol with extensive experiments on the main aspects influencing it, such as the number of participants used for hyperparameter search, the split into sequential simpler searches (multi-step search), the use of informed vs. non-informed search algorithms, and the number of random seeds for obtaining stable performance. The best protocol included 2-step hyperparameter search via an informed search algorithm, with the final training and evaluation performed using 10 random initializations. The optimal trade-off between performance and computational time was achieved by using a subset of 3-5 participants for hyperparameter search. Our protocol consistently outperformed baseline state-of-the-art pipelines, widely across datasets and models, and could represent a standard approach for neuroscientists for decoding EEG in a trustworthy and reliable way.

Deep learning for quantification of myocardial scar and microvascular obstruction in late gadolinium enhancement cardiovascular magnetic resonance images

Abstract Objectives Detecting and measuring myocardial scar and microvascular obstruction (MVO) accurately following reperfusion therapy is clinically vital for patients with acute myocardial infarction (AMI). Current clinical practice relies on manual delineation of endocardial borders as well as hyper- and hypo-enhanced regions on late gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) images for scar and MVO quantification, respectively. However, this approach is subjective, labor-intensive, and time-consuming. The objective of this study is to develop and assess a convolutional neural network (CNN)-based method for the automated quantification of LGE scar and MVO in AMI patients. Methods A total of 1836 LGE images from 252 patients with AMI were analyzed, with the cohort comprising 232 males and an average age of 56±9 years. Our approach, utilizing a combined architecture of cascaded YOLOv8 (You Only Look Once version 8) and nnU-Net, was implemented for the simultaneous segmentation of the left ventricular (LV) myocardium, scar, and MVO in LGE images. Independent datasets for training, validation, and testing were maintained in an 8:1:1 ratio. Performance evaluation was conducted by comparing the results to manual scar and MVO quantification, utilizing metrics such as the Dice similarity coefficient (DSC) and Bland-Altman analysis. Results The automated segmentation process took 0.05 seconds per image. The segmentation accuracy, as assessed by the DSC, yielded values of 0.96±0.04 (per-patient, n=25) and 0.96±0.05 (per-slice, n=184) for scar segmentation, and 0.86±0.17 (per-patient) and 0.89±0.15 (per-slice) for MVO segmentation. In per-patient analysis, the volumes of automatically segmented scar and MVO demonstrated strong agreement with manually derived results, with biases [limits of agreement] of -0.3 [-3.0, 2.4] cm³ and 0.1 [-0.2, 0.3] cm³, respectively. Furthermore, there was strong concordance between automatically and manually derived scar size as a percentage of LV mass (-0.2 [-3.1, 2.7] %) and MVO size as a percentage of scar (1.2 [-5.6, 8.0] %). Conclusions The findings demonstrate excellent agreement between our proposed automated deep-learning solution and clinical expert annotation, suggesting its potential as a predictive tool for prognosis and clinical outcomes in patients with AMI.

Equipping computational pathology systems with artifact processing pipelines: a showcase for computation and performance trade-offs

BackgroundHistopathology is a gold standard for cancer diagnosis. It involves extracting tissue specimens from suspicious areas to prepare a glass slide for a microscopic examination. However, histological tissue processing procedures result in the introduction of artifacts, which are ultimately transferred to the digitized version of glass slides, known as whole slide images (WSIs). Artifacts are diagnostically irrelevant areas and may result in wrong predictions from deep learning (DL) algorithms. Therefore, detecting and excluding artifacts in the computational pathology (CPATH) system is essential for reliable automated diagnosis.MethodsIn this paper, we propose a mixture of experts (MoE) scheme for detecting five notable artifacts, including damaged tissue, blur, folded tissue, air bubbles, and histologically irrelevant blood from WSIs. First, we train independent binary DL models as experts to capture particular artifact morphology. Then, we ensemble their predictions using a fusion mechanism. We apply probabilistic thresholding over the final probability distribution to improve the sensitivity of the MoE. We developed four DL pipelines to evaluate computational and performance trade-offs. These include two MoEs and two multiclass models of state-of-the-art deep convolutional neural networks (DCNNs) and vision transformers (ViTs). These DL pipelines are quantitatively and qualitatively evaluated on external and out-of-distribution (OoD) data to assess generalizability and robustness for artifact detection application.ResultsWe extensively evaluated the proposed MoE and multiclass models. DCNNs-based MoE and ViTs-based MoE schemes outperformed simpler multiclass models and were tested on datasets from different hospitals and cancer types, where MoE using (MobileNet) DCNNs yielded the best results. The proposed MoE yields 86.15 % F1 and 97.93% sensitivity scores on unseen data, retaining less computational cost for inference than MoE using ViTs. This best performance of MoEs comes with relatively higher computational trade-offs than multiclass models. Furthermore, we apply post-processing to create an artifact segmentation mask, a potential artifact-free RoI map, a quality report, and an artifact-refined WSI for further computational analysis. During the qualitative evaluation, field experts assessed the predictive performance of MoEs over OoD WSIs. They rated artifact detection and artifact-free area preservation, where the highest agreement translated to a Cohen Kappa of 0.82, indicating substantial agreement for the overall diagnostic usability of the DCNN-based MoE scheme.ConclusionsThe proposed artifact detection pipeline will not only ensure reliable CPATH predictions but may also provide quality control. In this work, the best-performing pipeline for artifact detection is MoE with DCNNs. Our detailed experiments show that there is always a trade-off between performance and computational complexity, and no straightforward DL solution equally suits all types of data and applications. The code and HistoArtifacts dataset can be found online at Github and Zenodo, respectively.

GBKPA and AuxShield: Addressing adversarial robustness and transferability in android malware detection

Android stands as the predominant operating system within the mobile ecosystem. Users can download applications from official sources like Google Play Store and other third-party platforms. However, malicious actors can attempt to compromise user device integrity through malicious applications. Traditionally, signatures, rules, and other methods have been employed to detect malware attacks and protect device integrity. However, the growing number and complexity of malicious applications have prompted the exploration of newer techniques like machine learning (ML) and deep learning (DL). Many recent studies have demonstrated promising results in detecting malicious applications using ML and DL solutions. However, research in other fields, such as computer vision, has shown that ML and DL solutions are vulnerable to targeted adversarial attacks. Malicious actors can develop malicious adversarial applications that can bypass ML and DL based anti-viruses. The study of adversarial techniques related to malware detection has now captured the security community’s attention. In this work, we utilise android permissions and intents to construct 28 distinct malware detection models using 14 classification algorithms. Later, we introduce a novel targeted false-negative evasion attack, Gradient Based K Perturbation Attack (GBKPA), designed for grey-box knowledge scenarios to assess the robustness of these models. The GBKPA attempts to craft malicious adversarial samples by making minimal perturbations without violating the syntactic and functional structure of the application. GBKPA achieved an average fooling rate (FR) of 77 % with only five perturbations across the 28 detection models. Additionally, we identified the most vulnerable android permissions and intents that malicious actors can exploit for evasion attacks. Furthermore, we analyse the transferability of adversarial samples across different classes of models and provide explanations for the same. Finally, we proposed AuxShield defence mechanism to develop robust detection models. AuxShield reduced the average FR to 3.25 % against 28 detection models. Our findings underscore the need to understand the causation of adversarial samples, their transferability, and robust defence strategies before deploying ML and DL solutions in the real world.

SCL: A sustainable deep learning solution for edge computing ecosystem in smart manufacturing

Edge computing empowered Deep Learning (DL) solutions have risen as the foremost facilitators of automation in a multitude of smart manufacturing applications. These models are implemented on edge devices with frozen learning capabilities to execute DL inference task(s). Nevertheless, the data they process is susceptible to intermittent alterations amidst the ever-changing landscape of dynamic smart manufacturing ecosystem. It sparks the demand for model maintenance solution(s) to address adaptability and dynamism issues to enhance the efficiency of smart manufacturing solutions. Moreover, additional issue(s), such as the non-availability of comprehensive data (or the availability of solely contemporary data), near-real-time execution of DL model maintenance task, etc., imposes daunting obstructions in devising an efficient DL model maintenance strategy. This work proposes a novel approach that encompasses the merits of Continual Learning (CL) and Split Learning (SL) driven by edge intelligence, amalgamating them into a hybrid solution aptly named Split-based Continual Learning (SCL). CL ensures the sustained performance of the DL model amidst constraints related to data availability. At the same time, SL empowers near-real-time execution at the edge to achieve improved efficiency. An extension of the SCL scheme, termed as Extended SCL (ESCL), is implemented to addresses the interaction soundness aspects among the mobile edge devices in a collaborative execution environment. Evaluation of a vision-based product-quality inspection use case in an emulated hardware test-bed setup signifies that the performance of SCL and ESCL schemes have the potential to meet the needs of smart manufacturing. SCL attains an appreciable reduction in the model maintenance cost in the range of 21 to 48 and 12 to 29 percent compared to the ECN-only and basic-SL schemes. The ESCL scheme further improved the performance by 18 to 34 and 20 to 36 percent respectively over the basic-SL and SCL.

DeepMarkerNet: Leveraging supervision from the Duchenne Marker for spontaneous smile recognition

Distinguishing between spontaneous and posed smiles from videos poses a significant challenge in pattern classification literature. Researchers have developed feature-based and deep learning-based solutions for this problem. To this end, deep learning outperforms feature-based methods. However, certain aspects of feature-based methods could improve deep learning methods. For example, previous research has shown that Duchenne Marker (or D-Marker) features from the face play a vital role in spontaneous smiles, which can be useful to improve deep learning performances. In this study, we propose a deep learning solution that leverages D-Marker features to improve performance further. Our multi-task learning framework, named DeepMarkerNet, integrates a transformer network with the utilization of facial D-Markers for accurate smile classification. Unlike past methods, our approach simultaneously predicts the class of the smile and associated facial D-Markers using two different feed-forward neural networks, thus creating a symbiotic relationship that enriches the learning process. The novelty of our approach lies in incorporating supervisory signals from the pre-calculated D-Markers (instead of as input in previous works), harmonizing the loss functions through a weighted average. In this way, our training utilizes the benefits of D-Markers, but the inference does not require computing the D-Marker. We validate our model’s effectiveness on four well-known smile datasets: UvA-NEMO, BBC, MMI facial expression, and SPOS datasets, and achieve state-of-the-art results.

Toponym resolution leveraging lightweight and open-source large language models and geo-knowledge

Toponym resolution is crucial for extracting geographic information from natural language texts, such as social media posts and news articles. Despite the advancements in current methods, including state-of-the-art deep learning solutions like GENRE and a sophisticated voting system that integrates seven individual methods, further enhancing their accuracy is essential. To achieve this goal, we propose a novel method that combines lightweight and open-source large language models and geo-knowledge. Specifically, we first fine-tune Mistral (7B), Baichuan2 (7B), Llama2 (7B & 13B), and Falcon (7B) to estimate toponyms’ unambiguous reference (e.g., city, state, country) given their contexts. Subsequently, we correct inaccuracies in generated references and determine their geo-coordinates via sequentially querying GeoNames, Nominatim, and ArcGIS geocoders until a successful geocoding result is achieved. Our methods demonstrate enhanced performance compared to 20 existing methods, as evidenced across seven challenging datasets including 83,365 toponyms worldwide, with the Mistral-based method leading, followed by Baichuan2, Llama2, and Falcon-based methods. Specifically, the Mistral-based method achieves an Accuracy@161km of 0.91, surpassing GENRE, the best individual method, by 17% and the seven-methods composite voting system by 7%. Moreover, our methods are computationally efficient, operable on one general GPU, have modest memory requirements (14 GB for 7B models and 27 GB for 13B models), and exceed both GENRE and the voting system in inferring speed.

TUMbRAIN: A transformer with a unified mobile residual attention inverted network for diagnosing brain tumors from magnetic resonance scans

Diagnosing tumors in Magnetic Resonance Imaging (MRI) brain scans is challenging and can lead to errors, even for radiologists. Deep learning, mainly through deep convolutional neural networks, has assisted in automating the diagnosis of these scans. However, there is still room for improvement. Researchers have shown that transformer models hold promise but often remain underutilized due to their need for large amounts of data and complexity compared to traditional neural networks. This paper introduces a new hybrid model called TUMbRAIN (Transformer with a Unified Mobile Residual Attention Inverted Network), which combines a lightweight transformer with a deep convolutional neural network to address these issues. TUMbRAIN incorporates innovative components designed for this purpose, such as the expanded inverted residual block and the unified self-attention mechanism. The results demonstrate that TUMbRAIN outperforms many existing state-of-the-art neural network models, achieving an impressive overall accuracy of 97.94 % with only 1.04 million parameters. These results suggest that hybrid transformer models like TUMbRAIN could significantly advance the automated diagnosis of brain tumors from MRI scans. The study also offers new insights into effectively integrating transformers into traditional neural network architectures, resulting in a cost-effective and accurate deep learning solution for medical imaging. By incorporating these advanced components, TUMbRAIN enhances support for radiological practice through improved diagnostic accuracy and efficiency.

DAMO: A Deep Solver for Arbitrary Marker Configuration in Optical Motion Capture

Marker-based optical motion capture (mocap) systems are increasingly utilized for acquiring 3D human motion, offering advantages in capturing the subtle nuances of human movement, style consistency, and ease of obtaining desired motion. Motion data acquisition via mocap typically requires laborious marker labeling and motion reconstruction, recent deep-learning solutions have aimed to automate the process. However, such solutions generally presuppose a fixed marker configuration to reduce learning complexity, thereby limiting flexibility. To overcome the limitation, we introduce DAMO, an end-to-end deep solver, proficiently inferring arbitrary marker configurations and optimizing pose reconstruction. DAMO outperforms state-of-the-art like SOMA and MoCap-Solver in scenarios with significant noise and unknown marker configurations. We expect that DAMO will meet various practical demands such as facilitating dynamic marker configuration adjustments during capture sessions, processing marker clouds irrespective of whether they employ mixed or entirely unknown marker configurations, and allowing custom marker configurations to suit distinct capture scenarios. DAMO code and pretrained models are available at https://github.com/CritBear/damo

An experimental system for detection and localization of hemorrhage using ultra-wideband microwaves with deep learning

Stroke is a leading cause of mortality and disability. Emergent diagnosis and intervention are critical, and predicated upon initial brain imaging; however, existing clinical imaging modalities are generally costly, immobile, and demand highly specialized operation and interpretation. Low-energy microwaves have been explored as a low-cost, small form factor, fast, and safe probe for tissue dielectric properties measurements, with both imaging and diagnostic potential. Nevertheless, challenges inherent to microwave reconstruction have impeded progress, hence conduction of microwave imaging remains an elusive scientific aim. Herein, we introduce a dedicated experimental framework comprising a robotic navigation system to translate blood-mimicking phantoms within a human head model. An 8-element ultra-wideband array of modified antipodal Vivaldi antennas was developed and driven by a two-port vector network analyzer spanning 0.6–9.0 GHz at an operating power of 1 mW. Complex scattering parameters were measured, and dielectric signatures of hemorrhage were learned using a dedicated deep neural network for prediction of hemorrhage classes and localization. An overall sensitivity and specificity for detection >0.99 was observed, with Rayleigh mean localization error of 1.65 mm. The study establishes the feasibility of a robust experimental model and deep learning solution for ultra-wideband microwave stroke detection.

Application of Generative Adversarial Networks in Semi-Annotated Defect Synthesis and Detection Under Limited Resources

Defect detection is a crucial technology that is extensively employed in the manufacturing industry to monitor and ensure the quality of output. Deep learning models have shown remarkable potential for defect detection. However, the success of these models heavily relies on voluminous training data. Collecting substantial amounts of defect data is challenging in practical settings, and the tedious process of pixel-level defect annotation further complicates the task. Among the common defects encountered in manufacturing, scratches are particularly significant. To address these challenges, this study proposes a two-phase generative adversarial network (GAN) approach for synthesizing defect images and generating semi-automatic pixel-wise labels for anomaly detection. The first phase primarily focuses on synthesizing images, while the second phase involves the pixel-wise labeling of the images. The synthesized paired images generated by the GANs serve as input to the semantic network. Notably, the proposed methodology requires only a few real defect samples for training and a small amount of annotated data, making it practical and computationally efficient for implementation in the manufacturing industry. Experimental results indicate the effectiveness of the proposed deep-learning solution in defect detection, specifically in identifying scratches on various textured and patterned surfaces. A notably high detection accuracy is achieved, validating the potential of the approach in real-world manufacturing scenarios.

DLAS: A Conceptual Model for Across-Stack Deep Learning Acceleration

Deep Neural Networks (DNNs) are very computationally demanding, which presents a significant barrier to their deployment, especially on resource-constrained devices. Significant work from both the machine learning and computing systems communities has attempted to accelerate DNNs. However, the number of techniques available and the required domain knowledge for their exploration continues to grow, making design space exploration (DSE) increasingly difficult. To unify the perspectives from these two communities, this paper introduces the Deep Learning Acceleration Stack (DLAS), a conceptual model for DNN deployment and acceleration. We adopt a six-layer representation that organizes and illustrates the key areas for DNN acceleration, from machine learning to software and computer architecture. We argue that the DLAS model balances simplicity and expressiveness, assisting practitioners from various domains in tackling co-design acceleration challenges. We demonstrate the interdependence of the DLAS layers, and thus the need for co-design, through an across-stack perturbation study, using a modified tensor compiler to generate experiments for combinations of a few parameters across the DLAS layers. Our perturbation study assesses the impact on inference time and accuracy when varying DLAS parameters across two datasets, seven popular DNN architectures, four compression techniques, three algorithmic primitives (with sparse and dense variants), untuned and auto-scheduled code generation, and four hardware platforms. The study observes significant changes in the relative performance of design choices with the introduction of new DLAS parameters (e.g., the fastest algorithmic primitive varies with the level of quantization). Given the strong evidence for the need for co-design, and the high costs of DSE, DLAS offers a valuable conceptual model for better exploring advanced co-designed accelerated deep learning solutions.

Achieving More with Less: A Lightweight Deep Learning Solution for Advanced Human Activity Recognition (HAR).

Human activity recognition (HAR) is a crucial task in various applications, including healthcare, fitness, and the military. Deep learning models have revolutionized HAR, however, their computational complexity, particularly those involving BiLSTMs, poses significant challenges for deployment on resource-constrained devices like smartphones. While BiLSTMs effectively capture long-term dependencies by processing inputs bidirectionally, their high parameter count and computational demands hinder practical applications in real-time HAR. This study investigates the approximation of the computationally intensive BiLSTM component in a HAR model by using a combination of alternative model components and data flipping augmentation. The proposed modifications to an existing hybrid model architecture replace the BiLSTM with standard and residual LSTM, along with convolutional networks, supplemented by data flipping augmentation to replicate the context awareness typically provided by BiLSTM networks. The results demonstrate that the residual LSTM (ResLSTM) model achieves superior performance while maintaining a lower computational complexity compared to the traditional BiLSTM model. Specifically, on the UCI-HAR dataset, the ResLSTM model attains an accuracy of 96.34% with 576,702 parameters, outperforming the BiLSTM model's accuracy of 95.22% with 849,534 parameters. On the WISDM dataset, the ResLSTM achieves an accuracy of 97.20% with 192,238 parameters, compared to the BiLSTM's 97.23% accuracy with 283,182 parameters, demonstrating a more efficient architecture with minimal performance trade-off. For the KU-HAR dataset, the ResLSTM model achieves an accuracy of 97.05% with 386,038 parameters, showing comparable performance to the BiLSTM model's 98.63% accuracy with 569,462 parameters, but with significantly fewer parameters.

Low probability of interception radar overlapping signal modulation recognition based on an improved you-only-look-once version 8 network

Low probability of interception (LPI) radar is widely used in modern electronic warfare. With the increased radiation sources, multiple signals will arrive simultaneously. The traditional feature extraction method has too many features, which brings great trouble to the subsequent data processing. Most modulation recognition methods based on deep learning only consider the single signal after preprocessing, and the generalization ability is weak. This paper proposes a deep learning solution based on an improved you only look once version 8 (YOLOv8) network with a global attention mechanism (GAM), achieving recognition accuracy over 98% in a -10 dB signal-to-noise ratio (SNR) scenario to address the above problems. We improved the performance of the original network, which can be used in electronic countermeasures.

Deep learning applications for quantitative and qualitative PET in PET/MR: technical and clinical unmet needs.

We aim to provide an overview of technical and clinical unmet needs in deep learning (DL) applications for quantitative and qualitative PET in PET/MR, with a focus on attenuation correction, image enhancement, motion correction, kinetic modeling, and simulated data generation. (1) DL-based attenuation correction (DLAC) remains an area of limited exploration for pediatric whole-body PET/MR and lung-specific DLAC due to data shortages and technical limitations. (2) DL-based image enhancement approximating MR-guided regularized reconstruction with a high-resolution MR prior has shown promise in enhancing PET image quality. However, its clinical value has not been thoroughly evaluated across various radiotracers, and applications outside the head may pose challenges due to motion artifacts. (3) Robust training for DL-based motion correction requires pairs of motion-corrupted and motion-corrected PET/MR data. However, these pairs are rare. (4) DL-based approaches can address the limitations of dynamic PET, such as long scan durations that may cause patient discomfort and motion, providing new research opportunities. (5) Monte-Carlo simulations using anthropomorphic digital phantoms can provide extensive datasets to address the shortage of clinical data. This summary of technical/clinical challenges and potential solutions may provide research opportunities for the research community towards the clinical translation of DL solutions.

Advanced emotion analysis: harnessing facial image processing and speech recognition through deep learning

The human face hides many secrets and is one of the most expressive human features. Human faces even contain hidden information about a person's personality. Considering the fundamental role of the human face, it is necessary to prepare appropriate deep-learning solutions that analyze human face data. This technology is becoming increasingly common in many industries, such as online retail, advertising testing, virtual makeovers, etc. For example, facial analysis technology now allows online shoppers to virtually apply makeup and try on jewelry or new glasses to get an accurate picture of what these products will look like. The human sense of hearing is a treasure trove of information about the current environment and the location and properties of sound-producing objects. For instance, we effortlessly absorb the sounds of birds singing outside the window, traffic passing in the distance, or the lyrics of a song on the radio. The human auditory system can process the intricate mix of sounds reaching our ears and create high-level abstractions of the environment by analyzing and grouping measured sensory signals. The process of obtaining segregation and identifying sources of a received complex acoustic signal, known as sound scene analysis, is a domain where the power of deep learning shines. The machine implementation of this functionality (separation and classification of sound sources) is pivotal in applications such as speech recognition in noise, automatic music transcription, searching and retrieving multimedia data, or recognizing emotions in statements.

Segmentation and characterization of macerated fibers and vessels using deep learning

PurposeWood comprises different cell types, such as fibers, tracheids and vessels, defining its properties. Studying cells’ shape, size, and arrangement in microscopy images is crucial for understanding wood characteristics. Typically, this involves macerating (soaking) samples in a solution to separate cells, then spreading them on slides for imaging with a microscope that covers a wide area, capturing thousands of cells. However, these cells often cluster and overlap in images, making the segmentation difficult and time-consuming using standard image-processing methods.ResultsIn this work, we developed an automatic deep learning segmentation approach that utilizes the one-stage YOLOv8 model for fast and accurate segmentation and characterization of macerated fiber and vessel form aspen trees in microscopy images. The model can analyze 32,640 x 25,920 pixels images and demonstrate effective cell detection and segmentation, achieving a mAP0.5-0.95\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\hbox {mAP}_{0.5-0.95}$$\\end{document} of 78 %. To assess the model’s robustness, we examined fibers from a genetically modified tree line known for longer fibers. The outcomes were comparable to previous manual measurements. Additionally, we created a user-friendly web application for image analysis and provided the code for use on Google Colab.ConclusionBy leveraging YOLOv8’s advances, this work provides a deep learning solution to enable efficient quantification and analysis of wood cells suitable for practical applications.