Raw Images Research Articles

Real-time data processing in colorimetry camera-based single-molecule localization microscopy via CPU-GPU-FPGA heterogeneous computation.

Because conventional low-light cameras used in single-molecule localization microscopy (SMLM) do not have the ability to distinguish colors, it is often necessary to employ a dedicated optical system and/or a complicated image analysis procedure to realize multi-color SMLM. Recently, researchers explored the potential of a new kind of low-light camera called colorimetry camera as an alternative detector in multi-color SMLM, and achieved two-color SMLM under a simple optical system, with a comparable cross-talk to the best reported values. However, extracting images from all color channels is a necessary but lengthy process in colorimetry camera-based SMLM (called CC-STORM), because this process requires the sequential traversal of a massive number of pixels. By taking advantage of the parallelism and pipeline characteristics of FPGA, in this paper, we report an updated multi-color SMLM method called HCC-STORM, which integrated the data processing tasks in CC-STORM into a home-built CPU-GPU-FPGA heterogeneous computing platform. We show that, without scarifying the original performance of CC-STORM, the execution speed of HCC-STORM was increased by approximately three times. Actually, in HCC-STORM, the total data processing time for each raw image with 1024 × 1024 pixels was 26.9 ms. This improvement enabled real-time data processing for a field of view of 1024 × 1024 pixels and an exposure time of 30 ms (a typical exposure time in CC-STORM). Furthermore, to reduce the difficulty of deploying algorithms into the heterogeneous computing platform, we also report the necessary interfaces for four commonly used high-level programming languages, including C/C++, Python, Java, and Matlab. This study not only pushes forward the mature of CC-STORM, but also presents a powerful computing platform for tasks with heavy computation load.

Study on the impact of camera ISP algorithms on stripe structured light 3D reconstruction and the improvement of 3D reconstruction accuracy by photon input

This paper investigates the impact of camera image signal processing (ISP) algorithms on stripe structured light 3D reconstruction and explores the improvement of 3D reconstruction accuracy by photon input. Existing 3D reconstruction technologies have broad applications in fields such as intelligent manufacturing, healthcare, and consumer electronics, but their high precision requirements often cannot be met by current ISP algorithms. By using handheld devices to capture images and combining key techniques such as the multi-step phase shifting method, multi-frequency phase unwrapping method, and triangulation method, this paper conducts an in-depth study on the application of photon input in 3D reconstruction. The study demonstrates that using non-visual information (RAW images) as input can significantly improve reconstruction accuracy, producing more accurate results compared to images processed by ISP. The paper also quantifies the impact of ISP processing on 3D reconstruction results by comparing the depth information and point cloud data of two sets of images. Experimental results show that disabling certain ISP algorithms, such as bilateral noise filtering (BNF), edge enhancement (EEH), and non-local means denoising (NLM), can further improve reconstruction accuracy and reduce errors. In conclusion, this paper proposes a photon image-based 3D reconstruction method that, combined with artificial intelligence technology and differentiable point cloud rendering techniques, holds promise for achieving higher precision and faster 3D reconstruction. This technology is of great significance in practical applications, particularly in the field of industrial close-range 3D reconstruction.

A new transferable deep learning approach for crop mapping

ABSTRACT Accurate crop mapping provides important information for government decision-making and agricultural management. Recent advancements in deep learning have significantly enhanced the capabilities of remote sensing-based crop mapping. In this study, we developed a new deep learning approach, DSH, which comprises three modules, DeepLabV3+ (D), channel self-attention (S), and histogram matching (H). The DeepLabV3+ module learns the spatial distribution of crops in the spatial and spectral dimensions. The channel self-attention module was used to enhance the weights of the important features. The histogram-matching module addresses the domain gap between the training and testing images and enhances the transferability of the DSH. In addition, the input data of the DSH are monthly synthesized Sentinel-2 data, thus relaxing the data collection requirements. The performance of the DSH was evaluated and benchmarked against HRNetV2, DeepLabV3+, and random forest (RF) at eight sites in the U.S. China, and France with different farming structures, climate conditions, and landscape complexities. Temporal transfer revealed that the classification performance of DSH improved as the growth season progressed, peaked in the peak growth season (August), and then declined at each study site. The DSH model trained during the peak growth season demonstrated superior temporal transferability. Spatial transfer experiments demonstrated that transferring DSH to sites with similar planting structures achieved higher precision than full spatial transfers. Spatiotemporal transfer experiments conducted at eight sites over three years (2020–2022) demonstrated an average overall accuracy (OA) of 88.1% for DSH, highlighting the importance of similar planting structures for effective spatiotemporal transfer. DSH outperformed HRNetV2, DeepLabV3+, and RF in OA, producer’s accuracy, and user’s accuracy at all eight study sites. Notably, DSH exhibited comparable performance during both the peak and full growth seasons, with an average OA of 93.9%. The classification accuracy of test images after histogram matching remained stable from 2020 to 2022, whereas the accuracy of raw test images fluctuated annually. This indicates that histogram matching is effective in improving the transferability of DSH. Embedding the channel self-attention module into the position of the high-level features had a more significant impact on improving DSH’s performance than other configurations. Visualizing features from the channel self-attention and high-level layers revealed that integrating the channel self-attention module with DeepLabV3+‘s high-level features significantly improved crop mapping by enhancing relevant semantic information. Overall, DSH exhibits robust spatiotemporal generalizability and requires minimal input data, making it suitable for large-scale crop mapping.

A Denoise Network for Structured Illumination Microscopy with Low-Light Exposure

Super-resolution structured illumination microscopy (SR-SIM) is one of the important techniques that are most suitable for live-cell imaging. The reconstructed SR-SIM images are noisy once the raw images are recorded with low-light exposure. Here, we propose a new network (entitled the ND-SIM network) to denoise the SR images reconstructed using frequency-domain algorithms (FDAs). We demonstrate that ND-SIM can yield artifact-free SR images using raw images with an average photon count down to 20 per pixel while achieving comparable resolution to the ground truth (GT) obtained with high-light exposure. We can envisage that the ND-SIM will be widely applied for the long-term, super-resolution live-cell imaging of various bioprocesses in the future.

Spatial-spectral feature extraction for in-field chlorophyll content estimation using hyperspectral imaging

In-situ leaf chlorophyll content (LCC) estimation based on hyperspectral imaging (HSI) is crucial to track the growth status of crops for field management. However, spatial and spectral features of HSI data, suffering from interference of growth dynamic effect and soil, pose the challenge on accuracy and robustness of LCC estimation in several years and growth stages. Therefore, a joint spectral-spatial feature extraction method was proposed by cascade of three-dimensional convolutional neural network (3DCNN) and long short-term memory (LSTM) to reduce the interference for optimising the LCC estimation. Firstly, crop pixels were separated from soil with vegetation index segmentation method. Secondly, when raw images and segmented pixels were input, sensitive bands were selected by random frog (RF bands), and 3DCNN-LSTM was used to extract the joint spectral-spatial features. Finally, models established by RF bands, 3DCNN and 3DCNN-LSTM were compared, and robustness in individual years and stages was validated. Results showed that RF bands and 3DCNN obtained RP2 of 0.76 and 0.84 when not segmented. After segmentation, performance of 3DCNN improved (RP2 = 0.85) compared to RF bands (RP2 = 0.80). Spectral-spatial features by 3DCNN reduced the interference of soil. 3DCNN-LSTM without and with segmentation obtained good performance with RP2 of 0.95 and 0.96, and the proposed method could reduce the image segmentation process. The optimal model achieved RP2 above 0.93 in individual years (RP2 = 0.96 in 2021, RP2 = 0.94 in 2021) and RP2 in the range of 0.87–0.97 at individual stages. This paper provides a method to track growth variability between soil and crop for the LCC estimation optimisation.

Bioinformatics Analysis of Differentially Expressed Genes in Carpal Tunnel Syndrome Using RNA Sequencing.

Carpal tunnel syndrome (CTS) is a common disease resulting from the median nerve entrapment at the wrist. Although CTS (prevalence=5%-10% in the general population) is the most common neuropathy, its molecular mechanisms need elucidation. We used bioinformatics to detect genes with differential expressions in CTS and introduce the molecular regulatory noncoding RNAs and signaling pathways involved. The raw files of the RNA sequencing of CTS patients and controls were obtained from GEO (accession: GSE108023), and the samples were analyzed. Differentially expressed genes were isolated using DESeq2 R. Functional analyses were conducted on the signaling pathways, biological processes, molecular functions, and cellular components of the differentially expressed genes. Additionally, interactions between the most differentially expressed genes and miRNAs and lncRNAs were investigated bioinformatically. Upregulation and downregulation were observed in 790 and 922 genes, respectively. The signaling pathway analysis identified the metabolism pathways of arachidonic acid, linoleic acid, and tyrosine as the most significant pathways in CTS. Moreover, PLA2G2D and PLA2G2A with upregulated expressions and PLA2G2F, PLA2G4F, PLA2G4D, PLA2G3, and PLA2G4E with downregulated expressions were genes from the phospholipase family playing significant roles in the pathways. Further analyses demonstrated that hsa-miR-3150b-3p targeted PLA2G2A and PLA2G4F, and RP11-573D15.8-018 lncRNA had regulatory interactions with the aforementioned genes. Molecular studies on CTS will clarify the involved signaling pathways and provide critical data for biomedical research, drug development, and clinical applications.

Advanced Quantification of Receptor-Ligand Interaction Lifetimes via Single-Molecule FRET Microscopy.

Receptor-ligand interactions at cell interfaces initiate signaling cascades essential for cellular communication and effector functions. Specifically, T cell receptor (TCR) interactions with pathogen-derived peptides presented by the major histocompatibility complex (pMHC) molecules on antigen-presenting cells are crucial for T cell activation. The binding duration, or dwell time, of TCR-pMHC interactions correlates with downstream signaling efficacy, with strong agonists exhibiting longer lifetimes compared to weak agonists. Traditional surface plasmon resonance (SPR) methods quantify 3D affinity but lack cellular context and fail to account for factors like membrane fluctuations. In the recent years, single-molecule Förster resonance energy transfer (smFRET) has been applied to measure 2D binding kinetics of TCR-pMHC interactions in a cellular context. Here, we introduce a rigorous mathematical model based on survival analysis to determine exponentially distributed receptor-ligand interaction lifetimes, verified through simulated data. Additionally, we developed a comprehensive analysis pipeline to extract interaction lifetimes from raw microscopy images, demonstrating the model's accuracy and robustness across multiple TCR-pMHC pairs. Our new software suite automates data processing to enhance throughput and reduce bias. This methodology provides a refined tool for investigating T cell activation mechanisms, offering insights into immune response modulation.

Defect Detection of GFRP Composites through Long Pulse Thermography Using an Uncooled Microbolometer Infrared Camera.

The detection of impact and depth defects in Glass Fiber Reinforced Polymer (GFRP) composites has been extensively studied to develop effective, reliable, and cost-efficient assessment methods through various Non-Destructive Testing (NDT) techniques. Challenges in detecting these defects arise from varying responses based on the geometrical shape, thickness, and defect types. Long Pulse Thermography (LPT), utilizing an uncooled microbolometer and a low-resolution infrared (IR) camera, presents a promising solution for detecting both depth and impact defects in GFRP materials with a single setup and minimal tools at an economical cost. Despite its potential, the application of LPT has been limited due to susceptibility to noise from environmental radiation and reflections, leading to blurry images. This study focuses on optimizing LPT parameters to achieve accurate defect detection. Specifically, we investigated 11 flat-bottom hole (FBH) depth defects and impact defects ranging from 8 J to 15 J in GFRP materials. The key parameters examined include the environmental temperature, background reflection, background color reflection, and surface emissivity. Additionally, we employed image processing techniques to classify composite defects and automatically highlight defective areas. The Tanimoto Criterion (TC) was used to evaluate the accuracy of LPT both for raw images and post-processed images. The results demonstrate that through parameter optimization, the depth defects in GFRP materials were successfully detected. The TC success rate reached 0.91 for detecting FBH depth defects in raw images, which improved significantly after post-processing using Canny edge detection and Hough circle detection algorithms. This study underscores the potential of optimized LPT as a cost-effective and reliable method for detecting defects in GFRP composites.

Enhancing prostate cancer segmentation on multiparametric magnetic resonance imaging with background information and gland masks.

The landscape of prostate cancer (PCa) segmentation within multiparametric magnetic resonance imaging (MP-MRI) was fragmented, with a noticeable lack of consensus on incorporating background details, culminating in inconsistent segmentation outputs. Given the complex and heterogeneous nature of PCa, conventional imaging segmentation algorithms frequently fell short, prompting the need for specialized research and refinement. This study sought to dissect and compare various segmentation methods, emphasizing the role of background information and gland masks in achieving superior PCa segmentation. The goal was to systematically refine segmentation networks to ascertain the most efficacious approach. A cohort of 232 patients (ages 61-73 years old, prostate-specific antigen: 3.4-45.6ng/mL), who had undergone MP-MRI followed by prostate biopsies, was analyzed. An advanced segmentation model, namely Attention-Unet, which combines U-Net with attention gates, was employed for training and validation. The model was further enhanced through a multiscale module and a composite loss function, culminating in the development of Matt-Unet. Performance metrics included Dice Similarity Coefficient (DSC) and accuracy (ACC). The Matt-Unet model, which integrated background information and gland masks, outperformed the baseline U-Net model using raw images, yielding significant gains (DSC: 0.7215vs. 0.6592; ACC: 0.8899vs. 0.8601, p<0.001). A targeted and practical PCa segmentation method was designed, which could significantly improve PCa segmentation on MP-MRI by combining background information and gland masks. The Matt-Unet model showcased promising capabilities for effectively delineating PCa, enhancing the precision of MP-MRI analysis.

Towards assessing and improving the reliability of ultrashort echo time quantitative magnetization transfer (UTE-qMT) MRI of cortical bone: In silico and ex vivo study.

To assess and improve the reliability of the ultrashort echo time quantitative magnetization transfer (UTE-qMT) modeling of the cortical bone. Simulation-based digital phantoms were created that mimic the UTE-qMT properties of cortical bones. A wide range of SNR from 25 to 200 was simulated by adding different levels of noise to the synthesized MT-weighted images to assess the effect of SNR on UTE-qMT fitting results. Tensor-based denoising algorithm was applied to improve the fitting results. These results from digital phantom studies were validated via ex vivo rat leg bone scans. The selection of initial points for nonlinear fitting and the number of data points tested for qMT analysis have minimal effect on the fitting result. Magnetization exchange rate measurements are highly dependent on the SNR of raw images, which can be substantially improved with an appropriate denoising algorithm that gives similar fitting results from the raw images with an 8-fold higher SNR. The digital phantom approach enables the assessment of the reliability of bone UTE-qMT fitting by providing the known ground truth. These findings can be utilized for optimizing the data acquisition and analysis pipeline for UTE-qMT imaging of cortical bones.

Tracking the movement of quartz sand particles with neural networks

Tracking the movement of particles is of paramount significance for studying the impact of particle breakage on the macroscopic mechanical behaviour of granular materials and remains a persistent challenge to date. This paper presents a novel particle tracking method that integrates Pointon and PointNetLK networks, enabling an accurate tracking of both intact and broken Leighton Buzzard sand (LBS) particles. Initially, morphological information of LBS particles was extracted from X-ray micro-tomography (CT) data collected from a miniature triaxial test. Various image processing techniques were applied to the raw CT images to achieve a realistic three-dimensional (3D) reconstruction. Subsequently, particle point cloud data was processed through sampling, Gaussian noise injection, and grouping for training and testing the Poynton and PointNetLK networks. Next, the correspondences among particles across different scans were established by PointConv, and the transformation matrix between two mutually matched particles was predicted using PointNetLK. Finally, an examination of the changes in the spatial distribution and morphological parameters of both tracked and untracked particles throughout the shearing process was conducted and followed by an analysis of particle kinematics.

Surface fatigue crack segmentation in steel box girder bridges using an improved Unet convolutional neural network

Despite the success of crack segmentation, the traditional segmentation strategy and networks are not necessarily suitable for fatigue cracks with unique visual features. Towards the indistinct crack features, a network named Fatigue Crack Unet (FC-Unet) was proposed, where the novel split attention mechanism was incorporated to boost diverse feature extraction. Scaled by various ratios, raw images were cropped and constituted three datasets to represent different segmentation strategies. FC-Unet was trained and tested on each dataset, where the class imbalance problem was alleviated by the adjustment of loss functions. Results showed that the segmentation performance is mainly limited to noise interference. Preserving abundant global contexts to distinguish noises, the global inference is the superior strategy to the previous local inference. Leveraging the wide receptive field in global inference, the down-sampling rate can be decreased to save resolution losses. Combining the focus-based and region-based loss functions, the accuracy of imbalanced data was further improved. Compared with nine classical networks, the proposed FC-Unet enjoyed a powerful backbone and achieved a superior metric of 83.1% mean intersection over union (MIoU).

Non-Thermal Effects of Terahertz Radiation on Gene Expression: Systematic Review and Meta-Analysis.

With the advancement of terahertz technology, unveiling the mysteries of terahertz has had a profound impact on the field of biomedicine. However, the lack of systematic comparisons for gene expression signatures may diminish the effectiveness and efficiency of identifying common mechanisms underlying terahertz effects across diverse research findings. We performed a comprehensive review and meta-analysis to compile patterns of gene expression profiles associated with THz radiation. Thorough bibliographic reviews were conducted, utilizing the PubMed, Embase, Web of Science, and ProQuest databases to extract references from published articles. Raw CEL files were obtained from Gene Expression Omnibus and preprocessed using Bioconductor packages. This systematic review (Registration No. CDR42024502937) resulted in a detailed analysis of 13 studies (14 papers). There are several possible mechanisms and pathways through which THz radiation could cause biological changes. While the established gene expression results are largely associated with immune response and inflammatory markers, other genes demonstrated transcriptional outcomes that may unravel unknown functions. The enrichment of genes primarily found networks associated with broader stress responses. Altogether, the findings showed that THz can induce a distinct transcriptomic profile that is not associated with a microthermal cellular response. However, it is impossible to pinpoint a single gene or family of genes that would accurately and reliably justify the patterns of gene expression response under THz exposure.

Implementation and Efficient Analysis of Preprocessing Techniques in Deep Learning for Image Classification

Deep learning models have recently been preferred to perform certain image-processing tasks. Recently, with the increasing radiation, heat, and poor lighting conditions, the raw image samples may contain noisy and ambiguous information. To process these images, the deep learning model requires a large number of data samples to learn the missing information from other clear data samples. This necessitates training the neural network with a huge dataset. The researchers are now attempting to filter and improve such noisy images via preprocessing in order to provide valid and accurate feature information to the neural network layers. However, certain research studies claim that some useful information may be lost when the image is not preprocessed with an appropriate filter or enhancement technique. The MSA (meta-synthesis and analysis) approach is utilized in this work to present the impact of the image processing applications done with and without preprocessing steps. Also, this work summarizes the existing deep learning-based image processing models utilizing or not preprocessing steps in their implementation. This work has also found that 85% of the existing techniques involve a preprocessing step while developing a deep learning model. However, a maximum accuracy of 96.89% is observed on Sine-Net when it is implemented without a preprocessing and the same model gave 96.85% when implemented with preprocessing. This research provides various research insights on the requirement and non-requirement of preprocessing steps in a deep learning-based implementation.

Hypoperfusion of hypothalamic subunits in medication-overuse headache using a 3D PCASL MRI

Hypothalamus is a crucial deep brain area that is responsible for the integration and coordination of various brain functions. The altered perfusion of hypothalamus during headache caused by medication-overuse headache (MOH) was previously unknown. In the current study, the altered perfusion of hypothalamic subregions in MOH patients was investigated using state-of-the-art 3D pseudo-continuous arterial spin labeling (PCASL) MR imaging. In this study, 29 normal controls subjects (NCs) and 29 MOH patients underwent 3D PCASL and brain structural MR imaging. The hypothalamus was automatically segmented into 10 subunits and the volume of each subunit was automatically determined using Freesurfer software (v7.4.1). All segmented hypothalamic subunits were converted to individual hypothalamic subunit masks. The cerebral blood flow (CBF) images were coregistered with the raw brain structural images and resliced. The CBF value of each hypothalamic subunit was extracted from the warped CBF images. The volume and CBF value of each hypothalamic subunit were analyzed using the independent sample T test and Mann–Whitney U test, receiver operating characteristic (ROC) curve analysis, and Pearson and Spearman correlation analysis. Hypothalamic subunits with significantly decreased perfusion were located in the left posterior, left tubular superior, right anterior–inferior, right tubular inferior, right tubular superior, right posterior subunit and the entire right hypothalamus [CBF value for MOH vs NC (mL/100 g·min): 48.41 ± 6.75 vs 54.08 ± 11.47, 44.44 ± 4.79 vs 48.11 ± 7.73, 41.49 (32.90, 61.46) vs 49.38 ± 10.47, 46.62 ± 7.04 vs 53.90 ± 11.75, 42.12 ± 5.74 vs 47.02 ± 9.99, 42.79 ± 5.15 vs 47.93 ± 10.48 and 43.58 ± 5.06 vs 48.65 ± 9.33, respectively] in MOH compared to NC (P < 0.05). ROC analysis for these positive subunits revealed that area under the curve was 0.658–0.693, and ROC curve for left posterior subunit had the highest specificity of 93.10% while the entire right hypothalamus had the highest sensitivity of 72.41%. Further correlation analysis showed that the CBF value of the left posterior, right anterior–inferior, right tubular superior, whole right hypothalamus presented significantly negative correlation with Hamilton Depression Scale (HAMD) score (P < 0.05). Hypoperfusion of hypothalamic subunits may contribute to the understanding of MOH pathogenesis, and the 3D PCASL could be considered as a potential diagnostic and assessment tool for MOH.

Dense object detection methods in RAW UAV imagery based on YOLOv8

Accurate, fast and lightweight dense target detection methods are highly important for precision agriculture. To detect dense apricot flowers using drones, we propose an improved dense target detection method based on YOLOv8, named D-YOLOv8. First, we introduce the Dense Feature Pyramid Networks (D-FPN) to enhance the model’s ability to extract dense features and Dense Attention Layer (DAL) to focus on dense target areas, which enhances the feature extraction ability of dense areas, suppresses features in irrelevant areas, and improves dense target detection accuracy. Finally, RAW data are used to enhance the dataset, which introduces additional original data into RAW images, further enriching the feature input of dense objects. We perform validation on the CARPK challenge dataset and constructed a dataset. The experimental results show that our proposed D-YOLOv8m achieved 98.37% AP, while the model parameters were only 13.2 million. The improved network can effectively support any task of dense target detection.

Autofluorescence-based tissue characterization enhances clinical prospects of light-sheet-microscopy

Light sheet fluorescence microscopy (LSFM) is a transformative imaging method that enables the visualization of non-dissected specimen in real-time 3D. Optical clearing of tissues is essential for LSFM, typically employing toxic solvents. Here, we test the applicability of a non-hazardous alternative, ethyl cinnamate (ECi). We comprehensively characterized autofluorescence (AF) spectra in diverse murine tissues—ocular globe, knee, and liver—employing LSFM under various excitation wavelengths (405–785 nm) to test the feasibility of unstained samples for diagnostic purposes, in particular regarding percutaneous biopsies, as they constitute to most harvested type of tissue sample in clinical routine. Ocular globe structures were best discerned with 640 nm excitation. Knee tissue showed complex variation in AF spectra variation influenced by tissue depth and structure. Liver exhibited a unique AF pattern, likely linked to vasculature. Hepatic tissue samples were used to demonstrate the compatibility of our protocol for antibody staining. Furthermore, we employed machine learning to augment raw images and segment liver structures based on AF spectra. Radiologists rated representative samples transferred to the clinical assessment software. Learning-generated images scored highest in quality. Additionally, we investigated an actual murine biopsy. Our study pioneers the application of AF spectra for tissue characterization and diagnostic potential of optically cleared unstained percutaneous biopsies, contributing to the clinical translation of LSFM.

A laminar spray group combustion experiment (LSGCE) with in situ image diagnostic methods and image analysis algorithms.

Spray combustion is important for different engines. The understanding of spray combustion should be further promoted especially in the non-dilute region, and there is lack of well-defined spray experiments. In this study, an experimental platform was developed. Using this platform, a cylindrical quasi-laminar spray can be formed and ignited by a thin and straight hot wire, making it a simple configuration. Two image diagnostic methods were also developed to capture in situ microscopic droplet images and macroscopic droplet-flame images synchronously. Different image analysis algorithms were developed to obtain droplet statistics (diameter, velocity, and number density) and flame information (size, location, and flame propagation speed) from the raw images. The design, diagnostic methods, and image analysis methods are detailedly presented. This experimental platform can cover a wide range of operating conditions, with Gig in a range of 0.01-0.06 and temperature in a range from room temperature to 1400K. In addition, this platform is small in size and is capable of further implanting into a ground-based microgravity facilitaty. The whole experimental system can be applied in spray ignition and combustion studies and can provide legitimate data for further model development.

Road-pavement classification by artificial neural network model based on tire-pavement noise and road-surface image

This study focuses on an artificial neural network (ANN) model for classifying pavement types using acoustic and image data. While conventional studies often use road-surface images for pavement classification, they face challenges with image quality degradation owing to external factors, such as sunlight angle, shadows, and lighting. Therefore, in this study, tire-pavement noise, which has different noise characteristics depending on the material and surface treatment, is used independently and in conjunction with image data for ANN training. To construct the training dataset, tire-pavement noise, and road-surface images are collected from 11 highway sampling sites in South Korea. Two simultaneous measurements are used: the tire-pavement noise is collected using the On-board sound intensity (OBSI) method, and the camera captures the road-surface images. 1/3 octave SIL, spectrum, MFCC, GLCM, and HOG are extracted from the raw data, and the ANN models are trained by these features. Using the spectrum as an input feature for the ANN yields a classification accuracy of 95.18%. However, the total number of parameters in the ANN is double that of the other models. To reduce the ANN size, 1/3 octave band SIL is used for training, and the model size is halved. However, the accuracy decreases by 13.47 percentage points. To overcome this significant decrease, the 1/3 octave bands SIL and image features were used to train ANN, simultaneously. This approach increases the accuracy by 93.85%. By training the ANN using MFCC, which is commonly used as an acoustic feature in other machine learning studies, the highest classification accuracy of 96.84% is achieved. Additionally, MFCC models are affected by the number of coefficients and the signal length. To include the dominant frequency of tire-pavement noise, more than 13 coefficients are used, a number generally known to be suitable for speech recognition. Increasing the number of coefficients from 13 to 40 improves accuracy by 1.17 percentage points. The interval for slicing raw WAV files is reduced to increase the training data and classify the pavement using shorter signals without statistically significant accuracy loss. Although accuracy does not decrease until the signal lengths reach 0.5 seconds, it rapidly decreases when the signal lengths become shorter than 0.4 seconds.

Data-driven prediction of future melt pool from built parts during metal additive manufacturing

Smart manufacturing in metal additive manufacturing (MAM) relies on real-time process optimization through the prediction of unbuilt parts using data from built parts. Melt pool dynamics are intimately associated with the development of various microstructures during the MAM. In this paper, we demonstrate the idea by establishing a machine learning (ML) model to predict the melt pool of future parts, based on the experimental data from the National Institute of Standards and Technology (NIST) where 80 % is used for the training (built parts) and 20 % for testing (future parts). The ML model integrates both data denoising and predictive modeling. First, a convolutional neural network (CNN) is used to process a large dataset of raw melt pool images, enabling the automatic removal of noises (e.g., splash and plume) and thereof extraction of high-quality melt pool data. Following that, a novel data-driven melt pool model based on multi-layer perceptron (MLP) is trained by incorporating raw, long scanning history as input features, which best accounts for the effects of printing history (e.g., intertrack heating) on melt pool development. It takes complete advantage of MLP in handling high-dimensional regression problems in conjunction with a large dataset. Upon testing under various manufacturing conditions, the average relative error magnitude (AREM) of predicting melt pool size drops to 2.8 %, compared to 14.8 % of the prior art – the Neighboring Effect Modeling Method model (NBEM). This research thus represents a significant step towards reliable melt-pool-guided AM process optimization for smart manufacturing, enabled by advanced, flexible, and maximum use of ML techniques.