Deep Learning-based Image Processing Research Articles

Quantification of litter in cities using a smartphone application and citizen science in conjunction with deep learning-based image processing

Cities are a major source of litter pollution. Determination of the abundance and composition of plastic litter in cities is imperative for effective pollution management, environmental protection, and sustainable urban development. Therefore, here, a multidisciplinary approach to quantify and classify the abundance of litter in urban environments is proposed. In the present study, litter data collection was integrated via the Pirika smartphone application and conducted image analysis based on deep learning. Pirika was launched in May 2018 and, to date, has collected approximately one million images. Visual classification revealed that the most common types of litter were cans, plastic bags, plastic bottles, cigarette butts, cigarette boxes, and sanitary masks, in that order. The top six categories accounted for approximately 80 % of the total, whereas the top three categories accounted for more than 60 % of the total imaged litter. A deep-learning image processing algorithm was developed to automatically identify the top six litter categories. Both precision and recall derived from the model were higher than 75 %, enabling proper litter categorization. The quantity of litter derived from automated image processing was also plotted on a map using location data acquired concurrently with the images by the smartphone application. Conclusively, this study demonstrates that citizen science supported by smartphone applications and deep learning-based image processing can enable the visualization, quantification, and characterization of street litter in cities.

Contactless weighing method based on deep learning and acoustic levitation

Abstract Acoustic weighing is a promising contactless method for screening the mass of micro-nano objects as it avoids contact contamination and losses. Existing acoustic weighing methods determine the mass of an object by detecting its oscillation trajectory with a laser sensor. However, this method suffers from several limitations, such as short measurement distance, poor accuracy in measuring transparent objects, and inducing damage to photosensitive samples. To solve these issues, this work proposes a contactless weighing method based on location-aware neural network (LANet) and acoustic levitation. The proposed LANet is a deep learning-based image processing method that detects object bit oscillation trajectories completely contactless, regardless of the color, shape, and oscillation distance of the levitated object. We employ a cross-stage aggregation module and cross-mixed feature pyramid strategy to build LANet network depth for enhanced feature extraction. In addition, to create a contactless environment, we built an acoustic levitation system, which drives the oscillation of objects. Finally, we verified the accuracy and effectiveness of the method. The results show that the proposed network can accurately detect the oscillation trajectories of various objects with high detection performance, even for small objects in low-contrast backgrounds. Meanwhile, the proposed method can accurately measure the mass of objects with a percentage error of no more than 7.83%.

FindD: AI-Driven Insights for Bone and Muscle Deficiencies Across Generations

This literature review examines a novel strategy for treating vitamin deficiencies, with a focus on vitamin D, vitamin K, and vitamin E. It also highlights the importance of deficiency of the vitamin D&K for bone and the vitamin E for muscle abnormalities in children and adults. The research uses an extensive methodology that combines deep learning-based image processing techniques with machine learning for text-based question-and-answer(Q&A) engagements. Through conversation, users are able to provide pertinent details about symptoms. Machine learning algorithms are then used to analyse the collected data. Concurrently, relevant images are processed by deep learning models to uncover subtle patterns that point to certain vitamin deficiencies. In order to address issues with bone caused by vitamin deficiencies, this research attempts to provide insights into the nexus of machine learning and deep learning for personalized diagnosis of vitamin deficiencies. The comprehensive method takes into account the multiplicity of variables affecting vitamin levels and how they affect bone health across age groups. This study's keywords are deep learning image processing, Q&A-based machine learning, vitamin D, vitamin K, and vitamin E deficiencies, as well as bone abnormalities.

Advanced analysis of disintegrating pharmaceutical compacts using deep learning-based segmentation of time-resolved micro-tomography images

The mechanism governing pharmaceutical tablet disintegration is far from fully understood. Despite the importance of controlling a formulation's disintegration process to maximize the active pharmaceutical ingredient's bioavailability and ensure predictable and consistent release profiles, the current understanding of the process is based on indirect or superficial measurements. Formulation science could, therefore, additionally deepen the understanding of the fundamental physical principles governing disintegration based on direct observations of the process. We aim to help bridge the gap by generating a series of time-resolved X-ray micro-computed tomography (μCT) images capturing volumetric images of a broad range of mini-tablet formulations undergoing disintegration. Automated image segmentation was a prerequisite to overcoming the challenges of analyzing multiple time series of heterogeneous tomographic images at high magnification. We devised and trained a convolutional neural network (CNN) based on the U-Net architecture for autonomous, rapid, and consistent image segmentation. We created our own μCT data reconstruction pipeline and parameterized it to deliver image quality optimal for our CNN-based segmentation. Our approach enabled us to visualize the internal microstructures of the tablets during disintegration and to extract parameters of disintegration kinetics from the time-resolved data. We determine by factor analysis the influence of the different formulation components on the disintegration process in terms of both qualitative and quantitative experimental responses. We relate our findings to known formulation component properties and established experimental results. Our direct imaging approach, enabled by deep learning-based image processing, delivers new insights into the disintegration mechanism of pharmaceutical tablets.

Phase-shifting determination and pattern recognition using a modified Sagnac interferometer with multiple reflections.

This work has implemented a diverse modification of the Sagnac interferometer to accommodate various measurement requirements, including phase shifting, pattern recognition, and a morphological analysis. These modifications were introduced to validate the adaptability and versatility of the system. To enable phase shifting using the multiple light reflection technique, a half-wave plate (HWP) was utilized with rotations at 0, π/8, π/4, and 3π/8radians, generating four interference patterns. It is possible to observe a distinct circular fringe width as the polarized light experiences diffraction at the interferometer's output as it travels through a circular aperture with various diameters ranging from 0.4 to 1mm. Further modifications were made to the setup by inserting a pure glass and a fluoride-doped tin oxide (FTO) transparent substrate into the common path. This modification aimed to detect and analyze a horizontal fringe pattern. Subsequently, the FTO substrate was replaced with a bee leg to facilitate morphology recognition. A deep learning-based image processing technique was employed to analyze the bee leg morphology. The experimental results showed that the proposed scheme succeeded in achieving the phase shift, measuring hole diameters with errors smaller than 1.6%, separating distinct transparent crystals, and acquiring the morphological view of a bee's leg. The method also has successfully achieved an accurate surface area and background segmentation with an accuracy over 87%. Overall, the outcomes demonstrated the potential of proposed interferometers for various applications, and the advantages of the optical sensors were highlighted, particularly in microscopic applications.

GoogleNet transfer learning with improved gorilla optimized kernel extreme learning machine for accurate detection of asphalt pavement cracks

Asphalt cracks are the initial indication and major pavement structural distress in the field of civil engineering because it might potentially intimidate road traffic and highway safety. Managing and maintaining pavement structures are the two crucial issues faced eventually by road administrators. It is essential to take immediate action regarding pavement structural damages to prevent its severity. In recent times, there established numerous deep learning-based image processing methods to detect pavement cracks, but due to the presence of interference factors such as noises and road markings in the images, those methods failed to provide high accuracy and efficiency. Therefore, with the aim to yield high accuracy of crack detection, this article designs a novel asphalt pavement crack identification system “GoogleNet transfer learning with improved gorilla optimized kernel extreme learning machine” (GNet TL with IGT-KELM). The road crack images used for perusal are acquired from NHA12D dataset, in which it consists of 40 asphalt pavement images with two different viewpoints. For the purpose of data balancing, some non-crack images are gathered by field survey. The images with large noise distortions and unwanted background information influence crack detection performance so that the preprocessing steps are executed before applying it to the prediction system. The most significant crack and non-crack features from the images are extracted using GNet TL model. With the extracted features, the KELM is well-trained to detect cracks and non-cracks separately. To increase crack detection performance of the classifier, an improved gorilla troop optimizer is introduced to optimize the KELM parameters. The experimental finding reveals that the proposed detection mechanism achieves crack recognition accuracy of about 98.93%, which is considerably greater than compared baseline approaches.

Simulating the cellular context in synthetic datasets for cryo-electron tomography.

Cryo-electron tomography (cryo-ET) allows to visualize the cellular context at macromolecular level. To date, the impossibility of obtaining a reliable ground truth is limiting the application of deep learning-based image processing algorithms in this field. As a consequence, there is a growing demand of realistic synthetic datasets for training deep learning algorithms. In addition, besides assisting the acquisition and interpretation of experimental data, synthetic tomograms are used as reference models for cellular organization analysis from cellular tomograms. Current simulators in cryo-ET focus on reproducing distortions from image acquisition and tomogram reconstruction, however, they can not generate many of the low order features present in cellular tomograms. Here we propose several geometric and organization models to simulate low order cellular structures imaged by cryo-ET. Specifically, clusters of any known cytosolic or membrane bound macromolecules, membranes with different geometries as well as different filamentous structures such as microtubules or actin-like networks. Moreover, we use parametrizable stochastic models to generate a high diversity of geometries and organizations to simulate representative and generalized datasets, including very crowded environments like those observed in native cells. These models have been implemented in a multiplatform open-source Python package, including scripts to generate cryo-tomograms with adjustable sizes and resolutions. In addition, these scripts provide also distortion-free density maps besides the ground truth in different file formats for efficient access and advanced visualization. We show that such a realistic synthetic dataset can be readily used to train generalizable deep learning algorithms.

Development of a deep learning based image processing tool for enhanced organoid analysis

Contrary to 2D cells, 3D organoid structures are composed of diverse cell types and exhibit morphologies of various sizes. Although researchers frequently monitor morphological changes, analyzing every structure with the naked eye is difficult. Given that deep learning (DL) has been used for 2D cell image segmentation, a trained DL model may assist researchers in organoid image recognition and analysis. In this study, we developed OrgaExtractor, an easy-to-use DL model based on multi-scale U-Net, to perform accurate segmentation of organoids of various sizes. OrgaExtractor achieved an average dice similarity coefficient of 0.853 from a post-processed output, which was finalized with noise removal. Correlation between CellTiter-Glo assay results and daily measured organoid images shows that OrgaExtractor can reflect the actual organoid culture conditions. The OrgaExtractor data can be used to determine the best time point for organoid subculture on the bench and to maintain organoids in the long term.

Physical characteristics of deep learning-based image processing software in computed tomography: a phantom study.

This study aimed to assess the image characteristics of deep-learning-based image processing software (DLIP; FCT PixelShine, FUJIFILM, Tokyo, Japan) and compare it with filtered back projection (FBP), model-based iterative reconstruction (MBIR), and deep-learning-based reconstruction (DLR). This phantom study assessed the object-specific spatial resolution (task-based transfer function [TTF]), noise characteristics (noise power spectrum [NPS]), and low-contrast detectability (low-contrast object-specific contrast-to-noise ratio [CNRLO]) at three different output doses (standard: 10mGy; low: 3.9mGy; ultralow: 2.0mGy). The processing strength of DLIPFBP with A1, A4, and A9 was compared with those of FBP, MBIR, and DLR. The standard dose with high-contrast TTFs of DLIPFBP exceeded that of FBP. Low-contrast TTFs were comparable to or lower than that of FBP. The NPS peak frequency (fP) of DLIPFBP shifts to low spatial frequencies of up to 8.6% at ultralow doses compared to the standard FBP dose. MBIR shifted the most fP compared to FBP-a marked shift of up to 49%. DLIPFBP showed a CNRLO equal to or greater than that of DLR in standard or low doses. In contrast, the CNRLO of the DLIPFBP was equal to or lower than that of the DLR in ultralow doses. DLIPFBP reduced image noise while maintaining a resolution similar to commercially available MBIR and DLR. The slight spatial frequency shift of fP in DLIPFBP contributed to the noise texture degradation suppression. The NPS suppression in the low spatial frequency range effectively improved the low-contrast detectability.

Implementation and Efficient Analysis of Preprocessing Techniques in Deep Learning for Image Classification

Deep learning models have recently been preferred to perform certain image-processing tasks. Recently, with the increasing radiation, heat, and poor lighting conditions, the raw image samples may contain noisy and ambiguous information. To process these images, the deep learning model requires a large number of data samples to learn the missing information from other clear data samples. This necessitates training the neural network with a huge dataset. The researchers are now attempting to filter and improve such noisy images via preprocessing in order to provide valid and accurate feature information to the neural network layers. However, certain research studies claim that some useful information may be lost when the image is not preprocessed with an appropriate filter or enhancement technique. The MSA (meta-synthesis and analysis) approach is utilized in this work to present the impact of the image processing applications done with and without preprocessing steps. Also, this work summarizes the existing deep learning-based image processing models utilizing or not preprocessing steps in their implementation. This work has also found that 85% of the existing techniques involve a preprocessing step while developing a deep learning model. However, a maximum accuracy of 96.89% is observed on Sine-Net when it is implemented without a preprocessing and the same model gave 96.85% when implemented with preprocessing. This research provides various research insights on the requirement and non-requirement of preprocessing steps in a deep learning-based implementation.

Integrating blockchain and deep learning for intelligent greenhouse control and traceability

This research presents a solution that combines deep learning-based image processing, blockchain technology, and the Internet of Things (IoT) to achieve smarter control and traceability in greenhouse operations within the agricultural sector. By integrating these technologies, the aim is to overcome challenges posed by climate change, plant growth, limited agricultural land, and water scarcity, while enhancing crop yields and ensuring efficient and secure operations. The proposed system automates image capture, measurement, storage, and monitoring of environmental parameters in greenhouses, utilizing highly accurate image processing techniques with a 98% success rate. The integration of blockchain technology establishes an immutable and transparent record of transactions and data points, thereby improving traceability across the agricultural supply chain. This comprehensive approach fosters accountability, transparency, and trust, empowering consumers to make well-informed decisions regarding the products they purchase. Ultimately, this research contributes to advancing efficient and sustainable agricultural practices.

Deep Learning Enhanced Volumetric Photoacoustic Imaging of Vasculature in Human.

The development of high-performance imaging processing algorithms is a core area of photoacoustic tomography. While various deep learning based image processing techniques have been developed in the area, their applications in 3D imaging are still limited due to challenges in computational cost and memory allocation. To address those limitations, this work implements a 3D fully-dense (3DFD) U-net to linear array based photoacoustic tomography and utilizes volumetric simulation and mixed precision training to increase efficiency and training size. Through numerical simulation, phantom imaging, and in vivo experiments, this work demonstrates that the trained network restores the true object size, reduces the noise level and artifacts, improves the contrast at deep regions, and reveals vessels subject to limited view distortion. With these enhancements, 3DFD U-net successfully produces clear 3D vascular images of the palm, arms, breasts, and feet of human subjects. These enhanced vascular images offer improved capabilities for biometric identification, foot ulcer evaluation, and breast cancer imaging. These results indicate that the new algorithm will have a significant impact on preclinical and clinical photoacoustic tomography.

Confirmation of Final Bolt Tightening via Deep Learning-Based Image Processing

In Japan, the final tightening of bolts in the bolt-tightening operations is guaranteed to have been performed correctly by visually determining the change in markings during the temporary tightening operation performed by the technician. However, the engineer must confirm many bolts; further, the amount of time needed for the confirmation work and the inability to keep an objective record of the confirmation results present problems. To solve these problems, we developed a system for automating the final tightening of bolts using deep learning-based image-processing technology. The proposed system takes as input videos of bolt fastening points, extracts individual bolts, extracts markings on the extracted bolts, and makes fastening decisions based on the markings. In the judgment stage, the system processes information on each bolt where a marking is detected; thus, it is possible to leave this information as objective data. In this paper, we evaluated the accuracy of each automated step using an actual bridge video. We also compared the confirmation time with human confirmation. As a result of the confirmation, our proposed method reduces the confirmation time by about 33% in comparison to human confirmation.

Measuring Surface Characteristics in Sustainable Machining of Titanium Alloys Using Deep Learning-Based Image Processing

Measuring Surface Characteristics in Sustainable Machining of Titanium Alloys Using Deep Learning-Based Image Processing

Image Feature Fusion Method Based on Edge Detection

Deep learning-based image processing algorithms have developed rapidly in the past decade and have shown significant improvements to extract image features when both sufficient computing power and big data are accessible. Thus, rapid advances in applications such as facial recognition and autonomous driving have been one of the implementation areas. On the other hand, edges as a low-level prevalence feature in images with independent semantics are practically adapted to attain better outcomes. However, neural network-based image feature extraction focusing on texture rather than shape leads to insufficient accuracy. To address this issue, an edge feature extraction method utilizing both conventional operators such as HDE and Sobel and a deep learning-based method is proposed to classify and retrieve images with better accuracy outcomes. By doing so, a large amount of data needed to conduct deep learning-based methods is decreased, the transferability of the model is achieved, classification and retrieval accuracies are enhanced, and the data is compressed. All these better results are attained with benchmark data sets. As a result, all these are achieved by proposing a novel method.

Computed Tomography of Flake Graphite Ore: Data Acquisition and Image Processing

A solid knowledge of the mineralogical properties (e.g., flake size, flake size distribution, purity, shape) of graphite ores is necessary because different graphite classes have different product uses. To date, these properties are commonly examined using well-established optical microscopy (OM), scanning electron microscopy with energy dispersive X-ray spectroscopy (SEM-EDS) and SEM-based automated image analysis. However, these 2D methods may be subject to sampling errors and stereological effects that can adversely affect the quality of the analysis. X-ray microcomputed tomography (CT) is a nondestructive imaging technique allowing for examination of the interior and exterior of solid materials such as rocks and ores in 3D. This study aimed to explore whether CT can provide additional mineralogical information for the characterisation of graphite ores. CT was used in combination with traditional techniques (XRD, SEM-EDS, OM) to examine a flake graphite ore in 3D. A scanning protocol for the examined graphite ore was established to acquire high-quality CT data. Quantitative mineralogical information on key properties of graphite was obtained by developing a deep learning-based image processing strategy. The results demonstrate that CT allows for the 3D visualisation of graphite ores and provides valid and reliable quantitative information on the quality-determining properties that currently cannot be obtained by other analytical tools. CT allows improved assessment of graphite deposits and their beneficiation.

Analyzing angiogenesis on a chip using deep learning-based image processing.

Angiogenesis, the formation of new blood vessels from existing vessels, has been associated with more than 70 diseases. Although numerous studies have established angiogenesis models, only a few indicators can be used to analyze angiogenic structures. In the present study, we developed an image-processing pipeline based on deep learning to analyze and quantify angiogenesis. We utilized several image-processing algorithms to quantify angiogenesis, including a deep learning-based cell nuclear segmentation algorithm and image skeletonization. This method could quantify and measure changes in blood vessels in response to biochemical gradients using 16 indicators, including length, width, number, and nuclear distribution. Moreover, this procedure is highly efficient for the three-dimensional quantitative analysis of angiogenesis and can be applied to diverse angiogenesis investigations.

Deep Learning based monocular fill level detection for an eKanban system

Timely material replenishment is a key requirement for realizing lean manufacturing. As the monitoring of inventory levels and triggering replenishment is still often carried out manually by human workers, the automation of this process offers great potential for optimization. Existing automation solutions have either a high cost or high integration effort. Therefore, by using modern advancements in deep learning-based image processing, we propose low-cost sensor modules for automated fill level detection in load carriers. An inexpensive RGB camera represents the main sensor for the measurement to keep costs low. A convolutional neural network model is used to determine the fill levels.In contrast to usual monocular depth estimation approaches, our method has just the determination of the fill level as a goal to achieve high precision. The algorithm was trained and tested on the base of two datasets and achieved an average deviation of 7% to the true fill level value, verifying the viability of this approach.

Immersive Metaverse and Digital Twin Technologies, Deep Learning-based Image Processing and Motion Planning Algorithms, and 3D Virtual Space Networking and Data Visualization Tools in Interactive Digital Worlds

Immersive Metaverse and Digital Twin Technologies, Deep Learning-based Image Processing and Motion Planning Algorithms, and 3D Virtual Space Networking and Data Visualization Tools in Interactive Digital Worlds

Avoiding Shortcut-Learning by Mutual Information Minimization in Deep Learning-Based Image Processing

Avoiding Shortcut-Learning by Mutual Information Minimization in Deep Learning-Based Image Processing