3D Image Recognition Research Articles

Synthetic Data Enhancement and Network Compression Technology of Monocular Depth Estimation for Real-Time Autonomous Driving System.

Accurate 3D image recognition, critical for autonomous driving safety, is shifting from the LIDAR-based point cloud to camera-based depth estimation technologies driven by cost considerations and the point cloud's limitations in detecting distant small objects. This research aims to enhance MDE (Monocular Depth Estimation) using a single camera, offering extreme cost-effectiveness in acquiring 3D environmental data. In particular, this paper focuses on novel data augmentation methods designed to enhance the accuracy of MDE. Our research addresses the challenge of limited MDE data quantities by proposing the use of synthetic-based augmentation techniques: Mask, Mask-Scale, and CutFlip. The implementation of these synthetic-based data augmentation strategies has demonstrably enhanced the accuracy of MDE models by 4.0% compared to the original dataset. Furthermore, this study introduces the RMS (Real-time Monocular Depth Estimation configuration considering Resolution, Efficiency, and Latency) algorithm, designed for the optimization of neural networks to augment the performance of contemporary monocular depth estimation technologies through a three-step process. Initially, it selects a model based on minimum latency and REL criteria, followed by refining the model's accuracy using various data augmentation techniques and loss functions. Finally, the refined model is compressed using quantization and pruning techniques to minimize its size for efficient on-device real-time applications. Experimental results from implementing the RMS algorithm indicated that, within the required latency and size constraints, the IEBins model exhibited the most accurate REL (absolute RELative error) performance, achieving a 0.0480 REL. Furthermore, the data augmentation combination of the original dataset with Flip, Mask, and CutFlip, alongside the SigLoss loss function, displayed the best REL performance, with a score of 0.0461. The network compression technique using FP16 was analyzed as the most effective, reducing the model size by 83.4% compared to the original while maintaining the least impact on REL performance and latency. Finally, the performance of the RMS algorithm was validated on the on-device autonomous driving platform, NVIDIA Jetson AGX Orin, through which optimal deployment strategies were derived for various applications and scenarios requiring autonomous driving technologies.

A 3D local feature description algorithm based on point distribution

It is well known that the local feature descriptor is playing an important role in 3D image recognition. But the current local feature descriptors have poor ability to describe target features in situations such as noise interference, be obscured, and changes in data resolution. On the basis of point distribution, we propose a new 3D local feature descriptor, the Point Distribution Signature Histogram(PDSH), which is relatively simple and easy to implement and has a fast identification speed by simplifying Local Reference Frame(LRF). We also rotate the local surface of the point cloud from multiple angles and project them onto three coordinate planes of the LRF. The projection planes are segmented at the same angle, and the distance information of the edge points and the count of projected points in each segment are statistically accumulated. Finally, the sub-feature vectors of the three coordinate planes are concatenated to obtain the feature descriptor of the key points. Experimental results show that our PDSH descriptor achieves a descriptiveness of 98 % on the pollution-free point cloud data set and better performance than the classical descriptors.

Recognition of 3D Images by Fusing Fractional-Order Chebyshev Moments and Deep Neural Networks.

In order to achieve efficient recognition of 3D images and reduce the complexity of network parameters, we proposed a novel 3D image recognition method combining deep neural networks with fractional-order Chebyshev moments. Firstly, the fractional-order Chebyshev moment (FrCM) unit, consisting of Chebyshev moments and the three-term recurrence relation method, is calculated separately using successive integrals. Next, moment invariants based on fractional order and Chebyshev moments are utilized to achieve invariants for image scaling, rotation, and translation. This design aims to enhance computational efficiency. Finally, the fused network embedding the FrCM unit (FrCMs-DNNs) extracts depth features to analyze the effectiveness from the aspects of parameter quantity, computing resources, and identification capability. Meanwhile, the Princeton Shape Benchmark dataset and medical images dataset are used for experimental validation. Compared with other deep neural networks, FrCMs-DNNs has the highest accuracy in image recognition and classification. We used two evaluation indices, mean square error (MSE) and peak signal-to-noise ratio (PSNR), to measure the reconstruction quality of FrCMs after 3D image reconstruction. The accuracy of the FrCMs-DNNs model in 3D object recognition was assessed through an ablation experiment, considering the four evaluation indices of accuracy, precision, recall rate, and F1-score.

A 3D motion image recognition model based on 3D CNN-GRU model and attention mechanism

Moving image recognition has become a well-explored problem in computer vision. However, it is difficult for the traditional convolutional neural network (CNN) model to effectively capture timing information in motion. For better use of video sequence features and to improve the accuracy of action recognition, Therefore, this paper proposes a Three-dimensional CNN (3DCNN) model based on Gated Recurrent Unit (GRU) with an attention mechanism. The model leverages 3DCNN for deep feature extraction from video frames, employs GRU to capture the temporal dynamics of feature sequences and incorporates an attention mechanism to emphasize key frames, which improves moving image recognition. Demonstrating superior accuracy in’Cross-Subject’ and’Cross-View’ evaluations, our model surpasses standard benchmarks with accuracies of 83.2% and 87.3% respectively.

Corrosion characteristics and evaluation of galvanized high-strength steel wire for bridge cables based on 3D laser scanning and image recognition

The study investigates the corrosion behavior of galvanized high-strength steel wire, which significantly affects the bearing capacity and residual fatigue life of cables in cable-supported bridges. Through standardized salt spray accelerated corrosion tests, steel wires with varying corrosion degrees are obtained. Three-dimensional laser scanning technology reconstructs the corroded wire surface, while image recognition techniques extract geometric characteristics of corrosion pits. Fitting expressions are established to correlate the minimum cross-sectional area, maximum pit depth and dip angle, and mass loss ratio of the wire. Statistical analysis reveals log-normal distributions for the lengths of major and minor axes of corrosion pits over time. Fitting curves for elongation after fracture, nominal yield strength, and nominal ultimate strength, along with elastic modulus distribution, are derived for different mass loss ratios. Comparing corrosion grading criteria, the study quantitatively describes corrosion characteristic parameters for distinct corrosion grades. These insights enhance understanding of corrosion behavior and mechanical property degradation in high-strength steel wires, offering a quantitative framework for assessing and classifying corrosion extent based on measurable parameters.

Discarded e-waste/printed circuit boards: a review of their recent methods of disassembly, sorting and environmental implications

AbstractThe improper disposal of discarded electronic and electrical equipment raises environmental and health concerns, spanning air pollution to water and soil contamination, underscoring the imperative for responsible management practises. This review explores the complex composition of discarded printed circuit boards (DPCBs), crucial components in electronic devices. Comprising substrates, electronic elements and solder, DPCBs showcase a heterogeneous structure with metal (30.0–50.0%) and non-metal (50.0–70.0%) fractions. Notably abundant in precious metals such as Au, Ag, and Pd, DPCBs offer a compelling avenue for recycling initiatives. The inclusion of heavy metals and flame retardants adds complexity, necessitating environmentally sound disposal methods. Ongoing research on smart disassembly, utilising 3D image recognition technology, underscores the importance of accurate identification and positioning of electronic components (ECs). The targeted approach of smart disassembly, centred on valuable components, highlights its significance, albeit with challenges in equipment costs and capacity limitations. In mechanical disassembly, techniques such as grinding and heat application are employed to extract ECs, with innovations addressing gas emissions and damage induced by overheating. Chemical disassembly methods, encompassing epoxy resin delamination and tin removal, present promising recovery options, whilst the integration of chemical and electrochemical processes shows potential. Efficient sorting, encompassing both manual and automated methods, is imperative post-disassembly, with smart sorting technologies augmenting accuracy in the identification and categorisation of ECs. In addition, explorations into NH3/NH4+ solutions for selective metal recovery underscore challenges and stress the necessity for meticulous process optimisation in environmentally sustainable PCB recycling. Challenges and future perspectives have also been expounded.

Explainable Feature Extraction and Prediction Framework for 3D Image Recognition Applied to Pneumonia Detection

Explainable machine learning is an emerging new domain fundamental for trustworthy real-world applications. A lack of trust and understanding are the main drawbacks of deep learning models when applied to real-world decision systems and prediction tasks. Such models are considered as black boxes because they are unable to explain the reasons for their predictions in human terms; thus, they cannot be universally trusted. In critical real-world applications, such as in medical, legal, and financial ones, an explanation of machine learning (ML) model decisions is considered crucially significant and mandatory in order to acquire trust and avoid fatal ML bugs, which could disturb human safety, rights, and health. Nevertheless, explainable models are more than often less accurate; thus, it is essential to invent new methodologies for creating interpretable predictors that are almost as accurate as black-box ones. In this work, we propose a novel explainable feature extraction and prediction framework applied to 3D image recognition. In particular, we propose a new set of explainable features based on mathematical and geometric concepts, such as lines, vertices, contours, and the area size of objects. These features are calculated based on the extracted contours of every 3D input image slice. In order to validate the efficiency of the proposed approach, we apply it to a critical real-world application: pneumonia detection based on CT 3D images. In our experimental results, the proposed white-box prediction framework manages to achieve a performance similar to or marginally better than state-of-the-art 3D-CNN black-box models. Considering the fact that the proposed approach is explainable, such a performance is particularly significant.

Temporal and periorbital depressions identified by 3D images are correlated with malnutrition phenotypes in cancer patients: A pilot study.

Prompt diagnosis of malnutrition and appropriate interventions can substantially improve the prognosis of patients with cancer; however, it is difficult to unify the tools for screening malnutrition risk. 3D imaging technology has been emerging as an approach to assisting in the diagnosis of diseases, and we designed this study to explore its application value in identifying the malnutrition phenotype and evaluating nutrition status. Hospitalized patients treating with maintenance chemotherapy for advanced malignant tumor of digestive system were recruited from the Department of Oncology, whose NRS 2002 score > 3. Physical examination and body composition data of patients at risk for malnutrition were analyzed by physicians trained to complete a subjective global assessment. The facial depression index was recognized using the Antera 3D® system, temporal and periorbital depression indexes were acquired using the companion software Antera Pro. This software captures quantitative data of depression volume, affected area, and maximum depth of temporal and periorbital concave areas. A total of 53 inpatients with malnutrition-related indicators were included. The volume of temporal depression was significantly negatively correlated with upper arm circumference (r = -0.293, p = 0.033) and calf circumference (r = -0.285, p = 0.038). The volume and affected area of periorbital depression were significantly negatively correlated with fat mass index (r = -0.273, p = 0.048 and r = -0.304, p = 0.026, respectively) and percent body fat (r = -0.317, p = 0.021 and r = -0.364, p = 0.007, respectively). The volume and affected area of temporal depression in patients with muscle loss phenotype (low arm circumference/low calf circumference/low handgrip strength/low fat-free mass index) were significantly higher than those in patients without muscle loss. Moreover, patients with fat mass loss phenotype (low fat mass index) showed a significant increase in the volume and affected area of periorbital depression. The facial temporal region, and periorbital depression indicators extracted by 3D image recognition technology were significantly associated with the phenotype of malnutrition-related muscle and fat loss and showed a trend of grade changes in the population of different subjective global assessment nutritional classifications.

Using Drones for Thermal Imaging Photography and Building 3D Images to Analyze the Defects of Solar Modules

In this research, drones were used to capture thermal images and detect different types of failure of solar modules, and MATLAB® image analysis was also conducted to evaluate the health of the solar modules. The processes included image acquisition and transmission by drone, grayscale conversion, filtering, 3D image construction, and analysis. The analyzed targets were the solar modules installed on buildings. The results showed that the employment of drones to monitor solar module farms could significantly improve inspection efficiency. Moreover, by combining the mean and median filtering techniques, an innovative box filtering method was successfully created. Additionally, this study compared the differences between the mean, median, and box filtering techniques, and proved that the 3D image improved by box filtering is a more convenient and accurate way to check the health of solar modules than the mean and median filtering methods. In addition, this new method can simplify the maintenance process, as it helps maintenance personnel to determine whether to replace the solar modules on site, achieving the goal of power generation efficiency enhancement. It is worth noting that 3D image recognition technology can enhance the clarity of thermal images, thereby providing maintenance personnel with better defect diagnosis capability. It is also able to provide the temperature value of the defect zone, and to indicate the scale of defects through the cumulative temperature chart, so the 3D image is qualified as a quantitative and qualitative indicator. The analysis of the transmitted image is innovative that it not only can locate the defect area of the module, but also can display the temperature of the module, providing more information for maintenance personnel.

ACE-SNN: Algorithm-Hardware Co-design of Energy-Efficient & Low-Latency Deep Spiking Neural Networks for 3D Image Recognition.

High-quality 3D image recognition is an important component of many vision and robotics systems. However, the accurate processing of these images requires the use of compute-expensive 3D Convolutional Neural Networks (CNNs). To address this challenge, we propose the use of Spiking Neural Networks (SNNs) that are generated from iso-architecture CNNs and trained with quantization-aware gradient descent to optimize their weights, membrane leak, and firing thresholds. During both training and inference, the analog pixel values of a 3D image are directly applied to the input layer of the SNN without the need to convert to a spike-train. This significantly reduces the training and inference latency and results in high degree of activation sparsity, which yields significant improvements in computational efficiency. However, this introduces energy-hungry digital multiplications in the first layer of our models, which we propose to mitigate using a processing-in-memory (PIM) architecture. To evaluate our proposal, we propose a 3D and a 3D/2D hybrid SNN-compatible convolutional architecture and choose hyperspectral imaging (HSI) as an application for 3D image recognition. We achieve overall test accuracy of 98.68, 99.50, and 97.95% with 5 time steps (inference latency) and 6-bit weight quantization on the Indian Pines, Pavia University, and Salinas Scene datasets, respectively. In particular, our models implemented using standard digital hardware achieved accuracies similar to state-of-the-art (SOTA) with ~560.6× and ~44.8× less average energy than an iso-architecture full-precision and 6-bit quantized CNN, respectively. Adopting the PIM architecture in the first layer, further improves the average energy, delay, and energy-delay-product (EDP) by 30, 7, and 38%, respectively.

Research on the analysis method of digital media art communication based on 3D image recognition

3D image has important applications in reverse engineering, pattern recognition, artificial intelligence and other fields. 3D image processing technology has become a research hotspot in the field of 3D image, and the dissemination of digital media technology has gradually become a key application content. Therefore, this paper mainly studies the key technologies such as 3D image mosaic, surface reconstruction and model simplification, studies the 2D image filtering, and analyzes the transmission of 3D technology and digital media technology. It can be seen from an example that digital media communication has a good application effect in 3D technology.

3D image recognition using new set of fractional-order Legendre moments and deep neural networks

3D Image representation is an important topic in computer vision and pattern recognition. Recently, 3D image analysis by fractional-order orthogonal moments has provided a new research direction, which has prompted researchers to think about efficient and fast classification. In this paper, the authors derived novel sets of fractional-order Legendre moment invariants (FrLMIs), for 3D object description and recognition. Therefore, an analysis of the performance of reconstruction and classification based on fractional-order moment invariants and Deep Neural Networks (DNNs) by changing the number of descriptors was presented. Accordingly, the performance of these proposed fractional-order moments and moment invariants are evaluated through several appropriate experiments, including 3D image reconstruction, region of interest feature extraction, invariance with respect to the geometric transformations and noisy, and 3D Object classification using different fractional parameters. The superiority of the proposed method is verified by comparing the classification percentages obtained by varying the amount of data used during the training process in comparison with the existing methods. The work presented will help to create new neural network architectures that take advantage of the descriptive capacity of 3D fractional-order moments. Finally, these fractional-order moments are very fast and computationally inexpensive which could be useful in many computer vision applications. Based on these characteristics, the proposed FrOLMs and FrLMIs outperformed all existing orthogonal moments.

Achieve accurate recognition of 3D point cloud images by studying the scattering characteristics of typical targets

Three-dimensional imaging lidar has been widely used in various fields due to its high measurement accuracy, strong directionality, and three-dimensional visualization. At present, the three-dimensional point cloud imaging method of lidar mainly depends on the distance information of the target, but the intensity information of the target can play an auxiliary role in the recognition of the target, and it has also received extensive attention. The combination of intensity information and distance information enables the system to make more accurate decisions and improve the redundancy of the system. When scanning non-cooperative targets, if there is only distance information, the target will not be accurately identified, which may cause errors in decision-making. This paper studies and analyzes the scattering characteristics of typical lidar targets. The intensity of the scattered echo signal of the target is related to the target material, target distance, gray value and detection angle, so the detector's receiving scattering rate is different. For the extraction of the intensity and the distance signal, we propose a double-scale intensity-weighted centroid algorithm to achieve it, which ensures the accuracy of the signal. To this end, we build a database of the scattering characteristics of related targets, combined with the original distance information to realize the accurate recognition of the target in the lidar 3D point cloud image.

RETRACTED: Fast 3D image reconstruction algorithm based on artificial intelligence technology

In order to improve the 3D reconstruction capability of high-resolution fine-grained 3D images, a fast 3D image reconstruction algorithm based on artificial intelligence technology is proposed. The cross-gradient sharpening detection method is used to collect features and extract information from high-resolution fine-grained three-dimensional images, and establish an edge contour feature detection model for high-resolution fine-grained three-dimensional images. Combining the salient feature analysis method and the subspace feature analysis method to cluster and analyze the high-resolution fine-grained three-dimensional image. In the artificial intelligence environment, the saliency of the three-dimensional image is detected and analyzed, and the multi-dimensional segmentation and gray histogram of the high-resolution fine-grained three-dimensional image are reconstructed through the subspace segmentation method. According to the reconstruction results of the gray histogram, fast 3D image reconstruction and image fusion processing are performed. Finally, the accurate detection and recognition of the reconstructed image is realized. The simulation results show that this method has a good effect on 3D image reconstruction, and the time cost of image reconstruction is relatively short. It improves the recognition and feature analysis capabilities of high-resolution fine-grained 3D images, and has good application value in the reconstruction, detection and recognition of high-resolution fine-grained 3D images.

New set of non-separable 2D and 3D invariant moments for image representation and recognition

The extraction of feature vectors of the images is the most important step in image recognition, because it allows a good description of the forms to be recognized. This step is an operation that makes it possible to convert an image into a vector of real or complex values that can serve as a signature for this image. For an accurate recognition system, the used feature vector must be invariant to the three image transformations (translation, rotation and scale), which means that, the descriptor vectors of the image and the transformed image by translation, rotation or scale must be equal. Given the importance of moments and their invariants in pattern recognition and imaging, based on Newton’s binomial and trinomial formulas and the normalized central moments, we construct in this paper four new series of moments for 2D and 3D image recognition, which are invariant to translation, scaling and rotation (TSR): the first is a set of non-orthogonal moment invariants for 2D images (2dMIs), the second is a set of orthogonal moment invariants for 2D images (2dOMIs), the third is a set of non-orthogonal moment invariants for 3D images (3dMIs) and the fourth is a set of orthogonal moment invariants for 3D images (3dOMIs). Using the proposed invariant moments (2dMIs), (2dOMIs), (3dMIs) and (3dOMIs), we construct two types of image descriptor vectors for 2D images and two types for 3D images, which are invariant to translation, rotation and scale. A series of experiments is performed to validate this new set of invariant moments and compare its performance with the existing invariants moments. The obtained results ensure the superiority of the proposed moments over all existing moments in image recognition. Experiments on processing time show that the proposed method is faster than the existing orthogonal invariant moments.

Three-Dimensional Image Recognition of Athletes' Wrong Motions Based on Edge Detection

The traditional 3D visual motion amplitude tracking algorithms cannot acquire the complete contour features, not to mention the correction of wrong motions in sports training. To solve the problem, this paper designs a 3D visual image recognition method based on contourlet domain edge detection, and applies it to the recognition of athlete’s wrong motions in sports training. Firstly, the visual reconstruction and feature analysis of human motions were carried out, and the edge detection features were extracted by edge detection algorithm. Then, a 3D visual motion amplitude tracking method was proposed based on improved inverse kinematics. The simulation results show that the proposed algorithm can effectively realize the recognition of 3D visual images of athlete motions, and improve the correction and judgment ability of athlete motions.

Accurate Recognition and Simulation of 3D Visual Image of Aerobics Movement

The structure of the deep artificial neural network is similar to the structure of the biological neural network, which can be well applied to the 3D visual image recognition of aerobics movements. A lot of results have been achieved by applying deep neural networks to the 3D visual image recognition of aerobics movements, but there are still many problems to be overcome. After analyzing the expression characteristics of the convolutional neural network model for the three-dimensional visual image characteristics of aerobics, this paper builds a convolutional neural network model. The model is improved on the basis of the traditional model and unifies the process of aerobics 3D visual image segmentation, target feature extraction, and target recognition. The convolutional neural network and the deep neural network based on autoencoder are designed and applied to aerobics action 3D visual image test set for recognition and comparison. We improve the accuracy of network recognition by adjusting the configuration parameters in the network model. The experimental results show that compared with other simple models, the model based on the improved AdaBoost algorithm can improve the final result significantly when the accuracy of each model is average. Therefore, the method can improve the recognition accuracy when multiple neural network models with general accuracy are obtained, thereby avoiding the complicated parameter adjustment process to obtain a single optimal network model.

Machine-Learning Image Recognition Enhances Rock Classification

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper SPE 196657, “Machine Learning for 3D Image Recognition To Determine Porosity and Lithology of Heterogeneous Carbonate Rock,” by Omar Al-Farisi, SPE, Hongtao Zhang, and Aikifa Raza, Khalifa University of Science and Technology, prepared for the 2019 SPE Reservoir Characterization and Simulation Conference and Exhibition, Abu Dhabi, 17-19 September. The paper has not been peer reviewed. Automated image-processing algorithms can improve the quality and speed in classifying the morphology of heterogeneous carbonate rock. Several commercial products have produced petrophysical properties from 2D images and, to a lesser extent, from 3D images. Images are mainly microcomputed tomography (μCT), optical images of thin sections, or magnetic resonance images (MRI). However, most successful work is from homogeneous and clastic rocks. In the complete paper, the authors have demonstrated a machine-learning-assisted image-recognition (MLIR) approach to determine the porosity and lithology of heterogeneous carbonate rock by analyzing 3D images from µCT and MRI. Introduction The authors’ literature review has revealed the pressing need to perform 3D image processing instead of 2D. Achieving this goal requires an interdisciplinary approach. This study deployed new analysis and verification approaches, including 3D micromodels (3DMM) with various micropore sizes and uses 3DMM as an image-processing calibration reference. Additionally, a new image-resolution enhancement for quality segmentation is developed. Porosity was determined mainly using two methods. The first is a standalone image processing, in which image-information extraction was successful. The second is MLIR. The difference between image processing and image analysis is important. If image processing enables extraction of meaningful data, then image analysis is the ability to interpret this data through numerical analysis.

A new fast algorithm to compute moment 3D invariants of generalized Laguerre modified by fractional-order for pattern recognition

Orthogonal moments are the projections of image functions on particular functions of the kernel. They play an essential role in image extraction: rotation, scaling, translation invariance, object recognition, image classification, image noise robustness, and low information redundancy. These moments are derived from orthogonal polynomials that can be continuous or discrete. This paper focuses on the fractional-order modified generalized Laguerre moment invariants (FMGLMIs), which is a generalization of the traditional integer order one. In this research, we have developed a new algorithm to compute the 3D invariant moments of FMGLMIs based on the 3D image cuboid representation, our proposed calculation method can improve the efficiency of 3D invariant moment calculation to maintain numerical stability and significantly reduce calculation time with very satisfactory accuracy. To check this new algorithm, the calculation of 3D invariant moments gives very encouraging results for the invariability property of the proposed method with respect to different geometric transformations and noise degradations of 3D images, classification and recognition of 3D images and the calculation time of fractional-order invariants proposed. Finally, the experimental results show that the proposed method makes it possible to construct fractional-order modified generalized Laguerre invariant moments offering better performances for image analysis and pattern recognition.

An improved CapsNet applied to recognition of 3D vertebral images

Deep learning is currently widely applied in medical image processing and has achieved good results. However, recognizing vertebrae via image processing remains a challenging problem due to their complex spatial structures. CapsNet is a newly proposed network whose characteristics compensate for some shortcomings of traditional CNNs, and it has been shown to perform well on many tasks, including medical image recognition. In this paper, we applied a modified CapsNet to recognise 3D vertebral images by introducing an RNN module into CapsNet to further enhance its learning ability. This new network is called RNNinCaps, and it achieves the highest recognition performance on 3D vertebral images (the average accuracy of RNNinCaps exceeds the accuracy of the original CapsNet by 46.2% and that of a traditional CNN by 12.6%). RNNinCaps also performs better than several mainstream networks. RNNinCaps can promotes CapsNet’s application in the field of 3D medical image recognition.