Popular Convolutional Neural Networks Research Articles

Automatic Classification of Simulated Breast Tomosynthesis Whole Images for the Presence of Microcalcification Clusters Using Deep CNNs.

Microcalcification clusters (MCs) are among the most important biomarkers for breast cancer, especially in cases of nonpalpable lesions. The vast majority of deep learning studies on digital breast tomosynthesis (DBT) are focused on detecting and classifying lesions, especially soft-tissue lesions, in small regions of interest previously selected. Only about 25% of the studies are specific to MCs, and all of them are based on the classification of small preselected regions. Classifying the whole image according to the presence or absence of MCs is a difficult task due to the size of MCs and all the information present in an entire image. A completely automatic and direct classification, which receives the entire image, without prior identification of any regions, is crucial for the usefulness of these techniques in a real clinical and screening environment. The main purpose of this work is to implement and evaluate the performance of convolutional neural networks (CNNs) regarding an automatic classification of a complete DBT image for the presence or absence of MCs (without any prior identification of regions). In this work, four popular deep CNNs are trained and compared with a new architecture proposed by us. The main task of these trainings was the classification of DBT cases by absence or presence of MCs. A public database of realistic simulated data was used, and the whole DBT image was taken into account as input. DBT data were considered without and with preprocessing (to study the impact of noise reduction and contrast enhancement methods on the evaluation of MCs with CNNs). The area under the receiver operating characteristic curve (AUC) was used to evaluate the performance. Very promising results were achieved with a maximum AUC of 94.19% for the GoogLeNet. The second-best AUC value was obtained with a new implemented network, CNN-a, with 91.17%. This CNN had the particularity of also being the fastest, thus becoming a very interesting model to be considered in other studies. With this work, encouraging outcomes were achieved in this regard, obtaining similar results to other studies for the detection of larger lesions such as masses. Moreover, given the difficulty of visualizing the MCs, which are often spread over several slices, this work may have an important impact on the clinical analysis of DBT images.

Using different training strategies for urban land-use classification based on convolutional neural networks

Urban land-use scene classification from high-resolution remote-sensing imagery at high quality and accuracy is of paramount interest for urban planning, government policy-making and urban change detection. In recent years, urban land-use classification has become an ongoing task in areas addressable primarily by remote sensing, and numerous deep learning algorithms have achieved high performance on this task. However, both dataset and methodology problems still exist in the current approaches. Previous studies have relied on limited data sources, resulting in saturated classification results, and they have difficulty achieving comprehensive classification results. The previous methods based on convolutional neural networks (CNNs) focused primarily on model architecture rather than on the hyperparameters. Therefore, to achieve more accurate classification results, in this study, we constructed a new large dataset for urban land-use scene classification. More than thirty thousand remote sensing scene images were collected to create a dataset with balanced class samples that includes both higher intra-class variations and smaller inter-class dissimilarities than do the previously available public datasets. Then, we analysed two possible strategies for exploiting the capabilities of three existing popular CNNs on our datasets: full training and fine tuning. For each strategy, three types of learning rate decay were applied: fixed, exponential and polynomial. The experimental results indicate that fine tuning tends to be the best-performing strategy, and using ResNet-V1-50 and polynomial learning rate decay achieves the best results for the urban land-use scene classification task.

Uncertainty measurement-guided iterative sample selection via shallow convolutional neural network for hyperspectral image classification

Continuously improving the accuracy is a hot topic in hyperspectral image (HSI) classification with small-scale samples, due to the high label noise of traditional labeling systems and the high cost of expert labeling systems. We focus on constructing a smaller and more informative training sample set, so an iterative sample selection method guided by uncertainty measurement (ISS-Un) is proposed. The method learns shallow and deep features in the spectral and spatial domains via a convolutional neural network (CNN), where an uncertainty measurement algorithm such as least confidence (LC), marginal sampling (MS) or entropy (Ent) is used to iteratively select high-quality samples for the training set. In addition, we propose a more efficient uncertainty measurement algorithm named margin-entropy fusion (MEF) algorithm to integrate multiple-criteria information. The proposed method is compared with the conventional random sampling method. Experimental results on three HSI datasets show that the proposed ISS-Un method can significantly alleviate the redundancy of training samples and form a more compact and efficient training set, thus improving the classification performance of pixel-oriented HSI. Meanwhile, training sets constructed based on different uncertainty measurement algorithms are applied to five popular CNN models to verify the quality and generalizability of the selected samples. The results show that these training sets work better than random training sampling. Moreover, the proposed MEF algorithm outperforms the LC, MS, and Ent algorithms in selecting samples and is the main recommended scheme.

Metamorphic Testing of Image Classification and Consistency Analysis Using Clustering

Testing deep learning systems requires expensive labeled data. In recent years, researchers began to leverage metamorphic testing to address this issue. However, metamorphic relations on image data remain poorly understood. To gain a deeper understanding of these metamorphic relations, we survey common image operations modeling covariate shift, manually classify and categorize the underlying metamorphic relations, and conduct experiments to validate our classifications. In our experiments, we train three popular convolutional neural network architectures on an image classification task. Next, we apply metamorphic operations on input test images and measure the change in classification accuracy and cross-entropy loss. A hierarchical clustering algorithm cluster these results and plots a dendrogram. We compare the groups from manual classification and the clusters from the algorithm to provide key insights. We find that Affine and Noise relations are consistent. Furthermore, we recommend metamorphic relationships to save time and better test deep learning systems in the future.

A Graph Guided Convolutional Neural Network for Surface Defect Recognition

Surface defect is a serious problem in real-world manufacturing system and it is important to use vision-based recognition to ensure the surface quality of products. Currently, due to the ability of automatic feature extraction, deep learning models, such as convolutional neural network (CNN), have been widely used in this area. However, these CNN-based models may not solve a problem well - inter-class similarities and intra-class variations (ISIV), which affect their ability of feature extraction and thus influence their recognition performance. To address this problem, this paper introduces a graph guidance mechanism into CNN to improve the ability of feature extraction, called Graph guided Convolutional Neural Network (GCNN). Firstly, GCNN defines a graph by computing the similarities between training samples. Secondly, the graph is introduced into VGG11, a popular CNN structure, to increase the inter-class distances and decrease the intra-class distances between defect samples. Meanwhile, a learnable coefficient is introduced into the training process to balance the effect of graph guidance automatically. The experimental results on four famous surface defect datasets demonstrate that the graph guidance helps CNN models have better ability of feature extraction and thus achieve better performance. Compared with state-of-the-art models, the proposed method can achieve the best performance. Furthermore, the final discussion shows that the learnable coefficient can help the proposed model to gain better performance, and that the proposed model increases a little computation cost compared to its original CNN model. <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Note to Practitioners</i> —This paper is motivated by the problem in real-world manufacturing process – inter-class similarities and intra-class variations. Most of current CNN-based models may not solve this problem well, which limits their applications in real-world system. This paper proposes a graph guidance mechanism and introduces it to the training of CNN. The proposal can improve the ability of feature extraction, and thus have better performance than those without the graph guidance. By applying the proposal to four examples, the results show that the proposal is more feasible and effective than the state-of-the-art models in surface defect recognition. Furthermore, the mechanism assists the training of CNN and is removed in the process of recognition, so it will not increase time and occupied memory in recognition, and meanwhile, only limited computation cost is increased in training process.

Detection of Plant Leaf Diseases using Transfer Learning Techniques

Agriculture is an essential field for meeting the country's increasing population's basic food needs. Meanwhile, the growth of grains and vegetables is essential for human nutrition and the global economy. Many farmers cultivate in distant places of the world, where reliable information and disease detection are lacking; yet, they rely on personal observation of grains and vegetables. Resulting in significant losses. This paper suggests an image processing based detection technique and preventive measures for plant leaf diseases in the agricultural field Using four popular convolutional neural network (CNN) models. such as the Xception model, VGG16, resNet-50, and one Custom CNN model. First, this technique is used to investigate the symptoms of diseased leaves using Kaggle datasets of several leaves. Then, using the image processing application and the Xception model, On dataset images, the feature extraction and classification procedure is used to find leaf diseases. In order to achieve better results, I used three additional CNN models: VGG16, Resnet50, and one custom CNN model.

CovidConvLSTM: A fuzzy ensemble model for COVID-19 detection from chest X-rays

The rapid outbreak of COVID-19 has affected the lives and livelihoods of a large part of the society. Hence, to confine the rapid spread of this virus, early detection of COVID-19 is extremely important. One of the most common ways of detecting COVID-19 is by using chest X-ray images. In the literature, it is found that most of the research activities applied convolutional neural network (CNN) models where the features generated by the last convolutional layer were directly passed to the classification models. In this paper, convolutional long short-term memory (ConvLSTM) layer is used in order to encode the spatial dependency among the feature maps obtained from the last convolutional layer of the CNN and to improve the image representational capability of the model. Additionally, the squeeze-and-excitation (SE) block, a spatial attention mechanism, is used to allocate weights to important local features. These two mechanisms are employed on three popular CNN models – VGG19, InceptionV3, and MobileNet to improve their classification strength. Finally, the Sugeno fuzzy integral based ensemble method is used on these classifiers’ outputs to enhance the detection accuracy further. For experiments, three chest X-ray datasets, which are very prevalent for COVID-19 detection, are considered. For all the three datasets, it is found that the results obtained by the proposed method are comparable to state-of-the-art methods. The code, along with the pre-trained models, can be found at https://github.com/colabpro123/CovidConvLSTM.

Deep transfer learning-based fully automated detection and classification of Alzheimer's disease on brain MRI.

To employ different automated convolutional neural network (CNN)-based transfer learning (TL) methods for both binary and multiclass classification of Alzheimer's disease (AD) using brain MRI. Herein, we applied three popular pre-trained CNN models (ResNet101, Xception, and InceptionV3) using a fine-tuned approach of TL on 3D T1-weighted brain MRI from a subset of ADNI dataset (n = 305 subjects). To evaluate power of TL, the aforementioned networks were also trained from scratch for performance comparison. Initially, Unet network segmentedthe MRI scans into characteristic components of gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF). The proposed networks were trained and tested over the pre-processed and augmented segmented and whole images for both binary (NC/AD + progressive mild cognitive impairment (pMCI)+stable MCI (sMCI)) and 4-class (AD/pMCI/sMCI/NC) classification. Also, two independent test sets from the OASIS (n = 30) and AIBL (n = 60) datasets were used to externally assess the performance of the proposed algorithms. The proposed TL-based CNN models achieved better performance compared to the training CNN models from scratch. On the ADNI test set, InceptionV3-TL achieved the highest accuracy of 93.75% and AUC of 92.0% for binary classification, as well as the highest accuracy of 93.75% and AUC of 96.0% for multiclass classification of AD on the whole images. On the OASIS test set, InceptionV3-TL outperformed two other models by achieving 93.33% accuracy with 93.0% AUC in binary classification of AD on the whole images. On the AIBL test set, InceptionV3-TL also outperformed two other models in both binary and multiclass classification tasks on the whole MR images and achieved accuracy/AUC of 93.33%/95.0% and 90.0%/93.0%, respectively. The GM segment as input provided the highest performance in both binary and multiclass classification of AD, as compared to the WM and CSF segments. This study demonstrates the potential of applying deep TL approach for automated detection and classification of AD using brain MRI with high accuracy and robustness across internal and external test data, suggesting that these models can possibly be used as a supportive tool to assist clinicians in creating objective opinion and correct diagnosis. We used CNN-based TL approaches and the augmentation techniques to overcome the insufficient data problem. Our study provides evidence that deep TL algorithms can be used for both binary and multiclass classification of AD with high accuracy.

A solution to supervised motor imagery task in the BCI Controlled Robot Contest in World Robot Contest

Background: One of the most prestigious competitions in the world is the World Robot Conference. This paper presents the winning solution to the supervised motor imagery (MI) task in the BCI Controlled Robot Contest in World Robot Contest 2021. Methods: Data augmentation, preprocessing, feature extraction, and model training are the main components of the solution. The model is based on EEGNet, a popular convolutional neural networks model for classifying electroencephalography data. Results: Despite the model’s lack of stability, this solution was the most successful in the task. The channels closest to the vertex were the most helpful in feature extraction. Conclusion: This solution is suitable for supervised MI tasks not only in this competition but also in future scenarios.

Building Efficient CNN Architectures for Histopathology Images Analysis: A Case-Study in Tumor-Infiltrating Lymphocytes Classification.

BackgroundDeep learning methods have demonstrated remarkable performance in pathology image analysis, but they are computationally very demanding. The aim of our study is to reduce their computational cost to enable their use with large tissue image datasets.MethodsWe propose a method called Network Auto-Reduction (NAR) that simplifies a Convolutional Neural Network (CNN) by reducing the network to minimize the computational cost of doing a prediction. NAR performs a compound scaling in which the width, depth, and resolution dimensions of the network are reduced together to maintain a balance among them in the resulting simplified network. We compare our method with a state-of-the-art solution called ResRep. The evaluation is carried out with popular CNN architectures and a real-world application that identifies distributions of tumor-infiltrating lymphocytes in tissue images.ResultsThe experimental results show that both ResRep and NAR are able to generate simplified, more efficient versions of ResNet50 V2. The simplified versions by ResRep and NAR require 1.32× and 3.26× fewer floating-point operations (FLOPs), respectively, than the original network without a loss in classification power as measured by the Area under the Curve (AUC) metric. When applied to a deeper and more computationally expensive network, Inception V4, NAR is able to generate a version that requires 4× lower than the original version with the same AUC performance.ConclusionsNAR is able to achieve substantial reductions in the execution cost of two popular CNN architectures, while resulting in small or no loss in model accuracy. Such cost savings can significantly improve the use of deep learning methods in digital pathology. They can enable studies with larger tissue image datasets and facilitate the use of less expensive and more accessible graphics processing units (GPUs), thus reducing the computing costs of a study.

Development of a multi-task learning V-Net for pulmonary lobar segmentation on CT and application to diseased lungs

To develop a multi-task learning (MTL) V-Net for pulmonary lobar segmentation on computed tomography (CT) and application to diseased lungs. The described methodology utilises tracheobronchial tree information to enhance segmentation accuracy through the algorithm's spatial familiarity to define lobar extent more accurately. The method undertakes parallel segmentation of lobes and auxiliary tissues simultaneously by employing MTL in conjunction with V-Net-attention, a popular convolutional neural network in the imaging realm. Its performance was validated by an external dataset of patients with four distinct lung conditions: severe lung cancer, COVID-19 pneumonitis, collapsed lungs, and chronic obstructive pulmonary disease (COPD), even though the training data included none of these cases. The following Dice scores were achieved on a per-segment basis: normal lungs 0.97, COPD 0.94, lung cancer 0.94, COVID-19 pneumonitis 0.94, and collapsed lung 0.92, all at p<0.05. Despite severe abnormalities, the model provided good performance at segmenting lobes, demonstrating the benefit of tissue learning. The proposed model is poised for adoption in the clinical setting as a robust tool for radiologists and researchers to define the lobar distribution of lung diseases and aid in disease treatment planning.

Generating a labeled data set to train machine learning algorithms for lithologic classification of drill cuttings

Despite significant developments in the past few years in the application of machine learning algorithms for the lithologic classification of rock samples, publicly available labeled data sets are very scarce. We open source a fully labeled data set containing more than 16,000 scanning electron microscopy (SEM) images of drill cutting samples—mounted on thin sections—from a low-permeability reservoir in western Canada. We develop a simplified image processing workflow to segment and isolate the rock chips into individual SEM images, which in turn are used to identify, classify, and quantify rock types based on textural characteristics. In addition, using this data set, we explore the use of convolutional neural networks (CNNs) as a baseline tool for acceleration and automatization of rock-type classification. Without significant modifications to popular CNN models, we obtain an accuracy of approximately 90% for the test set. Results demonstrate the potential of CNN as a fast approach for lithologic classification in low-permeability siltstone reservoirs. In addition to making the data set publicly available, we believe our workflow to segment and isolate drill cutting samples in individual images of rock chips will facilitate future research of drill cuttings properties (e.g., lithology, porosity, and particle size) using machine learning algorithms.

Applications of artificial neural networks in microorganism image analysis: a comprehensive review from conventional multilayer perceptron to popular convolutional neural network and potential visual transformer.

Microorganisms are widely distributed in the human daily living environment. They play an essential role in environmental pollution control, disease prevention and treatment, and food and drug production. The analysis of microorganisms is essential for making full use of different microorganisms. The conventional analysis methods are laborious and time-consuming. Therefore, the automatic image analysis based on artificial neural networks is introduced to optimize it. However, the automatic microorganism image analysis faces many challenges, such as the requirement of a robust algorithm caused by various application occasions, insignificant features and easy under-segmentation caused by the image characteristic, and various analysis tasks. Therefore, we conduct this review to comprehensively discuss the characteristics of microorganism image analysis based on artificial neural networks. In this review, the background and motivation are introduced first. Then, the development of artificial neural networks and representative networks are presented. After that, the papers related to microorganism image analysis based on classical and deep neural networks are reviewed from the perspectives of different tasks. In the end, the methodology analysis and potential direction are discussed.

Visibility classification and influencing-factors analysis of airport: A deep learning approach

This paper discusses the visibility classification and influencing-factor analysis of an airport, which were carried out through a data-driven deep learning approach and multiple nonlinear regression analysis. First, a deep integrated model for airport visibility classification was developed, and the relationship between runway visual range (RVR) and surface meteorological elements for different reasons was analyzed. To extract more low-level features from nighttime images, the original images were transformed into pseudo-color images, and the image pairs were input into the classification model. The proposed deep integrated model combined two popular convolutional neural network (CNN) models: VGG-16 and Xception. The results showed that the accuracy of the proposed model was 87.64%, with the F1-score being 88.58%; it achieved the best performance among other well-known experimental CNNs. Afterward, the relationships between RVR and meteorological optical range (MOR), source light intensity (I), and background light (BL) were analyzed, and the results showed that RVR increased with increasing I under the same MOR and BL conditions; however, it decreased with increasing log (BL) values under the same I and MOR conditions. RVR increased as MOR increased when other conditions remained unchanged. Subsequently, the multivariate regression analysis was conducted to fit the relationship between MOR and surface meteorological elements, and the results showed that the cubic curve was the best-fit curve with R-square values of 0.937 and 0.860 in December 2019 and March 2020, respectively. Finally, combined with the analysis of the factors influencing the RVR, a significant negative correlation was observed with relative humidity, along with a significant positive correlation with temperature and a positive correlation with average wind speed in the winter of 2019 and spring of 2020. However, the pressure factor showed the opposite correlation, possibly because the arrival of cold air in winter increases the ground pressure, which is conducive for the dissipation of pollutants. Meanwhile, the increase in temperature and decrease in pressure in spring increases the chance of disturbances in the atmosphere, thereby increasing the likelihood of adverse visibility factors. ◆ Pseudo-color images were found beneficial for improving the accuracy of visibility identification. ◆ An integrated model combined VGG-16 and Xception achieved the best performance compared to that of other popular models. ◆ The relationship between RVR and MOR, I, and BL was found. ◆ The multivariate regression analysis between MOR and surface meteorological elements in different seasons was conducted. ◆ There is a correlational relationship between RVR and surface meteorological elements.

Fast location and segmentation of high-throughput damaged soybean seeds with invertible neural networks.

Fast identification of damaged soybean seeds has undeniable importance in seed sorting and food quality. Mechanical vibration is generally used in soybean seed sorting, but this can seriously damage soybean seeds. The convolutional neural network (CNN) is considered an effective method for location and segmentation tasks. However, a CNN requires a large amount of ground truth data and has high computational cost. First, we propose a self-supervision manner to automatically generate ground truths, which can theoretically create an almost unlimited number of labeled images. Second, instead of using popular CNNs, a novel invertible convolution (involution)-enabled scheme is proposed by using the bottleneck block of the residual networks. Third, a feature selection feature pyramid network (FS-FPN) based on involution is designed, which selects features more flexibly and adaptively. We further merge involution-based backbones and FS-FPN into a unified network, achieving an end-to-end seed location and segmentation model; the best mean average precision of location and segmentation achieved was 85.1% and 81% respectively. The experimental results demonstrate that the proposed method greatly improves the performance of the baseline network with faster speed and fewer parameters, enabling it to detect soybean seeds more effectively. © 2022 Society of Chemical Industry.

Evaluating FFT-based Algorithms for Strided Convolutions on ARMv8 Architectures?

Convolutional Neural Networks (CNNs) have been widely adopted in all kinds of artificial intelligence applications. Most of the computational overhead of CNNs is mainly spent on convolutions. An effective approach to reducing the overhead is FFT-based fast algorithms for convolutions. However, current FFT-based fast implementations cannot be directly applied to strided convolutions with a stride size of greater than 1. In this paper, we first introduce rearrangement- and sampling-based methods for applying FFT-based fast algorithms on strided convolutions. Then, the highly optimized parallel implementations of the two methods on ARMv8-based many-core CPU are presented. Lastly, we benchmark the implementations against the two GEMM-based implementations on this ARMv8 CPU. Our experimental results with convolutions of different kernel, and feature maps, and batch sizes show that the rearrangementbased method generally exceed the sampling-based one under the same optimizations in most cases, and these two methods can get much better performance than GEMMbased ones when the kernel, feature maps and batch sizes are large. The experimental results with the convolutional layers in popular CNNs further demonstrate the conclusion above.

Deep learning and machine learning neural network approaches for multi class leather texture defect classification and segmentation

Modern leather industries are focused on producing high quality leather products for sustaining the market competitiveness. However, various leather defects are introduced during various stages of manufacturing process such as material handling, tanning and dyeing. Manual inspection of leather surfaces is subjective and inconsistent in nature; hence machine vision systems have been widely adopted for the automated inspection of leather defects. It is necessary develop suitable image processing algorithms for localize leather defects such as folding marks, growth marks, grain off, loose grain, and pinhole due to the ambiguous texture pattern and tiny nature in the localized regions of the leather. This paper presents deep learning neural network-based approach for automatic localization and classification of leather defects using a machine vision system. In this work, popular convolutional neural networks are trained using leather images of different leather defects and a class activation mapping technique is followed to locate the region of interest for the class of leather defect. Convolution neural networks such as Google net, Squeeze-net, RestNet are found to provide better accuracy of classification as compared with the state-of-the-art neural network architectures and the results are presented.Graphical

The detection of saffron adulterants using a deep neural network approach based on RGB images taken under uncontrolled conditions

Saffron is the most expensive spice in the world which is traded internationally. Hence, Saffron adulterants is always a challenging issue. It should be noted that traditional methods that are being used to solve this problem, are costly and time-consuming. In this study, for the first time, Saffron adulterants was detected using a deep neural network and RGB photos taken under uncontrolled/unstructured conditions. There were five classes of fake Saffron include mixed saffron stamen and dyed straw, mixed stigma and stamens, dyed orange blossom, Safflower and dyed stamens, and two classes of genuine Saffron consisting of freeze-drying and microwave-drying saffron used to design, develop and optimize a Convolutional Neural Network (CNN). 60% of the images were used to train and optimize the neural network model. For preventing over-training, the number of images used in the training phase increased by applying the augmentation method. The general structure of the model to detect saffron adulterants consisted of multiple convolutional (input layer), pooling layers and fully-connected layers, respectively. In this study, five structures (S1, S2, S3, S4 and S5) were proposed for CNN using Batch Normalization methods. Additionally, this paper compared these five structures with Popular Convolutional Networks consisted of VGG11, ResNet18, ResNet50, Inception V3 and DarkNet53. According to the results, the best configuration was S5 that achieved the accuracy of 99.8% and the cross-entropy loss of 0.019. Although the time process of this structure was 36.26 ms, the shortest time process was 19.38 ms related to the S1. This number was more than 300 ms for Popular CNN. Based on the results of this study, imaging without restrictions with CNN can detect Saffron adulterants.

Visual Positioning System Based on 6D Object Pose Estimation Using Mobile Web

Recently, the demand for location-based services using mobile devices in indoor spaces without a global positioning system (GPS) has increased. However, to the best of our knowledge, solutions that are fully applicable to indoor positioning and navigation and ensure real-time mobility on mobile devices, such as global navigation satellite system (GNSS) solutions, cannot achieve remarkable researches in indoor circumstances. Indoor single-shot image positioning using smartphone cameras does not require a dedicated infrastructure and offers the advantages of low price and large potential markets owing to the popularization of smartphones. However, existing methods or systems based on smartphone cameras and image algorithms encounter various limitations when implemented in indoor environments. To address this, we designed an indoor visual positioning system for mobile devices that can locate users in indoor scenes. The proposed method uses a smartphone camera to detect objects through a single image in a web environment and calculates the location of the smartphone to find users in an indoor space. The system is inexpensive because it integrates deep learning and computer vision algorithms and does not require additional infrastructure. We present a novel method of detecting 3D model objects from single-shot RGB data, estimating the 6D pose and position of the camera and correcting errors based on voxels. To this end, the popular convolutional neural network (CNN) is improved by real-time pose estimation to handle the entire 6D pose estimate the location and direction of the camera. The estimated position of the camera is addressed to a voxel to determine a stable user position. Our VPS system provides the user with indoor information in 3D AR model. The voxel address optimization approach with camera 6D position estimation using RGB images in a mobile web environment outperforms real-time performance and accuracy compared to current state-of-the-art methods using RGB depth or point cloud.

Subdivision-based Mesh Convolution Networks

Convolutionalneural networks (CNNs) have made great breakthroughs in two-dimensional (2D) computer vision. However, their irregular structure makes it hard to harness the potential of CNNs directly on meshes. A subdivision surface provides a hierarchical multi-resolution structure in which each face in a closed 2-manifold triangle mesh is exactly adjacent to three faces. Motivated by these two observations, this article presents SubdivNet , an innovative and versatile CNN framework for three-dimensional (3D) triangle meshes with Loop subdivision sequence connectivity. Making an analogy between mesh faces and pixels in a 2D image allows us to present a mesh convolution operator to aggregate local features from nearby faces. By exploiting face neighborhoods, this convolution can support standard 2D convolutional network concepts, e.g., variable kernel size, stride, and dilation. Based on the multi-resolution hierarchy, we make use of pooling layers that uniformly merge four faces into one and an upsampling method that splits one face into four. Thereby, many popular 2D CNN architectures can be easily adapted to process 3D meshes. Meshes with arbitrary connectivity can be remeshed to have Loop subdivision sequence connectivity via self-parameterization, making SubdivNet a general approach. Extensive evaluation and various applications demonstrate SubdivNet’s effectiveness and efficiency.