V2 Model Research Articles

Metric3D v2: A Versatile Monocular Geometric Foundation Model for Zero-Shot Metric Depth and Surface Normal Estimation.

We introduce Metric3D v2, a geometric foundation model designed for zero-shot metric depth and surface normal estimation from single images, critical for accurate 3D recovery. Depth and normal estimation, though complementary, present distinct challenges. State-of-the-art monocular depth methods achieve zero-shot generalization through affine-invariant depths, but fail to recover real-world metric scale. Conversely, current normal estimation techniques struggle with zero-shot performance due to insufficient labeled data. We propose targeted solutions for both metric depth and normal estimation. For metric depth, we present a canonical camera space transformation module that resolves metric ambiguity across various camera models and large-scale datasets, which can be easily integrated into existing monocular models. For surface normal estimation, we introduce a joint depth-normal optimization module that leverages diverse data from metric depth, allowing normal estimators to improve beyond traditional labels. Our model, trained on over 16 million images from thousands of camera models with varied annotations, excels in zero-shot generalization to new camera settings. As shown in Fig. 1, It ranks the 1st in multiple zero-shot and standard benchmarks for metric depth and surface normal prediction. Our method enables the accurate recovery of metric 3D structures on randomly collected internet images, paving the way for plausible single-image metrology. Our model also relieves the scale drift issues of monocular-SLAM (Fig. 3), leading to high-quality metric scale dense mapping. Such applications highlight the versatility of Metric3D v2 models as geometric foundation models.

Impact of Pre-trained Weights on MobileNet Architectures for Animal Classification

This study aims to address the problem of improving animal image classification accuracy using different versions of MobileNet. Accurate animal classification plays a vital role in biodiversity protection, environmental monitoring, and agriculture. The research is significant because existing studies focus on specific models and datasets, leaving a gap in the comparative performance analysis of MobileNet versions. To address this issue, MobileNet V1, V2, and V3 models were utilized, both with and without ImageNet pre-trained weights. The models were trained on a dataset composed of 30,179 images from two sources, covering 13 animal categories. The experiment involved training the models over 10 epochs using a standard configuration of the TensorFlow framework, with accuracy serving as the primary evaluation metric.The results showed that MobileNet V3Large, with pre-trained weights, achieved the highest accuracy (97.43%), outperforming V1 and V2. Using pre-trained weights consistently enhanced performance, as models without pre-training exhibited lower accuracy and slower convergence. This study contributes by providing a comprehensive comparison of MobileNet versions in animal classification tasks, demonstrating the importance of pre-training and model architecture optimization for achieving high accuracy in image classification.

Model-driven development for functional correctness of avionics systems: a verification framework for SysML specifications

AbstractCurrently, the most widespread software quality assurance methods in the avionics domain are semi-automated reviews and testing. However, their effort grows disproportionately to the size of the system under development. Also, these methods cannot achieve exhaustive coverage due to the complexity of today’s avionics systems and their potentially infinite set of combinations of possible inputs and system states. Furthermore, the later software issues are detected in the development process, the more expensive it is to fix them. To overcome these issues, a model-driven verification approach for modeling and analyzing avionics systems in early phases of the development is presented. To this end, semantics is given to SysML v2 models by a mapping to a theorem prover encoding. The development of a dedicated SysML v2 profile supporting event-driven data flow specifications, the encoding of corresponding structures in the theorem prover Isabelle, and a generator creating theorems from SysML v2 models are presented. The approach is evaluated by formally proving a representative liveness property of a hierarchical system model from the avionics domain. Since liveness properties can be negated only by infinite data sequences and thus cannot be covered exhaustively by testing, this case study demonstrates the added value for meeting typical safety requirements in the avionics domain. The results can be transferred from avionics to other domains, as well.

Research on the Method of Imperfect Wheat Grain Recognition Utilizing Hyperspectral Imaging Technology

As the primary grain crop in China, wheat holds a significant position in the country’s agricultural production, circulation, consumption, and various other aspects. However, the presence of imperfect grains has greatly impacted wheat quality and, subsequently, food security. In order to detect perfect wheat grains and six types of imperfect grains, a method for the fast and non-destructive identification of imperfect wheat grains using hyperspectral images was proposed. The main contents and results are as follows: (1) We collected wheat grain hyperspectral data. Seven types of wheat grain samples, each containing 300 grains, were prepared to construct a hyperspectral imaging system for imperfect wheat grains, and visible near-infrared hyperspectral data from 2100 wheat grains were collected. The Savitzky–Golay algorithm was used to analyze the hyperspectral images of wheat grains, selecting 261 dimensional effective hyperspectral datapoints within the range of 420.61–980.43 nm. (2) The Successive Projections Algorithm was used to reduce the dimensions of the 261 dimensional hyperspectral datapoints, selecting 33 dimensional hyperspectral datapoints. Principal Component Analysis was used to extract the optimal spectral wavelengths, specifically selecting hyperspectral images at 647.57 nm, 591.78 nm, and 568.36 nm to establish the dataset. (3) Particle Swarm Optimization was used to optimize the Support Vector Machines model, Convolutional Neural Network model, and MobileNet V2 model, which were established to recognize seven types of wheat grains. The comprehensive recognition rates were 93.71%, 95.14%, and 97.71%, respectively. The results indicate that a larger model with more parameters may not necessarily yield better performance. The research shows that the MobileNet V2 network model exhibits superior recognition efficiency, and the integration of hyperspectral image technology with the classification model can accurately identify imperfect wheat grains.

Identification of multimodal vortex optical orbital angular momentum in multimode fiber speckle patterns

Orbital angular momentum (OAM) multiplexing technology is a highly promising approach that can significantly increase the data capacity of optical communication systems. However, traditional optical communication systems are prone to atmospheric disturbances such as turbulence, which can degrade transmission quality. To mitigate this issue, multimode fibers (MMFs) have been introduced to reduce the impact of turbulence on signals. Moreover, previous studies have primarily focused on the identification of single vortex beams, facing challenges in accurately recognizing multiplexed modes. To address this challenge, this chapter proposes a multi-label image classification optimization algorithm based on transfer learning. By utilizing the pre-trained MobileNet V2 model as a feature extractor, this network structure can accurately identify 8-bit, 16-bit, and 24-bit multiplexed OAM from speckle patterns in multimode fibers, even with small sample datasets, achieving classification accuracies exceeding 95%. This method overcomes the limitations of traditional optical communication systems that are susceptible to atmospheric disturbances, providing new possibilities for long-distance transmission and increased data capacity in optical communication systems.

Few-shot learning to identify atypical endometrial hyperplasia and endometrial cancer based on transvaginal ultrasonic images

ObjectiveIt is challenging to accurately distinguish atypical endometrial hyperplasia (AEH) and endometrial cancer (EC) under routine transvaginal ultrasonic (TVU) detection. Our research aims to use the few-shot learning (FSL) method to identify non-atypical endometrial hyperplasia (NAEH), AEH, and EC based on limited TVU images. MethodsThe TVU images of pathologically confirmed NAEH, AEH, and EC patients (n = 33 per class) were split into the support set (SS, n = 3 per class) and the query set (QS, n = 30 per class). Next, we used dual pretrained ResNet50 V2 which pretrained on ImageNet first and then on extra collected TVU images to extract 1*64 eigenvectors from the TVU images in SS and QS. Then, the Euclidean distances were calculated between each TVU image in QS and nine TVU images of SS. Finally, the k-nearest neighbor (KNN) algorithm was used to diagnose the TVU images in QS. ResultsThe overall accuracy and macro precision of the proposed FSL model in QS were 0.878 and 0.882 respectively, superior to the automated machine learning models, traditional ResNet50 V2 model, junior sonographer, and senior sonographer. When identifying EC, the proposed FSL model achieved the highest precision of 0.964, the highest recall of 0.900, and the highest F1-score of 0.931. ConclusionsThe proposed FSL model combining dual pretrained ResNet50 V2 eigenvectors extractor and KNN classifier presented well in identifying NAEH, AEH, and EC patients with limited TVU images, showing potential in the application of computer-aided disease diagnosis.

Adaptive Filter Pruning via Sensitivity Feedback.

Filter pruning is advocated for accelerating deep neural networks without dedicated hardware or libraries, while maintaining high prediction accuracy. Several works have cast pruning as a variant of l1 -regularized training, which entails two challenges: 1) the l1 -norm is not scaling-invariant (i.e., the regularization penalty depends on weight values) and 2) there is no rule for selecting the penalty coefficient to trade off high pruning ratio for low accuracy drop. To address these issues, we propose a lightweight pruning method termed adaptive sensitivity-based pruning (ASTER) which: 1) achieves scaling-invariance by refraining from modifying unpruned filter weights and 2) dynamically adjusts the pruning threshold concurrently with the training process. ASTER computes the sensitivity of the loss to the threshold on the fly (without retraining); this is carried efficiently by an application of L-BFGS solely on the batch normalization (BN) layers. It then proceeds to adapt the threshold so as to maintain a fine balance between pruning ratio and model capacity. We have conducted extensive experiments on a number of state-of-the-art CNN models on benchmark datasets to illustrate the merits of our approach in terms of both FLOPs reduction and accuracy. For example, on ILSVRC-2012 our method reduces more than 76% FLOPs for ResNet-50 with only 2.0% Top-1 accuracy degradation, while for the MobileNet v2 model it achieves 46.6% FLOPs Drop with a Top-1 Acc. Drop of only 2.77%. Even for a very lightweight classification model like MobileNet v3-small, ASTER saves 16.1% FLOPs with a negligible Top-1 accuracy drop of 0.03%.

Real-Time Personal Protective Equipment Non-Compliance Recognition on AI Edge Cameras

Despite advancements in technology, safety equipment, and training within the construction industry over recent decades, the prevalence of fatal and nonfatal injuries and accidents remains a significant concern among construction workers. Hard hats and safety vests are crucial safety gear known to mitigate severe head trauma and other injuries. However, adherence to safety protocols, including the use of such gear, is often inadequate, posing potential risks to workers. Moreover, current manual safety monitoring systems are laborious and time-consuming. To address these challenges and enhance workplace safety, there is a pressing need to automate safety monitoring processes economically, with reduced processing times. This research proposes a deep learning-based pipeline for real-time identification of non-compliance with wearing hard hats and safety vests, enabling safety officers to preempt hazards and mitigate risks at construction sites. We evaluate various neural networks for edge deployment and find that the Single Shot Multibox Detector (SSD) MobileNet V2 model excels in computational efficiency, making it particularly suitable for this application-oriented task. The experiments and comparative analyses demonstrate the pipeline’s effectiveness in accurately identifying instances of non-compliance across different scenarios, underscoring its potential for improving safety outcomes.

Ovarian cancer identification technology based on deep learning and second harmonic generation imaging.

Ovarian cancer is among the most common gynecological cancers and the eighth leading cause of cancer-related deaths among women worldwide. Surgery is among the most important options for cancer treatment. During surgery, a biopsy is generally required to screen for lesions; however, traditional case examinations are time consuming and laborious and require extensive experience and knowledge from pathologists. Therefore, this study proposes a simple, fast, and label-free ovarian cancer diagnosis method that combines second harmonic generation (SHG) imaging and deep learning. Unstained fresh human ovarian tissues were subjected to SHG imaging and accurately characterized using the Pyramid Vision Transformer V2 (PVTv2) model. The results showed that the SHG imaged collagen fibers could quantify ovarian cancer. In addition, the PVTv2 model could accurately differentiate the 3240 SHG images obtained from our imaging collection into benign, normal, and malignant images, with a final accuracy of 98.4%. These results demonstrate the great potential of SHG imaging techniques combined with deep learning models for diagnosing the diseased ovarian tissues.

Skillful seasonal prediction of Afro-Asian summer monsoon precipitation with a merged machine learning and large ensemble approach

Afro-Asian summer monsoon precipitation (AfroASMP) is the life blood of billions of people living in many developing countries covering West Africa and Asia. Its complex variabilities are always accompanied by natural disasters like floods, landslides and droughts. Reliable AfroASMP prediction several months in advance is valuable for not only decision-makers but also regional socioeconomic sustainability. To address the current predicament of the AfroASMP seasonal prediction, this study provides an effective machine-learning model (Y-model). Y-model uses the monsoon related big climate data for searching the potential predictors, encompassing atmospheric internal factors and external forcings. Only the predictors associated with significant anomalies in summer horizonal winds at 850 hPa over the monsoon domain are retained. These selected predictors are then reorganized into a large ensemble based upon different thresholds of four fundamental principles. These principles include the focused sample sizes, the relationships between predictors and predictand, the independence among predictors, and the extremities of predictors in the forecast year. Real-time predictions can be generated based on the ensemble mean of skillful members during an independent hindcast period. Y-model skillfully predicts four monsoon precipitation indices of AfroASMP during 2011–2022 at lead 4–12 months, correlation skills range from 0.58 to 0.90 and root mean square error skills are reduced by 11–53% compared to CFS v2 model at lead 1 month. This study offers an effective method for preprocessing predictors in seasonal climate prediction.

Multiclass Classification using Enhanced MobileNet V2 Architecture

In computer vision applications including object detection, face recognition, image classification, etc., the Convolutional Neural Network (CNN) predominates. MobileNet v2 is one such CNN architecture which can significantly cut down the parameters and cost of calculation while giving comparatively higher accuracy. In this paper, we propose the enhanced MobileNet architecture which shows an improvement in the accuracy. We have applied transfer learning techniques to fine-tune the MobileNet V2 model on the food dataset. The proposed model’s generalization capacity, it’s robustness and recognition accuracy shows a considerable amount of improvement with respect to the base architecture. The findings demonstrate that the enhanced MobileNet architecture’s accuracy for food images recognition and classification has increased to 97.16%. Key Words: Convolutional Neural Network (CNN), MobileNet v2, transfer learning, image recognition and classification

Automated detection of COVID-19 and pneumonia diseases using data mining and transfer learning algorithms with focal loss from chest X-ray images

Pneumonia is a lung infection that produces severe inflammation, which can be caused by viruses, bacteria, or fungi. The global occurrence of pneumonia cases has been notably affected by the outbreak of Coronavirus Disease 2019 (COVID-19). Recent studies have demonstrated the capability of Deep Learning (DL) algorithms in detecting COVID-19 and pneumonia in medical images like X-rays. In this study, we employ four different Chest X-ray (CXR) datasets to evaluate four pre-trained Convolutional Neural Network (CNN) models: InceptionResNet V2, Inception V3, Xception, and MobileNet. These models were fine-tuned using either the Cross-Entropy (CE) loss function only for balanced datasets or separately using both the CE and Focal Loss (FL) functions for imbalanced datasets. For the first dataset, the InceptionResNet V2 model achieved the highest multi-classification accuracy of 88.63%. The superior model for the second and fourth datasets is Inception V3, with 94.35% and 97.67% multi-classification accuracy, respectively. For the third dataset, the best model is Xception, with a binary classification accuracy of 100.00%. Our results emphasize the significance of using the FL function in solving class imbalance problems in DL models. Additionally, it highlights the effectiveness of individual DL models in detecting different pneumonia infections using different CXR datasets.

Classification of Brain Tumor Diseases Using Data Augmentation and Transfer Learning

An accumulation of aberrant cells called a brain tumor is the outcome of unregulated cell division. Brain cancers can be found using magnetic resonance imaging (MRI). The exponential growth of deep learning networks has allowed us to tackle complex tasks, even in fields as complicated as medicine. However, using these models requires a large corpus of data for the networks to be highly generalizable and have high performance. This dearth of training data makes it critical to explore methods such as data augmentation. In this sense, data augmentation methods are widely used in strategies to train networks, and with small data sets being vital in medicine due to the limited access to data, this work aims to identify the best classification system by considering the prediction accuracy in this vein. Data augmentation is performed on the database and fed into the three convolutional neural network (CNN) models. A comparison line is drawn between the three models based on accuracy and performance on the Inception v3 models, Mobile Net V2, and Squeeze Net network for brain tumor detection and classifying 350 brain MR images. The statistical methods were modified in order to evaluate these algorithms. With 0.992% accuracy, 0.993% recall, 0.989% precision, and 0.994% F1 score, the Squeeze Net model performed the best. The Mobile Net V2 model, which had an accuracy of 0.964%, came next. When the research's findings were compared to those of related studies in the literature, they revealed better success rates than those of the majority of investigations.

Ensemble Learning-Based Pain Intensity Identification Model Using Facial Expressions

The pain identification model (PIM) can assist the healthcare professionals to render effective services to individuals. Disabled individuals can benefit from the automated PIM. Ensemble learning is widely employed for developing medical applications. A model for classifying the pain intensity using facial expression images is proposed in this study. A ShuffleNet V2 model is fine-tuned to extract features using fusion feature and class activation map techniques. CatBoost and XGBoost models are used as base models to predict pain intensities. The authors used the support vector machine (SVM) model as a meta-model to produce a final outcome. They optimize the SVM model in order to identify pain using the predictions of the base models. The model is generalized using the University of Northern British Columbia–McMaster dataset. The dataset encompasses 200 videos and 48,000 annotated images. The comparative analysis outcome highlights the exceptional performance of the proposed PIM. An optimal accuracy of 98.7% and an F1-score of 98.0% indicate the effectiveness of the proposed model. The uncertainty analysis outcome revealed that the model is reliable and can be deployed in healthcare centers. However, substantial training is required to boost the efficiency of the proposed model in real-time settings.

Advancing Fall Detection Utilizing Skeletal Joint Image Representation and Deformable Layers

Falls are a significant concern among the elderly population, with 25% of individuals over 65 years old experiencing a fall severe enough to require a visit to the emergency department each year. Early detection of falls can prevent serious injuries and complications, making it an important problem to address. There are various methods for detecting falls, utilizing different types of sensor input data. However, when considering factors such as ease of setup, accessibility, and accuracy, utilizing cameras for fall detection is a highly effective approach. In this study, a novel video-based fall detection algorithm that relies on skeleton joints is introduced. The results of pose estimation are preprocessed into an image representation and ShuffleNet V2 model with the addition of a Deformable Layer is employed for classification. Experiments were carried out on four distinct datasets: URFD, UP-Fall Detection, Le2i, and NTU RGB+D 60, which encompass individuals engaged in various activities, including falls. The results showcase exceptional performance across all these datasets, affirming the efficacy of the approach in accurately detecting falls in video footage.

HUMAN DETECTION IN SEARCH AND RESCUE OPERATIONS USING EMBEDDED ARTIFICIAL INTELLIGENCE

The paper discusses the use of unmanned aerial vehicles (drones) in search and rescue operations to detect humans in disaster areas where rescue teams cannot reach. The paper highlights the limitations of current methods, including high computational power, high cost, and dependence on internet connectivity. The paper proposes using transfer learning to develop a human detection model with a mean average precision (mAP@0.5) above 90% and compares two deep learning models, MobileNet v2 and EfficientDet. The study uses multi-datasets of aerial images of humans, namely SeaDronesee and SARD, and the TensorFlow version 2.8 framework. MobileNet v2 required less GPU usage for training and yielded a relatively high accuracy of 95.5%, while EfficientDet achieved higher accuracy (97.3%). The trained MobileNet v2 model size is compressed using quantization from 25.5 MB to 4.15 MB, making it suitable for deployment on an edge device for on-chip inference. The paper concludes that the proposed method can improve the efficiency and effectiveness of search and rescue operations.

Classification of human protein cell images using deep neural networks

Previously, single classification models were mainly studied to classify human protein cell images, i.e., to identify a certain protein based on a set of different cells. However, a classifier can identify only one protein, in fact, a single cell usually consists of multiple proteins, and the proteins are not completely independent of each other. In this paper, we build a human protein cell classification model by multi-label learning. The logical relationship and distribution characteristics among the labels are analyzed to determine the different proteins contained in a set of different cells (i.e., containing multiple elements in the output space). In this paper, using human protein image data, we conducted comparison experiments on pre-trained Xception and InceptionResnet V2 to optimize the two models in terms of data augmentation, channel settings, and model structure. The results show that the Optimized InceptionResnet V2 model achieves high performance in the classification task. The final accuracy of the Optimized InceptionResnet V2 model we obtained reached 96.1%, which is a 2.82% improvement relative to that before the optimized model.

I-CVSSDM: IoT Enabled Computer Vision Safety System for Disaster Management

INTRODUCTION: Around the world, individuals experience flooding more frequently than any other natural calamity.
 OBJECTIVES: The motivation behind this research is to provide an Internet of Things (IoT)-based early warning assistive system to enable monitoring of water logging levels in flood-affected areas. Further, the SSD-MobiNET V2 model is used in the developed system to detect and classify the objects that prevail in the flood zone.
 METHODS: The developed research is validated in a real-time scenario. To enable this, a customized embedded module is designed and developed using the Raspberry Pi 4 model B processor. The module uses (i) a pi-camera to capture the objects and (ii) an ultrasonic sensor to measure the water level in the flood area.
 RESULTS: The measured data and detected objects are periodically ported to the cloud and stored in the cloud database to enable remote monitoring and further processing.
 CONCLUSION: Also, whenever the level of waterlogged exceeds the threshold, an alert is sent to the concerned authorities in the form of an SMS, a phone call, or an email.

Research on detection of potato varieties based on spectral imaging analytical algorithm

Potatoes are popular among consumers due to their high yield and delicious taste. However, due to the numerous varieties of potatoes, different varieties are suitable for different processing methods. Therefore, it is necessary to distinguish varieties after harvest to meet the needs of processing enterprises and consumers. In this study, a new visible-near-infrared spectroscopic analysis method was proposed, which can achieve detection of five potato varieties. The method measures the transmission and reflection spectra of potatoes using a spectral acquisition system, encodes one-dimensional spectra into two-dimensional images using Gramian Angular Summation Field (GASF), Gramian Angular Difference Field (GADF), Markov Transition Field (MTF) and Recurrence Plot (RP), and improves the coordinated attention mechanism module and embeds the improved module into the ConvNeXt V2 model to build the ConvNeXt V2-CAP model for potato variety classification. The results show that compared with directly using one-dimensional classification models, image encoding of spectral data for classification greatly improves the accuracy. Among them, the best accuracy of 99.54% is achieved by using GADF image encoding of transmission spectra combined with the ConvNeXt V2-CAP model for classification, which is 16.28% higher than the highest accuracy of the one-dimensional classification model. The CAP attention mechanism module improves the performance of the model, especially when the dataset is small. When the training set is reduced to 150 images, the accuracy of the model is improved by 2.33% compared to the original model. Therefore, it is feasible to classify potato varieties using visible-near infrared spectroscopy and image encoding technology.

Study of the tightness of single seams of LOGICBASE™ / ECOBASE™ PVC membranes in engineering waterproofing systems

Objective. Currently, engineering waterproofing systems with polymer waterproofing membranes are widely used at various construction structures. At the same time, the usage of polymer membranes implies the operation of break welds, the strength of which should be no less than the strength of the waterproofing material itself. The purpose of the study is to evaluate the water resistance of single seams (without test channels, welded by manual or automatic equipment) for LOGICBASE™ / ECOBASE™ grade V-SL polymer membranes at a pressure of 1 MPa. Method. Testing of waterproofing seams for water resistance is carried out according to the requirements of GOST EN 1928-2011, method B. The GESTER model GT-C26B hydrostatic pressure testing device is used as a test installation. LEISTER welding equipment of Varimat V2 (automatic equipment) and Triac AT (manual/hand equipment) models was used for welding samples. The width of the weld was at least 30 mm. Result. At a hydrostatic head pressure of 1 MPa, within 24 hours from the start of the test, there were no traces of water penetration in the samples, which indicates that the seams are completely waterproof. Conclusion. Single seams of LOGICBASE™/ECOBASE™ engineered waterproofing PVC membranes are completely sealed at 1 MPa water pressure.