Image Feature Extraction Research Articles

Flexible Natural Language-Based Image Data Downlink Prioritization for Nanosatellites

Nanosatellites increasingly produce more data than can be downlinked within a reasonable time due to their limited bandwidth and power. Therefore, an on-board system is required to prioritize scientifically significant data for downlinking, as described by scientists. This paper determines whether natural language processing can be used to prioritize remote sensing images on CubeSats with more flexibility compared to existing methods. Two approaches implementing the same conceptual prioritization pipeline are compared. The first uses YOLOv8 and Llama2 to extract image features and compare them with text descriptions via cosine similarity. The second approach employs CLIP, fine-tuned on remote sensing data, to achieve the same. Both approaches are evaluated on real nanosatellite hardware, the VERTECS Camera Control Board. The CLIP approach, particularly the ResNet50-based model, shows the best performance in prioritizing and sequencing remote sensing images. This paper demonstrates that on-orbit prioritization using natural language descriptions is viable and allows for more flexibility than existing methods.

Enhancing Multimodal Emotional Information Extraction in Film and Television through Adaptive Feature Fusion with DenseNe, Transformer, and 3D CNN Models

ABSTRACT The extraction of multimodal emotional information enables a more nuanced representation of the emotional subtleties embedded in film and television works. However, conventional approaches that independently extract features from images and text fail to capture the rich semantic interplay between these modalities, impeding the feature learning process within each modality. To address this, this paper presents an innovative model for extracting emotional information from multimodal film and television content. The model utilizes DenseNe for image feature extraction, enhancing network depth via the MSC module. Text feature extraction is achieved through a Transformer encoder, while video feature extraction employs a 3D CNN model. Refinements are made to the number and placement of convolutional layers, planar convolution size, and 3D convolution depth. Moreover, a multi-head scaled dot-product attention mechanism is incorporated into the interaction module to compute the similarity between each image block within the sequence and every word in the text sequence. Experimental evaluations on the CMU-MOSEI and CMU-MOSI datasets showcase superior performance compared to the baseline model, achieving accuracy and F1 scores of 0.708 and 0.698, respectively. Noteworthy is the proposed adaptive feature fusion module, which enriches the expressiveness of pivotal emotional features while eliminating redundant data.

Grounded situation recognition under data scarcity

Grounded Situation Recognition (GSR) aims to generate structured image descriptions. For a given image, GSR needs to identify the key verb, the nouns corresponding to roles, and their bounding-box groundings. However, current GSR research demands numerous meticulously labeled images, which are labor-intensive and time-consuming, making it costly to expand detection categories. Our study enhances model accuracy in detecting and localizing under data scarcity, reducing dependency on large datasets and paving the way for broader detection capabilities. In this paper, we propose the Grounded Situation Recognition under Data Scarcity (GSRDS) model, which uses the CoFormer model as the baseline and optimizes three subtasks: image feature extraction, verb classification, and bounding-box localization, to better adapt to data-scarce scenarios. Specifically, we replace ResNet50 with EfficientNetV2-M for advanced image feature extraction. Additionally, we introduce the Transformer Combined with CLIP for Verb Classification (TCCV) module, utilizing features extracted by CLIP’s image encoder to enhance verb classification accuracy. Furthermore, we design the Multi-source Verb-Role Queries (Multi-VR Queries) and the Dual Parallel Decoders (DPD) modules to improve the accuracy of bounding-box localization. Through extensive comparative experiments and ablation studies, we demonstrate that our method achieves higher accuracy than mainstream approaches in data-scarce scenarios. Our code will be available at https://github.com/Zhou-maker-oss/GSRDS.

Adversarial Unsupervised Domain Adaptation for 3D Semantic Segmentation with 2D Image Fusion of Dense Depth

AbstractUnsupervised domain adaptation (UDA) is increasingly used for 3D point cloud semantic segmentation tasks due to its ability to address the issue of missing labels for new domains. However, most existing unsupervised domain adaptation methods focus only on uni‐modal data and are rarely applied to multi‐modal data. Therefore, we propose a cross‐modal UDA on multi‐modal datasets that contain 3D point clouds and 2D images for 3D Semantic Segmentation. Specifically, we first propose a Dual discriminator‐based Domain Adaptation (Dd‐bDA) module to enhance the adaptability of different domains. Second, given that the robustness of depth information to domain shifts can provide more details for semantic segmentation, we further employ a Dense depth Feature Fusion (DdFF) module to extract image features with rich depth cues. We evaluate our model in four unsupervised domain adaptation scenarios, i.e., dataset‐to‐dataset (A2D2 → SemanticKITTI), Day‐to‐Night, country‐to‐country (USA → Singapore), and synthetic‐to‐real (VirtualKITTI → SemanticKITTI). In all settings, the experimental results achieve significant improvements and surpass state‐of‐the‐art models.

Enhanced stimulated Raman and fluorescence imaging by single-frame trained BDN

Hyperspectral and multispectral imaging capture an expanded dimension of information that facilitates discoveries. However, image features are frequently obscured by noise generated from the limited photodamage threshold of the specimen. Although machine learning approaches demonstrate considerable promise in addressing this challenge, they typically require extensive datasets, which can be difficult to obtain. Here, we introduce BiFormer denoising network (BDN), designed to effectively and efficiently extract image features by utilizing both local and global level connections, sparse architectures, and fine-tuning. Experimental results indicate that BDN enhances the quality of stimulated Raman scattering (SRS) images by up to 16-fold in signal-to-noise ratio (SNR), particularly improving subtle features at higher spatial frequencies. Furthermore, BDN is successfully adapted to fluorescence imaging, achieving significant improvements in SNR and order-of-magnitude reduction in exposure time, thereby showcasing its versatility across various imaging modalities. Collectively, BDN exhibits substantial potential for spectroscopic imaging applications in the fields of biomedicine and materials science.

Unlocking the potential of simulated hyperspectral imaging in agro environmental analysis: a comprehensive study of algorithmic approaches

This study focuses on identifying and evaluating the severity of powdery mildew disease in tomato plants. The uniqueness of this work lies in combining the imaging and advanced deep learning methods to develop a technique that transforms Red Green Blue (RGB) images into Simulated Hyperspectral Images (SHSI) to perform spectral and spatial analysis for precise detection and assessment of powdery mildew severity, thereby enhancing disease management. Furthermore, this research evaluates three advanced pre-trained VGG16 models, ResNet50 and EfficientNet-B7 algorithms for image preprocessing and feature extraction. Extracted features are passed to a neural network generator model to convert RGB image features into SHSIs, providing insights into the spectrum. This method enables the image analysis to perform assessments from SHSIs for health classification using Normalized Difference Vegetation Index (NDVI) values, which are meticulously compared with accurate hyperspectral data using metrics like mean absolute error (MAE) and root mean squared error (RMSE). This strategy enhances precision farming, environmental monitoring, and remote sensing accuracy. Results show that ResNet50’s architecture offers a robust framework for this study’s spectral and spatial analysis, making it a suitable choice over VGG16 and EfficientNet-B7 for transforming RGB images into SHSI. These simulated hyperspectral images offer a scalable and affordable approach for precise assessment of crop disease severity.

Feature extraction and classification of microstructure and magnetic tomography images of an advanced Nd-Fe-B sintered magnet

Abstract Although the microstructure and magnetic tomography images of an advanced Nd-Fe-B sintered magnet were previously reported [Takeuchi, SO., NPG Asia Mater. 14, 70 (2022)], the relationship between these three-dimensional images has not been well analyzed. In this work, a feature extraction method of histogram of oriented gradients and a classification method of uniform manifold approximation and projection are employed for this issue. The microstructural features with superimposing the information of magnetic domain structure are classified into two groups depending on the external magnetic field, resulting in the different microstructural features for the different magnetization states are successfully classified. These differences of the microstructural features are difficult to be found by human recognition. Further detailed analysis of these microstructural features may clarify the key microstructures for the nucleation of reversed domains and their propagations.

Concatenated CNN-Based Pneumonia Detection Using a Fuzzy-Enhanced Dataset.

Pneumonia is a form of acute respiratory infection affecting the lungs. Symptoms of viral and bacterial pneumonia are similar. Rapid diagnosis of the disease is difficult, since polymerase chain reaction-based methods, which have the greatest reliability, provide results in a few hours, while ensuring high requirements for compliance with the analysis technology and professionalism of the personnel. This study proposed a Concatenated CNN model for pneumonia detection combined with a fuzzy logic-based image improvement method. The fuzzy logic-based image enhancement process is based on a new fuzzification refinement algorithm, with significantly improved image quality and feature extraction for the CCNN model. Four datasets, original and upgraded images utilizing fuzzy entropy, standard deviation, and histogram equalization, were utilized to train the algorithm. The CCNN's performance was demonstrated to be significantly improved by the upgraded datasets, with the fuzzy entropy-added dataset producing the best results. The suggested CCNN attained remarkable classification metrics, including 98.9% accuracy, 99.3% precision, 99.8% F1-score, and 99.6% recall. Experimental comparisons showed that the fuzzy logic-based enhancement worked significantly better than traditional image enhancement methods, resulting in higher diagnostic precision. This study demonstrates how well deep learning models and sophisticated image enhancement techniques work together to analyze medical images.

FG-UNet: fine-grained feature-guided UNet for segmentation of weeds and crops in UAV images.

Semantic segmentation of weed and crop images is a key component and prerequisite for automated weed management. For weeds in unmanned aerial vehicle (UAV) images, which are usually characterized by small size and easily confused with crops at early growth stages, existing semantic segmentation models have difficulties to extract sufficiently fine features. This leads to their limited performance in weed and crop segmentation of UAV images. We proposed a fine-grained feature-guided UNet, named FG-UNet, for weed and crop segmentation in UAV images. Specifically, there are two branches in FG-UNet, namely the fine-grained feature branch and the UNet branch. In the fine-grained feature branch, a fine feature-aware (FFA) module was designed to mine fine features in order to enhance the model's ability to segment small objects. In the UNet branch, we used an encoder-decoder structure to realize high-level semantic feature extraction in images. In addition, a contextual feature fusion (CFF) module was designed for the fusion of the fine features and high-level semantic features, thus enhancing the feature discrimination capability of the model. The experimental results showed that our proposed FG-UNet, achieved state-of-the-art performance compared to other semantic segmentation models, with mean intersection over union (MIOU) and mean pixel accuracy (MPA) of 88.06% and 92.37%, respectively. The proposed method in this study lays a solid foundation for accurate detection and intelligent management of weeds. It will have a positive impact on the development of smart agriculture. © 2024 Society of Chemical Industry.

Object detection model with efficient feature extraction and asymptotic feature fusion for unmanned aerial vehicle image

Object detection model with efficient feature extraction and asymptotic feature fusion for unmanned aerial vehicle image

Fostering ayurvedic plant wellness: Innovative leaf disease detection using computer vision and machine learning

Ayurveda is a conventional medicinal approach that has its roots in India and has been used for hundreds of years. It is still popular today since it is entirely natural and free of side effects. Although ayurvedic medicines are made from natural botanical substances, their safety depends on the way they are administered, taking into account the needs of the individual and the specific disease states they are treating. Diseases that affect the plants' leaves are regarded to be one of the main reasons for a decline in the output of ayurvedic plants in India's agricultural, economic, cosmetic, and pharmaceutical sectors. A plant exhibits signs of plant diseases in diverse sections of the plant, however, leaves are the most frequently observed component for spotting an infection. The objectives of this work are to modernize and innovate an autonomous ayurvedic plant leaf disease detection system by examining the scientific basis of computer vision. Additionally, a full analysis of computer vision techniques such as image pre-processing, segmentation, and feature extraction is covered to detect defects in plant leaves. This work investigates cutting-edge machine learning methods for identifying plant diseases that employ a variety of computer vision techniques. Based on the review, the article addresses the challenges and offers recommendations for changes that could be made in the future.

Fostering ayurvedic plant wellness: Innovative leaf disease detection using computer vision and machine learning

Ayurveda is a conventional medicinal approach that has its roots in India and has been used for hundreds of years. It is still popular today since it is entirely natural and free of side effects. Although ayurvedic medicines are made from natural botanical substances, their safety depends on the way they are administered, taking into account the needs of the individual and the specific disease states they are treating. Diseases that affect the plants' leaves are regarded to be one of the main reasons for a decline in the output of ayurvedic plants in India's agricultural, economic, cosmetic, and pharmaceutical sectors. A plant exhibits signs of plant diseases in diverse sections of the plant, however, leaves are the most frequently observed component for spotting an infection. The objectives of this work are to modernize and innovate an autonomous ayurvedic plant leaf disease detection system by examining the scientific basis of computer vision. Additionally, a full analysis of computer vision techniques such as image pre-processing, segmentation, and feature extraction is covered to detect defects in plant leaves. This work investigates cutting-edge machine learning methods for identifying plant diseases that employ a variety of computer vision techniques. Based on the review, the article addresses the challenges and offers recommendations for changes that could be made in the future.

A solution for the automatic detection of expansion joints in dam stilling pools using underwater robots

This research addresses the critical challenges of detecting and measuring underwater concrete structure expansion joints (CSEJ), with a particular focus on dam stilling basin environments. A full set of algorithms for detecting expansion joints with underwater robots is presented by the research, enabling real-time and offline analysis along with visualization. These algorithms introduce the use of Convolutional Neural Networks (CNNs) for image feature extraction and segmentation in CSEJ detection, significantly improving accuracy and adaptability over traditional methods. An automated approach known as the Underwater Dam Crack Detection Network (UDCD Network) was introduced by this research. And advanced attention gates and hybrid dilated convolutions to enhance its detection capabilities are incorporated in this system. Additionally, a three-phase deep transfer learning strategy during training was utilized in this research, which significantly improves the model's performance. The research suggests an underwater image restoration algorithm that uses a dark channel prior and dynamic white balance correction to improve image clarity. Furthermore, unique underwater environmental challenges were tackled by this research, such as non-contact measurement of expansion joint widths using a view-geometry-based algorithm and comprehensive detection of dam wall surfaces. Experimental validation demonstrates that the advanced imaging technologies of the proposed underwater robots hold significant potential for autonomous detection of expansion joints in dam stilling basins.

Classification of Rice Grains by Image Processing

General Background: The assessment of the surface quality of pre-treated rice grains is crucial for determining their market acceptability, storage stability, processing quality, and overall consumer satisfaction. Traditional evaluation methods are often time-consuming and yield subjective classifications. Specific Background: The lack of a comprehensive tagged image dataset hinders the application of advanced convolutional neural networks (CNN) for detailed damage classification of healthy rice grains. Knowledge Gap: Existing datasets and methods limit the effective exploration of sophisticated CNN models for categorizing rice types, particularly in identifying subtle damage characteristics. Aims: The study aims to create a robust rice grain classification system using image processing techniques, primarily deep learning algorithms, to improve the classification of rice varieties. Results: Utilizing a dataset of 75,000 images across five widely cultivated rice varieties in Turkey, we achieved classification accuracies of 100% for CNN, 99.95% for deep neural networks (DNN), and 99.87% for artificial neural networks (ANN). Novelty: The proposed approach represents a significant advancement in rice classification technology, employing a combination of image acquisition, feature extraction, and machine learning to streamline the process, effectively addressing the challenges faced in traditional methods. Implications: The findings underscore the potential for improved sorting and grading efficiency in the rice industry, facilitating better market outcomes and consumer satisfaction through enhanced quality control Highlights: Innovative: Uses CNN for accurate rice variety classification. Data-Driven: Analyzes 75,000 images for enhanced quality evaluation. Impactful: Increases efficiency in sorting and grading processes. Keywords: Rice Classification, Image Processing, CNN, Machine Learning, Agricultural Technology

A Transfer Learning-Based Framework for Classifying Lymph Node Metastasis in Prostate Cancer Patients.

Background: Prostate cancer is the second most common new cancer diagnosis in the United States. It is usually slow-growing, and when it is low-grade and confined to the prostate gland, it can be treated either conservatively (through active surveillance) or with surgery. However, if the cancer has spread beyond the prostate, such as to the lymph nodes, then that indicates a more aggressive cancer, and surgery may not be adequate. Methods: The challenge is that it is often difficult for radiologists reading prostate-specific imaging such as magnetic resonance images (MRIs) to differentiate malignant lymph nodes from non-malignant ones. An emerging field is the development of artificial intelligence (AI) models, including machine learning and deep learning, for medical imaging to assist in diagnostic tasks. Earlier research focused on implementing texture algorithms to extract imaging features used in classification models. More recently, researchers began studying the use of deep learning for both stand-alone feature extraction and end-to-end classification tasks. In order to tackle the challenges inherent in small datasets, this study was designed as a scalable hybrid framework utilizing pre-trained ResNet-18, a deep learning model, to extract features that were subsequently fed into a machine learning classifier to automatically identify malignant lymph nodes in patients with prostate cancer. For comparison, two texture algorithms were implemented, namely the gray-level co-occurrence matrix (GLCM) and Gabor. Results: Using an institutional prostate lymph node dataset (42 positives, 84 negatives), the proposed framework achieved an accuracy of 76.19%, a sensitivity of 79.76%, and a specificity of 69.05%. Using GLCM features, the classification achieved an accuracy of 61.90%, a sensitivity of 74.07%, and a specificity of 42.86%. Using Gabor features, the classification achieved an accuracy of 65.08%, a sensitivity of 73.47%, and a specificity of 52.50%. Conclusions: Our results demonstrate that a hybrid approach, i.e., using a pre-trainined deep learning model for feature extraction, followed by a machine learning classifier, is a viable solution. This hybrid approach is especially useful in medical-imaging-based applications with small datasets.

Hybrid Swin-CSRNet: A Novel and Efficient Fish Counting Network in Aquaculture

Real-time estimation of fish biomass plays a crucial role in real-world fishery production, as it helps formulate feeding strategies and other management decisions. In this paper, a dense fish counting network called Swin-CSRNet is proposed. Specifically, the VGG16 layer in the front-end is replaced with the Swin transformer to extract image features more efficiently. Additionally, a squeeze-and-excitation (SE) module is introduced to enhance feature representation by dynamically adjusting the importance of each channel through “squeeze” and “excitation”, making the extracted features more focused and effective. Finally, a multi-scale fusion (MSF) module is added after the back-end to fully utilize the multi-scale feature information, enhancing the model’s ability to capture multi-scale details. The experiment demonstrates that Swin-CSRNet achieved excellent results with MAE, RMSE, and MAPE and a correlation coefficient R2 of 11.22, 15.32, 5.18%, and 0.954, respectively. Meanwhile, compared to the original network, the parameter size and computational complexity of Swin-CSRNet were reduced by 70.17% and 79.05%, respectively. Therefore, the proposed method not only counts the number of fish with higher speed and accuracy but also contributes to advancing the automation of aquaculture.

A multi-class fundus disease classification system based on an adaptive scale discriminator and hybrid loss

Fundus images are crucial in the observation and detection of ophthalmic diseases. However, detecting multiple ophthalmic diseases from fundus images using deep learning techniques is a complex and challenging task One challenge is the complexity of fundus disease structures, which leads to low detection accuracy. Another challenge is the class imbalance problem common in multi-label image classification, which increases the difficulty of algorithm training and evaluation. To address these issues, this study leverages deep learning to propose an ophthalmic disease classification system. We first employ ResNet50 as the backbone network to extract image features, and then use our designed multi-dimensional attention module and adaptive scale discriminator to enhance the network's ability to detect disease features. During training, we innovatively propose a hybrid loss function method to improve the detection capability on imbalanced data. Finally, we conducted experiments on the ODRI-5K dataset with the proposed classification system. In the test set, our method achieved an AUC of 98.53 and an F1-score of 89.73. This result fully demonstrates the excellent disease classification capability of our method. In summary, the multi-label fundus image disease classification system we proposed exhibits outstanding recognition capability, providing an effective solution for the diagnosis of multi-label fundus image diseases.

Visual inspection and grasping methods based on deep learning

Aiming at the problems of existing robot grasping systems that have high hardware requirements, are difficult to adapt to different objects, and produce large harmful torques during the grasping process, a visual detection and grasping method based on deep learning is proposed. The channel attention mechanism is used to improve YOLO-V3, enhance the network's ability to extract image features, improve the effect of target detection in complex environments, and increase the average recognition rate compared with the original . Aiming at the problem of discreteness of the current pose estimation angle, a minimum area bounding rectangle (MABR) algorithm based on the main network embedded in the Visual Geometry Group 16 (VGG-16) is proposed to perform grasping pose estimation and angle optimization. The average error between the improved grasping angle and the actual angle of the target is less than , which greatly reduces the harmful torque applied by the two-finger manipulator to the object during the grasping process. A visual grasping system was built using UR5 robotic arm, pneumatic two-finger manipulator, Realsense D435 camera and ATI-Mini45 six-dimensional force sensor. Experiments show that the proposed method can effectively grasp and classify different objects, has low hardware requirements, and reduces harmful torque by about , thereby reducing damage to objects. It has good application prospects.

Design, Development and Testing of Electronic Class Attendance using Fingerprint

Electronic Class Attendance System (e-CAS) is an easy way to keep track of attendance of students and lecturers in School. It will cover the requirements of the personnel department in terms of day-to-day monitoring of students' and lecturers activities. Hence, students’ attendance is an important issue every institution must take into consideration to be productive. Hence, this research aims to design, develop and test using fingerprints. The methodologies used were miniature techniques and image processing techniques for design and development. This target was mainly decomposed into image pre-processing, feature extraction and feature match. For each sub-task, some classical and up-to-date methods in literature were analysed. Based on the analysis, an integrated solution for electronic class attendance was developed for demonstration. However, Poison probability process, Biometrics as well as NP-chart of Quality control were adopted for classical statistical analysis of quality and functionality of the e-Class Attendance System (e-CAS). The project on electronic class attendance system was coded by PHP and the hardware was developed with ATMEGA series of microcontrollers. The result when experimented achieved the objective of improving the performance and consistency of the electronic class attendance system based on template matching techniques as well as class attendance for both students and lecturers.

Defective Pennywort Leaf Detection Using Machine Vision and Mask R-CNN Model

Demand and market value for pennywort largely depend on the quality of the leaves, which can be affected by various ambient environment or fertigation variables during cultivation. Although early detection of defects in pennywort leaves would enable growers to take quick action, conventional manual detection is laborious and time consuming as well as subjective. Therefore, the objective of this study was to develop an automatic leaf defect detection algorithm for pennywort plants grown under controlled environment conditions, using machine vision and deep learning techniques. Leaf images were captured from pennywort plants grown in an ebb-and-flow hydroponic system under fluorescent light conditions in a controlled plant factory environment. Physically or biologically damaged leaves (e.g., curled, creased, discolored, misshapen, or brown spotted) were classified as defective leaves. Images were annotated using an online tool, and Mask R-CNN models were implemented with the integrated attention mechanisms, convolutional block attention module (CBAM) and coordinate attention (CA) and compared for improved image feature extraction. Transfer learning was employed to train the model with a smaller dataset, effectively reducing processing time. The improved models demonstrated significant advancements in accuracy and precision, with the CA-augmented model achieving the highest metrics, including a mean average precision (mAP) of 0.931 and an accuracy of 0.937. These enhancements enabled more precise localization and classification of leaf defects, outperforming the baseline Mask R-CNN model in complex visual recognition tasks. The final model was robust, effectively distinguishing defective leaves in challenging scenarios, making it highly suitable for applications in precision agriculture. Future research can build on this modeling framework, exploring additional variables to identify specific leaf abnormalities at earlier growth stages, which is crucial for production quality assurance.