Multispectral Image Processing Research Articles

Multi-spectral Remote Sensing Image Fusion Method Based on Gradient Moment Matching

Image fusion is a popular research direction in the field of computer vision. Traditional image fusion algorithms can achieve good results in fusing grayscale images, but it is difficult to achieve ideal results in processing multi-spectral images. To address the shortcomings of multi-spectral image fusion, this study proposes a low computational complexity and low latency multi-spectral image fusion model by utilizing a multi-step degree moment matching algorithm and a generative adversarial network for fusion. Through experiments, it was found that the F1 score of the GAN-MMN model on the TinyPerson dataset was 89.79%, with an average recall rate of 89.76%. The GAN-MMN performance was higher than that of the control model. Meanwhile, the GAN-MMN model also exhibited superior performance in high-frequency feature extraction and time delay compared to the control model. According to the experimental results, the proposed multi-spectral remote sensing image fusion model had a high feature extraction effect, and its recall rate and F1 score were better than the control model, so the research model had a certain progressiveness. The proposal of this model gives a new approach for the processing of multi-spectral remote sensing images, effectively promoting the development of the computer vision industry.

Geothermal potential area analysis using Landsat 8 OLI/TIRS and digital elevation model images (case study: Lawu Mount, Central Java)

Mount Lawu is one of the volcanoes in Central Java Province with hidden geothermal potential. This research aims to identify geothermal potential areas, especially hydrothermal alteration areas, using Landsat 8 imagery in Mount Lawu, Central Java. Hydrothermal alteration serves as a crucial indicator in the exploration of geothermal potential in an area. Multispectral image processing methods of Landsat 8 are employed to map the distribution patterns of hydrothermal alterations in the study area, together with the anomaly of temperature and heatflux in the surface. This approach involves principal component analysis and spectral alteration indications that combine information from various image channels. Lithology and lineament density analysis strengthen the delineation and analysis of the geothermal potential area. The western-northwestern part of Mount Lawu is the most potential region for geothermal resources based on the distribution of hydrothermal alteration areas and the high density of faulted regions, which are also supported by geothermal manifestations such as hotspring sites in the area.

The Uncertainty Assessment by the Monte Carlo Analysis of NDVI Measurements Based on Multispectral UAV Imagery.

This paper proposes a workflow to assess the uncertainty of the Normalized Difference Vegetation Index (NDVI), a critical index used in precision agriculture to determine plant health. From a metrological perspective, it is crucial to evaluate the quality of vegetation indices, which are usually obtained by processing multispectral images for measuring vegetation, soil, and environmental parameters. For this reason, it is important to assess how the NVDI measurement is affected by the camera characteristics, light environmental conditions, as well as atmospheric and seasonal/weather conditions. The proposed study investigates the impact of atmospheric conditions on solar irradiation and vegetation reflection captured by a multispectral UAV camera in the red and near-infrared bands and the variation of the nominal wavelengths of the camera in these bands. Specifically, the study examines the influence of atmospheric conditions in three scenarios: dry-clear, humid-hazy, and a combination of both. Furthermore, this investigation takes into account solar irradiance variability and the signal-to-noise ratio (SNR) of the camera. Through Monte Carlo simulations, a sensitivity analysis is carried out against each of the above-mentioned uncertainty sources and their combination. The obtained results demonstrate that the main contributors to the NVDI uncertainty are the atmospheric conditions, the nominal wavelength tolerance of the camera, and the variability of the NDVI values within the considered leaf conditions (dry and fresh).

Spectral Reflectance Measurements.

This chapter provides a methodology for evaluating plant health and leaf characteristics using spectral reflectance. It provides a step-by-step guide to using spectrometers for high-resolution point measurements of leaf spectral reflectance and multispectral imaging for capturing spatial data, emphasizing the importance of consistent measurement conditions. The chapter further explores the intricacies of multispectral imaging, including calibration, data collection, and image processing. Finally, this chapter delves into the application of various spectral indices for the quantification of key traits such as pigment content, the status of the xanthophyll cycle, water content, and how to identify spectral regions of interest for further research and development. Serving as a guide for researchers and practitioners in plant science, this chapter provides a straightforward framework for plant health assessment using spectral reflectance.

Multispectral Image Processing System for Precision Detection of Reheated Coconut Oil

In the pursuit of enhancing food safety protocols, this article explores a cutting-edge approach to quality control in the coconut oil industry. We present a multispectral image processing system designed specifically for the detection of reheated coconut oil, leveraging advancements in machine learning. Machine learning algorithms, fused with image classification techniques, provide a robust framework for accurately identifying reheated coconut oil. It is proposed to develop a spectral clustering-based classifier to determine the effect of reheating and reuse of coconut oil. Post-processing methods refine classification results, while validation ensures the system's adaptability to diverse datasets.

Drone agrotechnology’s for cotton (Gossypium hirsutum L.) pest and diseases management in Western of Burkina Faso

Drone technologies, also known as "unmanned aerial vehicles" (UAVs) or "unmanned aerial systems" (UAS), have a wide range of applications, including mapping, land-use planning, crop/infrastructure damage assessment, fertilizer spraying and precision crop protection. However, its use is still very limited in the agricultural sector in Africa in general and in Burkina Faso in particular. This study was initiated to explore the potential of UAVs for optimizing cotton yields through monitoring and phytosanitary efficiency. To this end, phytosanitary treatments and monitoring of vegetation status using multispectral drone imagery were carried out on cotton. The experimental sample consisted of four (04) cotton fields in four (04) locations. Phytosanitary treatments and aerial photography were respectively carried out using a DJI AGRAS T16 spray drone and a DJI Phantom 4 Multispectral drone. Phytosanitary treatment with DJI AGRAS T16 drone resulted in savings of 32.41% in treatment duration, 23.58% in water, 1% in pesticides and 10.5% in labor compared with manual spraying, but did not have a positive impact on boll yields due to jassid populations attack. The RGB and NDVI vegetation indices generated by photogrammetric processing of multispectral images acquired by drone enabled us to perceive phytosanitary state of cotton plants during their development cycle. Information provided by these vegetation indices and by this study in general can be used to improve the efficiency of phytosanitary treatments against cotton pests and diseases.

Method for maize plants counting and crop evaluation based on multispectral images analysis

The processing of multispectral images acquired with embedded cameras in unmanned aerial vehicles (drones) has brought new opportunities for precision agriculture. In this study a method for evaluating the number of corn plants (Zea mays L) in a crop area is presented. Plant density is one of the most important yield factors, yet its precise measurement after the emergence of plants is impractical in large and medium-scale production, since significant amount of labor is required. For validation, a dataset of spectral images was gathered from flights over an agricultural area, and digital image processing techniques were applied, taking into account the concept of intelligent processing. Therefore, pattern recognition and models to aid decision-making through machine learning were also used. After image acquisition, the processing of orthomosaics in the spectral channels, i.e., red (R), green (G), and blue (B), was performed, making it possible to register and organize all the images. Likewise, techniques for geometric transformation, brightness, and contrast adjustments were evaluated globally, whereas local adjustments were evaluated based on the use of adaptive equalization techniques, which were explored based in the choice of the HSV color space. For the post-processing step, segmentation based on the best observed color threshold technique, in conjunction with Gaussian filtering and morphological operations, were considered. To enable pattern recognition, techniques that use distance maps were evaluated, considering the use of Euclidean distance. Thus, the locations of canopy patterns in maize plants were studied using a template matching algorithm and Chamfer pattern mask. For feature extraction, chain code and circular pattern map techniques were considered. The analyses made it possible to establish vectors of features based on patterns related to the number of maize plants occurrences. Finally, three calibration steps were considered, one related to the plant height versus the canopy opening radius, other related to the number of maize plants for each position in the crop area versus the radii identified by the developed model, and the third related to the cross-correlation between the plant counting by human vision and the new method. In addition, the classification step was established using a set of classifiers based on support vector machine (SVM). Results have shown an accurate and timely counting methodology for maize plants, which can guide cultivation to ensure high yield. The results showed that as a new method it can effectively count the number of maize plants with an average accuracy rate equal to 88.47%. Besides, both selected SVM classifiers have presented accuracy higher than 84% and precision higher than 83%. Furthermore, the cross-correlation between the plant counting by human vision and the new method has presented a linear correlation coefficient equal to 0.98. Thus, the developed method proved to be adequate for counting the maize plants in the post-emergence stage.

Quantitative and Qualitative Analysis of Agricultural Fields Based on Aerial Multispectral Images Using Neural Networks.

This article presents an integrated system that uses the capabilities of unmanned aerial vehicles (UAVs) to perform a comprehensive crop analysis, combining qualitative and quantitative evaluations for efficient agricultural management. A convolutional neural network-based model, Detectron2, serves as the foundation for detecting and segmenting objects of interest in acquired aerial images. This model was trained on a dataset prepared using the COCO format, which features a variety of annotated objects. The system architecture comprises a frontend and a backend component. The frontend facilitates user interaction and annotation of objects on multispectral images. The backend involves image loading, project management, polygon handling, and multispectral image processing. For qualitative analysis, users can delineate regions of interest using polygons, which are then subjected to analysis using the Normalized Difference Vegetation Index (NDVI) or Optimized Soil Adjusted Vegetation Index (OSAVI). For quantitative analysis, the system deploys a pre-trained model capable of object detection, allowing for the counting and localization of specific objects, with a focus on young lettuce crops. The prediction quality of the model has been calculated using the AP (Average Precision) metric. The trained neural network exhibited robust performance in detecting objects, even within small images.

A new band selection method for multispectral data based on criterion function of information capability

Multispectral remote sensing is one of the most popular techniques in the earth observation, because this technique can provide information of ground objects on Earth’s surface using hundreds of narrow bands. However, multispectral images produces a very large volume of data. Processing the huge volume of information is one of most important and actual problems of remote sensing. The rapid development of the remote sensing demand to develop the data processing algorithms. But at present data processing techniques cannot give accurate results. If we use traditional methods to process multispectral images, the volume of the data increases. The main goal of the band selection is to choose the optimal combination of spectral bands for the solution of the particular remote sensing task. This process is important because different bands are sensitive to different objects. Selecting the right bands can help to optimize the detection of different ground objects. Some spectral bands are more sensitive to minerals, while others are more sensitive to vegetation or water bodies. Under a small number of training samples, the classification accuracy of multispectral images decreases when the volume of multispectral data increases. Usually adjacent bands are highly correlated, and some spectral bands may not carry unique information. That’s why it is necessarily to reduce the dimensionality of multispectral data. It helps to store, process, transmit information more efficiently and to reduce the computational costs while processing images. The different modern methods of multispectral band selection are also considered and analyzed in this work. It also is proposed a new method to select spectral bands, which is based on the concept of criterion function of information capability of spectral bands. In this article some examples using criterion function of information capability are considered too.

Dynamics of Land Salinization in the Commune of Fimela (Fatick, Senegal) from 1973 to 2020

In Senegal, the agricultural sector remains a major component of the economy and national growth. As the main subsistence activity for 60% of the population, agricultural activity is essential to reduce poverty and ensure food security for the population. In this context, land degradation is a major constraint. In the Fatick region, in the commune of Fimela, land salinization is a worrying environmental problem. The purpose of this study is to understand the dynamics of soil salinization in Fimela in the context of climate change that tends to modify the evolution of landscapes. It required direct observations in the field, socio-economic surveys based on a questionnaire administered in six (6) villages. The processing of the data from these surveys was carried out with Sphinx software for the extraction of data in numerical form and SPSS to carry out the correlations between the collected variables. Excel was also used to perform calculations and make tables and graphs. In addition, the acquisition and processing of Landsat multi-spectral satellite images from 1973, 1988 and 2020 allowed us to observe the evolution of landscape units according to determining climatic events such as drought. The areas of tans have experienced a positive evolution during the period 1973-2020 with an increase of 163.11 hectares or an evolution rate of 7.47%. The localities most affected are Ndangane, Fimela, Djilor, Simal and the villages of the Mar Islands. The overall dynamics of cultivable land are marked by a decline with a rate of change of −18%. Despite the multiple reforestation campaigns, the mangrove has recorded a continuous decrease of 54.58% or a loss estimated at 5335.59 ha during the period 1973-2020. Finally, the analysis of the results of our study shows that land salinization is a determining element of the dynamics of land use and deteriorates the already precarious living conditions of rural populations and compromises the future of the agropastoral production system.

Multilevel integrated studies and monitoring of natural and technical objects of Kola region

The relevance of research, diagnostics and study of natural and technical objects belonging to I–II hazard classes is shown. Methodological approaches to the application of interdisciplinary methods for a complex of observations and observation of such objects are proposed: visual observations and photodocumentation, geophysical survey, hydrological measurements, geomechanics methods, satellite geodesy, digital photography using UAVs, remote sensing, hydrogeomechanical modeling. Comprehensive natural observations of various incidents on the earth's surface: underground, explosive, high-altitude and satellite. The underground level includes hydrological measurements, seismic tomography and georadar sounding. At the surface level, geodetic surveys, horizontal and vertical measurements, geodetic referencing and precise positioning of objects are performed. Various types of digital aerial photography are used for the high-altitude level. The basis of the satellite level is the processing and analysis of multispectral and radar images (ERS). The data obtained as a result of field studies are used to develop hydrogeomechanical 3D models, with the help of which scenario modeling of various hydrogeological and geomechanical conditions is performed. Such an approach to solving monitoring problems, which consists in integrating interdisciplinary studies at various levels, makes it possible to control a natural and technical object in a hierarchy: from a structure as a whole to its individual structural elements and local sections. A description is given of examples of the application of multilevel integrated research and monitoring of natural and technical objects of mining enterprises in the Kola region.

Foreword

The eleven papers in this special section serve as a tribute to Professor David A. Landgrebe who is known for his work in the fundamentals of multispectral image processing and analysis. The papers are grouped into three categories: historical and future developments, methodological advancements, and survey and review.

WIANet: A Wavelet-Inspired Attention-Based Convolution Neural Network for Land Cover Classification

General convolutional neural networks (CNNs) process the data in the spatial domain and lack efficacy in modeling the spectral context in multispectral image processing task. To address this limitation, in this letter, we propose a wavelet-inspired attention-based convolution neural network (WIANet) architecture that combines the modeling of the spatio-spectral resolution of multispectral remote sensing (RS) images with a wavelet convolution and attention unit into a single deep learning architecture for land cover classification. Our aim is to add the characteristics of both wavelet transform (WT) and attention mechanism into an UNet-based architecture to better exploit the spectral and texture information for distinguishing classes with high similarity in the spectral signatures. We evaluate the performance of the proposed approach on a multilabel multispectral Sentinel-2 dataset. The experiment shows that the proposed approach performs better than the reference methods in different conditions on the number of training samples.

Hybrid endoscope with television and multispectral image processing for the internal organs cancer early diagnostics

Endoscopes are widely used to diagnose various internal diseases. The most advanced endoscopes contain miniature television cameras built into the probe which provide high spatial resolution. However, traditional television cameras significantly distort the spectral composition of the transmitted image because in the RGB system they reproduce only limited width spectral lines. However, the full optical spectrum of an image of an internal organ under examination provides a broader range of information that can be crucial in cancer diagnostics. Thus, the spectral resolution of the image plays an important role. Multispectral processing has been suggested for efficient detection of both spatial and spectral information simultaneously. As a result, a series of monochrome sub-images can be obtained, each of which corresponds to a selected resolved spectral interval. Acousto-optical tunable filters are used as a selective element allowing real-time extraction of sub-images. However, an increase in spectral resolution is accompanied by a decrease in spatial resolution. A new hybrid endoscope was suggested to solve this problem. It implements the method of combined application of a TV camera and a multispectral processing device. It is shown that the suggested method of hybrid endoscopy makes it possible to ensure high resolutions: spatial resolution with a camera and spectral resolution (no less than a hundred of resolvable intervals) due to multispectral processing. The resulting optical images reveal a clear outline of the neoplasm area. The results of multispectral processing diagnose the malignant nature of the neoplasm. A schematic diagram of the proposed endoscope is presented, and the principles of control are shown. The characteristics of the television and multispectral images are determined. An estimation of the linearity characteristics of the acousto-optical filters shows the possibility of transmitting the maximum amount of information due to the linearity of the signal transformation in the filter sections. Implementation of the proposed method in the design of a hybrid endoscope allowed obtaining the most complete spectral information about neoplasms and providing an opportunity of diagnostics of cancer of internal organs at earlier stages of development.

Low-cost system for multispectral image acquisition and its applicability to analysis of the physiological potential of soybean seeds

The use of multispectral images has great potential to assess seed quality and represents a significant technological advance in the search for fast and non-destructive analysis techniques. However, the devices currently available are expensive. Thus, this study aimed to propose a low-cost method for acquisition and processing of multispectral images of soybean seeds and to evaluate their potential for rapid determination of seed physiological potential. The study was conducted in three steps: implementation of the multispectral image acquisition system, development of an algorithm for automatic image processing, and evaluation of the relationship between the data obtained through image analysis and the results of standard tests used to evaluate seed physiological potential. A total of 43 variables were assessed, eight related to seed physiological potential (germination and vigor) and 35 obtained from the analysis of the multispectral images. Of the variables obtained from multispectral images, 21 were related to pixel values in the images in the different bands evaluated (green, red, and infrared) and 14 associated with seed morphometric characteristics. The proposed system is efficient in obtaining multispectral images and the algorithm developed was efficient to extract morphometric characteristics and pixel information from the images. The parameters obtained from the NIR spectrum region showed a good relationship with the physiological potential of soybean seeds.

Image Quality of Spectral Filter Arrays for Planetary Rover Applications: From Demosaicing to Color Correction

The exploration of the Solar System using unmanned probes and rovers has improved the understanding of our planetary neighbors. Despite a large variety of instruments, optical and near-infrared cameras remain critical for these missions to understand the planet’s surrounding, its geology but also to communicate easily with the general public. However, missions on planetary bodies must satisfy strong constraints in terms of robustness, data size, and amount of onboard computing power. Although this trend is evolving, commercial image-processing software cannot be integrated. Still, as the optical and a spectral information of the planetary surfaces is a key science objective, spectral filter arrays (SFAs) provide an elegant, compact, and cost-efficient solution for rovers. In this contribution, we provide ways to process multi-spectral images on the ground to obtain the best image quality, while remaining as generic as possible. This study is performed on a prototype SFA. Demosaicing algorithms and ways to correct the spectral and color information on these images are also detailed. An application of these methods on a custom-built SFA is shown, demonstrating that this technology represents a promising solution for rovers.

Multispectral image analysis for monitoring by IoT based wireless communication using secure locations protocol and classification by deep learning techniques

Land surface data is one of the entry points for a enormous resource management and human activity such as to monitor farming, ecology, urban management, and territories. This research aimed to improve the performance of the multispectral image processing technique using a novel deep learning based classification approaches used in classifying the surfaces in multispectral satellite images. The monitoring of Internet of Things (IoT) based wireless communication using secure locations sensor network protocol (SLSNP). The multispectral satellite images are collected based on developing and developed countries. Upon the collection of multispectral image data using deep convolution neural network (DCNN) deep learning system landmarks of the multispectral images are estimated. This will be effective for the identification of resources in a particular location. The simulation results show classification accuracy, precision, recall and the network lifetime, efficiency, throughput based on data transmission from satellite for secure location based wireless sensor networks (WSNs).

Polychromic light source for the realization of multispectral processing method of skin malignant lesions images

When diagnosing oncological skin diseases, the problem of untimely detection of pathologies remains relevant. Modern diagnostic methods have a number of disadvantages associated with the limited capabilities of human vision, high requirements for non-invasive studies and digital representation of the results obtained for their transmission over communication channels. The most important tasks of early binary diagnostics of skin neoplasms can be solved using bioradiophotonics methods. The paper considers the author’s multispectral method for processing images of skin areas using a new specialized polychrome light source with software control of the radiation wavelength. The principle of operation of such a source is analyzed and the use of a set of LEDs with a special arrangement is justified which makes it possible to direct a beam of light of any wavelength in one direction. The choice of the wavelength ranges used for LEDs is confirmed, the first version of the layout of such a light source is described, and its optical and structural diagrams are given. It is shown that the optimal way of the wavelength programmable control is its coarse tuning by switching LEDs and fine tuning by acousto-optic tunable filters installed at the source output. A number of test images of a pigment spot on the skin with a size of 40 × 30 mm with wavelengths ranging from near ultraviolet to near infrared ranges using a black and white camera are presented. With digital processing, received monochromatic images, histogram equalization and median filtering of images were carried out to expand the dynamic range of brightness levels and minimize distortion caused by the unevenness of the skin area under study and the presence of shadow illumination. Graphs of the amplitude intensity distribution in the original and filtered images are obtained for the selected wavelengths of the LEDs. It is shown that the presence of a skin neoplasm can be judged by the dips in the light intensity distribution curve. The radial arrangement of LEDs on the board is justified to minimize shadow illumination when illuminating the skin. The conducted studies of multispectral image processing using a realized polychrome light source contribute to the development of new methods for the early diagnosis of skin neoplasms based on photonics and optoelectronics and can find practical application in medical technologies.

Application of neural network and computer vision technologies for image analysis of skin lesion

Opportunity research of using neural networks and computer vision to analyze images of skin lesion and identify features of various pathologies, including oncological neoplasms. A methodology has been developed that makes it possible to evaluate the significance of combinations of color components and spaces in feature extraction using local binary patterns (LBP) and histogram of oriented gradients (HOG) computer vision technologies to extract features of skin changes binary classification of human skin lesions. Optimization of extracted feature makes it possible to more effectively solve the problem of data separability in classification. Research reveals an accessible way to classify skin lesions on a small dataset (less than 1000 images). Research is supposed to be applied to data sequences obtained using a new unique method of multispectral processing of skin lesions. In the course of the work, data from the ISIC-19 and ISIC-20 datasets were used. Samples were formed with a limit of 1000 images for training and validating the models. Additionally, a test sample of 250 images was formed. All images were reduced to 128 × 128 pixels and converted to YCrCb, BGR, Grayscale, HSV color spaces. Features were extracted for each color channel using the HOG and LBP methods. Mathematical models, including neural networks have been used for data classification. The effectiveness of features combinations by color channels and feature extraction methods was evaluated. The preprocessed images were divided into training and validation subsets in a 70/30 ratio. The accuracy, recall, precision and f1-score metrics were used to evaluate the models. The models were evaluated using stratified cross-validation and a test dataset. Optimization of model parameters was carried out based on the loss function represented by the average of cross-validation and evaluation on the validation set. In the process of research, more than 15 000 different optimizations of model parameters were executed. The most stable results on the validation dataset were achieved using ensemble of models, which were trained on a combination of features using local binary patterns (LBP) and histogram of oriented gradients (HOG) technologies. Models which used only local binary patterns technology had the best metrics values, but these models are not recommended to be used in practice without ensemble with stronger models. The results gained can be applied for usage with an ensemble of state-of-the-art convolutional and recurrent neural networks. The proposed approach is universal and applicable both for the analysis of individual images of skin neoplasms and for the analysis of their sequences obtained by the method of multispectral image processing. The technique can be applied to datasets with a limited amount of data. The results obtained will be of interest to specialists in the fields of computer vision and medical images analysis.

Climate Change Analysis Based on Satellite Multispectral Image Processing in Feature Selection Using Reinforcement Learning

Currently private and government agencies use remote sensing images (RSI) for various applications from military applications to agriculture growth. The images can be multispectral, panchromatic, ultra-spectral, or hyperspectral of terra bytes. RSI classification is considered one important application for remote sensing. Climate change detection especially affects numerous aspects of day-to-day lives, for instance, forestry management, weather forecasting, transportation, agriculture, road condition monitoring, and the detection of the natural atmosphere. Conversely, certain research works had a focus on classification of actual weather phenomenon images, generally depending on visual observations from humans. The conventional artificial visual difference between weather phenomena will take more time and error-prone. This paper develops a new reinforcement learning based climate change analysis on satellite multispectral image processing (RLCCA-SMSIP) technique. In order to properly determine climate change, the RLCCA-SMSIP technique employs residual network (ResNet-101) model for feature extraction. Next, deep reinforcement learning (DRL) approach is utilized for climate classification. Finally, parameter selection of the RLCCA-SMSIP technique involves sine cosine algorithm (SCA) for DRL model. For assuring the enhanced outcomes of the presented RLCCA-SMSIP model, comprehensive comparison results are assessed. The obtained values denote the supremacy of the RLCCA-SMSIP model on climate classification.