Multispectral Processing Research Articles

Spatial-spectral methods of increasing the efficiency of artillery reconnaissance

The author analyzes the evolution of reconnaissance-fire technology based on reconnaissance-strike (RSS) and reconnaissance-fire (RFS) systems and its transition to the ideology of reconnaissance-fire systems (RFS).
 Based on Boyd's cybernetic model, the situational model of RFS combat control and the issue of optimizing it in terms of the "observe" cycle are considered. Practical ways of implementing the developed approach based on multichannel spatial and multispectral processing of intelligence information are substantiated. The optimality of the developed approach is confirmed by the results of a model experiment.
 A methodology for adapting the characteristics of multi-channel processing of location information to a dynamically changing mobile phone environment has been developed. The optimality of the developed approach is confirmed by the results of the model experiment, and the practical feasibility is confirmed by the patent for the invention.The maximum combat effectiveness of the first stage of the Boyd cycle (artillery reconnaissance) – range, accuracy, target channel, immunity - is achieved by the integration of ground-based radar means of reconnaissance of firing positions and UAVs with multispectral surveillance equipment. 
 The practical significance of the obtained results is provided by the independence of targeting accuracy from the target range with simultaneous provision of round-the-clock operation, all-weather operation, interference resistance and potential informativeness of artillery reconnaissance. The attainable gain in artillery reconnaissance effectiveness indicators is 7.2 times the range, and 20 times the target channel. All analytical results are aimed at implementation in the practice of designing the production of equipment for ground and on-board reconnaissance components.

Hybrid endoscope with television and multispectral image processing for the internal organs cancer early diagnostics

Endoscopes are widely used to diagnose various internal diseases. The most advanced endoscopes contain miniature television cameras built into the probe which provide high spatial resolution. However, traditional television cameras significantly distort the spectral composition of the transmitted image because in the RGB system they reproduce only limited width spectral lines. However, the full optical spectrum of an image of an internal organ under examination provides a broader range of information that can be crucial in cancer diagnostics. Thus, the spectral resolution of the image plays an important role. Multispectral processing has been suggested for efficient detection of both spatial and spectral information simultaneously. As a result, a series of monochrome sub-images can be obtained, each of which corresponds to a selected resolved spectral interval. Acousto-optical tunable filters are used as a selective element allowing real-time extraction of sub-images. However, an increase in spectral resolution is accompanied by a decrease in spatial resolution. A new hybrid endoscope was suggested to solve this problem. It implements the method of combined application of a TV camera and a multispectral processing device. It is shown that the suggested method of hybrid endoscopy makes it possible to ensure high resolutions: spatial resolution with a camera and spectral resolution (no less than a hundred of resolvable intervals) due to multispectral processing. The resulting optical images reveal a clear outline of the neoplasm area. The results of multispectral processing diagnose the malignant nature of the neoplasm. A schematic diagram of the proposed endoscope is presented, and the principles of control are shown. The characteristics of the television and multispectral images are determined. An estimation of the linearity characteristics of the acousto-optical filters shows the possibility of transmitting the maximum amount of information due to the linearity of the signal transformation in the filter sections. Implementation of the proposed method in the design of a hybrid endoscope allowed obtaining the most complete spectral information about neoplasms and providing an opportunity of diagnostics of cancer of internal organs at earlier stages of development.

REMOTE SENSING OF THE STATE OF THE VEGETATION COVER OF THE TERRITORY OF THE BULGARIAN BLACK SEA

This paper is devoted to the multispectral processing of satellite images in order to establish the actual state of the vegetation cover. As a result of conducted experiments, the vegetation on the territory of the Bulgarian Black Sea coast was studied. For this purpose, new indexed images have been generated from appropriately selected multispectral images, which visually reveal the actual state of the vegetation cover and provide objective and timely data. They could serve to manage and other decisions (in the field of ecology, forestry, agriculture, etc.), for environmental assessments and expertise, for scientific research and other projects.

Polychromic light source for the realization of multispectral processing method of skin malignant lesions images

When diagnosing oncological skin diseases, the problem of untimely detection of pathologies remains relevant. Modern diagnostic methods have a number of disadvantages associated with the limited capabilities of human vision, high requirements for non-invasive studies and digital representation of the results obtained for their transmission over communication channels. The most important tasks of early binary diagnostics of skin neoplasms can be solved using bioradiophotonics methods. The paper considers the author’s multispectral method for processing images of skin areas using a new specialized polychrome light source with software control of the radiation wavelength. The principle of operation of such a source is analyzed and the use of a set of LEDs with a special arrangement is justified which makes it possible to direct a beam of light of any wavelength in one direction. The choice of the wavelength ranges used for LEDs is confirmed, the first version of the layout of such a light source is described, and its optical and structural diagrams are given. It is shown that the optimal way of the wavelength programmable control is its coarse tuning by switching LEDs and fine tuning by acousto-optic tunable filters installed at the source output. A number of test images of a pigment spot on the skin with a size of 40 × 30 mm with wavelengths ranging from near ultraviolet to near infrared ranges using a black and white camera are presented. With digital processing, received monochromatic images, histogram equalization and median filtering of images were carried out to expand the dynamic range of brightness levels and minimize distortion caused by the unevenness of the skin area under study and the presence of shadow illumination. Graphs of the amplitude intensity distribution in the original and filtered images are obtained for the selected wavelengths of the LEDs. It is shown that the presence of a skin neoplasm can be judged by the dips in the light intensity distribution curve. The radial arrangement of LEDs on the board is justified to minimize shadow illumination when illuminating the skin. The conducted studies of multispectral image processing using a realized polychrome light source contribute to the development of new methods for the early diagnosis of skin neoplasms based on photonics and optoelectronics and can find practical application in medical technologies.

Application of neural network and computer vision technologies for image analysis of skin lesion

Opportunity research of using neural networks and computer vision to analyze images of skin lesion and identify features of various pathologies, including oncological neoplasms. A methodology has been developed that makes it possible to evaluate the significance of combinations of color components and spaces in feature extraction using local binary patterns (LBP) and histogram of oriented gradients (HOG) computer vision technologies to extract features of skin changes binary classification of human skin lesions. Optimization of extracted feature makes it possible to more effectively solve the problem of data separability in classification. Research reveals an accessible way to classify skin lesions on a small dataset (less than 1000 images). Research is supposed to be applied to data sequences obtained using a new unique method of multispectral processing of skin lesions. In the course of the work, data from the ISIC-19 and ISIC-20 datasets were used. Samples were formed with a limit of 1000 images for training and validating the models. Additionally, a test sample of 250 images was formed. All images were reduced to 128 × 128 pixels and converted to YCrCb, BGR, Grayscale, HSV color spaces. Features were extracted for each color channel using the HOG and LBP methods. Mathematical models, including neural networks have been used for data classification. The effectiveness of features combinations by color channels and feature extraction methods was evaluated. The preprocessed images were divided into training and validation subsets in a 70/30 ratio. The accuracy, recall, precision and f1-score metrics were used to evaluate the models. The models were evaluated using stratified cross-validation and a test dataset. Optimization of model parameters was carried out based on the loss function represented by the average of cross-validation and evaluation on the validation set. In the process of research, more than 15 000 different optimizations of model parameters were executed. The most stable results on the validation dataset were achieved using ensemble of models, which were trained on a combination of features using local binary patterns (LBP) and histogram of oriented gradients (HOG) technologies. Models which used only local binary patterns technology had the best metrics values, but these models are not recommended to be used in practice without ensemble with stronger models. The results gained can be applied for usage with an ensemble of state-of-the-art convolutional and recurrent neural networks. The proposed approach is universal and applicable both for the analysis of individual images of skin neoplasms and for the analysis of their sequences obtained by the method of multispectral image processing. The technique can be applied to datasets with a limited amount of data. The results obtained will be of interest to specialists in the fields of computer vision and medical images analysis.

Towards rainbow portable Cytophone with laser diodes for global disease diagnostics

In vivo, Cytophone has demonstrated the capability for the early diagnosis of cancer, infection, and cardiovascular disorders through photoacoustic detection of circulating disease markers directly in the bloodstream with an unprecedented 1,000-fold improvement in sensitivity. Nevertheless, a Cytophone with higher specificity and portability is urgently needed. Here, we introduce a novel Cytophone platform that integrates a miniature multispectral laser diode array, time-color coding, and high-speed time-resolved signal processing. Using two-color (808 nm/915 nm) laser diodes, we demonstrated spectral identification of white and red clots, melanoma cells, and hemozoin in malaria-infected erythrocytes against a blood background and artifacts. Data from a Plasmodium yoelii murine model and cultured human P. falciparum were verified in vitro with confocal photothermal and fluorescent microscopy. With these techniques, we detected infected cells within 4 h after invasion, which makes hemozoin promising as a spectrally selective marker at the earliest stages of malaria progression. Along with the findings from our previous application of Cytophone with conventional lasers for the diagnosis of melanoma, bacteremia, sickle anemia, thrombosis, stroke, and abnormal hemoglobin forms, this current finding suggests the potential for the development of a portable rainbow Cytophone with multispectral laser diodes for the identification of these and other diseases.

Design of CGAN Models for Multispectral Reconstruction in Remote Sensing

Multispectral imaging methods typically require cameras with dedicated sensors that make them expensive. In some cases, these sensors are not available or existing images are RGB, so the advantages of multispectral processing cannot be exploited. To solve this drawback, several techniques have been proposed to reconstruct the spectral reflectance of a scene from a single RGB image captured by a camera. Deep learning methods can already solve this problem with good spectral accuracy. Recently, a new type of deep learning network, the Conditional Generative Adversarial Network (CGAN), has been proposed. It is a deep learning architecture that simultaneously trains two networks (generator and discriminator) with the additional feature that both networks are conditioned on some sort of auxiliary information. This paper focuses the use of CGANs to achieve the reconstruction of multispectral images from RGB images. Different regression network models (convolutional neuronal networks, U-Net, and ResNet) have been adapted and integrated as generators in the CGAN, and compared in performance for multispectral reconstruction. Experiments with the BigEarthNet database show that CGAN with ResNet as a generator provides better results than other deep learning networks with a root mean square error of 316 measured over a range from 0 to 16,384.

Analysis and classification of peanuts with fungal diseases based on real-time spectral processing

ABSTRACT The study presents an approach to the analysis and classification of peanuts performed in order to detect kernels with fungi diseases, i.e. kernels prone to contamination with mycotoxigenic Aspergillus flavus (Aspergillus parasiticus). The aim of this study was to evaluate the effectiveness of luminescent spectroscopy with a violet laser (405 nm wavelength) as the excitation source of the fluorescence when applied for real-time detection of mould in peanuts performed by means of multispectral processing based on machine learning methods. We suggest a laboratory unit used to form, register, and process the luminescence spectra of peanuts in visible and near-infrared wavelength ranges in the real-time mode. The study demonstrated that contaminated peanuts have increased luminous intensity and show a redshift in the fluorescence peaks of the contaminated samples as compared to the pure ones. The difference in the fluorescence spectra of pure and contaminated kernels is compatible with the results obtained when traditional UV-light sources are used (365 nm). To classify peanuts by their spectral characteristics, neural network algorithms were used combined with dimensionality reduction methods. The paper presents the probabilities of incorrect recognition of the peanuts’ type depending on the number of relevant secondary features determined when reducing the dimensionality of the initial data. When 10 spectral components were used, the error ratios were 0.7% or 0.3% depending on the method of reducing the dimensionality of the initial data.

Contribution of Earth Observation and Geospatial Information for Urban Planning of Historic Cities’ Centres: The Case Study of Nicosia, Cyprus

The Sustainable Development Goals (SDGs) of the United Nations state that cities and human settlements need to be more inclusive, safe and resilient. In Europe cities have experienced dramatic physical, social and economic changes during the last decades while historic centres of European cities, among the most important assets of the European cultural heritage, are living paradoxes. They are defined as “a collection of beauty, icon of well-being, model of sustainability, but abandoned”. This study investigates the changes in the urban landscape of Nicosia, a particular historical centre in the Mediterranean region (Cyprus). The city centre is characterised by exceptionally well-preserved Venetian fortifications. Due to political circumstances, the capital of Cyprus, Nicosia, is still divided and has been ruled by two different administrations for several decades. This study used optical multi-spectral satellite datasets processing, like the Landsat and the most recent Sentinel-2 products, to detect, identify and characterise significant morphological transformations within the walled city and around it. This paper’s central thesis promotes a more systematic use of earth observation products and derivatives in decision-making processes that regard planning, use and management of urban resources in Europe, especially in support of urban planning strategies of historic cities.

ПРИЛАДНЕ ОСНАЩЕННЯ БАГАТОСПЕКТРАЛЬНОГО ДИСТАНЦІЙНОГО ГЕОМОНІТОРИНГУ

On the basis of the performed theoretical researches variants of practical realization of means of multispectral remote monitoring for special and general technical application at the level of structural schemes and the basic technical characteristics are considered. The method of maximum likelihood was further developed, which synthesized optimal algorithms for multispectral processing of location signals for all stages of remote geomonitoring - exposure of enemy objects, as well as the method of hardware-algorithmic complexation of partial channels of geomonitoring to the level of integration of spatial-temporal processing. implementation. An integrated radio thermal system, a block diagram of a two-spectrum ground object detector with adaptation to telephone contrast and a block diagram of an integrated artillery reconnaissance complex are proposed. It is noted that the further development of the practice of multispectral geomonitoring is the development of a system for determining the physical characteristics of enemy observation objects (targets) and a method for determining the thermophysical characteristics of objects (targets) in remote monitoring and a system for its implementation. The main physical characteristics of materials suitable for mono and multispectral fairings and qualitative assessment of parameters on a five-point scale (excluding specific gravity) are presented at the level of advantages and disadvantages. The analysis of variants of the optimal on the criterion of efficiency / cost of constructive execution of components of the equipment of space-time processing of geoinformation is executed. Keywords: geomonitoring; location monitoring, remote monitoring, radar channel; heat channel; complexation of observation channels, synchronization of observation channels, integration, detection (detection, recognition, identification) of objects (targets), telephone contrast; detector, antenna fairing.

Оценивание состояния растительности на основе аппарата нечёткой логики

Assessment of forest vegetation is based on multispectral space imagery processing. The issues of the formation of initial data and selection of test sites for automated identification of landscape elements are considered. The aim of the work is to present an assessment technology for natural objects that applies the mathematical apparatus of fuzzy logic to automated processing of multispectral aerospace imagery. An example of assessing the forest state and determining the consequences of a forest fi re is presented.

Development of a blood oxygenation phantom for photoacoustic tomography combined with online pO2 detection and flow spectrometry.

.Photoacoustic tomography (PAT) is intrinsically sensitive to blood oxygen saturation () in vivo. However, making accurate measurements without knowledge of tissue- and instrumentation-related correction factors is extremely challenging. We have developed a low-cost flow phantom to facilitate validation of PAT systems. The phantom is composed of a flow circuit of tubing partially embedded within a tissue-mimicking material, with independent sensors providing online monitoring of the optical absorption spectrum and partial pressure of oxygen in the tube. We first test the flow phantom using two small molecule dyes that are frequently used for photoacoustic imaging: methylene blue and indocyanine green. We then demonstrate the potential of the phantom for evaluating using chemical oxygenation and deoxygenation of blood in the circuit. Using this dynamic assessment of the photoacoustic measurement in phantoms in relation to a ground truth, we explore the influence of multispectral processing and spectral coloring on accurate assessment of . Future studies could exploit this low-cost dynamic flow phantom to validate fluence correction algorithms and explore additional blood parameters such as pH and also absorptive and other properties of different fluids.

Abstract 1934: Development of spectrally distinct silica coated gold nanorods for detection of cancer using MSOT

Abstract Traditional cancer imaging devices are limited in their ability to screen for multiple contrast agents simultaneously in real time. Multispectral Optoacoustic Tomography (MSOT) is an emerging imaging modality capable of real-time imaging of numerous contrast agents with enhanced spatial resolution of 75µm at depths of 5 cm. The use of exogenous contrast agents in MSOT remains largely unexplored, so we developed two species of spectrally distinct gold nanorod as contrast agents for use in MSOT. Our goal was to evaluate the potential of MSOT to spectrally differentiate two exogenous contrast agents simultaneously. Two gold nanorod species were created using hydrogen peroxide (GNR-H2O2) or ascorbic (GNR-ASC) acid as reducing agents to modify the length of each species to create nanorods with individual light absorbance spectra in the IR range (680-900 nm). These gold nanorods were highly stabilized via encapsulation with mesoporous silica along with a subsequent chitosan capping. Human epidermal growth factor receptor 2 positive (HER2+) cells were specifically targeted by conjugating these mesoporous silica-coated chitosan capped gold nanorods (CMGs) to Trastuzumab resulting in TRA-CMG particles. Both TRA-CMG-ASC and TRA-CMG-H2O2 resulted in optoacoustic spectrally distinct signals when imaged in tissue phantoms both individually as well as mixed within the same well after multispectral processing using linear regression. Treatment of HER2+ breast cancer cell lines, DY36T2Q and SKBR3, with TRA-CMG-H2O2 resulted in 2.5x and 3.1x enhanced signal, respectively, as compared to HER2- MDA-MD468 cells. Treatment of DY36T2Q and SKBR3 cells with TRA-CMG-ASC demonstrated 3.7x and 6.9x, respectively, compared to MDA-MD468. In all three cell lines treated with a combination of TRA-CMG-H2O2’s and TRA-CMG-ASC’s clear and distinct signals were observed for each particle, demonstrating that each TRA-CMG possessed and maintained a detectibly distinct optoacoustic spectrum, in the IR range, allowing them to be detectable as separate contrast agents in MSOT while proximate to other targeted contrast agents. Both particles have demonstrated that they can be simultaneously administered and targeted at HER2+ cell while also maintaining distinct photoacoustic signals in MSOT upon consolidation. Each particle species, targeted to the same cells, were capable of being monitored individually in the presence of the other gold nanorod contrast agent. Citation Format: Karl N. Thomas, Abhilash Samykutty, Molly McNally, Lacey R. McNally. Development of spectrally distinct silica coated gold nanorods for detection of cancer using MSOT [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2019; 2019 Mar 29-Apr 3; Atlanta, GA. Philadelphia (PA): AACR; Cancer Res 2019;79(13 Suppl):Abstract nr 1934.

PeX 1. Multispectral expansion of residual speckles for planet detection

The detection of exoplanets in coronographic images is severely limited by residual starlight speckles. Dedicated post-processing can drastically reduce this "stellar leakage" and thereby increase the faintness of detectable exoplanets. Based on a multi-spectral series expansion of the diffraction pattern, we derive a multi-mode model of the residuals which can be exploited to estimate and thus remove the residual speckles in multi-spectral coronographic images. Compared to other multi-spectral processing methods, our model is physically grounded and is suitable for use in an (optimal) inverse approach. We demonstrate the ability of our model to correctly estimate the speckles in simulated data and demonstrate that very high contrasts can be achieved. We further apply our method to removing speckles from a real data cube obtained with the SPHERE IFS instrument.

基于仿人眼自适应调节的多光谱视觉图像处理方法

基于仿人眼自适应调节的多光谱视觉图像处理方法

Spectral sharpening of color sensors: diagonal color constancy and beyond.

It has now been 20 years since the seminal work by Finlayson et al. on the use of spectral sharpening of sensors to achieve diagonal color constancy. Spectral sharpening is still used today by numerous researchers for different goals unrelated to the original goal of diagonal color constancy e.g., multispectral processing, shadow removal, location of unique hues. This paper reviews the idea of spectral sharpening through the lens of what is known today in color constancy, describes the different methods used for obtaining a set of sharpening sensors and presents an overview of the many different uses that have been found for spectral sharpening over the years.

The self-guided missiles to the ground target at alternating sign of the target and background contrast

The main idea of the article is a solution to the problem of the self-guided missiles to the ground target at alternating sign of the target and background contrast based on the objective coordinator that implements the method of multi-spectral processing radar information.

Intra-pixel multispectral processing of magnetic resonance brain images for tissue characterisation

Magnetic resonance MR image analysis is generally performed by spatial domainbased image processing, referred to as inter-pixel image processing, which takes advantage of spatial correlation among sample pixels. Unfortunately, in many areas, several tissue substances are usually present and mixed in a single image pixel in which such an inter-pixel processing either fails or is ineffective. To resolve this dilemma, this paper develops an unconventional approach, called intra-pixel processing, which considers MR images as multispectral images where a multispectral MR image pixel is actually a pixel vector, of which each component is captured by a particular image pulse sequence used for MR image acquisition. Since the commonly used receiver operating characteristic ROC curves cannot directly deal with the issues arising in intra-pixel processing, a 3D ROC analysis is developed by including a parameter t as the third dimension that represents abundance fractions thresholded by τ.

Multispectral integral imaging acquisition and processing using a monochrome camera and a liquid crystal tunable filter

This paper presents an acquisition system and a procedure to capture 3D scenes in different spectral bands. The acquisition system is formed by a monochrome camera, and a Liquid Crystal Tunable Filter (LCTF) that allows to acquire images at different spectral bands in the [480, 680]nm wavelength interval. The Synthetic Aperture Integral Imaging acquisition technique is used to obtain the elemental images for each wavelength. These elemental images are used to computationally obtain the reconstruction planes of the 3D scene at different depth planes. The 3D profile of the acquired scene is also obtained using a minimization of the variance of the contribution of the elemental images at each image pixel. Experimental results show the viability to recover the 3D multispectral information of the scene. Integration of 3D and multispectral information could have important benefits in different areas, including skin cancer detection, remote sensing and pattern recognition, among others.

Abstract 2440: Glioblastoma molecular imaging in vivo using multi-spectral optoacoustic tomography

Abstract Brain research has been revolutionized by imaging technologies. X-ray CT and MRI provide high spatial resolution, revealing anatomical anomalies indicative of disease. PET and optical imaging have target specificity, allowing the visualization of molecular events. Intravital microscopy has specificity and resolution, providing key information on pathological micro-events, but lacks penetration depth. Multi-spectral optoacoustic tomography (MSOT) combines high resolution, molecular specificity and depth to achieve non-invasive in vivo anatomical, functional and molecular imaging in deep tissue. MSOT illuminates tissue with light pulses at multiple wavelengths and detects the acoustic waves generated by the thermoelastic expansion following light absorption. Using spectral analysis of the data collected, MSOT can then differentiate the spectral signatures of endogenous biomarkers such as oxy-/deoxy-hemoglobin and of photo-absorbing agents and quantify their concentration. In this work we explore the potential of MSOT in cross-sectional imaging of the mouse brain and contrast these results with MRI and ex vivo brain imaging to validate the MSOT in vivo findings. 8 week old nude CD-1 mice were used for stereotactic implantation of U87 glioblastoma cells and for imaging of hemoglobin contrast and ICG biodistribution. In vivo MSOT of the intact mouse head yielded unprecedented performance in cross-sectional imaging of the mouse brain by visualizing the overall brain outline and anatomy, and imaging temporal arteries and blood vessels beneath the skull. Additionally, NIR probes were injected into the 3rd ventricle, with an excellent correlation between MSOT and fluorescence imaging of cryoslices, demonstrating the capacity of MSOT to localize NIR probes in the brain through intact skin and skull with high accuracy. In addition, spectral decomposition of hemoglobin confirmed the MSOT ability to visualize well perfused and ischemic brain conditions following a CO2 challenge. Additionally, MSOT accurately visualized ICG bio-distribution injected into the tail vein, and followed in real time the ICG kinetics and clearance. Finally, spectral decomposition of deoxygenated hemoglobin allowed the observation of hypoxia related to the growth of U87 tumor cells injected into the striatum. Multispectral processing allowed the visualization of the true organ distribution of IntegriSense and AngioSense in the brain, with planar fluorescence imaging used for a reference comparison. The application of MSOT in in vivo brain imaging is demonstrated. MSOT can be used to follow changes in blood oxygenation, as well as the distribution of near-infrared probes. With the advent of new molecular probes, MSOT could also track molecular features of neurological disease and cancer in mouse models. Citation Format: {Authors}. {Abstract title} [abstract]. In: Proceedings of the 103rd Annual Meeting of the American Association for Cancer Research; 2012 Mar 31-Apr 4; Chicago, IL. Philadelphia (PA): AACR; Cancer Res 2012;72(8 Suppl):Abstract nr 2440. doi:1538-7445.AM2012-2440