Central Pixel Research Articles

On-Orbit Modulation Transfer Function Estimation Based on the Refined Image Kernel.

To overcome the limitations of traditional on-orbit modulation function transfer (MTF) measurement methods that are heavily dependent on natural features, scenery, artificial edges, and point source targets, this paper presents an on-orbit MTF measurement method of remote sensing imager based on the refined image kernel (RIK) acquired directly from remote sensing images. First, the kernel is estimated from some remote sensing sub-images with rich texture details by using an iterative support detection (ISD) algorithm; then, it is refined by central pixel energy concentration (EC) to obtain the RIK. Secondly, the MTF curves are calculated by interpolating RIK and Fourier transform. Finally, the final MTF is the average value of MTFs at Nyquist frequency acquired by each RIK. To demonstrate the feasibility and validity of this method, the MTFs were compared to the result of the ISO12233 edge method with an error of no more than 7%. The relative error of the measured results does not exceed 5% for image signal-to-noise ratio (SNR) above 20dB. The results obtained from the on-orbit MTF measurement using remote sensing images of the Jilin-1 satellite have a maximum error of less than 2% compared with the ISO12233 edge method. These demonstrate that the method proposed in this paper supplies highly accurate and robust results and can successfully increase the efficiency of on-orbit MTF measurement, providing a reference for high-frequency monitoring of satellite on-orbit stability and their optical imaging quality.

An Adaptive Feature Fusion Network with Superpixel Optimization for Crop Classification Using Sentinel-2 Imagery

Crop-type mapping is the foundation of grain security and digital agricultural management. Accuracy, efficiency and large-scale scene consistency are required to perform crop classification from remote sensing images. Many current remote-sensing crop extraction methods based on deep learning cannot account for adaptation effects in large-scale, complex scenes. Therefore, this study proposes a novel adaptive feature-fusion network for crop classification using single-temporal Sentinel-2 images. The selective patch module implemented in the network can adaptively integrate the features of different patch sizes to assess complex scenes better. TabNet was used simultaneously to extract spectral information from the center pixels of the patches. Multitask learning was used to supervise the extraction process to improve the weight of the spectral characteristics while mitigating the negative impact of a small sample size. In the network, superpixel optimization was applied to post-process the classification results to improve the crop edges. By conducting the crop classification of peanut, rice, and corn based on Sentinel-2 images in 2022 in Henan Province, China, the novel method proposed in this paper was more accurate, indicated by an F1 score of 96.53%, than other mainstream methods. This indicates our model’s potential for application in crop classification in large scenes.

Exploring the Relationship Between Center and Neighborhoods: Central Vector Oriented Self-Similarity Network for Hyperspectral Image Classification

To mine the spectral-spatial information of target pixel in hyperspectral image classification (HSIC), convolutional neural network (CNN)-based models widely adopt patch-based input pattern, where a patch represents its central pixel and the neighbor pixels play auxiliary roles in the classification process. However, compared to the central pixel, its neighbor pixels often have different contributions for classification. Although many existing patch-based CNNs could adaptively emphasize the spatial neighbor information, most of them ignore the latent relationship between the center pixel and its neighbor pixels. Moreover, efficient spectral-spatial feature extraction has been a difficult yet vital topic for HSIC. To address the mentioned problems, a central vector oriented self-similarity network (CVSSN) is proposed for HSIC. Specifically, based on two similarity measures, we firstly design an adaptive weight addition based spectral vector self-similarity module (AWA-SVSS) in input space and a Euclidean distance based feature vector self-similarity module (ED-FVSS) in feature space to fully mine the central vector oriented spatial relationships. Besides, a spectral-spatial information fusion module (SSIF) is formulated as a new pattern to fuse the central 1D spectral vector and the corresponding 3D patch for efficient spectral-spatial feature learning of the subsequent modules. Moreover, we implement a channel spatial separation convolution module (CSS-Conv) and a scale information complementary convolution module (SIC-Conv) for efficient spectral-spatial feature learning. Extensive experimental results on four popular HSI data sets demonstrate the effectiveness and efficiency of the proposed method compared with other state-of-the-art methods. The source code is available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/lms-07/CVSSN</uri>

Lightweight Parallel Octave Convolutional Neural Network for Hyperspectral Image Classification

Although most deep learning-based methods have achieved excellent performance for hyperspectral image (HSI) classification, they are often limited by complex networks and require massive training samples in practical applications. Therefore, designing an efficient, lightweight model to obtain better classification results under small samples situations remains a challenging task. To alleviate this problem, a novel, lightweight parallel octave convolutional neural network (LPOCNN) for HSI classification is proposed in this paper. First, the HSI data is preprocessed to construct two three-dimensional (3D) patch cubes with different spatial and spectral scales for each central pixel, removing redundancy and focusing on extracting spatial features and spectral features, respectively. Next, two non- deep parallel branches are created for the two inputs, which design octave convolution rather than classical 3D convolution to facilitate light weighting of the model. Then two-dimensional convolutional neural network is used to extract deeper spectral-spatial features when fusing spectral-spatial features from different parallel layers. Moreover, the spectral-spatial attention is designed to promote the classification performance even further by adaptively adjusting the weights of different spectral-spatial features according to their contribution to classification. Experiments show that our suggested LPOCNN acquires a significant advantage on classification performance over other competitive methods under small sample situations.

Hyperspectral image denoising based on the similar spectra approaching

The local-area-based methods that utilize the weighted average of the neighbor pixels to replace the central noisy pixel, are widely used for hyperspectral image (HSI) denoising due to their good spectrum preservation performance. However, most existing local-area-based denoising methods are originally designed for the RGB image without considering the special characteristic of HSI, and they usually require a parameter to obtain the weights of the neighbor pixels. In this paper, based on the linear mixed model of HSI, we derive a significant property of the noise-free HSI that each pixel can be linearly represented by its neighbor pixels. Inspired by this property, we develop a simple but effective local-area-based denoising method, in which the weights are obtained without any parameters. Moreover, our method can automatically select those neighbor pixels which are similar to the central pixel, so the overall shape of the spectrum to be denoised can be effectively maintained. Experiments on synthetic data and real data demonstrate that the proposed denoising method is superior to the state-of-the-art methods.

Spectral-Spatial Attention Rotation-Invariant Classification Network for Airborne Hyperspectral Images

An airborne hyperspectral imaging system is typically equipped on an aircraft or unmanned aerial vehicle (UAV) to capture ground scenes from an overlooking perspective. Due to the rotation of the aircraft or UAV, the same region of land cover may be imaged from different viewing angles. While humans can accurately recognize the same objects from different viewing angles, classification methods based on spectral-spatial features for airborne hyperspectral images exhibit significant errors. The existing methods primarily involve incorporating image or feature rotation angles into the network to improve its accuracy in classifying rotated images. However, these methods introduce additional parameters that need to be manually determined, which may not be optimal for all applications. This paper presents a spectral-spatial attention rotation-invariant classification network for the airborne hyperspectral image to address this issue. The proposed method does not require the introduction of additional rotation angle parameters. There are three modules in the proposed framework: the band selection module, the local spatial feature enhancement module, and the lightweight feature enhancement module. The band selection module suppresses redundant spectral channels, while the local spatial feature enhancement module generates a multi-angle parallel feature encoding network to improve the discrimination of the center pixel. The multi-angle parallel feature encoding network also learns the position relationship between each pixel, thus maintaining rotation invariance. The lightweight feature enhancement module is the last layer of the framework, which enhances important features and suppresses insignificance features. At the same time, a dynamically weighted cross-entropy loss is utilized as the loss function. This loss function adjusts the model’s sensitivity for samples with different categories according to the output in the training epoch. The proposed method is evaluated on five airborne hyperspectral image datasets covering urban and agricultural regions. Compared with other state-of-the-art classification algorithms, the method achieves the best classification accuracy and is capable of effectively extracting rotation-invariant features for urban and rural areas.

Vote Now Application

The E-voting system offers internet voting. To stop voter fraud, we use two different levels of security. As the first level of security, a web camera records the voter's face and uploads it to the database. After that, the person's face is compared to the face in the database to verify its authenticity. The two faces are compared with the help of the Local Binary Pattern algorithm. The plan's foundation is combining two images and assigning a value to a central pixel. Those central pixels will have values of either 0 or 1. If the value is less than a pixel, a histogram of the labels is calculated and used as a descriptor.

Enhanced Embedding Capacity for Data Hiding Approach Based on Pixel Value Differencing and Pixel Shifting Technology

Data hiding algorithms can achieve the issue of information security when secret data are transmitted via the public network. This paper proposes a novel data-hiding scheme based on pixel value differencing and pixel shifting to increase embedding capacity and visual quality. In the proposed method, the cover image is first divided into nonoverlapping blocks, and eight groups of different values are generated with the center pixel as the reference pixel to embed the secret message. The pixel shifting strategy is applied to adjust the stego image to improve its quality. Experimental results show that the proposed method has an embedding capacity of 740,000 bits with a peak signal-to-noise ratio value greater than 35 dB. Therefore, it is undetected by the human eye. Other compared state-of-the-art schemes have embedding capacities of 51,219 bits, 70,217 bits, and 104,055 bits, which are lower than the proposed methods’ 740,000 bits. The RS, chi-square, and rotation attack analyses prove that the proposed method can withstand security attacks. Thus, the proposed method performs better than other state-of-the-art methods with respect to embedding capacity and ability to withstand attacks.

An LDOP approach for face identification under unconstrained scenarios

ABSTRACT In unconstrained environments, it encounters a number of challenges when considering handcrafted features for face recognition, including changes in pose, illumination and facial expression and plastic surgery variations, look-alike faces and selfie images. As majority of the published works on local descriptors are based on the relationship between the centre pixel and neighbourhood pixels at different radial widths, by including the relationship between different neighbourhood pixels of a target pixel with it can facilitate the decoupling of the discriminatory features further. To exploit such relationships and to mitigate the challenges, the proposed study reports a novel local descriptor called Local Directional Octa Pattern (LDOP) that makes use of derivative operation and three well-known metrics, namely, histogram intersection, K-nearest neighbour and Euclidean distance for face identification. Rigorous experiments have been conducted on six benchmark face databases, namely, Extended Yale Face B, Labeled Faces in the Wild, Plastic Surgery, Look-alike, UMDAA-02-FD and ORL, confirming the superiority of the proposed local descriptor over state-of-the-art descriptors with accuracies of 97.37%, 49.09%, 75.47%, 31.2% and 93.31% at Rank–10 and 100% at Rank-5 determined on the databases, respectively.

An edge-located uniform pattern recovery mechanism using statistical feature-based optimal center pixel selection strategy for local binary pattern

The Local Binary Pattern (LBP) is a commonly used method for texture classification that performs well in terms of feature discrimination. However, (1) LBP can misclassify some important edge-located textures as non-uniform patterns with only one bin in the feature histogram, thus losing their discrimination capability. (2) When center pixel is contaminated by noise, a uniform pattern may be transformed into a non-uniform pattern, which can seriously affect the obtained LBP, thus degrading the classification results. To overcome these drawbacks, an edge-located uniform pattern recovery mechanism using statistical feature-based optimal center pixel selection strategy (SFB-OCPS) is proposed in this paper.To extract the correct edge pixels, we divide the whole texture image into 16 = 4 × 4 sub-images and propose an edge pixel selection strategy (EPSS) based on adaptive quantization with local threshold on each sub-image. Then 3 candidate center pixels constructed by statistical features of the local sampling neighborhood are generated for each edge-located center pixel. After the steps above, the SFB-OCPS strategy is introduced into the LBP-based algorithms. It is possible to recover some important edge-located non-uniform patterns to uniform patterns with an optimal center pixel selection, thus improving feature discrimination capability of the LBP-based algorithms.It should be emphasized that any LBP variants can introduce the proposed SFB-OCPS strategy to achieve the recovery of the edge-located uniform patterns. To validate the effectiveness of the proposed SFB-OCPS strategy, we introduce the SFB-OCPS strategy into the original LBP and 5 representative LBP-based algorithms. Experiments are conducted on 6 representative texture databases. Classification comparison reveals that the introduction of SFB-OCPS strategy can significantly improve the texture classification performance of LBP-based algorithms. Additionally, the noise-robustness of the proposed SFB-OCPS strategy is also verified through a series of experiments.

Ensquared Energy and Optical Centroid Efficiency in Optical Sensors: Part 1, Theory

High-performance megapixel focal plane arrays with small pixels have been widely used in modern optical remote sensing, astronomical, and surveillance instruments. In the prediction models applied in the traditional instrument performance analysis, the image of a point source is assumed to fall on the center of a detector pixel. A geometrical image of a point source in the realistic optical system may actually fall on any position on the detector pixel because the sensor’s line of sight includes pointing errors and jitter. This traditional assumption may lead to an optimistic error, estimated at between 10% and 20%. We present the critical factors that impact the performance estimate in a realistic instrument design based on the prediction for the noise-equivalent power (NEP). They are the optical centroid efficiency (OCE) and the ensquared energy, or more precisely, the energy on the rectangular detector pixel (EOD). We performed the simulation studies for imaging with an optical system with and without a generalized rectangular central obscuration.

A novel UAV-integrated deep network detection and relative position estimation approach for weeds

This paper aims at presenting a novel monocular vision–based approach for drones to detect multiple type of weeds and estimate their positions autonomously for precision agriculture applications. The methodology is based on classifying and detecting the weeds using a proposed deep neural network architecture, named fused-YOLO on the images acquired from a monocular camera mounted on the unmanned aerial vehicle (UAV) following a predefined elliptical trajectory. The detection/classification is complemented by a new estimation scheme adopting unscented Kalman filter (UKF) to estimate the exact location of the weeds. Bounding boxes are assigned to the detected targets (weeds) such that the centre pixels of the bounding box will represent the centre of the target. The centre pixels are extracted and converted into world coordinates forming azimuth and elevation angles from the target to the UAV, and the proposed estimation scheme is used to extract the positions of the weeds. Experiments were conducted both indoor and outdoor to validate this integrated detection/classification/estimation approach. The errors in terms of misclassification and mispositioning of the weeds estimation were minimum, and the convergence of the position estimation results was short taking into account the affordable platform with cheap sensors used in the experiments.

HoMeNL: A Homogeneity Measure-Based NonLocal Filtering Framework for Detail-Enhanced (Pol)(In)SAR Image Denoising

As an inherent problem in coherent imaging systems, the existence of speckle noise results in SAR images with strong signal-dependent variance and seriously hinders the related properties estimation and the image interpretation. Among many filtering methods, nonlocal means (NLM) have been proven to be effective in reducing noise while preserving details. However, traditional NLM filters still face two core problems: 1) it is difficult for homogeneous pixels selection to construct a patch adaptive to local structure for preventing the omission phenomenon; 2) most central pixel value estimators are still at the stage of suppressing the blurring effect rather than eliminating the wrongly selected heterogeneous pixels. To overcome these two problems in the (Pol)(In)SAR image denoising, this paper proposes a homogeneity measure-based nonlocal (HoMeNL) filtering framework based on the following three innovations: 1) to sufficiently select homogeneous pixels in the patch-wise matching processing, the shape-adaptive (SA) patch can be selected from multiple preset patches with the one-to-many matching strategy; 2) as a general extension of the Lee estimator in (Pol)(In)SAR image denoising, the homogeneity measure (HoMe)-based estimator can achieve an optimal bias-variance tradeoff for the central pixel value; 3) the highlight of the proposed method is that the iterative re-weighted (IRW) estimation combines the residuals statistics and the homogeneity measure to adaptively locate and remove the wrongly selected heterogeneous pixels. Simulated and real experimental results show that the proposed filtering framework owns a superior performance than most state-of-art filters in three aspects of noise reduction, detail enhancement, and coherence magnitude estimation.

From center to surrounding: An interactive learning framework for hyperspectral image classification

Owing to rich spectral and spatial information, hyperspectral image (HSI) can be utilized for finely classifying different land covers. With the emergence of deep learning techniques, convolutional neural network (CNN), fully convolutional network (FCN), and recurrent neural network (RNN) have been widely applied in the field of HSI classification. Recently, transformer-based approaches represented by Vision Transformer (ViT) have yielded promising performance on numerous tasks and have been introduced to classify HSI. However, existing methods based on the above architectures still face three crucial issues that limit the classification performance: 1) geometric constraints caused by input data, 2) contribution fuzziness of central pixels with details, and 3) interaction gap between local areas and further environments. To tackle the above problems, an interactive learning framework inspired by ViT is proposed from a center to surrounding perspective, namely the center-to-surrounding interactive learning (CSIL) framework. Different from existing works, the CSIL framework enables to achieve multi-scale, detail-aware, and space-interactive classification based on a well-designed hierarchical region sampling strategy, center transformer, and surrounding transformer. Specifically, a hierarchical region sampling strategy is first proposed to flexibly generate the center region, neighbor region, and surrounding region, respectively. Thus, multi-scale input data breaks the geometric constraints. Second, a center transformer is presented to obtain core characteristics in detail based on the center region. In this way, central pixels are remarkably highlighted and the details are easily perceived. Third, a surrounding transformer including interactive self-attention learning is formulated for interacting both locally fine-grained distributions in the neighbor region and further coarse-grained environments in the surrounding region. With this structure, short- and long-term dependencies can be modeled, emphasized, and exchanged to bridge the interaction gap. Finally, the features from center transformer and surrounding transformer are integrated, then fed into a multi-layer perceptron for the optimization of semantic representation. Extensive experiments on six HSI datasets including small-, medium-, and large-scale scenes demonstrate the superiority over state-of-the-art CNN–, FCN-, RNN- and transformer-based approaches, even with very few training samples (for example 0.19% in complex HanChuan city scene). The source code will be available soon at https://github.com/jqyang22/CSIL.

Likelihood-based bilateral filters for pre-estimated basis sinograms using photon-counting CT.

Noise amplification in material decomposition is an issue for exploiting photon-counting computed tomography (PCCT). Regularization techniques and neighborhood filters have been widely used, but degraded spatial resolution and bias areconcerns. This paper proposes likelihood-based bilateral filters that can be applied to pre-estimated basis sinograms to reduce the noise while minimally affecting spatial resolution andaccuracy. The proposed method needs system models (e.g., incident spectrum, detector response) to calculate the likelihood. First, it performs maximum likelihood (ML)-based estimation in the projection domain to obtain basis sinograms. The estimated basis sinograms suffer from severe noise but are asymptotically unbiased without degrading spatial resolution. Then it calculates the neighborhood likelihoods for a given measurement at the center pixel using the neighborhood estimates and designs the weights based on the distance of likelihoods. It is also analyzed in terms of statistical inference, and then two variations of the filter are introduced: one that requires the significance level instead of the empirical hyperparameter. The other is a measurement-based filter, which can be applied when accurate estimates are given without the system models. The proposed methods were validated by analyzing the local property of noise and spatial resolution and the global trend of noise and bias using numerical thorax and abdominal phantoms for a two-material decomposition (water and bone). They were compared to the conventional neighborhood filters and the model-based iterative reconstruction with an edge-preserving penalty applied in the basisimages. The proposed method showed comparable or superior performance for the local and global properties to conventional methods in many cases. The thorax phantom: The full width at half maximum (FWHM) decreased by -2%-31% (-2 indicates that it increased by 2% compared to the best performance from conventional methods), and the global bias was reduced by 2%-19% compared to other methods for similar noise levels (local: 51% of the ML, global: 49%) in the water basis image. The FWHM decreased by 8%-31%, and the global bias was reduced by 9%-44% for similar noise levels (local: 44% of the ML, global: 36%) in the CT image at 65 keV. The abdominal phantom: The FWHM decreased by 10%-32%, and the global bias was reduced by 3%-35% compared to other methods for similar noise levels (local: 66% of the ML, global: 67%) in the water basis image. The FWHM decreased by up to -11%-47%, and the global bias was reduced by 13%-35% for similar noise levels (local: 71% of the ML, global: 70%) in the CT image at 60 keV. This paper introduced the likelihood-based bilateral filters as a post-processing method applied to the ML-based estimates of basis sinograms. The proposed filters effectively reduced the noise in the basis images and the synthesized monochromatic CT images. It showed the potential of using likelihood-based filters in the projection domain as a substitute for conventional regularization or filteringmethods.

Parameter adaptive unit-linking pulse coupled neural network based MRI–PET/SPECT image fusion

Medical image fusion has many applications to healthcare that is accomplished by extracting and then combining the complementary information from multiple medical images into a single image. The pulse coupled neural network (PCNN) is greatly applied to image fusion due to its efficient coupling of surrounding neurons, but suffers from network complexity and manual parameter settings. This paper proposes a novel parameter adaptive unit-linking PCNN (PAULPCNN) model, whose parameters are automatically obtained from the external stimulus and exhibits a simpler structure than the original PCNN. The multi-scale decomposition-based methods, particularly based on the non-subsampled shearlet transform (NSST), are widely employed by researchers to perform medical image fusion due to their efficient separation of spatial details at different scales. Motivated by the advantages of the PCNN and multi-scale decomposition-based methods, a hybrid medical image fusion method based on the PAULPCNN model is introduced in the NSST domain that merges the salient complementary details from a gray-scale and the corresponding pseudo-color medical image captured at different modalities to produce a more informative image that is more useful to medical experts in computer-aided diagnosis of diseases. The proposed method first employs NSST to transform the source images into a low-pass and several high-pass sub-bands, respectively. The high-pass sub-bands are combined using the firing times of the proposed PAULPCNN model, whereas a new distance-weighted regional energy-based rule is applied to construct the fused low-pass sub-band, which while estimating regional energy, assigns weight to different neighboring pixels depending on their distance from the central pixel. Finally, inverse NSST is applied on the fused sub-bands to construct the fused image. The effectiveness of the proposed technique is shown using the fusion results of eleven state-of-the-art methods on thirty gray-scale and pseudo-color brain medical image pairs comprising mild Alzheimer’s disease, Huntington’s disease, motor neuron disease, sagittal plane, coronal plane, transaxial plane, and glioma image pairs, where eight objective metrics are considered for the quantitative assessment. Experimental results demonstrate that the proposed method is competitive with the state-of-the-art methods, which even outperforms some of these methods by providing fused images with better visual quality and greater objective performance, which are more suitable for specialists to diagnose brain diseases.

HYPSO-1 CubeSat: First Images and In-Orbit Characterization

The HYPSO-1 satellite, a 6U CubeSat carrying a hyperspectral imager, was launched on 13 January 2022, with the Goal of imaging ocean color in support of marine research. This article describes the development and current status of the mission and payload operations, including examples of agile planning, captures with low revisit time and time series acquired during a campaign. The in-orbit performance of the hyperspectral instrument is also characterized. The usable spectral range of the instrument is in the range of 430 nm to 800 nm over 120 bands after binning during nominal captures. The spatial resolvability is found empirically to be below 2.2 pixels in terms of Full-Width at Half-Maximum (FWHM) at 565 nm. This measure corresponds to an inherent ground resolvable resolution of 142 m across-track for close to nadir capture. In the across-track direction, there are 1216 pixels available, which gives a swath width of 70 km. However, the 684 center pixels are used for nominal captures. With the nominal pixels used in the across-track direction, the nadir swath-width is 40 km. The spectral resolution in terms of FWHM is estimated to be close to 5 nm at the center wavelength of 600 nm, and the Signal-to-Noise Ratio (SNR) is evaluated to be greater than 300 at 450 nm to 500 nm for Top-of-Atmosphere (ToA) signals. Examples of images from the first months of operations are also shown.

Robust Full-Screen Wavelength Calibration Algorithm

In spectrometer measurement, it is very important to accurately calibrate the wavelength of all target characteristic spectra. Although wavelength calibration methods have long been investigated, no techniques have been designed for the scanning, double-layer, secondary diffraction, linear-array CCD spectrometer, to the best of our knowledge. Based on the grating diffraction equation and experimental results, a mathematical model of wavelength calibration was established for the scanning, double-layer, secondary diffraction, linear-array CCD spectrometer. In this study, a robust, full-screen, wavelength calibration algorithm is proposed, based on the related working principle and the requirements of both accuracy and robustness. The detailed steps are as follows. First, we established a wavelength calibration model at central pixels, following the grating diffraction equation. Then, according to the relationship between the difference in pixels and the feedback values of the grating ruler, a model was established to show the association between these factors. Finally, we combined the two models and built a full-screen wavelength calibration model. We theoretically and experimentally demonstrate that the proposed calibration algorithm is an excellent calibration tool, which can conveniently and accurately calibrate the wavelengths of central and non-central pixels at the same time.

Research on plant leaf recognition method based on multi-feature fusion in different partition blocks

As an indispensable organism in nature, plants play an essential role in the ecological equilibrium. Ecological plant protection is also receiving increasing attention. In today's digital era, one of the important research topics is recognizing plants by using imaging equipment. In this study, we propose a multi-feature fusion approach based on the Local Binary Pattern (LBP) feature for plant leaf recognition by using the partition block strategy. By comparing the original LBP and Multiscale Block Local Binary Pattern (MB-LBP), we present an improved LBP feature descriptor. It extends the range of feature extraction, considering the effect that the multi-neighbourhood pixels have on the central pixels during the LBP encoding process and the central pixels are represented by double coding values. Therefore, it can extract more detailed information about plant leaves. Moreover, considering the leaf boundary and shape information extraction and the illumination variations, the Histogram of Oriented Gradient (HOG) feature and the colour feature are fused with the improved LBP feature descriptor. After dimensionality reduction by Principal Component Analysis (PCA), a mixed feature vector of plant leaves is used as input to an Extreme Learning Machine (ELM) for identifying plant leaves in two publicly available datasets, the Flavia dataset and the Swedish dataset. Experimental results show that our algorithm outperforms the conventional algorithms, with the recognition accuracy of 99.30% on the Flavia dataset where 4 × 4 partition blocks are used for the improved LBP feature and 2 × 2 partition blocks for the colour feature, 99.52% on the Swedish dataset where 4 × 4 partition blocks are used for the improved LBP feature and 2 × 2 partition blocks for the colour feature, respectively.

Semantic segmentation of superficial layer in intracoronary optical coherence tomography based on cropping-merging and deep learning

Manual labelling and qualitative analysis of lesion tissues in vessel optical coherence tomography (OCT) images are time-consuming and laborious for cardiovascular specialists. To beat these issues, a semantic segmentation method was presented based on deep learning on the OCT image analysis of the human vessel to reduce the diagnosis pressure of cardiovascular doctors. An outer border of ROI was segmented based on the level-set method to obtain the visible superficial layer containing useful information. And then, cropping square patches from the ROI with its patch center pixel inside the ROI and using patches as the input data. To reuse the preceding layers' feature maps, we used the dense block to replace the normal convolution layer, and simultaneously, employed a skip-connection from m the down-sampling path to the up-sampling path to keep the spatial information. With the advantage of SegNet on semantic segmentation, a dense-block-SegNet (DBSegNet) is constructed to complete the pixel-level segmentation. Training and testing were executed on 7 datasets (22, 210 images) to assess the model. Tenfold cross-validation method was implemented to measure the classification outcomes and the semantic segmentation capacity of our model. In deep learning experiment, sensitivities for calcified, fibrous and lipid plaques were 91.81 ± 3.60 %, 92.81 ± 2.72 % and 91.78 ± 1.62 %, respectively. A 3-D volume was created to insert each prediction slice along the depth axis to compute the maximum type number of each pixel in order to identify the final type of each pixel. Post-processing was used to refine the classification findings in order to eliminate classification errors. Semantic segmentation neural network based on the cropped input data and feature reusing was a feasible approach for the vessel tissue pixel-classification of IVOCT image. The proposed method has the potential to become a useful tool for specialists in analysing lesion kinds and determining clinical lesion characteristics (e.g., distribution position, angle and depth).