Local Window Research Articles

Convergence Results in Image Interpolation With the Continuous SSIM

Assessing the similarity of two images is a complex task that attracts significant efforts in the image processing community. The widely used structural similarity index measure (SSIM) addresses this problem by quantifying a perceptual structural similarity. In this paper we consider a recently introduced continuous SSIM (cSSIM), which allows one to analyze sequences of images of increasingly fine resolutions, and further extend the definition of the index to encompass the locally weighted version that is used in practice. For both the local and the global versions, we prove that the continuous index includes the classical SSIM as a special case, and we provide a precise connection between image similarity measured by the cSSIM and by the $L_2$ norm. Using this connection, we derive bounds on the cSSIM by means of bounds on the $L_2$ error, and we even prove that the two error measures are equivalent in certain circumstances. We exploit these results to obtain precise rates of convergence with respect to the cSSIM for several concrete image interpolation methods, and we further validate these findings by different numerical experiments. This newly established connection paves the way to obtain novel insights into the features and limitations of the SSIM, including on the effect of the local weighted window on the index performances.

Tracking by Joint Local and Global Search: A Target-Aware Attention-Based Approach.

Tracking-by-detection is a very popular framework for single-object tracking that attempts to search the target object within a local search window for each frame. Although such a local search mechanism works well on simple videos, however, it makes the trackers sensitive to extremely challenging scenarios, such as heavy occlusion and fast motion. In this article, we propose a novel and general target-aware attention mechanism (termed TANet) and integrate it with a tracking-by-detection framework to conduct joint local and global search for robust tracking. Specifically, we extract the features of the target object patch and continuous video frames; then, we concatenate and feed them into a decoder network to generate target-aware global attention maps. More importantly, we resort to adversarial training for better attention prediction. The appearance and motion discriminator networks are designed to ensure its consistency in spatial and temporal views. In the tracking procedure, we integrate target-aware attention with multiple trackers by exploring candidate search regions for robust tracking. Extensive experiments on both short- and long-term tracking benchmark datasets all validated the effectiveness of our algorithm.

Effects of Post-Anthesis Temperature and Radiation on Grain Filling and Protein Quality of Wheat (Triticum aestivum L.)

Due to climate change, global warming, and reduced radiation, there is an urgent need for research to explore the effects on wheat grain filling and protein-related quality. In this study, two spring and two semi-winter varieties were analyzed. Six sowing dates were set in the experiment, at 10-day intervals from the beginning of the local sowing window. The seedling population of the first sowing date (S1) was 300 × 104 plants ha−1, which was observed to have increased by about 10% by the subsequent sowing date. During the experiment, due to the different dates of sowing, the treatments were in different growth stages; so, all the treatments were grown under different day and night temperatures and radiation to study the effects on post-anthesis grain filling and protein quality. The results showed that the sowing date decreased the effective accumulated temperature (EAT) and the cumulative radiation after anthesis and increased the daily average, maximum, and minimum temperatures. The decrease in the EAT of 94.99 °C d and the increase in the daily average temperature of 1.59 °C after antrum resulted in a decrease in the wheat grain weight of 4.5 g and an increase in the grain filling rate of 0.029 mg d−1. This may be due to the shortening of the wheat filling time with the increase in the day/night temperatures. Compared with the normal sowing date, the later sowing date caused a decrease in the EAT and an increase in the Tmean, which led to an increase in the wheat protein content, wet gluten content, and sedimentation value. There was a positive correlation between the grain filling rate and the protein content in the wheat. Compared with radiation, temperature significantly regulates wheat grain filling and protein formation. These results can be used to guide the sowing date to obtain a higher quality of wheat protein in the future climate change.

The effect of nitrogen-rich ionic liquid [EMIMDCA] on the electronic structure of solid polymer electrolyte (PEO-LiTFSI).

In our work, the ab-initio methods based on density functional theory (DFT) formalism are used to investigate the effect of nitrogen-rich ionic liquid (EMIMDCA; 1-ethyl-3-methylimidazolium dicyanamide) on the electronic structure of SPE (solid polymer electrolyte) (PEO-LiTFSI) electrolyte with B3LYP method and LANL2DZ basis set. The differences in energy levels of highest occupied molecular orbital (HOMO) and lowest unoccupied molecular orbital (LUMO), i.e., energy gap ([Formula: see text]) of the PEO, [EMIMDCA], SPE, and ionogel (IG), are found 5.74, 6.59, 3.25, and 2.80eV, respectively. In SPE, the reduced band gap ([Formula: see text]) is due to the transportation of cation of LiTFSI (Li+ ion). Finally, the reduced band gap of IG confirms that IL enhances the transportation of cation of LiTFSI (Li+ ion) through the oxygen atom of PEO. The global and local chemical descriptors and electrochemical stability windows (ESWs) are calculated using the HOMO-LUMO energy levels. Partial charge analysis is studied by Mulliken population analysis and molecular electrostatic potential surface.

CAW: A Remote-Sensing Scene Classification Network Aided by Local Window Attention.

Remote-sensing image scene data contain a large number of scene images with different scales. Traditional scene classification algorithms based on convolutional neural networks are difficult to extract complex spatial distribution and texture information in images, resulting in poor classification results. In response to the above problems, we introduce the vision transformer network structure with strong global modeling ability into the remote-sensing image scene classification task. In this paper, the parallel network structure of the local-window self-attention mechanism and the equivalent large convolution kernel is used to realize the spatial-channel modeling of the network so that the network has better local and global feature extraction performance. Experiments on the RSSCN7 dataset and the WHU-RS19 dataset show that the proposed network can improve the accuracy of scene classification. At the same time, the effectiveness of the network structure in remote-sensing image classification tasks is verified through ablation experiments, confusion matrix, and heat map results comparison.

Using Discrete Wavelet Transform and Wiener filter for Image De-nosing

In this paper we proposed an algorithm for image de-nosing based on; the two level discrete wavelet transform (DWT), and Wiener filter, also this paper describe estimate noise power. At first The DWT transform noisy image into sub-bands, consist of lowfrequencyand high-frequencies, and then estimate noise power for each sub-band. The noise power is computed through two important computations; compute square of variance for each sub-band then compute the mean of the variance. After compute the variance apply the wiener filter on each sub-band by using local window nn , finally perform inverse DWT to obtain de-noised image. Our algorithm tested on the two color images and also compared with Normal Shrink filter and Wiener filter.

Collaborative representation with multipurification processing and local salient weight for hyperspectral anomaly detection

Anomalous objects detection for hyperspectral imagery is a significant branch in the area of remote sensing. Although enormous advancements have been developed, issues of redundancy of spectral information and correlation between pixels should be further explored and improved. To address these problems, we proposed a method that is on the basis of integrating collaborative representation with multipurification processing and local salient weight. Multipurification processing consists of spectral bands purification (SBP) and background purification (BGP). First, to alleviate the interference of redundant spectral information, we remove unnecessary spectral bands by adopting SBP based on considering the global spectral intensity of each band. Then, we remove the outliers in the local dual window by BGP to avoid the effect of heterogeneous pixels. Simultaneously, we obtain the local salient weight by calculating the similarity and difference of pixels in the dual window. Next, we obtain the initial detection result by a collaborative representation, which has been testified to be very effective. Finally, combined with the local salient weight map, the initial detection map is improved to the final detection map. To demonstrate the superiority of the proposed method, we conducted the comprehensive experiment on three public benchmark datasets that contain 15 hyperspectral images.

GLF-CR: SAR-enhanced cloud removal with global–local fusion

The challenge of the cloud removal task can be alleviated with the aid of Synthetic Aperture Radar (SAR) images that can penetrate cloud cover. However, the large domain gap between optical and SAR images as well as the severe speckle noise of SAR images may cause significant interference in SAR-based cloud removal, resulting in performance degeneration. In this paper, we propose a novel global–local fusion based cloud removal (GLF-CR) algorithm to leverage the complementary information embedded in SAR images. Exploiting the power of SAR information to promote cloud removal entails two aspects. The first, global fusion, guides the relationship among all local optical windows to maintain the structure of the recovered region consistent with the remaining cloud-free regions. The second, local fusion, transfers complementary information embedded in the SAR image that corresponds to cloudy areas to generate reliable texture details of the missing regions, and uses dynamic filtering to alleviate the performance degradation caused by speckle noise. Extensive evaluation demonstrates that the proposed algorithm can yield high quality cloud-free images and outperform state-of-the-art cloud removal algorithms with a gain about 1.7 dB in terms of PSNR on SEN12MS-CR dataset.

Modified Newton-residual interpolation for division of focal plane polarization image demosaicking.

With the improvement of semiconductor processing technology, polarization sensors using division of focal plane have gradually become the mainstream method of polarization imaging. Similar to the color restoration method of the Bayer array sensor, the spatial information of polarized image is also recovered through the polarization demosaicking algorithm. In this paper, we propose a new modified Newton-residual interpolation polarization image demosaicking algorithm based on residual interpolation, which is suitable for a monochrome or color polarization filter array. First, we use the modified Newton interpolation method to generate edge-sensitive guiding images. Then, we carry out the improvement of the guide process during the residual interpolation by performing variance statistics on the local window image in the guiding process, so that the edges and flat image blocks have different guiding weights. Finally, we obtain edge-preserving results by applying these two improvements, which reduces the zipper effect and edge confusion. We compare the results of various algorithms on experimental data, demonstrating that our algorithm has impactful improvements in the evaluation metrics based on the ground-truth images.

A deep relearning method based on the recurrent neural network for land cover classification

ABSTRACT Recent developments in deep learning (DL) techniques have provided a series of new methods for land cover classification. However, most DL-based methods do not consider the rich spatial association of land cover classes embedded in remote sensing images. In this research, a deep relearning method based on the recurrent neural network (DRRNN) is proposed for land cover classification. The relearning approach has great potential to improve classification, which has never been used in DL-based land cover classification. To utilize the spatial association of the pixels’ information classes, a class correlated feature (CCF) is first extracted in a local window from an initial classification result. This feature can reflect both the spatial autocorrelation and spatial arrangement of land cover classes. Since the recurrent neural network (RNN) is designed to process sequential data, the CCF is formed as a feature sequence, allowing RNN to model the dependency between class labels. The relearning process is then applied to iteratively classify remote sensing images based on the CCF and spectral-spatial feature. At each relearning iteration, the CCF is learned from the previous classification result until a stopping condition is satisfied. This method was tested on five remote sensing images with different sensors and diverse environments. It was observed that noise in the classification result can be filtered by considering spatial autocorrelation, and misclassified areas can be corrected by incorporating spatial arrangement in the relearning process. The classification results indicate that compared to other state-of-the-art DL methods, the proposed method consistently achieves the highest accuracy.

Machine learning in interpretation of electronic core-level spectra

Electronic transitions involving core-level orbitals offer a localized, atomic-site and element specific peek window into statistical systems such as molecular liquids. Although formally understood, the complex relation between structure and spectrum – and the effect of statistical averaging of highly differing spectra of individual structures – render the analysis of an ensemble-averaged core-level spectrum complicated. We explore the applicability of machine learning for molecular structure — core-level spectrum interpretation. We focus on the electronic Hamiltonian using the H2O molecule in the classical-nuclei approximation as our test system. For a systematic view we studied both predicting structures from spectra and, vice versa, spectra from structures, using polynomial approaches and neural networks. We find predicting spectra easier than predicting structures, where a tighter grid (even unphysical) of the spectrum improves prediction, possibly inviting for over-interpretation of the model. The accuracy of the structure prediction worsens when moving outwards from the center of mass of the training set in the structural parameter space, which cannot be overcome by model selection based on generalizability.

SAR Ship Image Speckle Noise Suppression Algorithm Based on Adaptive Bilateral Filter

Aiming at the shortcomings of traditional bilateral filtering in suppressing speckle noise in SAR ship images, especially strong speckle noise and loss of image edge details, it proposes an improved bilateral filtering algorithm based on fast adaptive threshold and variable window in this paper. The algorithm was used to suppress speckle noise in SAR ship images. The traditional bilateral filtering cannot effectively filter out the strong speckle noise, but the SAR image has strong speckle noise because of the defects of its imaging principle. To solve these problems, an image sample truncation method based on fast adaptive truncation statistical characteristics is designed to adjust the photometric similarity weight characteristics to realize the adaptive adjustment of spatial standard deviation and gray standard deviation. After the local reference window is modified and truncated according to the local characteristics of the image, the adjusted combined similarity weight value greatly reduces the impact of strong speckle noise on the image. It is smoothed into speckle signal with strong impulse noise. In the traditional bilateral filtering, in order to enhance the effect of smoothing noise, it is necessary to specify a large value of geometric diffusion factor and gray similarity diffusion factor, resulting in the loss of image details. Based on the variable window size filtering method, when the extended local reference window is in the case of nonuniform edge, its window can be enlarged to make the speckle noise stronger. When an extended window contains details such as edges and textures, its size is not expanded to maintain image detail. This method can further smooth the speckle noise in the uniform region while preserving the edge details of the image. Finally, the adaptive truncated sample is used as the input of the bilateral filter. The image sample truncation method based on fast adaptive threshold can effectively eliminate the strong speckle noise information that affects the photometric similarity and weight accuracy of the image. The method based on variable window can greatly enhance the smoothness of the edge area of the image. The experimental results show that improved adaptive bilateral filtering algorithm improves the speckle noise removal ability by 16.06% compared with the traditional bilateral filtering algorithm in the speckle noise suppression of the SAR ship image, and the preservation performance of the image edge after filtering is improved by 5.41%. Compared with the original image, the filtered image has a 1.2% improvement in structural similarity. The algorithm can effectively suppress speckle noise and has a good ability to retain edge and texture information of the SAR ship image, which has strong practicability.

Low-complexity trainable denoising filter structure for image recognition applications

Even in modern imaging equipment, different types of noise make their way into the images especially under bad lighting conditions and rough communication channels. While applying denoising filters, the preservation of the edge in images has always been an essential albeit a difficult task. This is particularly important for accuracy of further image processing tasks, such as recognition. We described a denoising approach to process noisy images specific for the task of classification with minimal impact on image resolution using low-complexity operations. Specifically, support vector machine-based machine learning classifiers have been proposed to dynamically select an appropriate filtering kernel while processing individual pixels in the input image. The selection is based on the image features in a local window. The provided experimental results on standard dataset prove that the proposed scheme performs as well as contemporary complex convolutional neural network (CNN)-based approaches for the task of denoising images before classification in the presence of a variety of mixed-type noise. Moreover, the proposed filtering approach can run at much higher speeds than their CNN-based counterparts while delivering similar or better performance. The proposed framework has been made publicly available as open-source code to facilitate further work by researchers and practitioners interested in this area.

The Evaluation Method of Community Emergency Resource Allocation Based on Coordination Sensor Information Collection

This paper analyzes the information collection data of coordination sensors and designs a corresponding evaluation method to conduct an in-depth assessment of the configuration of community emergency resources. Based on the principles of assessment index system construction and the actual research, an assessment index system of community emergency resource allocation level is established. In this paper, the G1 method was selected to determine the weights of the evaluation indexes, and the gray clustering method was applied to construct the whitening weight function, determine the gray level to which each index belongs and the affiliation degree of each gray class, and establish a comprehensive evaluation model of the community emergency resource allocation level. In response to the problem that the increase of state variables leads to a decrease in real-time map building, a dynamic local window size mechanism is proposed, which can reduce the time consumption and save computational resources under the condition of ensuring the accuracy of positioning and map building. Therefore, it is urgent to design an efficient self-interference cancellation mechanism to resist the influence of self-interference on the signal-to-noise ratio of the first-hop link. For example, the IMU pre integration theory is combined with the wheel range method to solve the problem of frequency synchronization; An initialization algorithm is proposed to recover the scale information of the camera and optimize the external parameters; Design a graphic optimization framework integrating IMU, wheel range finder and camera. The monitoring terminal sends the data information of coordination transportation to the monitoring platform in real-time, and the monitoring platform is responsible for storing and displaying the data information, thus realizing the real-time monitoring of the coordination transportation process. Finally, the functions of the monitoring terminal and the display system of the monitoring platform experiment, respectively, and the test results verify the effectiveness and integrity of the system data communication, which proves the correctness of the terminal design of the integrated coordination monitoring system based on multidimensional information and has practical engineering value.

Relation of local window size in a model of modules with estimation of the size of visual images and their segmentation

Subject of study. The fundamental mechanisms of visual image processing, such as size estimation and segmentation, are considered. The possibility of describing these mechanisms using the model of modules proposed by V. D. Glezer and based on electrophysiological data from studies of the receptive fields of neurons in the visual cortex was investigated. Aim of study. This study aimed to investigate mechanisms of size estimation and the segmentation processes by comparing the experimental data obtained by the authors with modeling results. Methods. Psychophysical methods were used for the experimental part. Two different paradigms were used for size estimation: a modified Ebbinghaus illusion with different surroundings of the test stimuli or a comparison of the sizes of upright and oblique crosses previously used in experiments on segmentation. Spatial-frequency filtering of images in local areas of the visual field was performed during modeling using a finite set of filters. Main results. The dependence of the size estimate on the distance between images and on their shape and the relationship of those estimates with segmentation was demonstrated for the first time to our knowledge. Another new result shows that images with different shapes are perceived as equal in size if the modules that optimally describe these images (i.e., maximum energy is preserved in images when filtered with a limited number of filters) are of equal sizes. The obtained results demonstrate that the model of modules can describe to a first approximation the mechanisms performing the estimation of the sizes of the images and their segmentation. Correspondence between the data of neurophysiological, psychophysical, and modeling investigations was shown for the first time to our knowledge. The conclusions were supported by a comparison of the experimental results, and the modeling, with patterns of traditional Byzatntine icon paintings, as well as Russian avant-garde paintings of the early 20th century, which were inspired by the former. Practical significance. Application and further enhancement of the model of modules as an artificial neural network that ensures the segmentation, image size estimation, and recognition of visual objects can be of practical significance.

An Unsupervised Fuzzy Clustering Approach for Early Screening of COVID-19 From Radiological Images.

A global pandemic scenario is witnessed worldwide owing to the menace of the rapid outbreak of the deadly COVID-19 virus. To save mankind from this apocalyptic onslaught, it is essential to curb the fast spreading of this dreadful virus. Moreover, the absence of specialized drugs has made the scenario even more badly and thus an early-stage adoption of necessary precautionary measures would provide requisite supportive treatment for its prevention. The prime objective of this article is to use radiological images as a tool to help in early diagnosis. The interval type 2 fuzzy clustering is blended with the concept of superpixels, and metaheuristics to efficiently segment the radiological images. Despite noise sensitivity of watershed-based approach, it is adopted for superpixel computation owing to its simplicity where the noise problem is handled by the important edge information of the gradient image is preserved with the help of morphological opening and closing based reconstruction operations. The traditional objective function of the fuzzy c-means clustering algorithm is modified to incorporate the spatial information from the neighboring superpixel-based local window. The computational overhead associated with the processing of a huge amount of spatial information is reduced by incorporating the concept of superpixels and the optimal clusters are determined by a modified version of the flower pollination algorithm. Although the proposed approach performs well but should not be considered as an alternative to gold standard detection tests of COVID-19. Experimental results are found to be promising enough to deploy this approach for real-life applications.

Binarization Algorithm Based on Side Window Multidimensional Convolution Classification.

Uneven illumination and space radiation can cause inhomogeneous grayscale distribution, low contrast, and noisy images in in-orbit cameras. A binarization algorithm based on morphological classification is proposed to solve the problem of inaccurate image binarization caused by space image degradation. Traditional local binarization algorithms generally calculate thresholds based on statistical information of gray dimensions within the local window, often ignoring the morphological distribution information, leading to poor results in degraded images. The algorithm presented in this paper demonstrates the property of the side window filtering (SWF) kernel on morphological clustering. First, the eight-dimensional SWF convolution kernel is used to describe the morphological properties of the pixels. Then, the positive and negative types of each pixel in the local window are identified, and the local threshold is calculated according to the difference between the two types. Finally, the positive pixel is used to filter the threshold of each pixel, with the binarization threshold satisfying the morphologically smooth and continuous property. A self-built dataset is used to evaluate the algorithm quantitatively and the results are compared with the three existing classical techniques using the quantitative measures FM, PSNR, and DRD. The experimental results show that the algorithm in this paper yields good binarization results for different degraded images, outperforms the comparison algorithm in terms of accuracy and robustness, and is insensitive to noise.

A local multi-block with various constraints approach for stereo vision in StereoPi

The conventional local block matching algorithms are fast but have an accuracy issue, making them unsuitable to be implemented on real-time applications. This paper studies a local block matching algorithm with different cost computations, combining a cost aggregation with various blocks' shape and sizes, using the computer and StereoPi. Compared to other local block matching algorithms, the Ranking-Gradient-Similarity Multi-Block (RGSMB) algorithm can extract information with a higher cost uniqueness within the predefined local window. Using KITTI 2015 as input data, RGSMB has better accuracy than other local block matching approaches. It is at least 5% better than Census Transform (CT), 23% better than Ranking Transform (RT), and 62% better than Sum of Absolute Differences (SAD) in terms of accuracy. Furthermore, it can achieve lower computational costs on the StereoPi, a low-cost computer vision embedded system. The RGSMB can achieve 2.5 fps with the right image resolution and parameter settings.

A novel unsupervised multiple change detection method for VHR remote sensing imagery using CNN with hierarchical sampling

ABSTRACT Detecting multiple changes from remote sensing imagery is a research hotspot. Very high resolution (VHR) images contain detailed spatial information and thus are often used in multiple change detection (CD). Compared with supervised multiple CD methods, unsupervised methods are more attractive, due to the ability of extracting changes automatically. However, many existing unsupervised methods fail to well adaptively make use of the high-level features relevant to multiple changes in VHR images in some cases. In this paper, a novel unsupervised multiple CD method for VHR images is proposed. First, the magnitude of spectral change vectors (SCVs) is calculated by change vector analysis, and fuzzy c-means clustering is performed to generate the unchanged and candidate changed samples. Secondly, the candidate changed samples are further clustered based on the direction of SCVs, and multiple changed samples are selected using a local window. Finally, image patches composed of neighbourhood areas of the generated samples are fed into a convolutional neural network (CNN) for training, and the multiple change map is obtained by the trained CNN. Experiments were performed on four data sets, and results indicated that the proposed unsupervised multiple CD approach outperformed some other state-of-the-art methods.

Edge-Preserving Image Filtering Based on Soft Clustering

Edge-preserving image filtering is an essential task in computational photography and imaging. In this paper, we propose a simple yet effective global edge-preserving filter based on soft clustering, and we propose a novel soft clustering algorithm based on a restricted Gaussian mixture model. Given specified parameters, the soft clustering process is firstly performed on the image to derive the partition matrix, from which the affinity matrix is then constructed for filtering. The filtering output is calculated as the weighted average of the pixels in the local window, so the proposed filter could suppress the intensity shift artifacts that impede most global filters. Besides, the weights in the proposed filter are derived by clustering, which properly separates dissimilar pixels, so the proposed filter could handle the halo artifacts that haunt many local filters. Moreover, our filter provides flexible control over the amount of smoothing that is deficient in the deep learning-based filters. Besides the efficacy in smoothing, the proposed filter naturally has low computational complexity. Qualitative and quantitative results suggest that the proposed filter benefits various applications, including edge-preserving smoothing, image enhancing, flash/non-flash fusion, HDR tone mapping, and dehazing.