Natural Scene Images Research Articles

Semi-supervised training using cooperative labeling of weakly annotated data for nodule detection in chest CT.

Machine learning algorithms are best trained with large quantities of accurately annotated samples. While natural scene images can often be labeled relatively cheaply and at large scale, obtaining accurate annotations for medical images is both time consuming and expensive. In this study, we propose a cooperative labeling method that allows us to make use of weakly annotated medical imaging data for the training of a machine learning algorithm. As most clinically produced data are weakly-annotated - produced for use by humans rather than machines and lacking information machine learning depends upon - this approach allows us to incorporate a wider range of clinical data and thereby increase the training set size. Our pseudo-labeling method consists of multiple stages. In the first stage, a previously established network is trained using a limited number of samples with high-quality expert-produced annotations. This network is used to generate annotations for a separate larger dataset that contains only weakly annotated scans. In the second stage, by cross-checking the two types of annotations against each other, we obtain higher-fidelity annotations. In the third stage, we extract training data from the weakly annotated scans, and combine it with the fully annotated data, producing a larger training dataset. We use this larger dataset to develop a computer-aided detection (CADe) system for nodule detection in chestCT. We evaluated the proposed approach by presenting the network with different numbers of expert-annotated scans in training and then testing the CADe using an independent expert-annotated dataset. We demonstrate that when availability of expert annotations is severely limited, the inclusion of weakly-labeled data leads to a 5% improvement in the competitive performance metric (CPM), defined as the average of sensitivities at different false-positiverates. Our proposed approach can effectively merge a weakly-annotated dataset with a small, well-annotated dataset for algorithm training. This approach can help enlarge limited training data by leveraging the large amount of weakly labeled data typically generated in clinical image interpretation.

Computer vision based method for severity estimation of tea leaf blight in natural scene images

Tea leaf diseases seriously affect the yield and quality of tea. Early warning and severity estimation of the diseases can be used to guide tea farmers to spray pesticide reasonably. Tea leaves infected with leaf blight are usually damaged, deformed, and occluded. An insufficient number of disease image samples will lead to overfitting of the estimated model. Thus, existing methods based on machine learning can only estimate the severity of tea diseases in natural scene images with low accuracy. Aiming to solve these problems, this study proposes a computer vision based method for the severity estimation of tea leaf blight in RGB images obtained under natural scenes. In this method, the influence of complex backgrounds is reduced by segmenting diseased tea leaves and spots, the problems of partial occlusion, deformation and damage of diseased leaves are solved by area fitting, and the severity of tea leaf blight is accurately estimated by the gradient boosting machine. Compared with classical machine learning methods and conventional convolution neural network methods, the method presented in this study only needs a small number of manually labeled samples and has better accuracy and robustness for the severity estimation of tea leaf blight in natural scene images.

Irregular Scene Text Detection Based on a Graph Convolutional Network.

Detecting irregular or arbitrary shape text in natural scene images is a challenging task that has recently attracted considerable attention from research communities. However, limited by the CNN receptive field, these methods cannot directly capture relations between distant component regions by local convolutional operators. In this paper, we propose a novel method that can effectively and robustly detect irregular text in natural scene images. First, we employ a fully convolutional network architecture based on VGG16_BN to generate text components via the estimated character center points, which can ensure a high text component detection recall rate and fewer noncharacter text components. Second, text line grouping is treated as a problem of inferring the adjacency relations of text components with a graph convolution network (GCN). Finally, to evaluate our algorithm, we compare it with other existing algorithms by performing experiments on three public datasets: ICDAR2013, CTW-1500 and MSRA-TD500. The results show that the proposed method handles irregular scene text well and that it achieves promising results on these three public datasets.

ORSI Salient Object Detection via Bidimensional Attention and Full-Stage Semantic Guidance

The application of optical remote sensing images (ORSIs) is prevalent in many fields. Accordingly, ORSI-oriented saliency object detection (SOD) has attracted more attention in recent years. However, yet many previously proposed methods present appealing performance in natural scene images (NSIs), they are difficult to be directly extended to remote sensing images due to the more complex scenes, such as blended backgrounds and diversiform topological shapes. Most specifically designed models often fail to achieve satisfactory results due to the weak usage of edge information and the ignorance of attention loss. Besides, computational inefficiency often causes poor applicability. To solve these problems, we propose a new model, namely Bidimensional Attention and Full-stage Semantic Guidance Network (BAFS-Net), containing an edge guidance branch and a mainstream detection branch. Concretely, edge guidance generates boundary information, in which a supervision with border labels is imposed to highlight the salient regions and plays a complementary role on the main branch. The mainstream detection branch involves two important components, i.e. bidimensional attention modules (BAM) and semantic-guided fusion modules (SGFM). Between these two, BAM uniformly assembles channel and spatial attention in an efficient and rational manner, addressing the open issue of dimension-wisely attention computation. SGFM hammers at the fusion of high-level features and low-level features. Moreover, the semantic maps are employed to interact with SGFM in full stages. Our approach surpasses most state-of-the-art RSIs-SOD methods proposed in recent years, with respect to accuracy, parameter size, computational cost, and FLOPS, as shown in Fig. 1. Code is available at https://github.com/ZhengJianwei2/BAFS-Net.

Text Recognition of Bangla and English Scripts in Natural Scene Images

Digital cameras on mobile devices have created several new computational hurdles. Text extraction from natural scene photos collected by such devices is an issue. This paper presents a method for recognizing English and Bangla text in scene photos. With this framework, it will be possible to recognize and locate text in its natural setting. Text recognition in natural settings is performed on multiple distinct tiers. To begin, a picture is taken using a smart device. Then text localization and text extraction are done. In the final stage, an artificial neural network is used to recognize the text from the image. This is a bilingual project, with results available in both Bengali and English. The system outperforms conventional recognition methods, producing image results with an average accuracy of 85%.

ORNAIC-TDNSI optimal RetinaNet with artificial immune classification for text detection on natural scene images

ORNAIC-TDNSI optimal RetinaNet with artificial immune classification for text detection on natural scene images

Enhanced Feature Model Based Hybrid Neural Network for Text Detection on Signboard, Billboard and News Tickers

Recognizing text from the nature scene images and videos has been the challenging task of computer vision and machine learning research community in recent years. These texts are difficult to recognize because of their shapes, complex backgrounds, color, shape and size variations. However, text recognition is very much useful in indexing, keyword-based image and video search, and information retrieval. In this research paper, a model is proposed to detect the isolated text characters in the photographic images of natural scenes. The proposed model uses the combination of Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN) for recognizing the text in natural images. The model uses two networks, where the first network combines the low-level and middle-level features to increase the feature size and passes the enriched information to the second network. Here, features are again widened by combining with high-level features, resulting in powerful and robust features. To evaluate the proposed model, ICDAR2003 (IC03), ICDAR2013 (IC13), SVT (Street View Text) datasets have been used. And an extensive Tamil news tickers image dataset has been developed to evaluate the model. The experimental results show that the combined feature fusion technique outperforms the other methods on the ICDAR2003, ICDAR2013, SVT and Tamil news tickers datasets.

Text Prior Guided Scene Text Image Super-resolution.

Scene text image super-resolution (STISR) aims to improve the resolution and visual quality of low-resolution (LR) scene text images, while simultaneously boost the performance of text recognition. However, most of the existing STISR methods regard text images as natural scene images, ignoring the categorical information of text. In this paper, we make an inspiring attempt to embed text recognition prior into STISR model. Specifically, we adopt the predicted character recognition probability sequence as the text prior, which can be obtained conveniently from a text recognition model. The text prior provides categorical guidance to recover high-resolution (HR) text images. On the other hand, the reconstructed HR image can refine the text prior in return. Finally, we present a multi-stage text prior guided super-resolution (TPGSR) framework for STISR. Our experiments on the benchmark TextZoom dataset show that TPGSR can not only effectively improve the visual quality of scene text images, but also significantly improve the text recognition accuracy over existing STISR methods. Our model trained on TextZoom also demonstrates certain generalization capability to the LR images in other datasets.

Embedded System Based Raspberry Pi 4 for Text Detection and Recognition

Detecting and recognizing text from natural scene images presents a challenge because the image quality depends on the conditions in which the image is captured, such as viewing angles, blurring, sensor noise, etc. However, in this paper, a prototype for text detection and recognition from natural scene images is proposed. This prototype is based on the Raspberry Pi 4 and the Universal Serial Bus (USB) camera and embedded our text detection and recognition model, which was developed using the Python language. Our model is based on the deep learning text detector model through the Efficient and Accurate Scene Text Detector (EAST) model for text localization and detection and the Tesseract-OCR, which is used as an Optical Character Recognition (OCR) engine for text recognition. Our prototype is controlled by the Virtual Network Computing (VNC) tool through a computer via a wireless connection. The experiment results show that the recognition rate for the captured image through the camera by our prototype can reach 99.75% with low computational complexity. Furthermore, our prototype is more performant than the Tesseract software in terms of the recognition rate. Besides, it provides the same performance in terms of the recognition rate with a huge decrease in the execution time by an average of 89% compared to the EasyOCR software on the Raspberry Pi 4 board.

Aerial images object detection method based on cross-scale multi-feature fusion.

Aerial image target detection technology has essential application value in navigation security, traffic control and environmental monitoring. Compared with natural scene images, the background of aerial images is more complex, and there are more small targets, which puts higher requirements on the detection accuracy and real-time performance of the algorithm. To further improve the detection accuracy of lightweight networks for small targets in aerial images, we propose a cross-scale multi-feature fusion target detection method (CMF-YOLOv5s) for aerial images. Based on the original YOLOv5s, a bidirectional cross-scale feature fusion sub-network (BsNet) is constructed, using a newly designed multi-scale fusion module (MFF) and cross-scale feature fusion strategy to enhance the algorithm's ability, that fuses multi-scale feature information and reduces the loss of small target feature information. To improve the problem of the high leakage detection rate of small targets in aerial images, we constructed a multi-scale detection head containing four outputs to improve the network's ability to perceive small targets. To enhance the network's recognition rate of small target samples, we improve the K-means algorithm by introducing a genetic algorithm to optimize the prediction frame size to generate anchor boxes more suitable for aerial images. The experimental results show that on the aerial image small target dataset VisDrone-2019, the proposed method can detect more small targets in aerial images with complex backgrounds. With a detection speed of 116 FPS, compared with the original algorithm, the detection accuracy metrics mAP0.5 and mAP0.5:0.95 for small targets are improved by 5.5% and 3.6%, respectively. Meanwhile, compared with eight advanced lightweight networks such as YOLOv7-Tiny and PP-PicoDet-s, mAP0.5 improves by more than 3.3%, and mAP0.5:0.95 improves by more than 1.9%.

Adjacent Context Coordination Network for Salient Object Detection in Optical Remote Sensing Images.

Salient object detection (SOD) in optical remote sensing images (RSIs), or RSI-SOD, is an emerging topic in understanding optical RSIs. However, due to the difference between optical RSIs and natural scene images (NSIs), directly applying NSI-SOD methods to optical RSIs fails to achieve satisfactory results. In this article, we propose a novel adjacent context coordination network (ACCoNet) to explore the coordination of adjacent features in an encoder-decoder architecture for RSI-SOD. Specifically, ACCoNet consists of three parts: 1) an encoder; 2) adjacent context coordination modules (ACCoMs); and 3) a decoder. As the key component of ACCoNet, ACCoM activates the salient regions of output features of the encoder and transmits them to the decoder. ACCoM contains a local branch and two adjacent branches to coordinate the multilevel features simultaneously. The local branch highlights the salient regions in an adaptive way, while the adjacent branches introduce global information of adjacent levels to enhance salient regions. In addition, to extend the capabilities of the classic decoder block (i.e., several cascaded convolutional layers), we extend it with two bifurcations and propose a bifurcation-aggregation block (BAB) to capture the contextual information in the decoder. Extensive experiments on two benchmark datasets demonstrate that the proposed ACCoNet outperforms 22 state-of-the-art methods under nine evaluation metrics, and runs up to 81 fps on a single NVIDIA Titan X GPU. The code and results of our method are available at https://github.com/MathLee/ACCoNet.

MISNet: Multiscale Cross-Layer Interactive and Similarity Refinement Network for Scene Parsing of Aerial Images

Although progress has been made in multisource data scene parsing of natural scene images, extracting complex backgrounds from aerial images of various types and presenting the image at different scales remain challenging. Various factors in high-resolution aerial images (HRAIs), such as imaging blur, background clutter, object shadow, and high resolution, substantially reduce the integrity and accuracy of object segmentation. By applying multisource data fusion, as in scene parsing of natural scene images, we can solve the aforementioned problems through the integration of auxiliary data into HRAIs. To this end, we propose a multiscale cross-layer interactive and similarity refinement network (MISNet) for scene parsing of HRAIs. First, in a feature fusion optimization module, we extract, filter, and optimize multisource features and further guide and optimize the features using a feature guidance module. Second, a multiscale context aggregation module increases the receptive field, captures semantic information, and extracts rich multiscale background features. Third, a dense decoding module fuses the global guidance information and high-level fused features. We also propose a joint learning method based on feature similarity and a joint learning module to obtain deep multilevel information, enhance feature generation, and fuse multiscale and global features to enhance network representation for accurate scene parsing of HRAIs. Comprehensive experiments on two benchmark HRAIs datasets indicate that our proposed MISNet is qualitatively and quantitatively superior to similar state-of-the-art models.

Few-Shot Object Detection Based on Adaptive Attention Mechanism and Large-Margin Softmax

Recently, a DCNet consisting of a dense relation distillation module and a context-aware aggregation module has achieved remarkable performance for the few-shot object detection task. In this article, we aim to improve the DCNet from the following two aspects. First, we design an adaptive attention module, which is equipped in the front of the dense relation distillation module, and can be trained together with the remainder parts of the DCNet. After training, the adaptive attention module helps to enhance foreground features and to suppress the background features. Second, we introduce a large-margin Softmax into the dense relation distillation module. The large-margin Softmax with a hyperparameter can normalize features without reducing the discriminability between different classes. We conduct extensive experiments on the PASCAL visual object classes and the Microsoft common objects in context data sets. The experimental results show that the proposed method can work under the few-shot scenario and achieves the mean average precision of 50.8% on the PASCAL visual object classes data set and 13.1% on the Microsoft common objects in context data set, which both outperform the existing baselines. Moreover, ablation studies and visualizations validate the usefulness of the adaptive attention module and the large-margin Softmax. The proposed method can be applied to recognize rare patterns in fabric images or detect clothes with new styles in natural scene images.

Boundary-Aware Salient Object Detection in Optical Remote-Sensing Images

Different from the traditional natural scene images, optical remote-sensing images (RSIs) suffer from diverse imaging orientations, cluttered backgrounds, and various scene types. Therefore, the object-detection methods salient to optical RSIs require effective localization and segmentation to deal with complex scenarios, especially small targets, serious occlusion, and multiple targets. However, the existing models’ experimental results are incapable of distinguishing salient objects and backgrounds using clear boundaries. To tackle this problem, we introduce boundary information to perform salient object detection in optical RSIs. Specifically, we first combine the encoder’s low-level and high-level features (i.e., abundant local spatial and semantic information) via a feature-interaction operation, yielding boundary information. Then, the boundary cues are introduced into each decoder block, where the decoder features are directed to focus more on the boundary details and objects simultaneously. In this way, we can generate high-quality saliency maps which can highlight salient objects from optical RSIs completely and accurately. Extensive experiments are performed on a public dataset (i.e., ORSSD dataset), and the experimental results demonstrate the effectiveness of our model when compared with the cutting-edge saliency models.

GoogLeNet based on residual network and attention mechanism identification of rice leaf diseases

Rice leaf diseases are a major cause of declining rice production and quality. The early identification and control of rice leaf diseases is critical for maintaining rice quality and production. However, early rice leaf diseases appear in natural scene images as small spots with irregular shapes, and existing models have difficulty accurately identifying rice leaf diseases. This study proposes a rE-GoogLeNet convolutional neural network model for accurately identifying rice leaf diseases in natural environments. rE-GoogLeNet is based on GoogLeNet and replaces the 7 × 7 convolution kernel in the first layer with three 3 × 3 convolution kernels and adds an ECA attention mechanism to the Inception module. This new module is named the E-Inception module, and a residual network is used to connect the E-Inception modules, addressing the issues of increased information loss and gradient loss due to increasing network depth. Finally, the ReLU activation function is replaced by the leaky ReLU activation function, improving the feature extraction ability for diseased leaves with irregularly shaped small spots. Moreover, the auxiliary classifier is simplified, reducing the complexity of the model without decreasing the recognition rate. Experiments show that compared with conventional models and advanced multiscale models, rE-GoogLeNet has a better classification performance on rice leaf diseases and an average accuracy of 99.58 %, an improvement of 1.72 % over the original GoogLeNet model. This evaluation shows that rE-GoogLeNet is robust, stable and highly accurate for detecting rice leaf diseases in practical applications. The evaluation also shows that rE-GoogLeNet has a good recall rate, F1 score and confusion matrix.

Blindness to the absence of color in natural scene images is mainly mediated by a perceptual/response bias

Blindness to the absence of color in natural scene images is mainly mediated by a perceptual/response bias

Interpretable mid-level encoding models of human visual cortex reveal associations between feature and semantic tuning for natural scene images

Interpretable mid-level encoding models of human visual cortex reveal associations between feature and semantic tuning for natural scene images

Different symmetries, different mechanisms

Three common symmetries exist in the natural visual world: (i) mirror symmetry, i.e., reflections around a vertical axis, (ii) radial symmetry, i.e., rotations around a point, and (iii) translational symmetry, i.e., shifted repetitions. Are these processed by a common class of visual mechanism? Using stimuli comprising arrays of Gaussian blobs we examined this question using a visual search protocol in which observers located a single symmetric target patch among varying numbers of random-blob distractor patches. The testing protocol used a blocked present/absent task and both search times and accuracy were recorded. Search times for mirror and radial symmetry increased significantly with the number of distractors, as did translational-symmetry patterns containing few repetitions. However translational-symmetry patterns with four repeating sectors produced search slopes close to zero. Fourier analysis revealed that, as with images of natural scenes, the structural information in both mirror- and radial-symmetric patterns is carried by the phase spectrum. However, for translational patterns with four repeating sectors, the amplitude spectrum appears to capture the structure, consistent with previous analyses of texture regularity. Modeling revealed that while the mirror and radial patterns produced an approximately Gaussian-shaped energy response profile as a function of spatial frequency, the translational pattern profiles contained a distinctive spike, the magnitude of which increased with the number of repeating sectors. We propose distinct mechanisms for the detection of different symmetry types: a mechanism that encodes local positional information to detect mirror- and radial-symmetric patterns and a mechanism that computes energy in narrowband filters for the detection of translational symmetry containing many sectors.

On strange images with application to lossy image compression

Single and three-channel images are widely used in numerous applications. Due to the increasing volume of such data, they must be compressed where lossy compression offers more opportunities. Usually, it is supposed that, for a given image, a larger compression ratio leads to worse quality of the compressed image according to all quality metrics. This is true for most practical cases. However, it has been found recently that images are called “strange” for which a rate-distortion curve like dependence of the peak signal-to-noise ratio on the quality factor or quantization step, behaves non-monotonously. This might cause problems in the lossy compression of images. Thus, the basic subject of this paper are the factors that determine this phenomenon. The main among them are artificial origin of an image, possible presence of large homogeneous regions, specific behavior of image histograms. The main goal of this paper is to consider and explain the peculiarities of the lossy compression of strange images. The tasks of this paper are to provide definitions of strange images and to check whether non-monotonicity of rate-distortion curves occurs for different coders and metrics. One more task is to put ideas and methodology forward of further studies intended to detect strange images before their compression. The main result is that non-monotonous behavior can be observed for the same image for several quality metrics and coders. This means that not the coder but image properties determine the probability of an image to being strange. Moreover, both grayscale and color images can be strange, and both the natural scene and artificial images can be strange. This depends more on image properties than on image origin and number of channels. In particular, the percentage of pixels that belong to large homogeneous regions and image entropy play an important role. As conclusions, we outline possible directions of future research that, in the first order, relate to the analysis of images in large databases to establish parameters that show that a given image can be considered as strange.

Two-stage natural scene image classification with noise discovering and label-correlation mining

In natural scene classification, it is common that one natural scene image may belong to multiple categories concurrently; as a result, multi-label learning has become a research hotspot. Despite recent rapid developments in multi-label learning, the increasing amount of high-dimensional data poses great challenges—such as redundant features and high computational costs—to conventional multi-label learning models. Most contemporary strategies for dealing with this issue depend on forcing feature learning into multi-label models. Notably, however, these approaches seldom pay attention to the label correlation and propagation in the feature subspace. To address this issue, we introduce an alternative multi-label feature learning solution that incorporates both labeled and unlabeled information. Unlike existing multi-label learning models , which rely on clean and trustworthy training datasets, we argue that in semi-supervised learning scenarios, the unlabeled data can be easily corrupted by noise or outliers, which causes the model performance to degrade. We next extract the label correlation and propagate the label information to discover the noise or outliers. Subsequently, our model adaptively searches the optimal feature subspace to reduce the influence of redundant features for high-dimensional data. The effectiveness of the proposed model is demonstrated by experimental observations on artificial and real-world datasets.