Irrelevant Images Research Articles

A Remote Sensing and Airborne Edge-Computing Based Detection System for Pine Wilt Disease

The pine wilt disease (PWD) is one of the most dangerous and destructive diseases to coniferous forests. The rapid spread trend and strong destruction directly threaten the security of forests. The complex spread pattern and the hard labor process of diagnosis call for an effective way to detect the infected areas. In this paper, an airborne edge-computing and lightweight deep learning based system are designed for PWD detection by using imagery sensors. Unmanned aerial vehicle (UAV) is firstly utilized to realize a large-scale coverage of forests, which can substantially reduce the hard labor. Except for infected trees, a large number of irrelevant images are also acquired by the UAV, which will overload the burden of process and transmission. Then a lightweight improved YOLOv4-Tiny based method (named as YOLOv4-Tiny-3Layers) is proposed to filter these uninterested images by leveraging the computation capability of edge computing, which can realize a fast coarse-grained detection with a low missing rate. Finally, all the remaining images are transmitted to the ground workstation for the final fine-grained detection. Experimental results show that the proposed system can implement a fast detection with superior performance as compared to other methods, which helps to detect the infected pine trees in a quick manner.

Design of Convolutional Neural Network with Cuckoo Search Algorithm for Super-Resolution UHD Systems on FPGA

Ultra high definition (UHD) video is considered as a technology that generates a video image with a resolution 16 times as many pixels as high definition television. Extensive and complex computational difficulties are there to generate a video in super-resolution (SR). Some extensive floating-point computations are needed for the direct implementation of area-pixel scaling, which is a factor for calculating one target pixel. In this paper, a hybrid Convolutional Neural Network and Cuckoo Search (CNN-CS) algorithm are proposed, which can replace the large scale of complex training and time-consuming machine learning technique. Cuckoo Search (CS) algorithm is based on bird’s levy-flight approach that can optimize the CNN. CS forecasting model isolates the redundant or irrelevant image patches and, therefore, it reduces the required memory space. The SR is performed effectively in hardware with less hardware utilization and memory space, where LR input data are processed line-by-line using the proposed hybrid CNN-CS algorithm. To optimize the design constraints, CS forecasting model estimates the required patches. Parallel sub pipeline (PSP) technique improves the parallelism process thereby reducing the computational complexity and improving the throughput rate and speed of execution. Finally, the proposed hybrid CNN-CS SR design is coded in Verilog HDL and is implemented with the XCKU040 FPGA device. To validate the proposed CNN-CS SR algorithm, standard datasets, such as Set-5, Set-14, BS-100 and Urban-100, are used. The visual quality of the proposed CNN-CS SR design is evaluated by the standard evaluation metrics such as peak signal-to-noise ratio (PSNR) and SSIM. The proposed hybrid CNN-CS SR design is compared with the existing state-of-the-art techniques both qualitatively and quantitatively. Experimental results demonstrate that the proposed method improved the throughput of 42.83%, memory space and power consumption are reduced to 16.16%, 63.81%, respectively, over the existing techniques.

Effects of Emotional Valence and Arousal on Comprehension and Assembly of Instructions

Research on multimedia comprehension from the emotional design perspective has suggested that including graphical elements enhances positive affect and learning. On the other hand, the seductive detail literature found that attractive but irrelevant images can hinder comprehension and learning. The present study has addressed the effect of decorative images with affective properties (valence and arousal) in procedural multimedia comprehension. First, a validation study determined the images’ affective properties; stimuli high or low in both dimensions were systematically introduced in the second experiment, which tested the images’ effect in mean study time when learning the instructions, and accuracy during object assembly. Low arousal images made participants study the instructions faster, and high arousal images led to more errors in performance. Valence did not have a differential effect. Overall, in line with the seductive detail effect, these findings argue against including irrelevant, highly arousing, emotional information in instructional materials.

Adaptive deformable convolutional network

Deformable Convolutional Networks (DCNs) are proposed to solve the inherent limited geometric transformation in CNNs, showing outstanding performance on sophisticated computer vision tasks. Though they can rule out irrelevant image content and focus on region of interest to some degree, the adaptive learning of the deformation is still limited. In this paper, we delve it from the aspects of deformable modules and deformable organizations to extend the scope of deformation ability. Concretely, on the one hand, we reformulate the deformable convolution and RoIpooling by reconsidering spatial-wise attention, channel-wise attention and spatial-channel interdependency, to improve the single convolution’s ability to focus on pertinent image contents. On the other hand, an empirical study is conducted on various and general arrangements of deformable convolutions (e.g., connection type) in DCNs. Especially on semantic segmentation, the study yields significant findings for a proper combination of deformable convolutions. To verify the effectiveness and superiority of our proposed deformable modules, we also provide extensive ablation study for them and compare them with other previous versions. With the proposed contribution, our refined Deformable ConvNets achieve state-of-the-art performance on two semantic segmentation benchmarks (PASCAL VOC 2012 and Cityscapes) and an object detection benchmark (MS COCO).

Discovering informative features in large-scale landmark image collection

One of the key problems in image retrieval systems is the presence of irrelevant and noisy image content. Such content can cause significant confusion for the system. Therefore, there is a need to represent images with only informative features in order to improve the retrieval performance of the system or any subsequent process. In this article, we propose a method to identify the informative features in a large-scale image collection. We apply the frequent itemset mining (FIM) approach to extract visual features patterns from a list of images of the same object. Then, we generate feature pairs to measure the significance of each feature depending on the co-occurrence with its neighbouring features. In addition, we apply this feature selection technique to localise the landmark in the image. The performance of the proposed method is evaluated in terms of average precision (AP) on two benchmark data sets and found that it gives a comparable retrieval performance over the bag of visual words baseline system and the previous methods.

Near-Duplicate Image Detection System Using Coarse-to-Fine Matching Scheme Based on Global and Local CNN Features

Due to the great success of convolutional neural networks (CNNs) in the area of computer vision, the existing methods tend to match the global or local CNN features between images for near-duplicate image detection. However, global CNN features are not robust enough to combat background clutter and partial occlusion, while local CNN features lead to high computational complexity in the step of feature matching. To achieve high efficiency while maintaining good accuracy, we propose a coarse-to-fine feature matching scheme using both global and local CNN features for real-time near-duplicate image detection. In the coarse matching stage, we implement the sum-pooling operation on convolutional feature maps (CFMs) to generate the global CNN features, and match these global CNN features between a given query image and database images to efficiently filter most of irrelevant images of the query. In the fine matching stage, the local CNN features are extracted by using maximum values of the CFMs and the saliency map generated by the graph-based visual saliency detection (GBVS) algorithm. These local CNN features are then matched between images to detect the near-duplicate versions of the query. Experimental results demonstrate that our proposed method not only achieves a real-time detection, but also provides higher accuracy than the state-of-the-art methods.

A Progressive Content-Based Image Retrieval in JPEG 2000 Compressed Remote Sensing Archives

Due to the dramatically increased volume of remote sensing (RS) image archives, images are usually stored in a compressed format to reduce the storage size. Existing content-based RS image retrieval (CBIR) systems require as input fully decoded images, thus resulting in a computationally demanding task in the case of large-scale CBIR problems. To overcome this limitation, in this article, we present a novel CBIR system that achieves a coarse-to-fine progressive RS image description and retrieval in the partially decoded Joint Photographic Experts Group (JPEG) 2000 compressed domain. The proposed system initially: 1) decodes the code blocks associated only to the coarse wavelet resolution and 2) discards the most irrelevant images to the query image based on the similarities computed on the coarse resolution wavelet features of the query and archive images. Then, the code blocks associated with the subsequent resolution of the remaining images are decoded and the most irrelevant images are discarded by computing similarities considering the image features associated with both resolutions. This is achieved by using the pyramid match kernel similarity measure that assigns higher weights to the features associated with the finer wavelet resolution than to those related to the coarse wavelet resolution. These processes are iterated until the codestreams associated with the highest wavelet resolution are decoded. Then, the final retrieval is performed on a very small set of completely decoded images. Experimental results obtained on two benchmark archives of aerial images point out that the proposed system is much faster while providing a similar retrieval accuracy than the standard CBIR systems.

Probabilistic robust regression with adaptive weights — a case study on face recognition

Robust regression plays an important role in many machine learning problems. A primal approach relies on the use of Huber loss and an iteratively reweighted l2 method. However, because the Huber loss is not smooth and its corresponding distribution cannot be represented as a Gaussian scale mixture, such an approach is extremely difficult to handle using a probabilistic framework. To address those limitations, this paper proposes two novel losses and the corresponding probability functions. One is called Soft Huber, which is well suited for modeling non-Gaussian noise. Another is Nonconvex Huber, which can help produce much sparser results when imposed as a prior on regression vector. They can represent any lq loss ($${1 \over 2}$$ ⩽ q < 2) with tuning parameters, which makes the regression model more robust. We also show that both distributions have an elegant form, which is a Gaussian scale mixture with a generalized inverse Gaussian mixing density. This enables us to devise an expectation maximization (EM) algorithm for solving the regression model. We can obtain an adaptive weight through EM, which is very useful to remove noise data or irrelevant features in regression problems. We apply our model to the face recognition problem and show that it not only reduces the impact of noise pixels but also removes more irrelevant face images. Our experiments demonstrate the promising results on two datasets.

Image Retrieval Scheme Using Quantized Bins of Color Image Components and Adaptive Tetrolet Transform

In this paper, a three stage hierarchical image retrieval scheme using a color, texture and shape visual contents (or descriptors) is proposed, since single visual content is not produce an adequate retrieval results effectively. This scheme has reduced the searching space during the image retrieval process at a certain extent due to the hierarchical mode. In initial stage, the shape feature descriptor has been computed by simple fusion of histograms of gradients and invariant moments of segmented image planes. The shape based retrieval process has reduced the search space by discarding the non-relevant images from the universal dataset (or original dataset) effectively and kept the retrieved images into the intermediate dataset. In the second stage, the texture feature descriptors have been computed from the intermediate sub-image dataset by applying the adaptive tetrolet transform on image plane of preprocessed HSV color image. This transform provides the multi-resolution images with finer details by employing the tetrominoes and the proper arrangement of tetrominoes contributes the effective local geometry of image plane. The gray level co-occurrence matrix based texture feature descriptor is obtained by computing second order statistical parameters from each decomposed sub-image. At this stage, the most of the irrelevant images are discarded by retrieving the images from intermediate dataset but still some undesired images are left, those will be handled at the last stage. At this stage, fused color information is captured by applying the color autocorrelogram on both the non-uniform quantized color components of the preprocessed HSV color image. Finally, the color feature descriptor produces the desired retrieval results by discarding the irrelevant images from the texture based sub-image dataset. The proposed scheme has also low computational overhead due to the use of three descriptors at different stages separately. The retrieved results show the better accuracy as compared to the other related visual contents based image retrieval schemes.

Active Learning Based Federated Learning for Waste and Natural Disaster Image Classification

The feasibility of Federated Learning (FL) is highly dependent on the training and inference capabilities of local models, which are subject to the availability of meaningful and annotated data. The availability of such data is in turn contingent on the tedious and time-consuming annotation job that typically requires the manual analysis of training samples. Active Learning (AL) provides an alternative solution allowing a Machine Learning (ML) model to automatically choose and label the data from which it learns without involving manual inspection of each training sample. In this work, we explore how FL can benefit from unlabelled data available at each participating client using AL. To this aim, we propose an AL-based FL framework by employing and evaluating several AL methods in two different application domains. Through an extensive experimentation setup, we show that AL is equally useful in federated and centralized learning by achieving comparable results with manually labeled data using fewer samples without involving human annotators in collecting training data. We also demonstrated that the proposed method is dataset/application independent by evaluating the proposed method in two interesting applications, namely natural disaster analysis and waste classification, having different properties and challenges. Promising results are obtained on both applications resulting in comparable results against the best-case scenario where each sample is manually analyzed and annotated (Baseline 1), and improvement of 3.1% and 4% with best methods respectively over the training sets with irrelevant images on natural disaster and waste classification datasets (Baseline 2).

Automatically Discovering Relevant Images From Web Pages

Web pages contain irrelevant images along with relevant images. The classification of these images is an error-prone process due to the number of design variations of web pages. Using multiple web pages provides additional features that improve the performance of relevant image extraction. Traditional studies use the features extracted from a single web page. However, in this study, we enhance the performance of relevant image extraction by employing the features extracted from different web pages consisting of standard news, galleries, video pages, and link pages. The dataset obtained from these web pages contains 100 different web pages for each 200 online news websites from 58 different countries. For discovering relevant images, the most straightforward approach extracts the largest image on the web page. This approach achieves a 0.451 F-Measure score as a baseline. Then, we apply several machine learning methods using features in this dataset to find the most suitable machine learning method. The best f-Measure score is 0.822 using the AdaBoost classifier. Some of these features have been utilized in previous web data extraction studies. To the best of our knowledge, 15 new features are proposed for the first time in this study for discovering the relevant images. We compare the performance of the AdaBoost classifier on different feature sets. The proposed features improve the f-Measure by 35 percent. Besides, using only the cache feature, which is the most prominent feature, corresponds to 7 percent of this improvement.

An Unordered Image Stitching Method Based on Binary Tree and Estimated Overlapping Area.

Aiming at the complex computation and time-consuming problem during unordered image stitching, we present a method based on the binary tree and the estimated overlapping areas to stitch images without order in this paper. For image registration, the overlapping areas between input images are estimated, so that the extraction and matching of feature points are only performed in these areas. For image stitching, we build a model of the binary tree to stitch each two matched images without sorting. Compared to traditional methods, our method significantly reduces the computational time of matching irrelevant image pairs and improves the efficiency of image registration and stitching. Moreover, the stitching model of the binary tree proposed in this paper further reduces the distortion of the panorama. Experimental results show that the number of extracted feature points in the estimated overlapping area is approximately 0.3∼0.6 times of that in the entire image by using the same method, which greatly reduces the computational time of feature extraction and matching. Compared to the exhaustive image matching method, our approach only takes about 1/3 of the time to find all matching images.

An Integrated Methods for Refined Facial-Image Retrieval using Sift and Click-Through Data

In the modern era, the emergence of social networking sites paved the way to the people to upload lot of images online. Social sites like Instagram and Flickr allow users to add semantic information to the images in the form of tags. Often these tags are the firsthand semantic data for retrieving the images from the Internet. When a user searches for images in the web, the images with tags relevant to user query are retrieved. Most of the time, these semantic data are not relevant to the content of the image and hence the user gets irrelevant images in contrast to their intended search. This is more common in facial image search. In this paper, we propose an integrated approach to refine the relevance of retrieved facial images. Using Scale Invariant Feature Extraction (SIFT) the facial features of semantically most relevant image and click-through data of all the retrieved images are used to rank and present a meaningful search result. Along with facial features and click through information, the co-occurrence of related tags is also considered. Also we propose the construction of inverted index structure to improve the search performance.

Transform domain representation-driven convolutional neural networks for skin lesion segmentation

Automated diagnosis systems provide a huge improvement in early detection of skin cancer, and consequently, contribute to successful treatment. Recent research on convolutional neural network has achieved enormous success in segmentation and object detection tasks. However, these networks require large amount of data that is a big challenge in medical domain where often have insufficient data and even a pretrained model on medical images can be hardly found. Lesion segmentation as the initial step of skin cancer analysis remains a challenging issue since datasets are small and include a variety of images in terms of light, color, scale, and marks which have led researchers to use extensive augmentation and preprocessing techniques or fine tuning the network with a pretrained model on irrelevant images. A segmentation model based on convolutional neural networks is proposed in this study for the tasks of skin lesion segmentation and dermoscopic feature segmentation. The network is trained from scratch and despite the small size of datasets neither excessive data augmentation nor any preprocessing to remove artifacts or enhance the images are applied. Alternatively, we investigated incorporating image representations of the transform domain to the convolutional neural network and compared to a model with more convolutional layers that resulted in 6% higher Jaccard index and has shorter training time. The model improved by applying CIELAB color space and the performance of the final proposed architecture is evaluated on publicly available datasets from ISBI challenges in 2016 and 2017. The proposed model has resulted in an improvement of as much as 7% for the segmentation metrics and 17% for the feature segmentation, which demonstrates the robustness of this unique hybrid framework and its future applications as well as further improvement.

Spatial-Temporal Person Re-Identification

Most of current person re-identification (ReID) methods neglect a spatial-temporal constraint. Given a query image, conventional methods compute the feature distances between the query image and all the gallery images and return a similarity ranked table. When the gallery database is very large in practice, these approaches fail to obtain a good performance due to appearance ambiguity across different camera views. In this paper, we propose a novel two-stream spatial-temporal person ReID (st-ReID) framework that mines both visual semantic information and spatial-temporal information. To this end, a joint similarity metric with Logistic Smoothing (LS) is introduced to integrate two kinds of heterogeneous information into a unified framework. To approximate a complex spatial-temporal probability distribution, we develop a fast Histogram-Parzen (HP) method. With the help of the spatial-temporal constraint, the st-ReID model eliminates lots of irrelevant images and thus narrows the gallery database. Without bells and whistles, our st-ReID method achieves rank-1 accuracy of 98.1% on Market-1501 and 94.4% on DukeMTMC-reID, improving from the baselines 91.2% and 83.8%, respectively, outperforming all previous state-of-theart methods by a large margin.

Content-based image retrieval based on supervised learning and statistical-based moments

Content-based image retrieval (CBIR) system generally retrieves images based on the matching of the query image from all the images of the database. This exhaustive matching and searching slow down the image retrieval process. In this paper, a fast and effective CBIR system is proposed which uses supervised learning-based image management and retrieval techniques. It utilizes machine learning approaches as a prior step for speeding up image retrieval in the large database. For the implementation of this, first, we extract statistical moments and the orthogonal-combination of local binary patterns (OC-LBP)-based computationally light weighted color and texture features. Further, using some ground truth annotation of images, we have trained the multi-class support vector machine (SVM) classifier. This classifier works as a manager and categorizes the remaining images into different libraries. However, at the query time, the same features are extracted and fed to the SVM classifier. SVM detects the class of query and searching is narrowed down to the corresponding library. This supervised model with weighted Euclidean Distance (ED) filters out maximum irrelevant images and speeds up the searching time. This work is evaluated and compared with the conventional model of the CBIR system on two benchmark databases, and it is found that the proposed work is significantly encouraging in terms of retrieval accuracy and response time for the same set of used features.

Orientation-Based Food Image Capture for Head Mounted Egocentric Camera.

Current head-mounted wearable sensors for monitoring of food intake operates by fusing multiple modalities such as inertial and image sensing. The image capture may be performed periodically, capturing a large number of irrelevant images, increasing power consumption and reducing the battery life. In this manuscript, we propose an efficient approach for food image capture, that captures the images only when the head tilt angle estimated from the accelerometer data matches that during ingestion of food. The method was developed and validated using data from 15 volunteers consuming unrestricted meals in a free-living environment between 12.5 to 18.5 hours. The tilt angle of the head was computed using 3D accelerometer data. A classifier for image capture was developed using a curve fitting approach on the tilt angles of the head. The proposed method achieved a sensitivity of 0.97 and specificity of 0.47 in predicting capture of food images, thus potentially improving the battery life of the wearable device.

The decorative images’ seductive effect in e-learning depends on attentional inhibition

Two studies sought to replicate and extend the seductive effect of decorative pictures in expository text comprehension to an e-learning environment. In the first study, undergraduate students read and answered questions about two texts, with and without decorative, irrelevant images, in an e-learning course. The presence of decorative images had a small detrimental effect on comprehension. In the second study, participants read more difficult texts (low prior knowledge texts in multiple screens) and completed working memory and inhibitory ability tests. A significant interaction between comprehension and perceptual/attentional inhibitory ability was found: Participants with lower inhibitory capacity were affected by irrelevant pictures. In conclusion, evidence supported the hypothesis of a detrimental effect of irrelevant, decorative images on comprehension in e-learning, particularly for students with low attentional inhibition.

Humans can decipher adversarial images

Does the human mind resemble the machine-learning systems that mirror its performance? Convolutional neural networks (CNNs) have achieved human-level benchmarks in classifying novel images. These advances support technologies such as autonomous vehicles and machine diagnosis; but beyond this, they serve as candidate models for human vision itself. However, unlike humans, CNNs are “fooled” by adversarial examples—nonsense patterns that machines recognize as familiar objects, or seemingly irrelevant image perturbations that nevertheless alter the machine’s classification. Such bizarre behaviors challenge the promise of these new advances; but do human and machine judgments fundamentally diverge? Here, we show that human and machine classification of adversarial images are robustly related: In 8 experiments on 5 prominent and diverse adversarial imagesets, human subjects correctly anticipated the machine’s preferred label over relevant foils—even for images described as “totally unrecognizable to human eyes”. Human intuition may be a surprisingly reliable guide to machine (mis)classification—with consequences for minds and machines alike.

Extracting Multiple Visual Senses for Web Learning

Labeled image datasets have played a critical role in high-level image understanding. However, the process of manual labeling is both time consuming and labor intensive. To reduce the dependence on manually labeled data, there have been increasing research efforts on learning visual classifiers by directly exploiting web images. One issue that limits their performance is the problem of polysemy. Existing unsupervised approaches attempt to reduce the influence of visual polysemy by filtering out irrelevant images, but do not directly address polysemy. To this end, in this paper, we present a multimodal framework that solves the problem of polysemy by allowing sense-specific diversity in search results. Specifically, we first discover a list of possible semantic senses from untagged corpora to retrieve sense-specific images. Then, we merge visual similar semantic senses and prune noise by using the retrieved images. Finally, we train one visual classifier for each selected semantic sense and use the learned sense-specific classifiers to distinguish multiple visual senses. Extensive experiments on classifying images into sense-specific categories and reranking search results demonstrate the superiority of our proposed approach.