Visualization Of Datasets Research Articles

Image Clustering: An Unsupervised Approach to Categorize Visual Data in Social Science Research

Automated image analysis has received increasing attention in social scientific research, yet existing scholarship has mostly covered the application of supervised learning to classify images into predefined categories. This study focuses on the task of unsupervised image clustering, which aims to automatically discover categories from unlabelled image data. We first review the steps to perform image clustering and then focus on one key challenge in this task—finding intermediate representations of images. We present several methods of extracting intermediate image representations, including the bag-of-visual-words model, self-supervised learning, and transfer learning (in particular, feature extraction with pretrained models). We compare these methods using various visual datasets, including images related to protests in China from Weibo, images about climate change on Instagram, and profile images of the Russian Internet Research Agency on Twitter. In addition, we propose a systematic way to interpret and validate clustering solutions. Results show that transfer learning significantly outperforms the other methods. The dataset used in the pretrained model critically determines what categories the algorithms can discover.

A Planted Clique Perspective on Hypothesis Pruning

Hypothesis pruning is an important prerequisite while working with outlier-contaminated data in many computer vision problems. However, the underlying random data structures are barely explored in the literature, limiting designing efficient algorithms. To this end, we provide a novel graph-theoretic perspective on hypothesis pruning exploiting invariant structures of data. We introduce the planted clique model, a central object in computational statistics, to investigate the information-theoretical and computational limits of the hypothesis pruning problem. In addition, we propose an inductive learning framework for finding hidden cliques that learns heuristics on synthetic graphs with planted cliques and generalizes to real vision problems. We present competitive experimental results with large runtime improvement on synthetic and widely used vision datasets to show its efficacy.

Plan Generation with Generative Adversarial Networks: Haeckel’s Drawings to Palladian Plans

In this study the application of deep learning networks in architectural design is explored via experimental plan generation. With image processing abilities of deep learning networks such as GAN (generative adversarial network), training generative models with architectural visual data is possible. One type of GANs called CycleGAN is specially chosen for the purposes of this study because of its flexibility on visual datasets and low requirement of preliminary labor. In the scope of this study, 2D plans and visuals are selected as datasets to train the CycleGAN model. Instead of training the model with only one dataset of plans and let it generate similar but novel outcomes, in this study two datasets are used to experiment on translations into plan-like images from a different dataset. For the dataset that consists of plans, Palladio’s plans are selected. Because the embedded spatial organizational data can be easily decoded and used as a training set for the CycleGAN algorithm, thanks to their potent and symmetrical representations on 2D. Second dataset is formed by Haeckel’s microorganism drawings, in order to investigate new possibilities of spatial organization when they are emerged from the visual data of organism structures. Instead of original microorganism images, Haeckel’s drawings are selected because of their idealized plan-like figures with rotational symmetry. The model was trained with these two datasets to perform image translation between them. Although the model can work both ways, this paper focused on and evaluated the translations from Haeckel’s microorganism drawings to Palladian-like plans. Eventually the model translated Haeckel’s drawings into plan-like images which shows the features of the forming patterns of Palladian plans. The outcomes can be beneficial and inspiring for the conceptual and preliminary design processes as well as studying the visual transformations between architectural and out of field visuals. This study, contributes to the field in terms of the application of AI methods -specifically GANs- in experimental plan generation tasks.

Extending the color discrimination metric with consideration of illuminance level.

Color discrimination is a crucial dimension for the visual assessment of white light sources. In this Letter, we took full advantage of data comparability among Farnsworth-Munsell 100 Hue (FM-100) color vision tests and modified our recent proposed color discrimination metric [Opt. Lett.45, 6062 (2020)10.1364/OL.400422] to make it effective in quantifying the color discrimination capability of lighting under multiple illuminance levels. The updated metric was developed based on a meta-analysis of 100 sets of combined visual data derived from three groups of visual studies containing 29 FM-100 trials and its superiority was comprehensively validated by 140 sets of combined data from five groups of studies with 42 FM-100 trials.

Postprocessing of Edge Detection Algorithms with Machine Learning Techniques

In this paper, machine learning (ML) techniques are applied at an early stage of Image Processing (IP). The learning procedures are usually applied from at least the image segmentation level, whereas, in this paper, this is done from a lower processing level: the edge detection level (ED). The main objective is to solve the edge detection problem through ML techniques. The proposed methodology is based on a classification of edges made pixel by pixel, but the predictors employed for the ML task include information about the pixel neighborhood and structures of connected pixels called edge segments. The Sobel operator is employed as input. Making use of 50 images that belong to the Berkeley Computer Vision data set, the average performance of the validation sets when employing our Neural Networks method reached an F-measure significatively higher than with the Sobel operator. The experiment results show that our post-processing technique is a promising new approach for ED.

PRNU-Based Video Source Attribution: Which Frames Are You Using?

Photo Response Non-Uniformity (PRNU) is reputed the most successful trace to identify the source of a digital video. However, its effectiveness is mainly limited by compression and the effect of recently introduced electronic image stabilization on several devices. In the last decade, several approaches were proposed to overcome both these issues, mainly by selecting those video frames which are considered more informative. However, the two problems were always treated separately, and the combined effect of compression and digital stabilization was never considered. This separated analysis makes it hard to understand if achieved conclusions still stand for digitally stabilized videos and if those choices represent a general optimum strategy to perform video source attribution. In this paper, we explore whether an optimum strategy exists in selecting frames based on their type and their positions within the groups of pictures. We, therefore, systematically analyze the PRNU contribute provided by all frames belonging to either digitally stabilized or not stabilized videos. Results on the VISION dataset come up with some insights into optimizing video source attribution in different use cases.

A cascaded CNN-based method for monocular vision robotic grasping

PurposeScattered parts are laid randomly during the manufacturing process and have difficulty to recognize and manipulate. This study aims to complete the grasp of the scattered parts by a manipulator with a camera and learning method.Design/methodology/approachIn this paper, a cascaded convolutional neural network (CNN) method for robotic grasping based on monocular vision and small data set of scattered parts is proposed. This method can be divided into three steps: object detection, monocular depth estimation and keypoint estimation. In the first stage, an object detection network is improved to effectively locate the candidate parts. Then, it contains a neural network structure and corresponding training method to learn and reason high-resolution input images to obtain depth estimation. The keypoint estimation in the third step is expressed as a cumulative form of multi-scale prediction from a network to use an red green blue depth (RGBD) map that is acquired from the object detection and depth map estimation. Finally, a grasping strategy is studied to achieve successful and continuous grasping. In the experiments, different workpieces are used to validate the proposed method. The best grasping success rate is more than 80%.FindingsBy using the CNN-based method to extract the key points of the scattered parts and calculating the possibility of grasp, the successful rate is increased.Practical implicationsThis method and robotic systems can be used in picking and placing of most industrial automatic manufacturing or assembly processes.Originality/valueUnlike standard parts, scattered parts are randomly laid and have difficulty recognizing and grasping for the robot. This study uses a cascaded CNN network to extract the keypoints of the scattered parts, which are also labeled with the possibility of successful grasping. Experiments are conducted to demonstrate the grasping of those scattered parts.

Statistical stegdetectors performance by message re-embedding

State-of-the-art stegdetectors for digital images are based on pre-processing (calibration) of analyzed image for increasing stego-to-cover ratio. In most cases, the calibration is realized by image processing with enormous set of high-pass filters to obtain good estimation of cover image from the stego one. Nevertheless, the efficiency of this approach significantly depends on careful selection of filters for reliably extraction of cover image alterations that are specific for each embedding method. The selection is non-trivial and laborious operation that is realized today by training of convolutional neural networks, such as Ye-Net, SR-Net to name but a few. The paper is devoted to performance analysis of alternative approach to image calibration, namely message re-embedding into analyzed image. The considered method is aimed to increasing stego-to-cover ratio by amplification of cover image alterations caused by message hiding. The analysis was performed on ALASKA and VISION datasets by usage of stegdetector based on SPAM model of covers. Messages were re-embedded according to state-of-the-art adaptive methods HUGO, S-UNIWARD, MG and MiPOD. Proposed approach allows significantly (up to 20%) decreasing detection error even in case of low payload of cover image (less than 10%) where modern stegdetectors are ineffective.

A Simulated Environment for Robot Vision Experiments

Training on simulation data has proven invaluable in applying machine learning in robotics. However, when looking at robot vision in particular, simulated images cannot be directly used no matter how realistic the image rendering is, as many physical parameters (temperature, humidity, wear-and-tear in time) vary and affect texture and lighting in ways that cannot be encoded in the simulation. In this article we propose a different approach for extracting value from simulated environments: although neither of the trained models can be used nor are any evaluation scores expected to be the same on simulated and physical data, the conclusions drawn from simulated experiments might be valid. If this is the case, then simulated environments can be used in early-stage experimentation with different network architectures and features. This will expedite the early development phase before moving to (harder to conduct) physical experiments in order to evaluate the most promising approaches. In order to test this idea we created two simulated environments for the Unity engine, acquired simulated visual datasets, and used them to reproduce experiments originally carried out in a physical environment. The comparison of the conclusions drawn in the physical and the simulated experiments is promising regarding the validity of our approach.

Analysis and Detection of Tomatoes Quality using Machine Learning Algorithm and Image Processing

Grading of agricultural products methods based on artificial intelligence is more important. Because these methods have the ability to learn and thus increase the flexibility of the system. In this paper, image processing systems, detection analysis methods, and artificial intelligence are used to grade tomatoes, and the success rate of grading these methods is compared with each other. However, the purpose of this study is to obtain a solution to detect appearance defects and grade and sort the tomato crop and provide an efficient system in this field. A visual dataset is created, to investigate the approach of image processing and machine learning based on a tomato image. Tomato models are placed individually under the camera and samples are classified in a lighting box away from the effects of ambient light. Data sets have been used in three types of first, second, and third quality categories. It should be noted that quality category one has the best quality and quality category two has the medium quality and category three has the worst quality, Also, each data class contains 80 samples. Using tomato appearance such as size, texture, color, shape, etc. Image processing is performed for extract features. Tomato images are pre-processed for optimization. Then, to prepare for classification, the dimensions of the images are reduced by principal component analysis (PCA). Three categories of an artificial neural network, a support vector machine, and a decision tree are compared to show the most efficient support machine. The analysis is examined in two classes and three classes. The support vector machine has the best accuracy compared to other methods so this rate is 99.9% for two classes and 99.79% for three classes.

Geofizičko postavljanje i seizmičnost u zoni Rift Malavi, Jugoistočna Afrika, analizirano kartografskim skriptiranjem

This paper describes a scripting cartographic techniques that automatically generate maps from open source spatial data using syntax of General Mapping Tools (GMT) and R. A case study present mapping East Africa with a focus on Malawi. In this study, two different approaches of scripting cartography using R programming language and GMT were studied for geophysical analysis aimed to visualize a series of eight new maps in Malawi: topography based on the GEBCO data, seismicity, geomorphometric modeling based on SRTM-90 m (slope, aspect, hillshade and elevation) and geophysical fields: geoid based on EGM-2008 and free-air Faye's gravity based on satellite derived gravity data from CryoSat-2 and Jason-1. In contrast to previous maps of Malawi, a scripting approach was introduced as a console-based cartographic mapping developed for plotting a series of thematic maps based on the high-resolution data. The maps demonstrate correlations between the topography and tectonic faults (Malawi Rift Zone) and earthquakes in the Malawi Lake and extent of landforms. The results demonstrate strong correspondence between the topography and geophysical fields (geoid and gravity): negative values of geoid (-15 to -20) are notable over the Malawi (Nyasa) Lake which corresponds with local topographic depressions. Free-air gravity fields reach the lowest values (-50 to -100) over the Malawi Lake. Local heights in gravity are compared with topographic mountain ranges in the NW and SW of the country on the borders with Zambia and Mozambique. The location of earthquakes vary with the majority located in the north. The geomorphological landforms demonstrate variability in slope steepness and aspect orientation shown on histogram. The techniques of scripts can be used to automatically map spatial data using raster datasets for geophysical visualization, and this paper demonstrated this through a variety of map from the presented thematic series of geophysical maps of Malawi. Full scripts used for mapping are available on the author's public GitHub repository with provided link to her open access codes.

Modeling Physical Activity in Children by Combining Raw Hip-Worn Accelerometry, 2D Pose Estimation, and Direct Observation

This work presents an inductive search of physical activity intensity features for developing computer vision data sets used in automatic physical activity observation systems. An online survey was conducted to calibrate the ground truth definitions in a data set of video synchronized with accelerometers. Experts from the research field were asked to classify 24 short video samples of children&#x2019;s physical activity relative to three metabolic equivalence of task units &#x2013; the presumed threshold of moderate to vigorous physical activity. Leave-one-out disagreement analysis was applied until moderate agreement was achieved (12 respondents remain, Light&#x2019;s <inline-formula> <tex-math notation="LaTeX">$\kappa $ </tex-math></inline-formula> &#x003D; 0.62). The predictive power of features from hip-worn ActiGraph wGT3X-BT 30Hz raw accelerations and several freely available 2D pose estimation models are explored by cut-point analyses and logistic regression while using several approaches to account for uncertainty. Features from the acceleration- and video pose estimation domains are combined in correlation analyses in a twelve-hour data set representing relatively unstructured behavior from 24 children ages 8&#x2013;13 filmed in four different locations. Results indicate that changes in the hip angles of pose-estimated kinematic skeletons across 10 fps video frames can supplement or possibly substitute accelerometers for estimating physical activity intensity in uncrowded indoor scenes. When taking such an approach to labeling physical activity recognition data sets, good joint tracking capacity of the pose estimation method should increase the robustness of the hip angle features.

Unsupervised Single-Scene Semantic Segmentation for Earth Observation

Earth observation data has huge potential to enrich our knowledge about our planet. An important step in many Earth observation tasks is semantic segmentation. Generally, a large number of pixelwise labeled images are required to train deep models for supervised semantic segmentation. On the contrary, strong inter-sensor and geographic variations impede the availability of annotated training data in Earth observation. In practice, most Earth observation tasks use only the target scene without assuming availability of any additional scene, labeled or unlabeled. Keeping in mind such constraints, we propose a semantic segmentation method that learns to segment from a single scene, without using any annotation. Earth observation scenes are generally larger than those encountered in typical computer vision datasets. Exploiting this, the proposed method samples smaller unlabeled patches from the scene. For each patch an alternate view is generated by simple transformations, e.g., addition of noise. Both views are then processed through a two-stream network and weights are iteratively refined using deep clustering, spatial consistency, and contrastive learning in the pixel space. The proposed model automatically segregates the major classes present in the scene and produces the segmentation map. Extensive experiments on four Earth observation datasets collected by different sensors show the effectiveness of the proposed method. Implementation is available at https://gitlab.lrz.de/ai4eo/cd/-/tree/main/unsupContrastiveSemanticSeg.

Deep Neural Network Driven Automated Underwater Object Detection

Object recognition and computer vision techniques for automated object identification are attracting marine biologist's interest as a quicker and easier tool for estimating the fish abundance in marine environments. However, the biggest problem posed by unrestricted aquatic imaging is low luminance, turbidity, background ambiguity, and context camouflage, which make traditional approaches rely on their efficiency due to inaccurate detection or elevated false-positive rates. To address these challenges, we suggest a systemic approach to merge visual features and Gaussian mixture models with You Only Look Once (YOLOv3) deep network, a coherent strategy for recognizing fish in challenging underwater images. As an image restoration phase, pre-processing based on diffraction correction is primarily applied to frames. The YOLOv3 based object recognition system is used to identify fish occurrences. The objects in the background that are camouflaged are often overlooked by the YOLOv3 model. A proposed Bi-dimensional Empirical Mode Decomposition (BEMD) algorithm, adapted by Gaussian mixture models, and integrating the results of YOLOv3 improves detection efficiency of the proposed automated underwater object detection method. The proposed approach was tested on four challenging video datasets, the Life Cross Language Evaluation Forum (CLEF) benchmark from the F4K data repository, the University of Western Australia (UWA) dataset, the bubble vision dataset and the DeepFish dataset. The accuracy for fish identification is 98.5 percent, 96.77 percent, 97.99 percent and 95.3 percent respectively for the various datasets which demonstrate the feasibility of our proposed automated underwater object detection method.

Deep Learning-Based Approach for Arabic Visual Speech Recognition

Lip-reading technologies are rapidly progressing following the breakthrough of deep learning. It plays a vital role in its many applications, such as: human-machine communication practices or security applications. In this paper, we propose to develop an effective lip-reading recognition model for Arabic visual speech recognition by implementing deep learning algorithms. The Arabic visual datasets that have been collected contains 2400 records of Arabic digits and 960 records of Arabic phrases from 24 native speakers. The primary purpose is to provide a high-performance model in terms of enhancing the preprocessing phase. Firstly, we extract keyframes from our dataset. Secondly, we produce a Concatenated Frame Images (CFIs) that represent the utterance sequence in one single image. Finally, the VGG-19 is employed for visual features extraction in our proposed model. We have examined different keyframes: 10, 15, and 20 for comparing two types of approaches in the proposed model: (1) the VGG-19 base model and (2) VGG-19 base model with batch normalization. The results show that the second approach achieves greater accuracy: 94% for digit recognition, 97% for phrase recognition, and 93% for digits and phrases recognition in the test dataset. Therefore, our proposed model is superior to models based on CFIs input.

Dynamic Classifier Alignment for Unsupervised Multi-Source Domain Adaptation

Unsupervised domain adaptation leverages the previously gained knowledge from a labeled source domain to tackle the task from a different but similar unlabeled target domain. Most existing methods focus on transferring knowledge from a single source domain, but the information from a single domain may be inadequate to complete the target task. Some previous studies have turned to multi-view representations to enrich the transferable information. However, they simply concatenate multi-view features, which may result in information redundancy. In this paper, we propose a dynamic classifier alignment (DCA) method for multi-source domain adaptation, which aligns classifiers driven from multi-view features via a sample-wise automatic way. As proposed, both the importance of each view and the contribution of each source domain are investigated. To determine the important degrees of multiple views, an importance learning function is built by generating an auxiliary classifier. To learn the source combination parameters, a domain discriminator is developed to estimate the probability of a sample belonging to multiple source domains. Meanwhile, a self-training strategy is proposed to enhance the cross-domain ability of source classifiers with the assistance of pseudo target labels. Experiments on real-world visual datasets show the superiority of the proposed DCA.

CDFKD-MFS: Collaborative Data-Free Knowledge Distillation via Multi-Level Feature Sharing

Recently, the compression and deployment of powerful deep neural networks (DNNs) on resource-limited edge devices to provide intelligent services have become attractive tasks. Although knowledge distillation (KD) is a feasible solution for compression, its requirement on the original dataset raises privacy concerns. In addition, it is common to integrate multiple pretrained models to achieve satisfactory performance. How to compress multiple models into a tiny model is challenging, especially when the original data are unavailable. To tackle this challenge, we propose a framework termed collaborative data-free knowledge distillation via multi-level feature sharing (CDFKD-MFS), which consists of a multi-header student module, an asymmetric adversarial data-free KD module, and an attention-based aggregation module. In this framework, the student model equipped with a multi-level feature-sharing structure learns from multiple teacher models and is trained together with a generator in an asymmetric adversarial manner. When some real samples are available, the attention module adaptively aggregates predictions of the student headers, which can further improve performance. We conduct extensive experiments on three popular computer visual datasets. In particular, compared with the most competitive alternative, the accuracy of the proposed framework is 1.18&#x0025; higher on the CIFAR-100 dataset, 1.67&#x0025; higher on the Caltech-101 dataset, and 2.99&#x0025; higher on the mini-ImageNet dataset.

A Framework of Lightweight Deep Cross-Connected Convolution Kernel Mapping Support Vector Machines

Deep kernel mapping support vector machines have achieved good results in numerous tasks by mapping features from a low-dimensional space to a high-dimensional space and then using support vector machines for classification. However, the depth kernel mapping support vector machine does not take into account the connection of different dimensional spaces and increases the model parameters. To further improve the recognition capability of deep kernel mapping support vector machines while reducing the number of model parameters, this paper proposes a framework of Lightweight Deep Convolutional Cross-Connected Kernel Mapping Support Vector Machines (LC-CKMSVM). The framework consists of a feature extraction module and a classification module. The feature extraction module first maps the data from low-dimensional to high-dimensional space by fusing the representations of different dimensional spaces through cross-connections; then, it uses depthwise separable convolution to replace part of the original convolution to reduce the number of parameters in the module; The classification module uses a soft margin support vector machine for classification. The results on 6 different visual datasets show that LC-CKMSVM obtains better classification accuracies on most cases than the other five models.

GasBotty: Multi-Metric Extraction in the Wild

The lived environment, particularly when proximal to roadways, is filled with multi-digit and multi-numbered values corresponding to advertised commodities. The reliable detection of single multi-digit values from natural imagery has been widely studied through the last decade (e.g. Street View House Numbers [SVHN]); however, extraction and assignment of the contextual meaning of those values are far more difficult given the diversity and unstructured nature of advertisements. To operationalize information extracted from detected values in the wild, the contextual meaning that those values represent is critical. To our knowledge, no large-scale visual dataset comprising multi-digit, multi-number values with associated context labels exists; we denote this class of problems as &#x201C;multi-metric extraction in the wild&#x201D;. In this work, we focus on the accurate detection and reading of gas prices, and their contextual association to gas grade and payment type. We provide complete annotations for the Gas Prices of America (GPA) dataset, comprising 2,048 training and 512 test images sampled across the United States including over 2,600 signs, 6,000 prices, 27,000 digits, and 7,800 gas grade and payment type labels. With these data, we develop the GasBotty predictor, a composite neural network model, and evaluate it over eight benchmark tasks of increasing difficulty. Finally, we define a new highly stringent, binary-type metric, denoted &#x201C;All-or-Nothing Accuracy&#x201D; (ANA), requiring that a predictor perfectly extract and correctly associate all information in a gas sign. Our proposed model achieves 72.9&#x0025; ANA over the independent test set of 512 images, where conventional state-of-the-art models object detection models and both a uniform random and biased random predictor would all tend towards 0&#x0025; ANA. GasBotty and the GPA dataset will serve as a valuable benchmark for the development of future multi-metric extraction in the wild systems.

Real-time strawberry detection using deep neural networks on embedded system (rtsd-net): An edge AI application

Computer vision is a key technique to make agricultural machinery smart. Deep neural network has achieved great success in computer vision. How to use it at a small size, low cost, low power consumption device with high accuracy and speed on strawberry harvesting machinery has drawn much research attention. Since the infield situation has reduced number of objects and that they are easier to be distinguished from the background compared to other computer vision datasets, the huge neural network structure can be simplified in order to speed up the detection inference without penalizing the detection accuracy. In this research, a new deep neural network called RTSD-Net is proposed based on stat-of-art light-weighted YOLOv4-tiny with reduced layers and modified structure for real-time strawberry detection under infield condition. The original CSPNet was replaced by 2 types of CSPNet designed with reduced parameters and a simplified structure and 4 new network structures are designed by combining these 2 types. The performances of the 4 networks were evaluated. It was observed that the number of parameters of these 4 networks and the detection speed of the model is negatively correlated. Simplified structure and reduced parameters can contribute to faster operational speed. The last one was selected and named as RTSD-Net. Comparing with YOLOv4 tiny, the accuracy of RTSD-Net is only reduced by 0.62% but the speed is increased by 25FPS, which is 25.93% higher than that of YOLOv4-tiny. Embedded system Jetson Nano was selected as the evaluation platform to evaluate the RTSD-Net’s performance for edge computing. The original Open Neural Network Exchange (ONNX) model was loaded on Jetson Nano and the speed of RTSD-Net was 13.1FPS, which is 19.0% higher than that of YOLOv4-tiny. After speeded up by TensorRT method, the transformed model reached 25.20fps, which is twice as fast as the ONNX model, and 15% faster than the YOLOv4-tiny model. After speeding up, the efficiency of RTSD-Net is enough for computer vision based strawberry detection and harvesting. In summary, the proposed RTSD-Net has good potential in smart strawberry harvesting machinery and the idea of redesigning neural structure and reducing parameters to speed up the detection rate of deep neural network is expected to have good application in edge computing.