Traditional Computer Vision Techniques Research Articles

Deep Learning Based Hand Wrist Segmentation using Mask R-CNN

Deep learning is one of the trending technologies in computer vision to identify and classify objects. Deep learning is a subset of Machine Learning and Artificial Intelligence. Detecting and classifying the object was a challenging task in traditional computer vision techniques, and now there are numerous deep learning Techniques scaled up to achieve this. The primary purpose of the research is to detect and segment the human hand wrist region using deep learning methods. This research is widespread to deep learning enthusiasts who needs to segment custom objects using instance segmentation. We demonstrated a segmented hand wrist using the Mask Regional Convolutional Neural Network (R-CNN) technique with an average accuracy of 99.73%. This work also compares the performance evaluation of baseline and custom Hand Wrist Mask R-CNN. The achieved validation class loss is 0.00866 training and 0.02736 validation; both the values are comparatively deficient compared with baseline Mask R-CNN.

Vision-based positioning of Unmanned Surface Vehicles using Fiducial Markers for automatic docking

This paper describes a method of using fiducial markers to aid Unmanned Surface Vehicles (USVs) as an additional positioning source during auto-docking. Vision-based techniques allow USVs to localize themselves relative to their surroundings without relying on external communication. This paper shows and evaluates a vision-based strategy to localize USVs to a pier. We used the Global Navigation Satellite System (GNSS) with Real-Time kinematic (RTK) and Inertial Navigation System (INS) with a base station on the pier to validate the vision-based position estimates. The experiment shows that traditional computer vision techniques using fiducial markers can give accurate outdoor position estimates in good conditions. We also highlight some adverse conditions where the performance decreased considerably.

Model Assumptions and Data Characteristics: Impacts on Domain Adaptation in Building Segmentation

Studies on domain adaptation (DA) for remote sensing (RS) imagery analysis lack consistency in selection and description of evaluation scenarios. Without properly characterizing datasets, model assumptions, and evaluation scenarios, it is difficult to objectively compare DA methods and reach conclusions about their suitability across different applications. With this motivation, this work seeks to empirically assess to which extent the interaction between data characteristics and model assumptions influence the effectiveness of DA methods. Using the widely explored task of building footprint segmentation as case study, we perform a large-scale study across over 200 domain adaptation scenarios that include variations across view angles, areas observed, and sensors used for data acquisition. Rather than adopting different model architectures or optimization criteria, we contrast the performances of two DA methods based on adversarial learning that differ only in their assumptions about source and target domains. Informed by metadata and data characteristics unveiled using traditional computer vision techniques as well as pre-trained deep models, we provide a detailed meta-analysis of experiments highlighting the importance of accurately considering data assumptions for DA in RS segmentation tasks. As a “cherry-picking” exercise demonstrates, different claims regarding which model is best could be made by selecting different subsets of evaluation scenarios. While well-calibrated assumptions can be beneficial, mismatching assumptions can lead to negative biases in DA applications. This study intends to motivate the community towards more consistent evaluation protocols, while providing recommendations and insights toward creating novel benchmark datasets, documenting data characteristics, application-specific knowledge, and model assumptions.

Deep Neural Network Training and Testing Datasets for License Plate Recognition

Modern society has made tremendous progress towards automation to increase the quality of life and reduce the margin of human error. Intelligent transportation systems are a critical aspect of this evolution. The core technology of these systems is the automatic identification of vehicles' license plates to monitor safety and control violations of traffic rules and other crimes. The research on license plate detection and recognition has gone a long way, from traditional computer vision techniques to features (color, shape, text, etc.) based classification and finally to modern deep learning structures. The deep networks comprising hundreds of layers require enormous amounts of training data. The training dataset should contain plates from different countries; otherwise, the system will be specific to only certain types of plates (from a country or province). There are several datasets collected by researchers containing large numbers of license plates from different countries. This paper provides a detailed survey of such datasets available in the public domain. Sample images from each dataset are shown, and details such as the dataset size, size of images, download link, and country of origin are provided. This survey will be a helpful reference for new researchers in the field for the tasks of training new networks and benchmarking their performances.

LICENSE PLATE RECOGNITION TECHNIQUES: COMPARATIVE STUDY

Vehicle License Plate Recognition (LPR) has become a crucial system for various applications such as security monitoring, parking access, law enforcement and so on. LPR is employed for the identification of vehicles using their license plate. Recently, LPR technology has evolved progressively where vast improvement had been made towards the development of the image recognition’s quality and speed, as well as its state of the art methods. Although several research studies managed to resolve most of the issues that arise in LPR systems, more studies need to be conducted to improve the performance of LPR. This paper aims to provide a comprehensive analysis and comparison of different methods used in LPR. It summarizes each of the methods in terms of their accuracy, performance, strengths and weaknesses. Based on the recognition techniques used, LPR is then characterized into two categories, namely Traditional Computer Vision and Deep Learning Techniques.

DCRN: An Optimized Deep Convolutional Regression Network for Building Orientation Angle Estimation in High-Resolution Satellite Images

Recently, remote sensing satellite image analysis has received significant attention from geo-information scientists. However, the current geo-information systems lack automatic detection of several building characteristics inside the high-resolution satellite images. The accurate extraction of buildings characteristics helps the decision-makers to optimize urban planning and achieve better decisions. Furthermore, Building orientation angle is a very critical parameter in the accuracy of automated building detection algorithms. However, the traditional computer vision techniques lack accuracy, scalability, and robustness for building orientation angle detection. This paper proposes two different approaches to deep building orientation angle estimation in the high-resolution satellite image. Firstly, we propose a transfer deep learning approach for our estimation task. Secondly, we propose a novel optimized DCRN network consisting of pre-processing, scaled gradient layer, deep convolutional units, dropout layers, and regression end layer. The early proposed gradient layer helps the DCRN network to extract more helpful information and increase its performance. We have collected a building benchmark dataset that consists of building images in Riyadh city. The images used in the experiments are 15,190 buildings images. In our experiments, we have compared our proposed approaches and the other approaches in the literature. The proposed system has achieved the lowest root mean square error (RMSE) value of 1.24, the lowest mean absolute error (MAE) of 0.16, and the highest adjusted R-squared value of 0.99 using the RMS optimizer. The cost of processing time of our proposed DCRN architecture is 0.0113 ± 0.0141 s. Our proposed approach has proven its stability with the input building image contrast variation for all orientation angles. Our experimental results are promising, and it is suggested to be utilized in other building characteristics estimation tasks in high-resolution satellite images.

Detection of Cervical Cancer Cells in Whole Slide Images Using Deformable and Global Context Aware Faster RCNN-FPN.

Cervical cancer is a worldwide public health problem with a high rate of illness and mortality among women. In this study, we proposed a novel framework based on Faster RCNN-FPN architecture for the detection of abnormal cervical cells in cytology images from a cancer screening test. We extended the Faster RCNN-FPN model by infusing deformable convolution layers into the feature pyramid network (FPN) to improve scalability. Furthermore, we introduced a global contextual aware module alongside the Region Proposal Network (RPN) to enhance the spatial correlation between the background and the foreground. Extensive experimentations with the proposed deformable and global context aware (DGCA) RCNN were carried out using the cervical image dataset of “Digital Human Body” Vision Challenge from the Alibaba Cloud TianChi Company. Performance evaluation based on the mean average precision (mAP) and receiver operating characteristic (ROC) curve has demonstrated considerable advantages of the proposed framework. Particularly, when combined with tagging of the negative image samples using traditional computer-vision techniques, 6–9% increase in mAP has been achieved. The proposed DGCA-RCNN model has potential to become a clinically useful AI tool for automated detection of cervical cancer cells in whole slide images of Pap smear.

Wave Peel Tracking: A New Approach for Assessing Surf Amenity and Analysis of Breaking Waves

The creation and protection of surfing breaks along populated coastlines have become a consideration for many councils and governments as surfing breaks are a major driver of tourism. To assess the surf amenity of surfing breaks, a quantitative and objective assessment method is required. A new wave peel tracking (WPT) method has been developed using a shore-based camera to assess surf amenity by measuring and quantifying potential surfing ride rate, length, duration, speed and direction on a wave-by-wave basis. The wave peel (or “curl” below the wave peak) is the optimal surfing region on a wave, and each wave peel track represents a surfable ride. Wave peel regions are identified, classified and tracked using traditional and machine learning-based computer vision techniques. The methodology is validated by comparing the rectified wave peel tracks with GPS-measured tracks from surfers in the wave peel regions. The WPT methodology is evaluated with data from a reef and adjacent natural beach at the Gold Coast, Australia. The reef produced longer ride lengths than the nearshore region and showed a consistent breaking location along the reef crest. Spatial maps of the wave peel tracks show the influence of tides on the wave breaking patterns and intensity. The WPT algorithm provides a robust, automated method for quantifying surf amenity to provide baseline data for surf break conservation. The methodology has potential uses to verify numerical modelling of surf breaks and to assess the impact of coastal development on surf breaks.

Single Image Façade Segmentation and Computational Rephotography of House Images Using Deep Learning

Rephotography is the process of recapturing the photograph of a location from the same perspective in which it was captured earlier. A rephotographed image is the best presentation to visualize and study the social changes of a location over time. Traditionally, only expert artists and photographers are capable of generating the rephotograph of any specific location. Manual editing or human eye judgment that is considered for generating rephotographs normally requires a lot of precision, effort and is not always accurate. In the era of computer science and deep learning, computer vision techniques make it easier and faster to perform precise operations to an image. Until now many research methodologies have been proposed for rephotography but none of them is fully automatic. Some of these techniques require manual input by the user or need multiple images of the same location with 3D point cloud data while others are only suggestions to the user to perform rephotography. In historical records/archives most of the time we can find only one 2D image of a certain location. Computational rephotography is a challenge in the case of using only one image of a location captured at different timestamps because it is difficult to find the accurate perspective of a single 2D historical image. Moreover, in the case of building rephotography, it is required to maintain the alignments and regular shape. The features of a building may change over time and in most of the cases, it is not possible to use a features detection algorithm to detect the key features. In this research paper, we propose a methodology to rephotograph house images by combining deep learning and traditional computer vision techniques. The purpose of this research is to rephotograph an image of the past based on a single image. This research will be helpful not only for computer scientists but also for history and cultural heritage research scholars to study the social changes of a location during a specific time period, and it will allow users to go back in time to see how a specific place looked in the past. We have achieved good, fully automatic rephotographed results based on façade segmentation using only a single image.

Towards Lifespan Automation for Caenorhabditis elegans Based on Deep Learning: Analysing Convolutional and Recurrent Neural Networks for Dead or Live Classification.

The automation of lifespan assays with C. elegans in standard Petri dishes is a challenging problem because there are several problems hindering detection such as occlusions at the plate edges, dirt accumulation, and worm aggregations. Moreover, determining whether a worm is alive or dead can be complex as they barely move during the last few days of their lives. This paper proposes a method combining traditional computer vision techniques with a live/dead C. elegans classifier based on convolutional and recurrent neural networks from low-resolution image sequences. In addition to proposing a new method to automate lifespan, the use of data augmentation techniques is proposed to train the network in the absence of large numbers of samples. The proposed method achieved small error rates (3.54% ± 1.30% per plate) with respect to the manual curve, demonstrating its feasibility.

Determining Chess Game State from an Image.

Identifying the configuration of chess pieces from an image of a chessboard is a problem in computer vision that has not yet been solved accurately. However, it is important for helping amateur chess players improve their games by facilitating automatic computer analysis without the overhead of manually entering the pieces. Current approaches are limited by the lack of large datasets and are not designed to adapt to unseen chess sets. This paper puts forth a new dataset synthesised from a 3D model that is an order of magnitude larger than existing ones. Trained on this dataset, a novel end-to-end chess recognition system is presented that combines traditional computer vision techniques with deep learning. It localises the chessboard using a RANSAC-based algorithm that computes a projective transformation of the board onto a regular grid. Using two convolutional neural networks, it then predicts an occupancy mask for the squares in the warped image and finally classifies the pieces. The described system achieves an error rate of 0.23% per square on the test set, 28 times better than the current state of the art. Further, a few-shot transfer learning approach is developed that is able to adapt the inference system to a previously unseen chess set using just two photos of the starting position, obtaining a per-square accuracy of 99.83% on images of that new chess set. The code, dataset, and trained models are made available online.

Dual attention-guided feature pyramid network for instance segmentation of group pigs

In respect of pig instance segmentation, the application of traditional computer vision techniques is constrained by sundries barrier, overlapping, and different perspectives in the pig breeding environment. In recent years, the attention-based methods have achieved remarkable performance. In this paper, we introduce two types of attention blocks into the feature pyramid network (FPN) (see nomenclature table) framework, which encode the semantic interdependencies in the channel (named channel attention block (CAB)) (see nomenclature table) and spatial (named spatial attention block (SAB)) (see nomenclature table) dimensions, respectively. By integrating the associated features, the CAB selectively emphasizes the interdependencies among the channels. Meanwhile, the SAB selectively aggregates the features at each position through a weighted sum of the features at all positions. A dual attention block (DAB) (see nomenclature table) is proposed to integrate CAB features with SAB information flexibly. A total of 45 pigs with 8 pens are captured as the experiment subjects. In comparison with such state-of-art attention modules as convolutional block attention module (CBAM) (see nomenclature table), bottleneck attention module (BAM) (see nomenclature table), and spatial-channel squeeze & excitation (SCSE) (see nomenclature table), embedding DAB can contribute to the most significant performance improvement in different task networks with distinct backbone networks. Especially with HTC-R101-DAB (hybrid task cascade) (see nomenclature table), the best performance is produced, with the AP0.5 (average precision) (see nomenclature table) AP0.75, AP0.5:0.95, and AP0.5:0.95-large reaching 93.1%, 84.1%, 69.4%, and 71.8%, respectively. Also, as indicated by ablation experiments, the SAB contributes more than CAB. Meanwhile, the predictive results appear a trend of increasing initially and decreasing afterwards after different numbers of SAB are merged. Besides, as revealed by the visualization of attention maps, attention blocks can extract regions with similar semantic information. The attention-based models also produce outstanding segmentation performance on public dataset, which evidences the practicability of our attention blocks. Ourbaseline models are available11https://github.com/zhiweihu1103/pig-instance-segmentation.

Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras

Thermal infrared cameras are increasingly being used in various applications such as robot vision, industrial inspection and medical imaging, thanks to their improved resolution and portability. However, the performance of traditional computer vision techniques developed for electro-optical imagery does not directly translate to the thermal domain due to two major reasons: these algorithms require photometric assumptions to hold, and methods for photometric calibration of RGB cameras cannot be applied to thermal-infrared cameras due to difference in data acquisition and sensor phenomenology. In this paper, we take a step in this direction, and introduce a novel algorithm for online photometric calibration of thermal-infrared cameras. Our proposed method does not require any specific driver/hardware support and hence can be applied to any commercial off-the-shelf thermal IR camera. We present this in the context of visual odometry and SLAM algorithms, and demonstrate the efficacy of our proposed system through extensive experiments for both standard benchmark datasets, and real-world field tests with a thermal-infrared camera in natural outdoor environments.

Deep learning-based high-throughput phenotyping can drive future discoveries in plant reproductive biology

Key messageAdvances in deep learning are providing a powerful set of image analysis tools that are readily accessible for high-throughput phenotyping applications in plant reproductive biology.High-throughput phenotyping systems are becoming critical for answering biological questions on a large scale. These systems have historically relied on traditional computer vision techniques. However, neural networks and specifically deep learning are rapidly becoming more powerful and easier to implement. Here, we examine how deep learning can drive phenotyping systems and be used to answer fundamental questions in reproductive biology. We describe previous applications of deep learning in the plant sciences, provide general recommendations for applying these methods to the study of plant reproduction, and present a case study in maize ear phenotyping. Finally, we highlight several examples where deep learning has enabled research that was previously out of reach and discuss the future outlook of these methods.

Real-Time Hair Segmentation Using Mobile-Unet

We described a real-time hair segmentation method based on a fully convolutional network with the basic structure of an encoder–decoder. In one of the traditional computer vision techniques for hair segmentation, the mean shift and watershed methodologies suffer from inaccuracy and slow execution due to multi-step, complex image processing. It is also difficult to execute the process in real-time unless an optimization technique is applied to the partition. To solve this problem, we exploited Mobile-Unet using the U-Net segmentation model, which incorporates the optimization techniques of MobileNetV2. In experiments, hair segmentation accuracy was evaluated by different genders and races, and the average accuracy was 89.9%. By comparing the accuracy and execution speed of our model with those of other models in related studies, we confirmed that the proposed model achieved the same or better performance. As such, the results of hair segmentation can obtain hair information (style, color, length), which has a significant impact on human-robot interaction with people.

A Deep Learning-Based Benchmarking Framework for Lane Segmentation in the Complex and Dynamic Road Scenes

Automatic lane detection is a classical task in autonomous vehicles that traditional computer vision techniques can perform. However, such techniques lack reliability for achieving high accuracy while maintaining adequate time complexity in the context of real-time detection in complex and dynamic road scenes. Deep neural networks have proved their ability to achieve competing accuracy and time complexity while training them on manually labeled data. Yet, the unavailability of segmentation masks for host lanes in harsh road environments hinders fully supervised methods’ operability on such a problem. This work proposes integrating traditional computer vision techniques and deep learning methods to develop a reliable benchmarking framework for lane detection tasks in complex and dynamic road scenes. Firstly, an automatic segmentation algorithm based on a sequence of traditional computer vision techniques has been experimented. This algorithm precisely segments the semantic region of the host lane in the complex urban images of nuScenes dataset used in this framework; hence corresponding weak labels are generated. After that, the developed data is qualitatively evaluated to be used in training and benchmarking five state-of-the-art FCN-based architectures: SegNet, Modified SegNet, U-Net, ResUNet, and ResUNet++. The performance evaluation of the trained models is done visually and quantitatively by considering lane detection a binary semantic segmentation task. The output results show robust performance, especially ResUNet++, which outperforms all the other models while testing them in different complex road scenes with dynamic scenarios and various lighting conditions.

Study on Intelligent Image Recognition of Non-linear Short Pointer SF6 Meter Readings

Currently widely used SF6 inflatable equipment pressure gauge are short pointer and the pointer is not connected to the center of the dial (non-linear pointer), The position of the hands on this dial cannot be identified using traditional computer vision techniques. In order to solve the problem, This paper proposes a method for SF6 pressure gauge pointer and reading recognition based on digital image processing and Mask-RCNN neural network image segmentation technology. The method first pre-processes the SF6 pressure gauge image and Canny edge detection, while using Mask-RCNN network to extract the pointer feature information and scale feature information, and uses the SF6 pressure gauge pointer feature and scale feature to calculate the pressure gauge reading. The effectiveness of the algorithm is verified through the scale identification of 180 short pointer SF6 pressure meters actually operated in a 220kV substation with 100% accuracy.

Hair Segmentation and Removal in Dermoscopic Images Using Deep Learning

Melanoma and non-melanoma skin cancers have shown a rapidly increasing incidence rate, pointing to skin cancer as a major problem for public health. When analyzing these lesions in dermoscopic images, the hairs and their shadows on the skin may occlude relevant information about the lesion at the time of diagnosis, reducing the ability of automated classification and diagnosis systems. In this work, we present a new approach for the task of hair removal on dermoscopic images based on deep learning techniques. Our proposed model relies on an encoder-decoder architecture, with convolutional neural networks, for the detection and posterior restoration of hair’s pixels from the images. Moreover, we introduce a new combined loss function in the network’s training phase that combines the $L_{1}$ distance, the total variation loss, and a loss function based on the structural similarity index metric. Currently, there are no datasets that contain the same images with and without hair, which is necessary to quantitatively evaluate our model. Thus, we simulate the presence of hair in hairless images extracted from publicly known datasets. We compare our results with six state-of-the-art algorithms based on traditional computer vision techniques by means of similarity measures that compare the reference hairless image and the one with simulated hair. Finally, the Wilcoxon signed-rank test is used to compare the methods. The results, both qualitatively and quantitatively, demonstrate the effectiveness of our model and how our loss function improves the restoration ability of the proposed model.

Deep learning approach for automatic microplastics counting and classification

The quantification of microplastics is a needed task to monitor its evolution and model its behavior. However, it is a time demanding task traditionally performed using expensive equipment. In this paper, an architecture based on deep learning networks is presented with the aim of automatically count and classify microplastic particles in the range of 1–5 mm from pictures taken with a digital camera or a mobile phone with a resolution of 16 million pixels or higher. The proposed architecture comprises a first stage, implemented with the U-Net neural network, in charge of making the segmentation of the particles in the image. After the different particles have been isolated, a second stage based on the VGG16 neural network classifies them into three types: fragments, pellets and lines. These three types have been selected for being the most common in the range size under consideration. The experimental evaluation was carried out using images taken with two digital cameras and one mobile phone. The particles used in experiments correspond to samples collected on the beach of Playa del Poris in Tenerife Island, Spain, (28° 09′ 51″ N, 16° 25′ 54″ W) in August 2018. A Jaccard index value of 0.8 is achieved in the experiments of particles segmentation and an accuracy of 98.11% is obtained in the classification of the microplastic particles. The proposed architecture is remarkable faster than a similar previously published system based on traditional computer vision techniques.

Fi-Fo Detector: Figure and Formula Detection Using Deformable Networks

We propose a novel hybrid approach that fuses traditional computer vision techniques with deep learning models to detect figures and formulas from document images. The proposed approach first fuses the different computer vision based image representations, i.e., color transform, connected component analysis, and distance transform, termed as Fi-Fo image representation. The Fi-Fo image representation is then fed to deep models for further refined representation-learning for detecting figures and formulas from document images. The proposed approach is evaluated on a publicly available ICDAR-2017 Page Object Detection (POD) dataset and its corrected version. It produces the state-of-the-art results for formula and figure detection in document images with an f1-score of 0.954 and 0.922, respectively. Ablation study results reveal that the Fi-Fo image representation helps in achieving superior performance in comparison to raw image representation. Results also establish that the hybrid approach helps deep models to learn more discriminating and refined features.