Vision-based Applications Research Articles

Hand Gesture Recognition and Voice Conversion for Speech Impaired

Speech impairment may be infirmities that affect a human’s capability to reach out others via conversation and listening. Humankind in such community, suffering from the above said disability, adopts sign language as an alternate mode of conversation. Even though speech and hearing impaired are using sign language everywhere, the non-sign language speakers face challenges in communicating with speech impaired. Modern approaches in deep learning and computer vision have resulted in a tremendous improvement in the areas of motion and hand sign identification. The main motive presented work is to develop vision-based applications that provide sign language transliteration to speech enabling communication among verbal and non-verbal communicators. The proposed framework uses a methodology of collecting video sequences followed by extracting temporal and spatial features. Recognition of spatial features is compassed using “Convolutional Neural Networks” and to train on temporal features, “Recurrent Neural Networks” were utilized. The databases employed are the American Sign Language datasets.

Unsupervised Moving Object Segmentation from Stationary or Moving Camera based on Multi-frame Homography Constraints.

Moving object segmentation is the most fundamental task for many vision-based applications. In the past decade, it has been performed on the stationary camera, or moving camera, respectively. In this paper, we show that the moving object segmentation can be addressed in a unified framework for both type of cameras. The proposed method consists of two stages: (1) In the first stage, a novel multi-frame homography model is generated to describe the background motion. Then, the inliers and outliers of that model are classified as background trajectories and moving object trajectories by the designed cumulative acknowledgment strategy. (2) In the second stage, a super-pixel-based Markov Random Fields model is used to refine the spatial accuracy of initial segmentation and obtain final pixel level labeling, which has integrated trajectory classification information, a dynamic appearance model, and spatial temporal cues. The proposed method overcomes the limitations of existing object segmentation algorithms and resolves the difference between stationary and moving cameras. The algorithm is tested on several challenging open datasets. Experiments show that the proposed method presents significant performance improvement over state-of-the-art techniques quantitatively and qualitatively.

An R based interface to understand cognitive ability of different participants using fixation and saccade detection

Eye-tracking is an emerging area of science in a wide range of computer vision-based applications. Eye-tracking mainly deals with where the person is looking at and for what duration. In this work, we propose an R based interface to visualize the eye-tracking data as fixations and saccades that depicts where the person looking at –fixations and saccades and what duration – fixation duration. Through the eye-tracking metrics that are visualized in our work, one can visualize the difference between the viewing behaviour of various participants. The differences thus depicted can later be studied in order to understand the cognitive abilities of the participants. The paper contains a detailed survey of the existing literature and the experimental results generated using the R interface.

Review of Traffic Sign Detection and Recognition Techniques

Text, as one of the most compelling developments of humankind, has assumed a significant job in human life, so distant from antiquated occasions. The rich and exact data epitomized in content is extremely helpful in a wide scope of vision-based applications; along these lines content detection and recognition in regular scenes have turned out to be significant and dynamic research points in PC vision and report investigation. Traffic sign detection and recognition is a field of connected PC vision research worried about the programmed detection and grouping or recognition of traffic signs in scene pictures procured from a moving vehicle. Driving is an assignment dependent on visual data handling. The traffic signs characterize a visual language translated by drivers. Traffic signs convey much data important for effective driving; they portray current traffic circumstance, characterize option to proceed, preclude or grant certain headings. In this paper, talked about different detection and recognition schemes.

Visual Saliency Detection Using a Rule-Based Aggregation Approach

In this paper, we propose an approach for salient pixel detection using a rule-based system. In our proposal, rules are automatically learned by combining four saliency models. The learned rules are utilized for the detection of pixels of the salient object in a visual scene. The proposed methodology consists of two main stages. Firstly, in the training stage, the knowledge extracted from outputs of four state-of-the-art saliency models is used to induce an ensemble of rough-set-based rules. Secondly, the induced rules are utilized by our system to determine, in a binary manner, the pixels corresponding to the salient object within a scene. Being independent of any threshold value, such a method eliminates any midway uncertainty and exempts us from performing a post-processing step as is required in most approaches to saliency detection. The experimental results on three datasets show that our method obtains stable and better results than state-of-the-art models. Moreover, it can be used as a pre-processing stage in computer vision-based applications in diverse areas such as robotics, image segmentation, marketing, and image compression.

An Improved Multi-layer Cooperation Routing in Visual Sensor Network for Energy Minimization

Recent development on visual sensor networks (VSN) have shown the emergence of VSN as an important class of sensor-based distributed intelligent systems, with unique performance, complexity, and quality of service challenges. Furthermore, the popularity of VSN had attracted researchers and industries from many applications. Basically, VSN consists of a large number of low-power camera nodes, visual sensor networks support a great number of novel vision-based applications. The information provided by the camera nodes from a monitored site were collected data and processed in distributed and collaboratively. The use of multiple cameras in the network provides different views of the scene which enhances the reliability of the captured events. However, the cameras combined with the network's resource produce large amount of image data which require more energy consumption. Many research experts are working and publishing research papers on energy minimization in visual sensor network. However, there are no special attentions given by researchers on multi-layer cooperation routing in visual sensor network (VSN) particularly on the deficiency of forward-thinking innovation. This research proposed a new method that put forward the step and use (CO-VSN) multi-layer routing arrangement in visual sensor network. The performance of the proposed method is then compared with RASE protocol in different parameters and the result shows some meditation change in the stability of the network, increase the network life, reduce packet loss and energy drainage reduce in the VSNs. Even though, the RASE scheme used channel condition for movement of data and yet the proposed CO-VSN used cooperation and demonstrated better results.

Logistic Regression Confined by Cardinality-Constrained Sample and Feature Selection.

Many vision-based applications rely on logistic regression for embedding classification within a probabilistic context, such as recognition in images and videos or identifying disease-specific image phenotypes from neuroimages. Logistic regression, however, often performs poorly when trained on data that is noisy, has irrelevant features, or when the samples are distributed across the classes in an imbalanced setting; a common occurrence in visual recognition tasks. To deal with those issues, researchers generally rely on ad-hoc regularization techniques or model a subset of these issues. We instead propose a mathematically sound logistic regression model that selects a subset of (relevant) features and (informative and balanced) set of samples during the training process. The model does so by applying cardinality constraints (via l0-'norm' sparsity) on the features and samples. l0 defines sparsity in mathematical settings but in practice has mostly been approximated (e.g., via l1 or its variations) for computational simplicity. We prove that a local minimum to the non-convex optimization problems induced by cardinality constraints can be computed by combining block coordinate descent with penalty decomposition. On synthetic, image recognition, and neuroimaging datasets, we show that the accuracy of the method is higher than alternative methods and classifiers commonly used in the literature.

Reliable Visual Exploration System with Fault Tolerance Structure

Reliability of visual tracking and mapping is a challenging problem in robotics research, and it limits the promotion of vision-based mobile robot applications to a great extent. In this paper, we propose to improve the reliability of visual exploration in terms of its fault tolerance. Three modules are involved in our visual exploration system: visual localization and mapping, active controller and termination condition. High maintainability of mapping is obtained by the submap-based visual mapping module, persistent driving is achieved by a semantic segmentation based active controller, and robustness of re-localization is guaranteed by a novel completeness evaluation method in the termination condition. All the modules are integrated tightly for maintaining mapping and improving visual tracking. The system is verified with simulations and real world experiments, and all the solutions to fault tolerance are verified to overcome the failure conditions of visual tracking and mapping.

Comprehensive evaluation of static and dynamic obstacle detection method for safe navigation of the visually impaired people

Among the most challenging difficulties that hinder the visually impaired is a safe navigation from one place to another. We will provide a comprehensive framework to evaluate our smartphone-based obstacle detection method for safe navigation for the visually impaired people. This method aims to provide a safe navigation by detecting static and dynamic obstacles in unknown environments while offering maximum flexibility to the user and using the most cost-effective equipment possible. Our obstacle detection method relies on a multiregion analysis technique to examine the results of analyzing different regions of video frames. The user is notified about the existence of an obstacle by receiving alert messages. We (1) introduce a rich dataset, which is available for research purposes and can be used for the future evaluation and development of computer vision-based applications pertinent to static and dynamic obstacle detection in the unknown environment via smartphone. (The dataset is made available by King Abdulaziz University for research purposes only. Please, contact the authors for further information.) In addition, (2) we demonstrate the efficiency of the proposed method through a comprehensive framework of testing and evaluation in various complex and unknown scenes. The usability of the method has been tested by allowing a set of visually impaired people to use our system in their navigation. The experimental results show that our obstacle detection method is easy to use, operates efficiently, can be deployed on an average range smartphone device, and has a high accuracy rate ranging from 85% to 96%.

Review of Scene Text Detection and Recognition

Scene texts contain rich semantic information which may be used in many vision-based applications, and consequently detecting and recognizing scene texts have received increasing attention in recent years. In this paper, we first introduce the history and progress of scene text detection and recognition, and classify conventional methods in detail and point out their advantages as well as disadvantages. After that, we study these methods and illustrate the corresponding key issues and techniques, including loss function, multi-orientation, language model and sequence labeling. Finally, we describe commonly used benchmark datasets and evaluation protocols, based on which the performance of representative scene text detection and recognition methods are analyzed and compared.

Learning Robust Features for Planar Object Tracking

Visual tracking of planar objects with multiple degrees of freedom is a core component for numerous vision-based applications. Generally, since direct methods use all raw pixels in the region of interest to estimate the motion model directly, these methods are sensitive to illumination changes, partial occlusion, and motion blur. Recently, the deep convolutional network has demonstrated remarkable ability in visual tracking via learning robust deep features. In this paper, deep features are used for improving the robustness of direct methods. To learn suitable features in an end-to-end fashion, we employ a novel network architecture, the efficient second-order minimization (ESM) layer, which performs the ESM algorithm on deep feature maps. We train and validate the convolutional features on a synthetic dataset generated from the MS-COCO dataset and evaluate the tracking performance on two challenging, real-world datasets. The experimental results show that the proposed approach outperforms most state-of-the-art methods in various tracking challenges.

Remote sensing image fusion using fuzzy logic and gyrator transform

ABSTRACTImage fusion is a way to merge high spatial resolution image with low-resolution image, which is extensively utilized in several vision-based applications. The primary objective is to improve the radiometric quality of fused image compared to multi-spectral images with low resolution. Existing methods are found to be efficient, but if the similar radiometric information is blended into every image, it produces redundant high frequency of pixels. Therefore, to overcome this issue, in this paper a Fuzzy and Gyrator Transform (GT) based image fusion technique is proposed. It decomposes remotely sensed image into the GT domain; then it preserves the radiometric data by using fuzzy if-then rules. In the last phase, proposed technique injects high-frequency blocks from input images and returns a high-resolution multi-spectral image with maximum radiometric data. In this work, to efficiently tune the required fuzzification parameters we have utilized well-known Genetic algorithm. Genetic algorithm iteratively tune the required parameters for fuzzification in such a way that the overall entropy is maximized. Therefore, entropy is used as fitness function. Quantitative analysis has shown that the proposed method has better structural detail, spatial resolution, and spectral information than existing methods.

Brain tumor segmentation on Multimodal MRI scans using EMAP Algorithm.

The utilization of digital images is becoming popular in multiple areas such as clinical applications. There are multiple diagnostic and machine vision-based applications, where image processing plays a vital role in analyzing, interpreting, and solving the problem. Digital image processing techniques are used to increase the quality of images for human interpretation and machine perception. Tumor segmentation in brain magnetic resonance (MRI) volumes is considered as a complex task because of tumor shape, location, and texture. Manual segmentation is a time-consuming task that can be inaccurate due to an increasing volume of MR scanning performed. The goal of this research is to propose an automated method that can identify the whole tumor in each slice in volumetric MRI brain images, and find out the sub-tumor (core tumor, enhancing and non-enhancing) regions. The proposed algorithm is fully automated to segment out both high-grade glioma (HGG) and low-grade glioma (LGG), using the information provided by a sequence of MRI volumes. The designed algorithm does not require any training database and estimates the tumor regions independently using image processing techniques based on expectation maximization and K-mean clustering. The method is evaluated on BRATS 2015 dataset of LGG and HGG MR volumes. The average DICE score achieved by using the proposed technique is 0.92 and is comparable to state-of-the-art techniques which rely on computationally expensive algorithms.

Visual-based change detection in scene regions using statistical-based approaches

Detecting changes of the same scene taken at different time instances is crucial and demanding for medical, remote sensing, infrastructure, agriculture, and planogram compliance applications. We propose a statistical-based approach by exploiting the linear relationship. Initially, region of interest is identified using a graph-cut-based technique followed by geometrical alignment via area-based registration. To perform statistical correlation, we adopt features such as block-wise average coefficient value of the first level of the discrete wavelet transform (DWT-LL1) and the map obtained using hybrid saliency approaches. In the former approach, Pearson’s correlation measure is calculated for the DWT-LL1, and in the latter, PCC has been calculated using the saliency value. Change has been detected using optimal PCC value while minimizing the error rate. Experimental results on datasets reveal that saliency feature and DWT-LL1 perform better for normal and noise corrupted images, respectively. The efficiency of the proposed method is validated by user study with average mean opinion score of 70%. Hybrid saliency-based change detection gives 92.9% of correct classification and hence useful for the vision-based applications like damage detection in a car.

Patchmatch-Based Robust Stereo Matching Under Radiometric Changes.

In the real world, the two challenges of stereo vision system include a robust system under various radiometric changes and real-time process. To extract depth information from stereoscopic images, this paper proposes Patchmatch-based robust and fast stereo matching under radiometric changes. For this, a cost function was designed and minimized for estimating an accurate disparity map. Specifically, we used a prior probability to minimize the occlusion region and a smoothness term that considers convexity of objects to extract a fine disparity map. For evaluating the performance of the proposed scheme, we used Middlebury stereo data sets with radiometric changes. The experimental result showed that the proposed method outperforms state-of-the-art methods by up to 3.35 percent better and a range of 4.71 - 27.24 times faster result in terms of bad pixel error and processing time, respectively. Therefore, we believe that the proposed scheme can be a useful tool for computer vision-based applications.

Resource-Efficient Object-Recognition Coprocessor With Parallel Processing of Multiple Scan Windows in 65-nm CMOS

Object recognition offers a more general implementation for vision-based applications. This paper reports a resource-efficient recognition coprocessor with embedded cell-based simplified speeded up robust feature descriptor extraction unit and parallel scan-window (SW) recognition engine, applicable for various mobile scenarios and image sensor types. The feature extraction circuitry with pixel-based pipelined architecture describes the target objects among complex backgrounds, only relying on the pixel frequency from the image sensor. A cell-based SW algorithm enables parallelized recognition in multiple SWs and compatibility to different image sizes. The proposed hardware-friendly object-recognition coprocessor was implemented in 65-nm Silicon on thin BOX CMOS technology with 1.26 mm2 core area and can operate down to low supply voltage of 0.5 V. For video graphics array image sizes, the energy efficiency is determined as $910~\mu \text{J}$ per frame at 200 MHz and 1-V supply voltage. The coprocessor’s classification performance is demonstrated for pedestrian and car detection.

Human Detection and Segmentation Using Automatic Geodesic Active Contours for Vision Based Activity Recognition Applications

Vision-based activity recognition applications are in need of Automatic human detection and segmentation for surveillance purposes. Though there are many state-of-art methods in the literature, there still exist many challenges such as self-occlusion, illumination variations and sensitive to light conditions, appearance, and variations due to clothing. In this paper, a new novel framework for automatic detection and segmentation of the human region in a video sequence using Automatic Geodesic Active Contours is proposed. Normally geodesic active contours have static Region of coincidence but in this work, a dynamic Region of coincidence is proposed to draw the initial contours on the human region. To detect the human region the histogram of oriented gradients are computed and trained using SVM classifier. Once the human region is detected, the contour is drawn on the human region alone to segment the human region from the background. The proposed algorithm achieves an accuracy rate of 98%, 99% and 98% for KTH, Weizmann and U t-interaction dataset respectively.

Light invariant real-time robust hand gesture recognition

Computer vision has spread over different domains to facilitate difficult operations. It works as the artificial eye for many industrial applications to observe elements, process, automation and to find defects. Vision-based systems can also be applied to normal human life operations but changing light conditions is a big problem for these systems. Hand gesture recognition can be embedded with many existing interactive applications/games to make interaction natural and easy but changing illumination and non-uniform backgrounds make it very difficult to perform operations with good image segmentation. If a vision based system is installed in public domain, different people are supposed to work on the application.This paper demonstrates a light intensity invariant technique for hand gesture recognition which can be easily applied to other vision-based applications also. The technique has been tested on different people in different light conditions with the extreme change in intensity. This was done as one skin color looks different in changed light intensity and different skin colors may look same in changed light intensity. The orientation histogram was used to identify unique features of a hand gesture and it was compared using supervised ANN. The overall accuracy of 92.86% is achieved in extreme light intensity changing environments.

Joint upsampling of random color distance maps for fast salient region detection

The human visual system is capable of rapid response, even in the presence of massive quantities of visual information. This is possible because it restricts the operation of further processing stages to a small, potentially important, subset of the incoming information. This mechanism is called visual attention and is drawn by distinctive, visually salient, regions of the scene. Detection of visually salient regions is widely employed in vision-based applications, since a reduction in visual search space can lead to significant improvement in computational performance. Despite recent advances in salient region detection, most efforts have focused on improving accuracy, at the expense of increased execution time, significantly hindering their applicability. To address this, a fast and accurate salient region detection method is presented in this work, based on an efficient saliency estimate called random color distance map. This estimate is joint upsampled into an accurate saliency map, which is assessed and compared to saliency maps obtained by other four state-of-the-art methods on the MSRA1K, MSRA10K and SED2 datasets, showing that it is highly competitive in both accuracy and execution time.

Machine Vision-Based Measurement Systems for Fruit and Vegetable Quality Control in Postharvest.

Individual items of any agricultural commodity are different from each other in terms of colour, shape or size. Furthermore, as they are living thing, they change their quality attributes over time, thereby making the development of accurate automatic inspection machines a challenging task. Machine vision-based systems and new optical technologies make it feasible to create non-destructive control and monitoring tools for quality assessment to ensure adequate accomplishment of food standards. Such systems are much faster than any manual non-destructive examination of fruit and vegetable quality, thus allowing the whole production to be inspected with objective and repeatable criteria. Moreover, current technology makes it possible to inspect the fruit in spectral ranges beyond the sensibility of the human eye, for instance in the ultraviolet and near-infrared regions. Machine vision-based applications require the use of multiple technologies and knowledge, ranging from those related to image acquisition (illumination, cameras, etc.) to the development of algorithms for spectral image analysis. Machine vision-based systems for inspecting fruit and vegetables are targeted towards different purposes, from in-line sorting into commercial categories to the detection of contaminants or the distribution of specific chemical compounds on the product's surface. This chapter summarises the current state of the art in these techniques, starting with systems based on colour images for the inspection of conventional colour, shape or external defects and then goes on to consider recent developments in spectral image analysis for internal quality assessment or contaminant detection.