Open Source Computer Vision Library Research Articles

Generalized Omnipresence Detection

Identification and Tracking of a person in a video are useful in applications such as video surveillance. Two levels of tracking are carried out. They are Classification and monitoring of individuals. The human body’s color histogram is used as the basis for monitoring individuals. Our project can detect a human face in a video and store the detected facial features of the Local Binary Pattern Histogram (LBPH). In a video, once a person is detected, it automatically track that individual and assigns a label to that individual. We use the stored LBPH features to track him in any other videos. In this paper, we proposed and compared the efficiency of two algorithms. One constantly updates the background to make it suitable for illumination changes and other uses depth information with RGB. This is the first step in many complex algorithms in computer vision, such as identification of human activity and behavior recognition. The main challenges in human/object detection and tracking are changing illumination and background. Our work is based on image processing and also it learns the activities and stores them using machine learning with the help of OpenCV, an open source computer vision library.

GPU-Accelerated Vision for Robots: Improving System Throughput Using OpenCV and CUDA

OpenCV is an open source computer vision and machine learning library for C/C++/Python available for Windows, Linux, macOS, and Android platforms. It contains low-level image processing functions as well as high-level algorithms such as object identification, face recognition, and action classification in videos. OpenCV has become very popular, with more than 47,000 people in its user community and 18 million downloads (see https://opencv.org/about/). Under a Berkeley Software Distribution (BSD) license, it can be used for both academic and commercial applications.

OpenCV Basics: A Mobile Application to Support the Teaching of Computer Vision Concepts

Contribution: Open Source Computer Vision Library (OpenCV) Basics is an application designed with the purpose of facilitating the initiation of industrial engineering students in the field of Computer Vision, making the learning process easier, more dynamic and more direct. To this end, an application has been developed for the Android operating system with which users can make use of a wide variety of algorithms available in the OpenCV library. Background: Teaching topics related to Computer Vision can rely on the use of new technologies such as mobile applications. With this type of support, students can learn concepts that might otherwise be difficult to understand. Intended Outcomes: The objective is to facilitate the assimilation of concepts related to Computer Vision by taking advantage of the camera and the processing power of a mobile device to observe in real time the effects produced on an image by many of the image processing algorithms included in OpenCV. This application is currently available to be downloaded for free through the Google Play Store so that anyone interested in the field of Computer Vision can make use of it. Application Design: The proposed approach introduces students to concepts related to Computer Vision by making use of the developed application, complementing the theoretical contents taught by the teacher with specific examples. Findings: The degree of satisfaction of OpenCV Basics users has been evaluated within the framework of the course advanced robotized systems, taught in the industrial engineering degree at the University of La Laguna.

SUBSTANTIATION OF PERSPECTIVES OF USING THE LIBRARY OF COMPUTER VISION OpenCV IN THE DEVELOPMENT OF SOFTWARE FOR PROCESSING VIDEO INFORMATION TESTS

The article highlights the practicability of developing our own software for processing video information obtained during testing of weapons and military equipment (WME) specimens. To obtain measurement information of processes which are distributed in space and time, namely: motion trajectories, velocity, acceleration of the object, propagation of gun muzzle blast ejection, dynamics of explosion of ammunition of various purposes.
 The basic operations of video information processing were outlined:
 1) perspective correction;
 2) compensation (correction) of distortion (optical effect "fish eye”);
 3) scale determination of the video material’s frame;
 4) conducting of static measurements;
 5) position determination of the image object in a sequence of video stream’s frames.
 A brief overview of major open-access computer libraries (Accord.NET, VXL (Vision-something-Libraries), OpenCV (Open Source Computer Vision Library)) were conducted.
 To demonstrate how the above operations were performed by the OpenCV library, examples of the results of using it in Kinovea program were presented.
 An analysis of the popularity of computer vision libraries was conducted, which made it possible to assess the prospects for their further development and information support.
 The rationale for using OpenCV in software was fulfilled, that can be developed to process the video information obtained during testing of weapons and military equipment specimens.

On development of models and algorithms for automated metallographic measurement of visible metal slice grain sizes

The article proposes an approach to metallographic research based on solving the problem of semantic segmentation using a trained neural network classifier. To solve the problem of isolating grains in micrographs of longitudinal and transverse slices of metal, the neural network model U-Net was adapted. In order to obtain closed image contours, a post-processing algorithm was developed using the OpenCV open source computer vision library. The article describes the training of a neural network and the conversion of its results, as well as the comparative analysis of the histograms between the reference grain area distribution and the distribution obtained using the proposed algorithm.

Computer Vision-based Reader for analogue Energy/Water Meters in low-cost embedded System: a Case Study in an Office Building in Scotland

Implementation of cost-effective energy conservation measures (ECMs) is expected to generate up to 18% of carbon emissions reductions in office buildings. In order to determine adequate ECMs for a specific building, operational data is required. However, buildings generally lack operational data in the form of time series that can limit a breath of analysis required for determining adequate ECMs. Energy time-series data is commonly lacking in the UK due to uneven availability of smart meters (heat, gas, water), security restrictions in Energy Information Systems (EIS) and building management systems (BMS), restrictions and costs associated for automated reporting from utility companies, etc. This work presents a non-intrusive computer vision-based reader to generate energy readings at 10-minute resolution using a Raspberry-Pi, a traditional webcam and an LED light. OpenCV, an open source computer vision library, is used to detect and interpret numeric values from a heat meter, which are in turn uploaded to a cloud-based energy platform to create a complete operational data set enabling detailed analytics, fault detection and diagnostics (FDD) and model calibration. A case study of an office building in Scotland is presented. The building has a heat meter with no remote access capabilities. The accuracy of the method, i.e. the ability of the script to accurately derive the rate of change between readings, resulted on a 92% percent during a test done for 100 samples. Recommendations for accuracy improvements are included in the conclusions.

КОМП’ЮТЕРНА ДІАГНОСТИКА ДЕФОРМАЦІЙ ЗВОДІВ СТОПИ

Today, about 60-70% of Ukrainians have some degree of flatfoot. Timely diagnosis deformity of the arches of the foot allows the doctor to take a number of rehabilitation actions that will stop the progression of this pathology. The main method for diagnosing deformities of the arches of the foot is to evaluate a plantogram obtained using a simple mechanical plantographer, which takes a lot of time and does not exclude the human factor. Therefore, it is advisable to develop a program that will analyze the contour of the fit of the foot on the scan - image, calculating the index, with which you can later determine the degree of flatfoot. For development, Visual Studio 2017 programming environment, an open-source computer vision library Open CV, implemented in C ++, was chosen. As a result of the work, a program has been developed to search for the contour and control points, display the results in a separate window, register patient data in the database, if the algorithm does not work correctly, it is possible to manually point the image and measure the distance between them

Multi-Variant Accuracy Evaluation of UAV Imaging Surveys: A Case Study on Investment Area.

The main focus of the presented study is a multi-variant accuracy assessment of a photogrammetric 2D and 3D data collection, whose accuracy meets the appropriate technical requirements, based on the block of 858 digital images (4.6 cm ground sample distance) acquired by Trimble® UX5 unmanned aircraft system equipped with Sony NEX-5T compact system camera. All 1418 well-defined ground control and check points were a posteriori measured applying Global Navigation Satellite Systems (GNSS) using the real-time network method. High accuracy of photogrammetric products was obtained by the computations performed according to the proposed methodology, which assumes multi-variant images processing and extended error analysis. The detection of blurred images was preprocessed applying Laplacian operator and Fourier transform implemented in Python using the Open Source Computer Vision library. The data collection was performed in Pix4Dmapper suite supported by additional software: in the bundle block adjustment (results verified using RealityCapure and PhotoScan applications), on the digital surface model (CloudCompare), and georeferenced orthomosaic in GeoTIFF format (AutoCAD Civil 3D). The study proved the high accuracy and significant statistical reliability of unmanned aerial vehicle (UAV) imaging 2D and 3D surveys. The accuracy fulfills Polish and US technical requirements of planimetric and vertical accuracy (root mean square error less than or equal to 0.10 m and 0.05 m).

Research on Image Navigation Algorithms for UAV

UAV (unmanned aerial vehicle) is used in agriculture filed to inspect water conservancy facilities, it not only can greatly reduce labor, but also effectively save relevant costs. The choice of UAV navigation mode is the key problem to achieve patrol of water conservancy facilities. In order to solve this problem, a new method of UAV image navigation is proposed which is different from traditional image navigation, and experimental research is carried out. Firstly, the acquired image is corrected and preprocessed to reduce image noise by perspective transformation. Then, the target information is extracted by threshold segmentation and ROI (region of interest) method. Finally, the attitude information of UAV is calculated and displayed in the original image through OpenCV (Open Source Computer Vision Library) library function. Based on the calculation results, the attitude of UAV can be adjusted by flight controller.

Development of autonomous target recognition and scanning technology for pulse-echo ultrasonic propagation imager

This article proposes a pulse-echo ultrasonic propagation imaging system that is capable of autonomous target recognition and scanning an area of a customizable shape for the non-destructive evaluation of structural defects. The proposed system employs through-the-thickness bulk waves for ultrasonic inspection, which is achieved by joining two laser beams: one each for the ultrasonic wave generation and sensing. Moreover, the system is capable of autonomously suggesting a suitable inspection area for a specimen placed in front of the scanning head. The scan area delimitation algorithm uses the specimen image and ascertains the specimen border by means of edge and contour detection operations, following which a scan area closely conforming to the specimen boundary is suggested. The system can scan an area of any arbitrary shape, thereby preventing any wasteful operations that may result from fixed shape (rectangular or square) scanning. A Q-switched laser is used for generating the aforementioned ultrasonic waves, while a laser Doppler vibrometer is used for sensing these signals. A dual-axis automated translation stage is applied for raster scanning of the specimen, and a data acquisition card is employed for taking measurements. A camera mounted on the scan head is used for imaging the specimen for the scan area detection. Graphical user interface software controls all the individual blocks of the system, while implementing the required image processing, scan area detection, signal acquisition, signal processing, and result display. The graphical user interface is created in C++ using the Qt framework. Moreover, Qt Widgets for Technical Applications is used for the result display, and the Open Source Computer Vision Library is employed for the implementation of basic image processing algorithms. Multi-threading is used for real-time updating of the scan results while the scanning is ongoing.

Research on key issues of gesture recognition for artificial intelligence

Gesture recognition has become a hot spot in the direction of artificial intelligence and has great research significance. At present, some classical algorithms, such as the neural network method and the hidden Markov method, have the disadvantages of large computational complexity and long training time. This paper proposes the support vector machine (SVM) algorithm to realize gesture recognition. In order to make the recognition more accurate, SVM is combined with the principal component analysis (PCA) algorithm, performs the dimensionality reduction on the gesture image to form the PCA + SVM algorithm for gesture recognition. At the same time, a new dynamic gesture recognition processing method is proposed, and its effectiveness is proved by various methods. Using open-source computer vision library (OPENCV), the algorithm is simulated on visual studio 2015 environment. The results show that the algorithm has an excellent recognition effect.

A Low-Cost Portable Dynamic Droplet Sensing System for Digital Microfluidics Applications

A simple and low-cost open electrowetting on dielectric (EWOD)-based dynamic droplet sensing system has been designed and fabricated to measure the droplet parameters. The system has the capability of monitoring and controlling the droplet position by using electrical and optical methods. The droplet position is measured electrically by sensing the capacitance between the bottom electrode and top ground wire with a capacitance input range of ±8 pF and resolution of 24 bit. A high-voltage control unit (5–400 VDC) is designed for actuation of voltage on different electrode pads. The printed circuit board (PCB) technology is used for the cost-effective open EWOD device fabrication, whereas polypropylene (PP) is used as a dielectric as well as a hydrophobic layer. Open source computer vision (OpenCV) libraries are used in the optical method for processing of the droplet images. The speed of droplet movement at 300 VDC is 7.1 mm/s. Because of its portability, automation, and low cost, the dynamic sensing system has the potential to be useful in many of the existing lab-on-chip (LOC) applications.

OpenCV 라이브러리 기반 FSK OCC 시스템의 성능 측정

With the development of semiconductor especially, nano technology, image processing is playing an important role in the multimedia display industry. It is also one of the most considered research topic which has been combined with the industrial production quickly with the on-chip integration. As an important step in optical camera communication (OCC) operation, image processing makes decision for the reliable system. There are some open-source computer vision libraries for image processing applications. One of the earliest libraries to be considered by many researchers is OpenCV. In this paper, we will propose a new OCC system based on frequency shift OOK modulation and evaluate the performance of proposed system based OpenCV library for a real time application on android smartphone scenario. From the measurement on consumption time, accuracy for every phase, we make a comparison with other implements to prove the efficiency of OpenCV library.

Robotic ARM using Computer Vision

A Bot which pursues Human hand developments. Its unlimited authority lies with the client and doesn't have any knowledge of its own. Programmed robots having man-made brainpower are a danger to society and may cause hurt in certain situations. Subsequently, having full oversight over the robot is a protected method to work with such robots. In this paper, we have proposed a comparable arrangement of a robot. catching pictures from the PC web cam progressively condition and procedure them as we are required. By utilizing open source Computer vision library (OpenCV for short), a picture can be caught on the basis of its Hue saturation value (HSV) extend. The fundamental library capacities for picture dealing with and handling are utilized. Fundamental library capacities are utilized for stacking a picture, making windows to hold picture at run time, sparing pictures, and to separate pictures dependent on their shading values. I have additionally connected capacity to edge the yield picture so as to diminish the twisting in it. While handling, the pictures are changed over from their essential plain Red, Green, and Blue (RGB) to an increasingly reasonable one that is HSV.

Image Analysis and Processing of Skin Cell Injury Based on OpenCV

OpenCV is an open source computer vision and machine learning software library that provides a common infrastructure for computer vision applications and accelerates their use of machine awareness in commercial products[1].This paper uses the visual processing algorithm provided by OpenCV to analyze and process the image of skin cells in detail, and quantitatively determine the percentage of damaged cells in the image. First, dilation, opening operation, closing operation, coarsening, refinement, skeleton, tailoring, boundary extraction, holes filling, connected components and so on of a variety of morphological methods are used in the process of corrosion for the graphics pretreatment. Then the watershed algorithm is adopted to image segmentation. This paper expounds the ridge algorithm to water in detail. Finally, in order to determine the proportion of damaged cells in the segmented cells, the polygonal fitting method in graphic image processing was creatively used, and the polygonal fitting method was described in detail, which successfully solved the task of the research topic.

A New Native Video Filtering based on OpenGL ES for Mobile Platform

In the last five years, there have been many Android applications implementing video filter or video effect as an excellent feature. Open CV is an open source computer vision library that can be simply and easily used for video filtering in Android application. However, using OpenCV library for video filtering commonly yields a bigger size of Android application. The concept of “Develop for Billion People” has enforced the developers to optimize the size of their applications to preserve resources and size of memory—as not all Android devices come with sufficiently large memory. On the other hand, OpenGL ES does not burden the filtering process because of its smaller size when it is implemented during the application development. In this research, we present a new native video processing technique using OpenGL ES. We implement the proposed method on a native video file without decreasing its quality before video filtering process. The experiments were conducted with five different mobile devices. We compared several metrics including: quality of the resulted video, file size of the apk, power consumption, and memory usage. Based on the experimental results, OpenGL ES produces smaller file size of apk (2 MB) compared with the produced file size of apk by Open CV (20MB). The resulted file after video filtering possesses same properties as observed before video filtering. Additionally, OpenGL ES uses more efficient power with 0.1965 mAh, while OpenCV consumes 0.283 mAh. Finally, video filtering with OpenGL ES uses 29.3% lesser memory than video filtering with OpenCV. The proposed method is proven to be more appropriate with “Develop for Billion People” as it preserves more computational resources compared with the existing video filtering technique in Android.

Effects of Varying Noise Levels and Lighting Levels on Multimodal Speech and Visual Gesture Interaction with Aerobots

This paper investigated the effects of varying noise levels and varying lighting levels on speech and gesture control command interfaces for aerobots. The aim was to determine the practical suitability of the multimodal combination of speech and visual gesture in human aerobotic interaction, by investigating the limits and feasibility of use of the individual components. In order to determine this, a custom multimodal speech and visual gesture interface was developed using CMU (Carnegie Mellon University) sphinx and OpenCV (Open source Computer Vision) libraries, respectively. An experiment study was designed to measure the individual effects of each of the two main components of speech and gesture, and 37 participants were recruited to participate in the experiment. The ambient noise level was varied from 55 dB to 85 dB. The ambient lighting level was varied from 10 Lux to 1400 Lux, under different lighting colour temperature mixtures of yellow (3500 K) and white (5500 K), and different background for capturing the finger gestures. The results of the experiment, which consisted of around 3108 speech utterance and 999 gesture quality observations, were presented and discussed. It was observed that speech recognition accuracy/success rate falls as noise levels rise, with 75 dB noise level being the aerobot’s practical application limit, as the speech control interaction becomes very unreliable due to poor recognition beyond this. It was concluded that multi-word speech commands were considered more reliable and effective than single-word speech commands. In addition, some speech command words (e.g., land) were more noise resistant than others (e.g., hover) at higher noise levels, due to their articulation. From the results of the gesture-lighting experiment, the effects of both lighting conditions and the environment background on the quality of gesture recognition, was almost insignificant, less than 0.5%. The implication of this is that other factors such as the gesture capture system design and technology (camera and computer hardware), type of gesture being captured (upper body, whole body, hand, fingers, or facial gestures), and the image processing technique (gesture classification algorithms), are more important in developing a successful gesture recognition system. Some further works were suggested based on the conclusions drawn from this findings which included using alternative ASR (Automatic Speech Recognition) speech models and developing more robust gesture recognition algorithm.

SOFTWARE AND HARDWARE AUTOMATED SYSTEM OF CASTS DEFECTS NON-DESTRUCTIVE MONITORING

Introduction of the “Automated system for operational control of casts production (OCCP AS)” makes the basis of an integrated automated production control system (APCS). It performs three main tasks: control and recording (production, products, materials, etc.), improving quality of casts and operational management of technological processes. Solution of these tasks was accomplished through automating data collection in real time for all production operations, recording material flows, creating operational communication channels, as well as centralized collection, processing and representation of data by the process information server. The next step in building an effective automated control system is to stabilize product quality in changing external conditions, for example, quality of materials, and to optimize production (technology change in order to reduce costs for constant or higher product quality). The second stage is based on mathematical processing and analysis of data coming from OCCP AS, it allows to determine optimal ranges of parameters of technological processes – “Automated system for optimization and analysis of production progress (OAPP AS)”. OAPP AS consists of two subsystems: quality analysis and technology management. The first solves the problem of data analysis and modeling, the second – calculation of real-time optimal process parameters and real time prediction. The stages tasks compete for access to different hardware resources. The most critical parameter for OCCP AS is performance of server disk arrays, for OAPP AS it is processor performance. In either case, system scaling is effectively solved by parallelizing operations across different servers, forming a cluster, and across different processors (cores) on the same server. To process defect images and to obtain cause-and-effect characteristics, you can use OpenCV software package, which is an open source computer vision library. In course of processing, Sobel operator, Gauss filter and binarization were used. They are based on processing pixels using matrices. Operations on pixels are independent and can be performed in parallel. The task of clustering is reduced to definition of an expert method or using various mathematical algorithms for defects belonging to a specific cluster (data block) through a set of values of dependent factors. Thus, data blocks are formed by the criterion of the defect cause. Calculation of a data block to which a product defect belongs can be very resource-intensive operation. To increase efficiency of image recognition systems and parallelization ofsearch operations, it makes sense to place data clusters on different servers. As a result, there is a need for a distributed database. This is a special class of DBMS, which requires appropriate software. Generation of OAPPAS based on a multi-node cluster with ApacheCassandra DBMS installed and using Nvidia video cards supporting CUDA technology on each node will be the cheapest and most effective solution. Video card is selected based on required number of graphics processors on the node.

Motion Control System of Unmanned Railcars Based on Image Recognition

The main purpose of this paper is to construct an autopilot system for unmanned railcars based on computer vision technology in a fixed luminous environment. Four graphic predefined signs of different colors and shapes serve as motion commands of acceleration, deceleration, reverse and stop for the motion control system of railcars based on image recognition. The predefined signs’ strong classifiers were trained based on Haar-like feature training and AdaBoosting from Open Source Computer Vision Library (OpenCV). Comprehensive system integrations such as hardware, device drives, protocols, an application program in Python and man machine interface have been properly done. The objectives of this research include: (1) Verifying the feasibility of graphic predefined signs serving as commands of a motion control system of railcars with computer vision through experiments; (2) Providing reliable solutions for motion control of unmanned railcars, based on image recognition at affordable cost. The experiment results successfully verify the proposed methodology and integrated system. In the main program, every predefined sign must be detected at least three times in consecutive images within 0.2 s before the system confirms the detection. This digital filter like feature can filter out false detections and make the correct rate of detections close to 100%. After detecting a predefined sign, it was observed that the system could generate new motion commands to drive the railcars within 0.3 s. Therefore, both real time performance and the precision of the system are good. Since the sensing and control devices of the proposed system consist of computer, camera and predefined signs only, both the implementation and maintenance costs are very low. In addition, the proposed system is immune to electromagnetic interference, so it is ideal to merge into popular radio Communication Based Train Control (CBTC) systems in railways to improve the safety of operations.

Mobile system for road sign detection and recognition with template matching

This paper explores the effective approach to road sign detection and recognition based on mobile devices. Detecting and recognising road signs is a challenging matter because of different shapes, complex background and irregular sign illumination. The main goal of the system is to assist drivers by warning them about the existence of road signs to increase safety during driving. In this paper, the system for detection and recognition of road signs was implemented and tested with the use of Open Source Computer Vision Library (OpenCV). The system consists of two parts. The first part is the detection stage, which is used to detect the signs from the whole image frame and includes the modules: data-image acquisition, image pre-processing and sign detection. During this stage, the impact of Canny edge detector and Hough transform parameters on the quality-level of sign detection was tested. The second part is the recognition stage, whose role is to match the detected object with a priori models of signs in the dataset. In the research, the authors also compared the influence of various image processing algorithms parameters to the time of road sign recognition. The discussion part answers also the question whether the mobile system (smartphone) is robust enough to detect and recognise road sings in real time.