Vision Techniques Research Articles

Intelligent Infrastructure for Traffic Monitoring Based on Deep Learning and Edge Computing

In the field of traffic management and control systems, we are witnessing a symbiotic evolution, where intelligent infrastructure is progressively collaborating with smart vehicles to produce benefits for traffic monitoring and security, by rapidly identifying hazardous behaviours. This exponential growth is due to the rapid development of deep learning in recent years, as well as the improvements in computer vision models. These technologies allow for monitoring tasks without the need to install numerous sensors or stop the traffic, using the extensive camera network of surveillance cameras already present in worldwide roads. This study proposes a computer vision-based solution that allows for real-time processing of video streams through edge computing devices, eliminating the need for Internet connectivity or dedicated sensors. The proposed system employs deep learning algorithms and vision techniques that perform vehicle detection, classification, tracking, speed estimation, and vehicle geolocation.

Rocking of single-layer armour units measured by embedded sensors

Single layer randomly placed armour units are used in many rubble mound breakwaters around the world. For these armour layers first extraction of units starts at high loads and can then progress quickly. Before the first extraction of a unit, typically no quantitative description of damage can be given. But additional to extraction, breakage of armour units due to rocking could be a major damage mechanism. This paper treat novel embedded Rocking Sensors. The technique is used to obtain the first measurements of rocking-impact velocities of single-layer units. They are also the first tests where the instrumented units can naturally move with the compacting layer during storm build-up. Physical model tests were performed on an armour layer with XBloc units. With 8 to 10 instrumented units per test run, in total 640 single measurements of the rocking motion of a unit during a 1200 wave test run were obtained, for three water levels and five wave heights. From the Rocking Sensors the number of impacts and rotational impact velocities were obtained. From an image analysis the along-slope settlement of the units during the tests was quantified. The rotational motion expressed by was found to be most convenient to express the motion. It can be seen that the units in the armour layer rock much more often than visually observed. Settlement seems to be a continuous progress, with most units rocking intermittently. Highest impact velocities are seen to occur around the water line, and in the uprush phase of the waves. A maximum impact velocity for all tests of 0.34 m/s (model scale) was measured. A preliminary design expression for rocking impact velocities of single layer units (Xblocs) is given. The paper shows that novel measurement techniques like the Rocking Sensors and vision techniques can and should be used to quantify damage mechanisms to rubble mound single-layer armour, additional to counting the extracted number of intact units.

Analyzing lower half facial gestures for lip reading applications: Survey on vision techniques

Lip reading has gained popularity due to the proliferation of emerging real-world applications. This article provides a comprehensive review of benchmark datasets available for lip-reading applications and pioneering works that analyze lower facial cues for lip-reading applications. A comprehensive review of lip reading applications is broadly classified into five distinct applications: Lip Reading Biometrics (LRB), Audio Visual Speech Recognition (AVSR), Silent Speech Recognition (SSR), Voice from Lips, and Lip HCI (Human–computer interaction). LRB entails extensive research in the fields of authentication and liveness detection. AVSR covers key findings that have contributed significantly to applications such as voice assistants, video-to-text transcription, hearing aids, and pronunciation-correcting systems. SSR analyzes the efforts made for silent-video-to-text transcription and surveillance camera applications. The voice from lips section discusses applications such as voice for the voiceless and vision-infused speech inpainting. In lip HCI, LR-HCI for smartphones, smart TVs, computers, robots, and musical instruments is reviewed in detail. Comprehensive coverage is given to cutting-edge techniques in computer vision, signal processing, machine learning, and deep learning. The advancements that aid the system in learning to lip-read and authenticate lip gestures, generate text transcription, synthesize voice based on lip movements, and control systems via lip movements (lip HCI) are covered. The work concludes by highlighting the limitations of existing frameworks, the road maps of each application illustrating the evolution of techniques employed over time, and future research avenues in lip-reading applications.

Real-time pedestrian pose estimation, tracking and localization for social distancing.

The corona virus pandemic has introduced limitations which were previously not a cause for concern. Chief among them are wearing face masks in public and constraints on the physical distance between people as an effective measure to reduce the virus spread. Visual surveillance systems, which are common in urban environments and initially commissioned for security surveillance, can be re-purposed to help limit the spread of COVID-19 and prevent future pandemics. In this work, we propose a novel integration technique for real-time pose estimation and multiple human tracking in a pedestrian setting, primarily for social distancing, using CCTV camera footage. Our technique promises a sizeable increase in processing speed and improved detection in very low-resolution scenarios. Using existing surveillance systems, pedestrian pose estimation, tracking and localization for social distancing (PETL4SD) is proposed for measuring social distancing, which combines the output of multiple neural networks aided with fundamental 2D/3D vision techniques. We leverage state-of-the-art object and pose estimation algorithms, combining their strengths, for increase in speed and improvement in detections. These detections are then tracked using a bespoke version of the FASTMOT algorithm. Temporal and analogous estimation techniques are used to deal with occlusions when estimating posture. Projective geometry along with the aforementioned posture tracking is then used to localize the pedestrians. Inter-personal distances are calculated and locally inspected to detect possible violations of the social distancing rules. Furthermore, a "smart violations detector" is employed which estimates if people are together based on their current actions and eliminates false social distancing violations within groups. Finally, distances are intuitively visualized with the right perspective. All implementation is in real time and is performed on Python. Experimental results are provided to validate our proposed method quantitatively and qualitatively on public domain datasets using only a single CCTV camera feed as input. Our results show our technique to outperform the baseline in speed and accuracy in low-resolution scenarios. The code of this work will be made publicly available on GitHub at https://github.com/bilalze/PETL4SD.

Sewer defect instance segmentation, localization, and 3D reconstruction for sewer floating capsule robots

Most automated sewer inspection tasks are based on closed-circuit television (CCTV) methods and focus on image classification or object detection but fail to obtain information on fine-grained sewer defects. Targeting the sewer defect detection from a video and performing the sewer inspection in a lightweight, low-cost, and practical manner, this study develops a sewer detection framework based on computer vision technology for the sewer floating capsule robot. The proposed framework performs three main sub-tasks, including instance segmentation, real-time sewer inspection device localization, and real three-dimensional (3D) model reconstruction, to realize sewer defect detection. An improved mask regional convolutional neural network (Mask RCNN), which integrates split attention module and balanced L1 loss module, is proposed for robust feature representation. In addition, an improved data augmentation method developed according to the sewer defect instance segmentation tasks is introduced. The 3D reconstruction and real-time localization of a sewer scene are achieved using the structure-from-motion techniques for the sewer floating capsule robot. Extensive experiments are conducted, and the experimental results show that the proposed method is effective and robust. The average precision of instance segmentation at the intersection of the union value of 0.5 is 92.7%, and the maximum 3D model measurement error is 1 m. However, video sequence information and multi-sensor fusion, combining the inertial measurement unit and vision technique, could be studied in the future to achieve better generalization and robust results.

Forensic palynology: computer vision and geotechnologies to support criminalistics expertise

Pollen grains can provide valuable information to forensic palynology, such as the time of death or the possible origin of a corpse. Forensic Palynology is a vital tool to be used in a criminal investigation because the different environment has distinct pollen signatures. Brazil has a rich and diversified flora that is suitable for the application of forensic palynology. The purpose of this research is to introduce palynology automation as a tool to improve the investigative method in forensic palynology and apply it to forensic palynology automation. The studied city has different vegetation types, in which we performed assessments to identify its correspondent pollen types. PALINOVIC algorithm was developed using computer vision and geotechnology techniques. Our results show that it is possible to correlate pollen grains found in forensic samples by automatic pollen identification and with a mapping of the likely vegetation. Our results show that it is possible to relate the presence of pollen grains found in forensic samples through the automatic identification of images together with a database of georeferenced plant species. It was possible to analyze the pollen grains collected in eight bodies, where the algorithm presented a performance of 90.51% in the pollen grain classification tests. Furthermore, pollen grains could be correlated with the type of vegetation where the body was found. Thus, the technique developed can be applied in other urban centers from a previous georeferencing of plants, as well as a pollen database.

EARLY PEST DETECTION FROM CROP USING IMAGE PROCESSING AND COMPUTATIONAL INTELLIGENCE

Early pest detection is a major challenge in agriculture field. The easiest way, to control the pest infection is the use of pesticides. But the excessive use of pesticides are harmful to plants, animals as well as human beings. Integrated pest management combines biological and physical methods to prevent pest infection. The techniques of machine vision and digital image Processing are extensively applied to agricultural science and it have great perspective especially in the plant protection field, which ultimately leads to crops management. This paper deals with a new type of early detection of pest’s system. Images of the leaves affected by pests are acquired by using a digital camera. The leaves with pest images are processed for getting a gray colored image and then using feature extraction, image classification techniques to detect pests on leaves. The images are acquired by using a digital camera. The images are then transferred to a PC and represented in python software. The RGB image is then converted into gray scale image and the feature extraction techniques are applied on that image. The Support Vector Machine classifier is used to classify the pest types. Keywords- Parking Slot, Deep Learning, Automated Parking, CNN, Mask R-CNN, YOLO, Image Processing

Evaluation of modal identification under base motion excitation using vision techniques

In certain situations, employing a movable base acting as the excitation of a mechanical system is the best or even the only way to determine the response model for modal analysis. However, the obtained transmissibility functions must be modified prior to modal identification with a conventional procedure based on frequency response functions. Moreover, when employing vision techniques, the response curves are noisier and even poorly defined as the sensitivity is significantly lower than traditional sensors. Using the right model for curve-fitting is particularly relevant in this case. The current study performs an analysis of the adaptation of transmissibility functions, obtained by a vision technique, to improve the accuracy of the modal data estimation with conventional procedures. Two sets of transmissibility functions were evaluated: the originally obtained in the experiment, and the adapted one. After modal identification, significant differences were found concerning mode shapes and curve synthesis. The adaptation improved the accuracy of the identification in all the measurement points, proved by statistical indicators of the curve-fitting procedure like the correlation coefficient and the error between the synthesised and the experimental curves.

Vision-based estimation of the number of occupants using video cameras

Although occupancy information is critical to energy consumption of existing buildings, it still remains to be a major source of uncertainty. For reliable and accurate occupant modeling with minimal uncertainties, capturing precise occupant information on occupants is essential. This paper proposes a computer vision-based approach that utilizes deep learning architectures to estimate of the number of people in large, crowded spaces using multiple cameras. Various vision techniques (head detection, background elimination, head tracking) are implemented in three methods: (i) a method that instantaneously counts people in a scene, (ii) a method that incrementally counts people entering/exiting a room and (iii) a combination of the first two methods. These methods were applied in a classroom with heavy occlusions, and resulted in a high prediction capacity when compared to ground truth measurements. Future work in video-analytical approaches can address problems regarding lowering the computational cost of analysis, capturing occupancy data in complex room geometries and addressing concerns in privacy preservation.

Forest Fire Detection using Machine Learning with Raspberry PI

Laptop Vision primarily based fireplace detection utilize image process has the potential to be helpful in conditions during which standard ways cannot be adopted. the fire detection rule uses visual characteristics of fires like brightness, colour, spectral texture, spectral flicker, and edge trembling to discriminate them from different visible stimuli. There are numerous varied fireplace detection techniques like infrared device, a thermal detector, smoke detector, flame detector, and optical smoke detector. These techniques aren't constantly reliable as they are doing not perpetually notice {the fireplace the fire |the fireplace} itself however notice one or additional phenomena ensuing from fire, like smoke, heat, infrared, UV radiation or gas, that might be created in different ways and hence, produces several false alarms. By the assistance of laptop vision and image process techniques, it's achievable to get higher results than standard systems as a result of pictures will offer additional reliable data.

Accurate visual localization with semantic masking and attention

Visual localization is the task of accurate camera pose estimation within a scene and is a crucial technique for computer vision and robotics. Among the various approaches, relative pose estimation has gained increasing interest because it can generalize to new scenes. This approach learns to regress relative pose between image pairs. However, unreliable regions that contain objects such as the sky, persons, or moving cars are often present in real images, causing noise and interference to localization. In this paper, we propose a novel relative pose estimation pipeline to address the problem. The pipeline features a semantic masking module and an attention module. The two modules help suppress interfering information from unreliable regions, while at the same time emphasizing important features with an attention mechanism. Experiment results show that our framework outperforms alternative methods in the accuracy of camera pose prediction in all scenes.

Virtual Spy (Automating the Role of Third Umpire in the Game of Gully Cricket)

A fair decision is important in any game to give justice to the game. Any wrong decision due to a person’s misunderstanding can be caused by the outcome of the game. Computer vision and image processing techniques have been cited in book reviews that have used multiple cameras to display. This paper focuses on a system that helps the umpire to make decisions such as no ball, Running, out, etc. with the help of a good quality smartphone camera. The Decision Review System (DRS) aims to provide decisions such as run-out and stump-out. Tkinter is used to develop the GUI of DRS. Object classification and object recognition is implemented using Histogram of Gradients (HOG) and Support Vector Machine (SVM). To detect the cricket ball from the video we optimized and used frame subtraction, contour detection and minimum enclosing circle algorithms using OpenCV library. Linear regression and quadratic regression are used to track and predict the motion of the ball from video source. VPython is used for the visual representation.

Automated Textile Circuit Generation using Machine Vision and Embroidery Technique

An automated textile circuit generation method was developed using machine vision and embroidery technique. For this, an image acquisition device was designed to capture the physical layout image of electronic devices on the fabric. A series of image analysis algorithms were developed to calibrate the image and recognize the type, location, and orientation of each device from the image. Dedicated computer-aided design software was developed that can design a circuit and modify it according to the actual device layout on the fabric. It can also convert the circuit elements such as conductive path and insulation layer into data for a computerized embroidery machine. An example circuit consisting of seven electronic devices was designed and repeatedly produced to verify the developed method. The method has shown promising results.

Метод гауссова обучения нейронной сети для решения задачи распознавания многоспектральных спутниковых изображений

The methods of Gaussian computer vision and machine learning used to process multispectral satellite images in the detection and recognition of research objects --- banana plantations in the area of El Porvenir on the coast of Ecuador --- are considered. Using computer vision and machine learning techniques, processing of multispectral satellite images and indirect raster data after calculating NDVI and NDWI vegetation indices, classification of vegetation and water objects was performed. The analysis of multispectral satellite image data is complicated by the presence of clouds and rasters that do not contain data other than the object of study. The machine learning method when labeling data with validation by vegetation indices is used in segmentation of classes that do not participate in the computer vision analysis in the detection and recognition of the research object. The probability of the given positive experimental results is ~ 96 %. The proposed methods of objects detection and recognition on multispectral satellite images can be implemented in agricultural systems to improve, analyze and assess the needs of crops, which are part of agriculture as the primary sector of the economy, to meet the needs of food and raw materials production. Thus, with the increased demand for agricultural products, it is necessary to introduce technologies that guarantee the quality and high productivity of agricultural machinery

Finite Element Analysis and Experimental Investigation of Cut Surface Formation of Magnetic Silicon Steel in Shear Cutting.

Shear cutting allows for shaping materials with any length of cutting line with high efficiency and without negative thermal effects, but it causes stresses and deformations in the cutting zone of the material. This has a negative effect on the magnetic properties of the sheet in the areas of the cut edge. The main problem on production lines is to ensure appropriate control of the process so as to obtain the appropriate technological quality of the cut edge, free of not only defects in the form of burrs and shape deviations, but also the minimum deformed zone. This task is difficult due to the large number of control variables, the influence of which on the shaping of the material and the formation of the cut edge is not fully understood. The article attempts to determine the course of the cutting process and to examine the influence of control variables on the formation of the cut edge in the shear-slitting process in which the tools perform a rotary motion. For this purpose, FEM modeling, vision techniques and experimental studies were used. A 3D model of the process was developed, which enables a detailed analysis of the states of stresses, strains, displacements and fracture mechanisms of the material. The simulation results were verified using vision techniques, which were used in the work to observe the flow and cracking mechanisms of the material. Parametric analyses were performed for the process control variables. The research showed a significant influence of the cutting velocity and the clearance between the tools on the formation of the cut edge. The most homogeneous surface of the cut edge with the minimum burr height was obtained for the following parameters: rake angle α = 15–30°, horizontal clearance hc = 0.03 mm and slitting velocity v2 = 15 m/min. The developed results can be useful for controlling the cutting process on production lines in terms of maximum process efficiency while maintaining the appropriate technological quality of the cut edge.

3D reconstruction of non-textured surface by combining shape from shading and stereovision

For the improvement of robot-assisted minimally invasive surgery, 3D reconstruction of the visual field in surgery is demanding nowadays. This task faces the difficulty that the tissue surfaces in the visual field usually appear to be non-textured. How to reconstruct such kind of surfaces well under this environment is unsolved. Among many vision techniques, Shape From Shading is a unique method which infers surface shape (normal field) based on the reflectance model without requiring any texture. One major limitation is that the accurate surface depth (not shape) is unable to compute when illumination parameters are not calibrated. Stereovision is a long-standing method which estimates depth by establishing the pixel correspondences between two camera images. But it fails to find correspondences when encountered with non-textured surfaces. It is intuitive to combine these two methods to recover depth of non-textured surfaces and calibrate the illumination parameters as well. However, how to bridge these two methods efficiently is challenging. In this paper, the combination of these two methods is accomplished by a 3D matching framework based on Fast Point Feature Histogram (FPFH). We first reconstruct initial surface point clouds of left image and right image using SFS method. The pixel correspondences can be built by aligning SFS calculated point clouds using FPFH 3D descriptors. Therefore, stereo method can reconstruct depth of the surface. Furthermore, we employ a classic variational fusion algorithm which refines SFS-shape and stereovision-depth. Results of the recovered depth are shown through experiments.

Classification of Macronutrient Deficiencies in Maize Plant using Machine Learning

Detection of nutritional deficiencies in plants is vital for improving crop productivity. Timely identification of nutrient deficiency through visual symptoms in the plants can help farmers take quick corrective action by appropriate nutrient management strategies. Manual checking of plant nutrient deficiency does not give adequate result as naked eye observation is old method and requires more time for deficiency recognition, and also need expert person. which is very costly for farmers hence it is non-effective. To overcome disadvantages of traditional eye observing technique, we used the application of computer vision and machine learning techniques offers new prospects in nondestructive field-based analysis for nutrient deficiency detection. In the proposed system, we are using unsupervised machine learning algorithm, which learns features on its own from given dataset. Initially, we perform preprocessing on image to be tested. preprocessing includes image enhancement, image restoration, image segmentation. classifier predicts the result based on the image we passed. The system will provide the definite result about a nutrient deficiency along with disease if any and also provide solution for nutrient deficiency.

Optical motion capture dataset of selected techniques in beginner and advanced Kyokushin karate athletes

Human motion capture is commonly used in various fields, including sport, to analyze, understand, and synthesize kinematic and kinetic data. Specialized computer vision and marker-based optical motion capture techniques constitute the gold-standard for accurate and robust human motion capture. The dataset presented consists of recordings of 37 Kyokushin karate athletes of different ages (children, young people, and adults) and skill levels (from 4th dan to 9th kyu) executing the following techniques: reverse lunge punch (Gyaku-Zuki), front kick (Mae-Geri), roundhouse kick (Mawashi-Geri), and spinning back kick (Ushiro-Mawashi-Geri). Each technique was performed approximately three times per recording (i.e., to create a single data file), and under three conditions where participants kicked or punched (i) in the air, (ii) a training shield, or (iii) an opponent. Each participant undertook a minimum of two trials per condition. The data presented was captured using a Vicon optical motion capture system with Plug-In Gait software. Three dimensional trajectories of 39 reflective markers were recorded. The resultant dataset contains a total of 1,411 recordings, with 3,229 single kicks and punches. The recordings are available in C3D file format. The dataset provides the opportunity for kinematic analysis of different combat sport techniques in attacking and defensive situations.

A Wide-Ranging Review on Diabetic Retinotherapy

Diabetic Retinopathy (DR) is one of the major eye diseases that causes damage to retina of the human eye ball due to the rupture of tiny blood vessels. DR is identified by the ophthalmologists on the basis of various specifications i.e., textures, blood vessels and pathologies. The ophthalmologists are recently considering software for eye diseases detection based on image processing designed by the computing techniques and bio-medical images. In the analysis of medical imaging, traditional techniques of image processing and computer vision have played an important role in the field of ophthalmology. From the past two decades, there is a tremendous advancement in the development of computerized system for DR detection. This paper comprises the five parts of analysis on image based retinal detection DR, named as review of low varying contrast techniques of the retinal fundus Images (RFI), review of noise effect in the fundus images, review of pathology detection method from the retinal fundus images, review of blood vessels extraction from the RFI, and review of automatic algorithm for the DR detection. This paper presents a comprehensive detail to each problem in the retinal images. The procedures that are currently utilized to analyze the contrast issue and noise issues are discussed in detail. The paper also explains the techniques used for segmentation. In the end, the recent automated detection system of related eye diseases or DR is described.

Iterative-learning error compensation for autonomous parking of mobile manipulator in harsh industrial environment

In wide-area and multi-sites manufacturing scenarios, the mobile manipulator suffers from inadequate autonomous parking performance due to the harsh industrial environment. Instead of struggling to model various errors or calibrate multiple sensors, this paper resolves the above challenge by proposing an iterative-learning error compensation scheme that consists of offline pre-regulation and online compensation, which can improve the compensation efficiency and accommodate the error fluctuations caused by environmental fluctuations. Integrating an improved Monte-Carlo localization and eye-in-hand vision technique, an effective measurement system is firstly developed to accurately obtain the parking data without requiring superfluous facilities or cumbersome measurement. Then, after removing the data outliers utilizing the Grubbs test, offline pre-regulation is achieved to give a suitable initial value and increase the compensation convergence. To reduce the time-varying systematic errors and parking error fluctuations, online compensation is presented by offering an efficacious estimation of environmental fluctuations using fuzzy logic rules and providing an adaptive iterative-learning law. Finally, the feasibility and effectiveness of the presented compensation method are validated by extensive experiments implemented on a self-developed mobile manipulator.