Vision-based Techniques Research Articles

Real-Time Performance Comparison of Vision-Based Autonomous Landing of Quadcopter on a Ground Moving Target

The tremendous applications of unmanned aerial vehicles (UAVs), such as inspection in complex environments, search, and rescue missions, have established this area of research. The domain of UAVs autonomous navigation and landing has received considerable amount of interest of robotics researchers in the past decades. Nevertheless, the limited capability of UAVs for autonomous landing severely hampers the use of aerial vehicles specially in GPS-denied areas. Fortunately, with the rapid development of computer vision, vision-based techniques have become an important and cheap tool to be used in UAVs for moving target detection and landing. In this paper, four major vision-based techniques namely PID controller, fuzzy logic, sliding mode control and model predictive control have been investigated and their landing performance is compared to each other. These landing techniques are implemented on a quadcopter for the purpose of ground moving target detection and then landing on it. The Raspberry Pi board with two Pi cameras for 3D scene construction and depth estimation along with ultrasonic sensor have been used in the quadcopter. A wheeled mobile robot with landing platform is taken as the target. The landing performance of different techniques has been observed and compared in terms of landing displacement error, time taken and the distance travelled to finish the operation in an open and obstacle-free environment. The simulation results are further verified by the hardware experiments.

Uncertainty Evaluation in Vision-Based Techniques for the Surface Analysis of Composite Material Components.

In this paper, a methodology is discussed concerning the measurement of yarn’s angle of two different glass-reinforced polypropylene matrix materials, widely used in the production of automotive components. The measurement method is based on a vision system and image processing techniques for edge detection. Measurements of angles enable, if accurate, both useful suggestions for process optimization to be made, and the reliable validation of the simulation results of the thermoplastic process. Therefore, uncertainty evaluation of angle measurement is a mandatory pre-requisite. If the image acquisition and processing is considered, many aspects influence the whole accuracy of the method; the most important have been identified and their effects evaluated with reference to two different materials, which present different optical-type characteristics. The influence of piece geometry has also been taken into account, carrying out measurements on flat sheets and on a semi-spherical object, which is a reference standard shape, to verify the effect of thermoforming and to tune the process parameters. Complete uncertainty in the order of a few degrees has been obtained, which is satisfactory for purposes of simulation validation and consequent process optimization. The uncertainty budget also allowed individuation of the most relevant causes of uncertainty for measurement process improvement.

Evaluation of a Computer Vision-Based System to Analyse Behavioral Changes in High School Classrooms

The objectives of this study were to investigate the feasibility of applying computer vision techniques and to analyse changes in behaviour and movement of high school students during class. The study is performed over two phases. Phase one focuses on developing a feasible method to use computer vision-based techniques in high school classes and phase two focuses on the testing of aromatherapy to affect student’s movement. All camera data was processed and analysed by OpenPose, Matlab and EyesWeb. Movement features such as velocity, acceleration, and kinetic energy and postural variables, spinal extension and neck flexion were calculated. Results of phase one, shows significant differences in the overall segment velocity, acceleration, energy, and neck flexion. Similarly, the second phase shows significant differences in velocity, acceleration and jerk for the left shoulder and elbow joints of the group exposed to aroma. In conclusion, the results show the feasibility of using computer vision techniques to apply in a classroom setting.

A Design of Deep Learning Experimentation for Fruit Freshness Detection

Indonesia is a country with a tropical climate so that fruit and vegetable plants can grow easily in Indonesia.Fruits have many good nutrients such as vitamins, proteins and others. But the fruit also has a period where the fruit is said to be fresh fruit.During this time there are still many fruit supplier companies that send fruit unfit for consumption due to lack of accuracy in the process of sorting the fruit when the fruit is taken from the plantation and the entry of other fruit into an improper packaging. Thus, it makes detecting food spoilage from the production stage to consumption is very important. We propose a design of computer vision-based technique usingdeep learning with the Convolutional Neural Network (CNN) model to detect fruit freshness. The specially designed CNN model is then evaluated with public datasets of fruits fresh and rotten for classification derived from Kaggle.

Towards Personalised Gaming via Facial Expression Recognition

In this paper we propose an approach for personalising the space in which a game is played (i.e., levels) dependent on classifications of the user's facial expression — to the end of tailoring the affective game experience to the individual user. Our approach is aimed at online game personalisation, i.e., the game experience is personalised during actual play of the game. A key insight of this paper is that game personalisation techniques can leverage novel computer vision-based techniques to unobtrusively infer player experiences automatically based on facial expression analysis. Specifically, to the end of tailoring the affective game experience to the individual user, in this paper we (1) leverage the proven InSight facial expression recognition SDK as a model of the user's affective state InSight, and (2) employ this model for guiding the online game personalisation process. User studies that validate the game personalisation approach in the actual video game Infinite Mario Bros. reveal that it provides an effective basis for converging to an appropriate affective state for the individual human player.

Smart, comfortable wearable system for recognizing Arabic Sign Language in real-time using IMUs and features-based fusion

Automatic sign language recognition systems have been presented in the literature to avail as a helpful device for deaf and mute individuals to incorporate them into the community without impediments. Gesture recognition is usually done by using vision-based techniques or hand-attached sensors that track hand and finger movements. This paper presents a sensors-based Arabic Sign Language recognition system. The proposed system depends on the information fusion from a 3-axis accelerometer and gyroscope Inertial Measurement Units (IMUs) sensors. The architecture of the proposed system consists of two main modules: a wearable device (glove), which includes sensors, a communication unit, and micro-controller unit, and a software module running on a mobile device. The proposed glove system utilizes six IMUs sensors where five of them placed at the fingers and the sixth sensor placed at the back of the palm. The collected data from sensors are transmitted to a mobile device through a Bluetooth low-energy wireless transmission for data analysis and gesture recognition. For each gesture, the beginning and endpoints of the significative gesture part are discovered automatically using the intensity of the gyroscope signal. The system employs a features-based sensor fusion that combines accelerometer and gyroscope time-domain and frequency-domain features. In a comprehensive set of experiments with seven participants, the system was trained and tested with 28 Arabic alphabets, and the best recognition accuracy achieved was 98.6% for the user-dependent data and 96% for the user-independent data under support vector machine classifier. The achieved results are outperforming existing systems that have been presented in recent decades for alphabet-level gesture recognition.

Rice Grain Quality Detection

The quality of grain is of great importance for human beings as it directly impacts human health. Hence there is a great need to measure the quality of grain and identifying non-quality elements. Analysing the grain samples manually is a more time-consuming and complicated process, and having more chances of errors with the subjectivity of human perception. To achieve uniform standard quality and precision, machine vision-based techniques are evolved. Rice quality is nothing but a combination of physical and chemical characteristics. So, to get the physical characteristics of the rice grains, image processing techniques are applied. Grain size and shape are some physical characteristics. The obtained all physical features grades the rice grains using canny edge detection.

User Localization and Upstream Signaling for Beam-Steered Infrared Light Communication System

Beam-steered infrared light can provide ultra-high wireless capacity to individual user devices for indoor communication systems. The optical beams used to obtain such high capacities are relatively narrow with the average beam divergence of less than 2.55°, hence an accurate localization system to pinpoint multiple user devices at the same time is required. In addition, an indoor communication should provide mobile coverage to all users, therefore a dynamic and fast localization equipped with a capability of conveying the channel link status of the user devices to optical access points is necessary. In this work, we propose a vision-based technique to simultaneously localize the user devices in a room. A LED-tag system positioned around a photodetector enables the access points to recognize active users. A localization accuracy of 0.018° is achieved within milliseconds. Furthermore, taking advantage of this vision-based system, upstream signaling for channel status information is proposed using M-ary intensity modulation. Experimental results show that an upstream signaling rate of 2.4kbps can be achieved with our simple and low-cost prototype, enabling >10Gbit/s link per user.

More views of a one-sided surface: mechanical models and stereo vision techniques for Möbius strips

Möbius strips are prototypical examples of ribbon-like structures. Inspecting their shapes and features provides useful insights into the rich mechanics of elastic ribbons. Despite their ubiquity and ease of construction, quantitative experimental measurements of the three-dimensional shapes of Möbius strips are surprisingly non-existent in the literature. We propose two novel stereo vision-based techniques to this end—a marker-based technique that determines a Lagrangian description for the construction of a Möbius strip, and a structured light illumination technique that furnishes an Eulerian description of its shape. Our measurements enable a critical evaluation of the predictive capabilities of mechanical theories proposed to model Möbius strips. We experimentally validate, seemingly for the first time, the developable strip and the Cosserat plate theories for predicting shapes of Möbius strips. Equally significantly, we confirm unambiguous deficiencies in modelling Möbius strips as Kirchhoff rods with slender cross-sections. The experimental techniques proposed and the Cosserat plate model promise to be useful tools for investigating a general class of problems in ribbon mechanics.

Automatic Discovery of Political Meme Genres with Diverse Appearances

Forms of human communication are not static --- we expect some evolution in the way information is conveyed over time because of advances in technology. One example of this phenomenon is the image-based meme, which has emerged as a dominant form of political messaging in the past decade. While originally used to spread jokes on social media, memes are now having an outsized impact on public perception of world events. A significant challenge in automatic meme analysis has been the development of a strategy to match memes from within a single genre when the appearances of the images vary. Such variation is especially common in memes exhibiting mimicry. For example, when voters perform a common hand gesture to signal their support for a candidate. In this paper we introduce a scalable automated visual recognition pipeline for discovering political meme genres of diverse appearance. This pipeline can ingest meme images from a social network, apply computer vision-based techniques to extract local features and index new images into a database, and then organize the memes into related genres. To validate this approach, we perform a large case study on the 2019 Indonesian Presidential Election using a new dataset of over two million images collected from Twitter and Instagram. Results show that this approach can discover new meme genres with visually diverse images that share common stylistic elements, paving the way forward for further work in semantic analysis and content attribution.

Vision-based modal parameter identification for bridges using a novel holographic visual sensor

Noncontact monitoring using holography can be quite useful in vision-based structural health monitoring owing to its advantages in terms of data visualization and real-time applications. Aimed at exploring the potential of vision-based sensors for structural health monitoring in more general applications, this paper has developed the holographic visual sensor (HVS) with a vision-based modal parameter identification method. This strategy is an extension of the current vision-based techniques and holography. Based on the newly introduced spatial and temporal sequence data measured by HVS, the vision-based modal parameter identification method was proposed and applied to structural health monitoring for the first time. Accordingly, the feature point set describing the geometric morphology of the structural holography and the time-state-space mathematical model of the structural dynamics system was constructed. In this way, the mechanical behavior characteristics could be extracted from serial transient holographic responses following excitation, and the structural modal parameters were then calculated and quantitatively analyzed. Laboratory experiments and field tests were employed for validation based on the comparison between the results determined by the proposed HVS and those determined by conventional, high-precision contact sensors. The developed HVS and identification algorithms were able to visually collect and identify modal parameters at a higher data resolution and provide smoother modal shapes. The proposed method can thus be used to complement and improve existing computer-vision-based measurement techniques and damage detection.

Exploring the solutions via Retinex enhancements for fruit recognition impacts of outdoor sunlight: a case study of navel oranges

Machine vision-based techniques are one of the critical means to realize intelligent orchard management. Additionally, the third wave of artificial intelligence guided by deep learning has promoted the application of machine vision technology in fruit recognition. Currently, multiple detection models can extract the fruits from collected images, yet the accuracy has often existed deviation, due to different observation times cause corresponding changes of sunlight throughout a day. Consequently, exploring a method to solve this problem is of great significance to facilitating smart orchards. On this basis, this article takes the navel orange as a study object, dividing four observation periods (10:00–11:00, 12:00–13:00, 14:00–15:00, 16:00–17:00) and two viewing distances of one-meter and two-meter to collect image data. Corresponding algorithms are designed according to each Retinex processing modes to assist the detection of YOLOv5, including single-scale Retinex (SSR), multi-scale Retinex (MSR), multi-scale Retinex with color restoration (MSRCR), multi-scale Retinex with chromaticity preservation (MSRCP), and MSRCR with automatic color gradation adjustment (AutoMSRCR). The experimental results showed that two observation periods of 10:00–11:00 and 14:00–15:00 are more conducive to the data collection, where MSR-based and MSRCR-based models respectively from the two periods improved 9.28% and 6.32% of mean average precision (MAP) than original YOLOv5 under one-meter viewing distance. Also, MSRCR-based and AutoMSRCR-based models achieved 4.92% and 16.91% of MAP higher than the original under two-meter viewing distance. Simultaneously, this article also provides technical selection schemes and analyzes the sensitivity of each model in typical impact scenarios.

A Study on Deep Learning Methods for Skin Disease Classification

Dermatological disorders are one among the foremost widespread diseases within the world. Despite being common its diagnosis is extremely difficult due to its complexities of skin tone, color, presence of hair. This paper provides an approach to use various computer vision-based techniques (deep learning) to automatically predict the varied sorts of skin diseases. The system makes use of deep learning technology to coach itself with the varied skin images. the most objective of this technique is to realize maximum accuracy of disease of the skin prediction. The people health quite the other diseases. Skin diseases are mostly caused by mycosis, bacteria, allergy, or viruses, etc. The lasers advancement and Photonics based medical technology is employed in diagnosis of the skin diseases quickly and accurately. The medical equipment for such diagnosis is restricted and costliest. So, Deep learning techniques helps in detection of disease of the skin at an initial stage. The feature extraction plays a key role in classification of skin diseases. The usage of Deep Learning algorithms has reduced the necessity for human labor, like manual feature extraction and data reconstruction for classification purpose.

A BEHAVIORAL APPROACH TO DETECT SOMNOLENCE OF CAB DRIVERS USING CONVOLUTIONAL NEURAL NETWORK

The Road Traffic Accident Statistics concluded that fatal road accident 60% is caused by vehicle collision of taxi drivers. World Health Organization (WHO) is constantly initiating global road safety measures to minimize road accidents but, the cause for fatal injuries is primarily due to driving fatigue. Most people rely on cabs as the main transport. To provide obligate care of passengers, a computer vision-based technique is needed to detect the somnolence of drivers. Our proposed model CabSafety is based non-intrusive computer vision technique using Convolutional Neural Network (CNN). A tiny camera is fixed focusing the driver’s face to monitor the behavioral changes like an eye blink, yawing, watery eye, mouth movement, and head position. The measures of the driver’s eye are concentrated to identify sleepiness under stimulator or test conditions. The efficiency of the proposed model provides better results compared to the existing technique. The image from the camera is processed by OpenCV and Keras/Tensor flow. CNN classifier is used to detect eye status. The prediction from the CNN classifier produces an alarm to alert the driver.

A Novel vision based embedded framework system to detect and track dynamic vehicles

A vehicle tracking system is a smart device installed in vehicles to detect and track the moving vehicle. In the emerging world the number of vehicles on road has been increasing globally. Intelligent transport system has been explored better to make an efficient transport system and also for traffic arrangements. Hence there is a need for a system that tracks the vehicle and provide assistance to the driver. The study aims at developing a novel embedded driver assistance framework that could analyze the dynamics of vehicle in the front and rear surround views. In the proposed study, to detect and track the moving vehicles Gaussian Mixture Model (GMM) and Kalman filter is implemented thereby critical safety to the vehicles is enabled by calculating the distance along with the directions. The developed prototype contains 4 cameras each equipped with an embedded processor. In order for the system to run at a near real time pace, optimized vision-based techniques are used that detect vehicles that are near-by. The only modality used for sensing are the camera sensors which analyze the surroundings of the ego or the host vehicle. Object detection by GMM technique proves to be more accurate and reliable in light traffic conditions and during day time whereas during night time the detection technique is not that accurate. Hence the developed prototype includes ultrasonic sensors that calculates the distance between the objects by sound wave. Further the developed detection system is validated by analyzing Receiver Operating Characteristics. Thus vision based embedded computing framework proves to be more efficient method for tracking objects.

A Review of Vision-Based Techniques Applied to Detecting Human-Object Interactions in Still Images

Due to the rising demand for automatic interpretation of visual relationships in several domains, human-object interaction (HOI) detection and recognition have also gained more attention from researchers over the last decade. This survey paper concentrates on human-centric interactions, which can be categorized as human-to-human and human-to-objects. Although an extensive amount of research work has been done in this area, real-world constraints like the domain of possible interactions make the research a challenging task. This paper provides an analysis of conventional hand-crafted representation-based methods and recent deep learning-based methods, ongoing advancements taking place in the field of HOI recognition and detection, and challenges faced by the researchers. Moreover, we present a detailed picture of publicly available datasets for HOI evaluations. At the end, the future scope of the study is discussed.

Device-Free Localization: A Review of Non-RF Techniques for Unobtrusive Indoor Positioning

Accurate, reliable indoor localization or positioning is the key enabler for location-based services. Indoor localization can be broadly classified into two distinct categories. Active localization entails tracking a tag attached to or carried by the target. Passive localization, on the other hand, involves positioning a device-free or untagged target. While passive or device-free localization is comparatively more difficult to achieve, it is the preferred option for many applications. Vision-based techniques can accurately localize an untagged target. However, privacy is a significant concern and thus have limited usability for many applications, especially in noncommercial and residential settings. Passive localization using radio frequency (RF) or wireless sensing is an unobtrusive option and has seen extensive research efforts in recent years leading to a saturated research field but no consensus solution. Researchers have been investigating alternative solutions that can facilitate robust passive localization. The rapid proliferation of Internet of Things (IoT) is bringing ubiquitous networked devices and ambient sensors into modern buildings. The consequential pervasive ambient intelligence and signals of opportunity can enable unobtrusive device-free positioning. This article presents a comprehensive review of non-RF solutions covering visible light-, infrared-, physical excitation-and electric field sensing-based techniques. Limitations of the state-of-the-art and potential future research directions are also outlined.

Abstract PR-06: Utilizing biological domain knowledge and machine learning methods to improve cellular segmentation on multiplex fluorescence and imaging mass cytometry datasets improves the quality of single-cell data obtained

Abstract Currently, many existing multi-modality image analysis methods rely heavily on manual annotations for the extraction of quantitative readouts. This limits both the analytical throughput and the complexity to which one can analyze the data – for every hour of imaging time, several additional hours of tedious manual labor are require to extract even rudimentary metrics. Recent advances in computer vision-based techniques, including the use of machine and deep learning methods, have shown tremendous promise for solving previously intractable image segmentation and classification problems. This presents an opportunity for the development of image analysis pipelines that utilize these technical advances to solve known biomedical imaging challenges. We have developed a series of image analysis pipelines for multiplex image segmentation that draws from several interdisciplinary collaborations between biological scientists, pathologists, and biomedical image analysis specialists. For example, we have recently utilized biomedical domain knowledge to develop a customized cellular segmentation methodology for Imaging Mass Cytometry, to identify invading immune cells of various biomarker-identified types, and measure their density and proximity to blood vessels (a marker of active invasion) in multiple sclerosis lesions in the brain. A robust image segmentation and classification pipeline permits us to move these complex datasets from the format of “pixels embedded in spatial coordinates” towards the format of a “single-cell proteomic” dataset, that also permits spatial relationships between markers or tissue regions to be queried. Likewise, we have illustrated a number of methods for single-cell and per-vessel analysis of hypoxia and proliferation gradients within solid tumor tissue sections. Development of single-cell “tissue cytometric” methods permits the in-depth study of spatial relationships that would be difficult or impossible to quantify with more rudimentary whole-tissue analysis approaches. The spatial relationships between tissue components are tightly integrated with tissue metabolism and biomedical transport phenomena, so the study of these relationships permits us to better model tissue physiology in silico, through more accurate measurement of relevant physiological parameters. Clinical validation and deployment of analytical methodologies developed in the laboratory, along with robust methods for quality control and validation of the analytical outputs, allows us to move these promising research tools towards the ultimate goal of clinically approved diagnostic algorithms and medical devices. Such efforts hold the promise to deliver a profound positive impact on the healthcare system, reducing tedious manual steps like counting cells or reviewing scans, through an optimal combination of the advantages of validated automated methods with clinical wisdom and experience. Citation Format: Trevor D. McKee, Mark Zaidi, Veronica Cojocari. Utilizing biological domain knowledge and machine learning methods to improve cellular segmentation on multiplex fluorescence and imaging mass cytometry datasets improves the quality of single-cell data obtained [abstract]. In: Proceedings of the AACR Virtual Special Conference on Artificial Intelligence, Diagnosis, and Imaging; 2021 Jan 13-14. Philadelphia (PA): AACR; Clin Cancer Res 2021;27(5_Suppl):Abstract nr PR-06.

Computer vision-based displacement measurement with m-sequence target

The development of image sensors enables the application of vision-based techniques to the non-contact displacement measurement of large-scale structures. The features of the physical targets are critical to the accuracy, stability and anti-interference of the displacement measurement results. In this study, a novel m-sequence target and the associated circular correlation processing technique are developed for real-time displacement measurement. The properties of the m-sequence as a pseudo-random sequence are introduced. The vision-based displacement calculation method is then derived from the correlation property of the m-sequence. The algorithms and measurement systems are integrated in the LabVIEW environment. To verify the anti-interference performance of the developed system, static and dynamic experimental tests are carried out with various forms of interference, such as partial occlusion, uneven illumination, out of focus and smoke effect. Experimental results indicate that the developed system cannot only accurately measure structural displacement, but also has outstanding antiinterference performance, even if 30% of the target is masked.

RCNet: road classification convolutional neural networks for intelligent vehicle system

Vision-based techniques for intelligent vehicles in heterogeneous road environments are gaining significant attention from researchers and industrialists. Unfortunately, the mechanisms in this domain suffer from limited performance due to scene complexity, varying road structure, and improper illumination conditions. These challenging situations may lead an intelligent vehicle into dangerous situations such as collisions or road accidents and may cause higher mortality. The application of intelligent methods and other machine learning techniques for road surface classification is little explored in the existing literature. Thus, we propose a convolutional neural network-based road classification network (RCNet) for the accurate classification of road surfaces. This procedure includes the classification of five major categories of road surfaces: curvy, dry, ice, rough, and wet roads. The experimental results reveal the behavior of the proposed RCNet under various optimizer techniques. The standard performance evaluation measures have been used to test and validate the proposed method on the Oxford RobotCar dataset. RCNet achieves classification accuracy, precision, and sensitivity of 99.90%, and 99.97% of specificity. Results of implemented work are significantly higher than available state-of-the-art techniques and show accurate and effective performance in the complex road environment.