Optical Flow Sensor Research Articles

A multi-sensor fusion SLAM algorithm for indoor aerial robots

Nowdays, existing Light Detection and Ranging (LiDAR) based unknown-space-exploration Simultaneous Localization and Mapping (SLAM) methods have the problems of unstable mapping in feature-degraded environments and poor estimation accuracy in the vertical direction. To solve these problems, this study presents a multi-sensor fusion SLAM algorithm for indoor aerial robots as well as Unmanned Aerial Vehicles (UAVs). The sensors used in the multi-sensor fusion algorithm include optical flow sensor, barometer, Inertial Measurement Unit (IMU), and LiDAR. The proposed algorithm is named LiDAR-IMU-optical flow odometry fusion algorithm (Lioo). We derived and established a tightly coupled multi-sensor fusion framework based on factor graph optimisation. The IMU-optical flow pre-integration factor is proposed, and the barometer factor to the algorithm is added innovatively. Engineering applications are also realised in this paper. Flight experiments demonstrated that the proposed algorithm achieves more precise mapping and localisation of UAVs in indoor environments.

Design and Performance Analysis of a Torque-Based Optical Fiber Flow Sensor

Design and Performance Analysis of a Torque-Based Optical Fiber Flow Sensor

Optimizing an Autonomous Robot’s Path to Increase Movement Speed

The goal of this study is to address the challenges associated with identifying and planning a mobile land robot’s path to optimize its speed in a stationary environment. Our focus was on devising routes that navigate around obstacles in various spatial arrangements. To achieve this, we employed MATLAB R2023b for trajectory simulation and optimization. On-board data processing was conducted, while obstacle detection relied on the omnidirectional video processing system integrated into the robot. Odometry was facilitated by engine encoders and optical flow sensors. Additionally, an external video system was utilized to verify the experimental data pertaining to the robot’s movement. Last but not least, the algorithms and hardware equipment used enabled the robot to go along the path at greater speeds. Limiting the amount of time and energy required to travel allowed us to avoid obstacles.

An Optical Fiber Multiphase Flow Sensor Based on Identification of Bubble Collision and Bounce

An Optical Fiber Multiphase Flow Sensor Based on Identification of Bubble Collision and Bounce

Improving Optical Flow Sensor Using a Gimbal for Quadrotor Navigation in GPS-Denied Environment.

This paper proposes a new sensor using optical flow to stabilize a quadrotor when a GPS signal is not available. Normally, optical flow varies with the attitude of the aerial vehicle. This produces positive feedback on the attitude control that destabilizes the orientation of the vehicle. To avoid this, we propose a novel sensor using an optical flow camera with a 6DoF IMU (Inertial Measurement Unit) mounted on a two-axis anti-shake stabilizer mobile aerial gimbal. We also propose a robust algorithm based on Sliding Mode Control for stabilizing the optical flow sensor downwards independently of the aerial vehicle attitude. This method improves the estimation of the position and velocity of the quadrotor. We present experimental results to show the performance of the proposed sensor and algorithms.

Improved modeling and fast in-field calibration of optical flow sensor for unmanned aerial vehicle position estimation

To better reveal the principle of off-the-shelf OFS that used for UAV position estimation, two measurement models are derived, namely the continuous-time and discrete-time models. Then the intrinsic equivalence of these two models is also proven. Based on these two models, a calibration method for OFS is introduced, which only makes use of in-situ rotation to determine the OFS’s parameters including its focal length and installation angles with respect to the UAV. Furthermore, an OFS-based UAV position estimation algorithm is presented, which stems from the proposed discrete-time model. The proposed OFS models, calibration method, and position estimation algorithm form a full solution for UAV displacement measurement via OFS, and its feasibility and effectiveness is verified through flight tests on a quadcopter UAV in outdoor environment.

A fully printed sensor with optical readout for real-time flow monitoring

In recent years, there has been a growing interest in the development of flexible thermal flow sensing devices due to their wide-ranging applications. In this study, we present the fabrication of a screen-printed flow sensor with optical readout on a 125 μm polyethylene terephthalate substrate in a three-layer configuration. The device comprises electrodes made from a commercial silver (Ag) ink, a heating area using a commercial carbon ink, and a thermochromic (TC) layer employing a commercial ink with a standard activation temperature of 31 °C. We designed a specialized experimental setup to evaluate the performance of the optical flow sensor under static and dynamic conditions. To analyze the device’s thermal response and performance across various flow conditions, we utilized a combination of electrical measurements and infrared (IR)-optical imaging techniques. The all-printed device operates on the basis of a thermodynamic cycle frequency, which activates the TC ink, causing it to blink at a frequency related to the flow passing over the sensor. The results of our preliminary testing are highly promising, as the sensor successfully demonstrated a clear relationship between flow and optical duty cycle. This innovative device offers a contactless, low-cost, easy-to-use flow detection method and holds significant potential for various practical applications.

Optical flow-based closed-loop control of a multi-degree-of-freedom clamping-type ultrasonic motor.

Rotor speed and position detection are integral parts of the closed-loop control system for multi-degree-of-freedom (multi-DOF) ultrasonic motors. The non-contact speed detection method is important for the high-precision control of the spherical rotor. This paper proposes a closed-loop control method based on an optical flow sensor for the velocity-position control of a multi-DOF clamping-type ultrasonic motor. The optical flow information is analyzed using the conical L-K optical flow algorithm to obtain the rotor speed and position. An incremental PID control method is used to perform dual closed-loop positioning control of the motor regarding speed and position. An experimental platform for the optical flow sensor is designed, and the method's feasibility is verified experimentally.

Unmanned Aerial Vehicle Position Estimation Augmentation Using Optical Flow Sensor

Flight control of unmanned aerial vehicle (UAV) requires reliable measurements of UAV’s position and attitude. Optical flow sensor is able to detect UAV’s motion with respect to the ground, and thus it can be incorporated in the integrated navigation system to enhance the positioning accuracy. However, the commonly used measurement model of optical flow sensor is actually fit for continuous-time condition only, since it defines the optical flow as the instantaneous velocity of a pixel on the image plane. For commercial optical flow sensors that work under discrete-time condition, a novel measurement model is proposed in this paper, which gives a vectorized symmetrical description of the optical flow measurement between every two successive frames in the image sequence. Moreover, a cubature transform-based data fusion scheme is also presented in this paper, which can directly augment the UAV’s position estimation with optical flow data, rather than extracting UAV’s velocity information from optical flow for dead-reckoning, and hence it can be easily added to the UAV’s navigation system without changing the existing algorithm flow. Flight tests are conducted using a quadcopter UAV that equipped with PIXHAWK autopilot and PX4Flow optical flow sensor, and the test results prove that the proposed optical flow model and data fusion scheme can effectively improve the accuracy of UAV position estimation in various outdoor environments.

Automated IV Infusion Control System Design and Development for Medical Use

For patients suffering from pathological conditions, Intravenous infusion serves to be the go-to treatment, it requires for a medical practitioner to set up the IV infusion device up as per the doctors’ instructions, followed by periodic supervision as abnormalities may occur. The objective of this paper is to demonstrate the construction of an automated intravenous infusion control system. Medical and laboratory fluid control has previously been achieved using pressure-based flow sensing, ultrasonic flow measurement, optical flow sensors, capacitive flow sensing, and thermal flow sensing. Due to its reduced susceptibility to external factors, our approach utilizing an IR sensor is particularly advantageous in IV scenarios for accurately detecting fluid drops. An IR sensor, microcontroller, servo motor/peristaltic pump, and an affordable microcontroller are used in this system to allow precise saline circulation rate control. An input control system can be implemented through either a matrix keypad or a single control network server. Controls the drops per minute and allows internet-based server-side control via IR sensor, 3x4 matrix keypad, and Bluetooth module. To accurately measure the saline flow rate and count drops, an IR sensor is integrated into the drip chamber of the saline container. In order to achieve the desired blood flow rate, the microcontroller continuously compares the sensor readings with the commands and adjusts the servo motor/peristaltic pump accordingly. Testing of the completed hardware provided satisfactory results, proving the device's ability to enhance patient care.

Optical fiber flow sensor based on a lever-hinge configuration.

Based on a lever-hinge structure, a target-type fiber Bragg grating (FBG) flow sensor is proposed. Differential measurements of temperature and pressure are achieved using two FBGs. The design idea of the sensor is demonstrated from both theoretical and experimental aspects, and the relationship between FBG wavelength and temperature and the relationship between FBG wavelength and volume flow rate were established, respectively. The sensor is compact with good resolution, high stability, wide measurement range, and easy fabrication, and can be applied to measure temperature and volume flow rate in injection wells.

CNN-Based QR Code Reading of Package for Unmanned Aerial Vehicle.

This paper plans to establish a warehouse management system based on an unmanned aerial vehicle (UAV) to scan the QR codes printed on packages. This UAV consists of a positive cross quadcopter drone and a variety of sensors and components, such as flight controllers, single-board computers, optical flow sensors, ultrasonic sensors and cameras, etc. The UAV stabilizes itself by proportional-integral-derivative (PID) control and takes pictures of the package as it reaches ahead of the shelf. Through convolutional neural networks (CNNs), the placement angle of the package can be accurately identified. Some optimization functions are applied to compare system performance. When the angle is 90°, that is, the package is placed normally and correctly, the QR code will be read directly. Otherwise, image processing techniques that include Sobel edge computing, minimum circumscribed rectangle, perspective transformation, and image enhancement is required to assist in reading the QR code. The experimental results showed that the proposed algorithm provided good performance of a recognition rate of 94% for the stochastic gradient descent (SGD) and 95% for Adadelta optimization functions. After that, successful QR code reading was presented.

Augmentation for unmanned aerial vehicle position estimation using MARG and optical flow sensors

The optical flow sensor can detect the movement of the unmanned aerial vehicle (UAV) on the ground; thus, it is widely used for UAV flight control. The commonly used pinhole model of optical flow sensor describes the optical flow measurement in linear and angular velocities. It is not suitable in the case of discrete-time processing. A novel measurement model for the optical flow sensor is proposed, which directly gives the relationship between the optical flow and the UAV’s translational/angular motions in each sampling period. A data fusion scheme based on cubature transform is also presented, which can augment UAV’s position estimation using optical flow data. The proposed method is proven to be effective through flight tests on UAVs.

Autonomous Flight Using UWB-Based Positioning System with Optical Flow Sensors in a GPS-Denied Environment

This study presents the positioning method and autonomous flight of a quadrotor drone using ultra-wideband (UWB) communication and an optical flow sensor. UWB communication obtains the distance between multiple ground stations and a mobile station on a robot, and the position is calculated based on a multilateration method similar to global positioning system (GPS). The update rate of positioning using only UWB communication devices is slow; hence, we improved the update rate by combining the UWB and inertial measurement unit (IMU) sensor in the prior study. This study demonstrates the improvement of the positioning method and accuracy by sensor fusion of the UWB device, an IMU, and an optical flow sensor using the extended Kalman filter. The proposed method is validated by hovering and position control experiments and also realizes a sufficient rate and accuracy for autonomous flight.

Elastic Organic Crystals as Bioinspired Hair-Like Sensors.

One of the typical haptic elements are natural hairy structures that animals and plants rely on for feedback. Although these hair sensors are an admirable inspiration, the development of active flow sensing components having low elastic moduli and high aspect ratios remains a challenge. Here, we report a new sensing approach based on a flexible thin and optically transmissive organic crystal of high aspect ratio, which is stamped with fluorescent dye for tracking. When subjected to gas flow and exposed to laser, the crystal bends due to exerted pressure and acts as optical flow sensor with low detection limit (~1.578 ms-1) and fast response time (~2.70 s). The air-flow-induced crystal deformation and flow dynamics response are modelled by finite element analysis. Due to the simple design, light-weight, and mechanical robustness, this prototypical crystal hair-like sensor opens prospects for a new class of sensing devices from wearable electronics to aeronautics.

Elastic Organic Crystals as Bioinspired Hair‐Like Sensors

AbstractOne of the typical haptic elements are natural hairy structures that animals and plants rely on for feedback. Although these hair sensors are an admirable inspiration, the development of active flow sensing components having low elastic moduli and high aspect ratios remains a challenge. Here, we report a new sensing approach based on a flexible, thin and optically transmissive organic crystal of high aspect ratio, which is stamped with fluorescent dye for tracking. When subjected to gas flow and exposed to laser, the crystal bends due to exerted pressure and acts as an optical flow (hair) sensor with low detection limit (≈1.578 m s−1) and fast response time (≈2.70 s). The air‐flow‐induced crystal deformation and flow dynamics response are modelled by finite element analysis. Due to having a simple design and being lightweight and mechanically robust this prototypical crystal hair‐like sensor opens prospects for a new class of sensing devices ranging from wearable electronics to aeronautics.

Characteristics of optical flow from aerial thermal imaging, “thermal flow”

AbstractThis study explores the utility of optical flow calculated from thermal imaging cameras, “thermal flow,” mounted on an aircraft for localization in day and night conditions. Our sensor implementation utilizes a long wave infrared (LWIR) micro sensor to capture sequences of thermal images and an on‐board computer to compute an optical flow estimate. We compared the performance of optical flow from the LWIR camera with the output of visible spectrum optical flow sensor. Flights were conducted spanning a 24 h window to explore how thermal flow performs relative to optical flow as the ground heats and cools. Agreement between optical and thermal flow was found during daylight when both sensors were functional. Additionally, thermal flow results were reliable in the middle of the day through to late evening, gradually degrading until shortly after sunrise.

Forklift Tracking: Industry 4.0 Implementation in Large-Scale Warehouses through UWB Sensor Fusion

This article addresses the problem of determining the location of pallets carried by forklifts inside a warehouse, which are recognized thanks to an onboard Radio Frequency IDentification (RFID) system at the ultra-high-frequency (UHF) band. By reconstructing the forklift trajectory and orientation, the location of the pallets can be associated with the forklift position at the time of unloading events. The localization task is accomplished by means of an easy-to-deploy combination of onboard sensors, i.e., an inertial measurement unit (IMU) and an optical flow sensor (OFS), with a commercial ultra-wideband (UWB) system through an Unscented Kalman Filter (UKF) algorithm, which estimates the forklift pose over time. The proposed sensor fusion approach contributes to the localization error mitigation by preventing drifts in the trajectory reconstruction. The designed methos was at first evaluated by means of a simulation framework and then through an experimental analysis conducted in a large warehouse with a size of about 4000 m2.

Combining inertial sensors and optical flow to assess finger movements: Pilot study for telehealth applications.

Parkinson's disease is the fastest growing neurological disorder worldwide. Traditionally, diagnosis and monitoring of its motor manifestations depend on examination of the speed, amplitude, and frequency of movement by trained providers. Despite the use of validated scales, clinical examination of movement is semi-quantitative, relatively subjective and it has become a major challenge during the ongoing pandemic. Using digital and technology-based tools during synchronous telehealth can overcome these barriers but it requires access to powerful computers and high-speed internet. In resource-limited settings without consistent access to trained providers, computers and internet, there is a need to develop accessible tools for telehealth application. We simulated a controlled asynchronous telehealth environment to develop and pre-test optical flow and inertial sensors (accelerometer and gyroscope) to assess sequences of 10 repetitive finger-tapping movements performed at a cued frequency of 1 Hz. In 42 sequences obtained from 7 healthy volunteers, we found positive correlations between the frequencies estimated by all modalities (ρ=0.63-0.93, P<0.01). Test-retest experiments showed median coefficients of variation of 7.04% for optical flow, 7.78% for accelerometer and 11.79% for gyroscope measures. This pilot study shows that combining optical flow and inertial sensors is a potential telehealth approach to accurately measure the frequency of repetitive finger movements.Clinical relevance- This pilot study presents a comparative analysis between inertial sensors and optical flow to characterize repetitive finger-tapping movements in healthy volunteers. These methods are feasible for the objective evaluation of bradykinesia as part of telehealth applications.

OF-VO: Efficient Navigation Among Pedestrians Using Commodity Sensors

We present a modified velocity-obstacle (VO) algorithm that uses probabilistic partial observations of the environment to compute velocities and navigate a robot to a target. Our system uses commodity visual sensors, including a mono-camera and a 2D Lidar, to explicitly predict the velocities and positions of surrounding obstacles through optical flow estimation, object detection, and sensor fusion. A key aspect of our work is coupling the perception (OF: optical flow) and planning (VO) components for reliable navigation. Overall, our OF-VO algorithm using learning-based perception and model-based planning methods offers better performance than prior algorithms in terms of navigation time and success rate of collision avoidance. Our method also provides bounds on the probabilistic collision avoidance algorithm. We highlight the realtime performance of OF-VO on a Turtlebot navigating among pedestrians in both simulated and real-world scenes. A demo video is available at https://gamma.umd.edu/ofvo/