Onboard Video Research Articles

Real-time assessment of ship collision risk using image processing techniques

The poor quality or the miss of Automatic Identification System (AIS) data may cause erroneous judgement of the potential navigational risk. Therefore, this study proposes a real-time framework for assessing ship collision risk using onboard video data in order to improve the risk perception ability of navigators. Firstly, the Squeeze-and-Excitation (SE) attention mechanism and the K-means algorithm are simultaneously utilized for the framework to enhance the multi-scale ship detection capability. The Deep-SORT is employed to complete multi-ship feature matching. Secondly, the distances between two ships and their speeds are measured using the pinhole imaging principle based on the ship visual feature extraction results. Moreover, the ship distance-speed correction method is designed to improve the reliability of estimated results. Finally, the effectiveness of the framework is validated using naturalistic driving data from the “He Hua Hai” ship. The results show that the proposed framework could demonstrate an excellent performance in assessing ship collision risk using the onboard video data. The proposed framework could help precisely detect and promptly provide warnings about potential ship collision risks. This could help prevent catastrophic accidents that pose a threat to oceans and coasts, particularly in situations when AIS data proves to be unreliable or ineffective.

Evaluating rail transit assignment models in the temporal dimension: The problem and its solution

Passenger flow is the foundation for urban rail transit (URT) operations. However, its calculated results from assignment models may deviate from the actual situation in both spatial and temporal dimensions, which arouses more attention and needs to be evaluated in particular. On the other hand, onboard video data from URT trains provides a potential way for model evaluation. This study defines the evaluation problem and proposes a methodological solution for evaluating rail transit assignment models in the temporal dimension, including qualitative validation and difference quantification. A suitable time granularity is determined for the best effectiveness and onboard video data is used for actual passenger flow extraction. The gap between the actual and the calculated by the model is identified with nonparametric statistical techniques (NPSTs) and quantified with time series similarity measurement (TSSM) methods. A case study on the Shanghai metro demonstrates the performance of the proposed approach, and several practice implications for URT operation agencies are also discussed.

Passive Visual Odometry of Mobile Transport Robots in Production Areas Using Anchor Points

The article considers the problem of determining the orientation of mobile transport robots, which are now actively used in almost all areas of industry. They make it possible to intensify production, free personnel from performing routine operations, and exclude the influence of the human factor from work. Based on the analysis of various variants of active orientation methods involving the use of binocular video systems installed on the vehicles themselves, it is proposed to use a passive orientation detection system. It includes a single stationary monocular video system, the replacement of onboard video systems on mobile robots with simple radiation sensors, as well as the use of anchor points in the area of their movement. The order of choice of anchor points in solving the problem of orientation of a mobile transport robot using a monocular video system, in particular for a rectangular interior space, is considered. Calculation formulas for determining the coordinates of the sensor from the pixel coordinates of its image obtained from a monocular video camera are determined. The general sequence of actions for determining the orientation of a mobile robotic platform is also considered. Unlike active systems, this method makes it possible to significantly simplify the hardware, significantly facilitate the analysis of the current position of the mobile robot and thereby reduce the computational complexity of calculations due to the fact that there is no need to use a complex mathematical apparatus. It is being replaced by simpler two-dimensional geometric calculations. This approach, due to the unified management, makes it possible to effectively coordinate the actions of a group of mobile transport robots when they are used together, greatly simplifies the solution of a number of tasks for optimizing the intra-shop movement of mobile vehicles.

Practice of methods for remote control of walking robotic mini-dredgers moving along the bottom

The results of experimental testing of methods for remote control of underwater walking mini-dredgers are discussed. Tests were carried out in real water conditions on the basis of a prototype of a 6-legged underwater walking vehicle. During the tests, technological operations for clearing and deepening small channels and eriks, as well as the extraction of sapropel from the bottom of the lake, were simulated. During the experiments, a mini-dredger has been controlled using video information of on-board video cameras. Due to poor visibility in the zone of hydraulic erosion of bottom soil, management was carried out in conditions of an incomplete and ambiguous understanding of the external environment. Local positioning was carried out according to natural landmarks. It is shown that fairly accurate remote control of the movement of a mini-dredger can be organized on the basis of public satellite maps of the area, even in conditions of a lack of sensory information. Moreover, the operation of an underwater dredger can be realized with minimal external control influences. The research results may be demanded in the development of walking robotic systems intended for the rehabilitation of degrading small water bodies and for other underwater technical work.

The use of vision system to determine lateral deviation from landing trajectory

PurposeThis study presents an image processing algorithm capable of calculating selected flight parameters requested by flight control systems to guide aircraft along the horizontal projection of the landing trajectory. The parameters identified based on the basics of the image of the Calvert light system appearing in the on-board video system are used by flight control algorithms that imitate the pilot’s schematics of control. Controls were generated using a fuzzy logic expert system. This study aims to analyse an alternative to classical solutions that can be applied to some specific cases.Design/methodology/approachThe paper uses theoretical discussions and breakdowns to create the basics for the development of structures for both image processing algorithms and control algorithms. An analytical discussion on the first stage was transformed into laboratory rig tests using a real autopilot unit. The results of this research were verified in a series of software-in-the-loop computer simulations.FindingsThe image processing method extracts the most crucial parameters defining the relative position of the aircraft to the runway, as well as the control algorithm that uses it.Practical implicationsIn flight control systems that do not use any dedicated ground or satellite infrastructure to land the aircraft.Originality/valueThis paper presents the original approach of the author to aircraft control in cases where visual signals are used to determine the flight trajectory of the aircraft.

Modelling Aero-Structural Deformation of Flexible Membrane Kites

Airborne wind energy systems using flexible membrane wings have the advantages of a low weight, small packing volume, high mobility and rapid deployability. This paper investigates the aero-structural deformation of a leading edge inflatable kite for airborne wind energy harvesting. In the first step, a triangular two-plate representation of the wing is introduced, leading to an analytical description of the wing geometry depending on the symmetric actuation state. In the second step, this geometric constraint-based model is refined to a multi-segment wing representation using a particle system approach. Each wing segment consists of four point masses kept at a constant distance along the tubular frame by linear spring-damper elements. An empirical correlation is used to model the billowing of the wing’s trailing edge. The linear spring-damper elements also the model line segments of the bridle line system, with each connecting two point masses. Three line segments can also be connected by a pulley model. The aerodynamic force acting on each wing segment is determined individually using the lift equation with a constant lift coefficient. The particle system model can predict the symmetric deformation of the wing in response to a symmetric actuation of the bridle lines used for depowering the kite (i.e., changing the pitch angle). The model also reproduces the typical twist deformation of the wing in response to an asymmetric line actuation used for steering the kite. The simulated wing geometries are compared with photogrammetric information taken by the onboard video camera of the kite control unit, focusing on the wing during flight. The results demonstrate that a particle system model can accurately predict the geometry of a soft wing at a low computational cost, making it an ideal structural building block for the next generation of soft wing kite models.

Video Observations and Large Accessible Transit Vehicle Operator Evaluations of Two Wheelchair Securement Systems

Wheelchairs need to be secured properly during transit to reduce the risk of passenger injury. New wheelchair securement systems have been developed in an attempt to provide increased usability and independence for passengers who need to use them. This field study evaluated newer wheelchair securement system usage through a retrospective review of on-board video surveillance recordings and large accessible transit vehicle (LATV) operators’ perceived usability of two newer wheelchair securement systems on fixed-route LATVs. Results showed that nonuse and misuse of securement systems were more prevalent with trips using the 3-point, forward-facing (3P-FF) tiedown system compared with trips using the semi-automated, rear-facing (SA-RF) system. While a majority of nonuse and misuse of the 3P-FF system was because of LATV operator error, the reasons for nonuse and misuse of the SA-RF system were more complex and mostly involved system design and transit policy issues. Operators most preferred using the 3P-FF system on the LATV rather than the SA-RF system. Findings support the need for further development and implementation of wheelchair securement with improved usability, innovative securement systems that can be operated independently by wheelchair users, and improved securement training for bus operators and wheelchair passengers.

Examining the associations between urban bus drivers’ rule violations and crash frequency using observational data

Urban bus crashes have been an increasing concern in China thanks to the expansion of urban bus systems. Bus drivers’ rule violation behavior appears to be an important factor of bus crashes. However, existing studies rely heavily on questionnaire survey of rule violation behaviors, which may suffer from self-reporting bias. Using observational data of 1,356 bus drivers from onboard video recordings, this paper develops a random parameter zero-inflated Poisson model (with heterogeneity in means and variances) to uncover the relationship between drivers’ rule violation behaviors and bus crashes. Model results show that rule violation behaviors such as using a cell phone, driving outside the bus lane, and touching solid line with wheel while driving significantly correlate with crash frequencies with large marginal effects, whereas some other violation behaviors (e.g., running red lights, changing lanes suddenly) show a minor role. This finding highlights the importance of differentiating various rule violations when examining bus crashes instead of aggregating them altogether. Furthermore, the effects of rule violation behavior show substantial heterogeneity. For example, the association between not yielding to pedestrians and bus crashes is enhanced when drivers also show records of touching solid line with wheel while driving, driving outside the bus lane, or are internally trained. Findings of this study provide insights for countermeasures to improve bus safety by targeting critical bus driver groups with specific rule violations, and inform future research to collect observational data on drivers’ rule violation behaviors.

Evaluating the Effectiveness of Onboard Video Feedback Systems on Reducing Transit Collisions and Injuries

In the mid-2000s, public transit agencies began testing onboard video feedback systems on buses, which capture short video clips when triggered by an unusual event such as hard braking, a sharp turn, or impact with an object. The objective of this study was to determine whether the systems have enhanced passenger safety by reducing the frequency and severity of collisions and injuries and to identify lessons learned from the implementation of such systems. The study concludes that the systems appear to have a positive impact on transit safety achieved through a reduction in collisions and injuries and the risky driving behaviors that contribute to them. The systems provide transit mangers with a wealth of information about their employees’ driving habits that was not previously available. Transit agencies should consider investing in such systems as one component of an overall safety and training program.

The AUV-Follower Control System Based on the Prediction of the AUV-Leader Movement Using Data from the Onboard Video Camera

The paper proposes a new method for the synthesis of spatial motion control systems of the AUV-leader and a group of AUV-followers during their cooperative movement in a desired formation. This system allows the provision of an accurate positioning of the followers relative to the leader using information received with a low frequency only from their onboard video cameras. To improve the accuracy of the created system it proposes a method of estimation of the parameters of the movement of the AUV-leader (its speeds and accelerations) to predict its movement relative to the AUV-follower in the time intervals between the updates of information received from video cameras.

The PAPI Lights-Based Vision System for Aircraft Automatic Control during Approach and Landing

The paper presents the concept of a component of an aircraft’s automatic flight control system, controlling the airplane when in longitudinal motion (i.e., pitch angle, sink rate, airspeed channels) during automatic landing, from a final approach until a touchdown. It is composed of two key parts: a vision system and an automatic landing system. The first part exploits dedicated image-processing algorithms to identify the number of red and white PAPI lights appearing on an onboard video camera. Its output data—information about an aircraft’s position on a vertical profile of a landing trajectory—is used as one of the crucial inputs to the automatic landing system (the second part), which uses them to control the landing. The control algorithms implemented by the automatic landing system are based on the fuzzy logic expert system and were developed to imitate the pilot’s control actions during landing an aircraft. These two parts were teamed together as a component of a laboratory rig, first as pure software algorithms only, then as real hardware modules with downloaded algorithms. In two test campaigns (software in the loop and hardware in the loop) they controlled an aircraft model in a simulation environment. Selected results, presenting both control efficiency and flight precision, are given in the final section of the paper.

Performance Assessment of the CCSDS-123 Standard for Panchromatic Video Compression on Space Missions

The use of next-generation and high-resolution imaging sensors is gaining interest for space missions, because of their properties for identification and exploration purposes. It is expected that the demand for video sensors in the space industry will increase during the next years, mainly for monitoring and exploration missions. In this context, onboard video compression techniques emerge as mandatory, because memory resources are available on space missions and the downlink bandwidth with ground prevents for the transmission of raw video, especially when near real-time capabilities are required. The complexity of the current commercial video encoders supposes a challenge for their implementation on the hardware used on space missions, because of their high architectural complexity and computational requirements, deriving in an area overhead and an unacceptable power consumption. In this context, we propose the use of the CCSDS-123.0-B-2 standard, originally focused on the near-lossless on-board compression of multispectral and hyperspectral images, for compressing panchromatic video. The proposed solution presents low complexity, because it is specifically designed for working on space missions and also has the capability of compressing both 3-D images and panchromatic video by using a single processing core. Results demonstrate that this solution achieves high compression ratios, maintaining a considerable video quality after reconstruction on ground and achieving a fine-grain control of the losses introduced in the compression chain.

Adaptation of the CCSDS 123.0-B-2 Standard for RGB Video Compression

The integration of video sensors on-board satellites is becoming a trend in the space industry, since they provide extra information in the temporal domain when compared with traditional remote sensing imaging acquisition equipment. The inclusion of the temporal dimension together with the constant increase in the sensor resolution supposes a challenge for on-board processing, taking into account the limited computational and storage resources on-board satellites and that it is unfeasible to directly transmit raw video to ground, due to downlink bandwidth limitations. For these reasons, on-board video compression is needed. However, the inherent complexity of the video encoders used on ground limits their implementation on environments with high constraints in terms of computational burden, area, and power consumption. This article proposes an extended compression chain that implements as compression core the CCSDS 123.0-B-2 standard, originally developed for near-lossless compression of multi- and hyperspectral images. In addition, some preprocessing stages are included to manage the temporal dimension of RGB videos efficiently. The proposed solution guarantees low complexity and flexibility to compress both multi- and hyperspectral images and panchromatic and RGB videos by using a single compression instance, which is adapted by adding or removing the appropriate stages. Results demonstrate the viability of this solution to be implemented on space payloads, since high compression ratios are achieved without incurring in a penalty in terms of video quality.

Development of Poor Visibility Assessment Method in Winter Using Images Taken by On-Board Video Camera

Development of Poor Visibility Assessment Method in Winter Using Images Taken by On-Board Video Camera

An Efficient Cascaded Model for Ship Segmentation in Aerial Images

In this work, we propose a new method for real-time ship segmentation during maritime surveillance missions using aircrafts with onboard video cameras. We propose a cascade model with a detection stage followed by a segmentation stage. The detection stage selects candidate regions (bounding boxes) likely to contain ships. These bounding boxes are passed to the segmentation stage, where the ship segmentation mask is then obtained. By focusing the segmentation effort only in the image regions recommended by the previous detection stage, it is possible to improve the overall image processing time and, simultaneously, to obtain a better segmentation score than the one obtained by monolithic segmentation models that are trained in an end-to-end way. Additionally, we test the viability of using a Conditional Random Field model as final boundary refinement stage: although such model can improve the segmentation results when a full segmentation approach is used, our experiments did not show any significant improvements when using our proposed cascade model. We trained the detection and segmentation models with aerial ship images from publicly available maritime datasets. We tested the cascade model on the Airbus ship detection challenge, showing real-time performance and accurate maritime ship segmentation, comparable to state-of-the-art results.

Automatic Processing of the Information Received from the Space Rocket Onboard Video Control System.

The paper discusses the information processing method developed by the Mis-sion Control Center (AO TSNIImash) specialists, used to process the infor-mation received from the Soyuz space rocket onboard video control system. Specific features of the information being received are presented. Methods of restoring corrupt information are proposed. The results of information pro-cessing during real Soyuz-FG and Soyuz-2 space rocket launches are given.

An Edge-Cloud Approach for Video Surveillance in Public Transport Vehicles

A Video Surveillance System (VSS) is of primary importance in public transport hubs such as airports, train or bus stations but also inside the vehicle itself. In this paper, we present a heuristic architecture model for on-board video surveillance system based on Internet of Video Things (IoVT) devices which addresses the need for delivering smart video surveillance in public transport vehicles (e.g., buses) minimizing the impact on network performance. A proof-of-concept was implemented using a public Cloud Service Provider (CSP) and two Raspberry Pi as edge computing nodes. On the edge nodes, a Machine Learning(ML) application was deployed along with a network-efficient video streaming system. On the other hand, laboratory tests are included to understand the network traffic dynamic, furthermore results are enhanced with a set of simulations in order to analyze the performance of a video streaming application and6differentcongestion control algorithms in terms of packet loss and delay.

THE MODELS OF AUTO TARGETING STRIKE UAV AT THE MOVING TARGET

A special kind of use of UAVs as transport is their use to deliver the destruction means to the target on the battlefield, i.e. as strike unmanned aerial vehicles (UAVs), capable to inflict missile and bomb strikes on the enemy.
 Of particular interest are such drones, which do not require constant aiming at the target by the flight operator (ground pilot) with the help of a video transmitting system, but are targeted independently. Such UAVs have onboard a target sighting sensor (coordinator), with which the onboard control system determines the angular position of the target relative to the UAV. It is necessary to determine the most optimal targeting algorithms to be used in onboard control systems.
 UAVs due to the successful combination of cheap goods, technological availability and availability of other technical indicators (load capacity, efficiency and duration of flight, maneuverability) can be used as vehicles for cargo delivery. For example, UAVs are used to deliver ammunition and medicine to victims.
 The mechanism of the primary notification to the on-board control system of the UAV of the coordinates of the target to be hit, ie the task of "primary targeting", remains not completely solved by the technical and organizational task when using strike UAVs. Targeting methods using the on-board video broadcast system and direct commands of the ground pilot require, firstly, the installation of appropriate payload on board with an understanding of the loss of this equipment in combat use of UAVs, and secondly, effective avoidance of radio suppression by the enemy.
 The article presents the results of modeling with the help of Simulink models of the processes of guidance of a strike UAV with targeting on a moving aim by two methods: direct guidance and dynamic bias. The trajectories of UAV flights during diving to the target were obtained. The changes in the main working parameters during the guidance process were obtained as well. The conclusion about the advantage of the method of dynamic bias is made.

Fixed Wing Aircraft Automatic Landing with the Use of a Dedicated Ground Sign System

The paper presents automatic control of an aircraft in the longitudinal channel during automatic landing. There are two crucial components of the system presented in the paper: a vision system and an automatic landing system. The vision system processes pictures of dedicated on-ground signs which appear to an on-board video camera to determine a glide path. Image processing algorithms used by the system were implemented into an embedded system and tested under laboratory conditions according to the hardware-in-the-loop method. An output from the vision system was used as one of the input signals to an automatic landing system. The major components are control algorithms based on the fuzzy logic expert system. They were created to imitate pilot actions while landing the aircraft. Both systems were connected with one another for cooperation and to control an aircraft model in a simulation environment. Selected results of tests presenting control efficiency and precision are shown in the final section of the paper.

Geoinformation laboratory for determining objects by unmanned aerial vehicles

The paper considers the problem of positioning unmanned aerial vehicles (UAVs) without using satellite navigation systems. The application of positioning occurs through the visual channel of flight control by means of an on-Board video camera on the basis of the terrain plan or satellite photos. The aim of the work is to improve the accuracy of the positioning method in accordance with several flight control positions, predetermined landmarks and closed route flight (both open or obstructed ways), for implementation in control computer systems or mobile applications.The obtained results will improve the effectiveness of UAV technology in applications for environmental monitoring, construction monitoring, agriculture, maritime industry, forestry, law enforcement, urban and municipal economy.