Multi-camera System Research Articles

The Concept of Big Data Management with Various Transportation Systems Sources as a Key Role in Smart Cities Development

An increasing number of devices and their communication with each other generates huge amounts of data. The efficiency of processing such large and heterogeneous data is crucial for extracting the reliable and consistent information that is needed for the effective management of smart cities within the field of transport. Data heterogeneity and volume as well as its integration and analytics are big challenges for decision-makers. The development of urban agglomerations is largely dependent on the proper management of such data. Therefore, this paper explores the role of these data repositories, their acquisition from different sources, and the ways to combine them. The main goal of this paper is to propose a concept of Smart City management based on Big Data Analytics and technology related to UAVs (Unmanned Aerial Vehicle) which may reduce costs and resource consumption. The presented concept includes successive data generation and collection, data type identification, problem and requirement identification, filtering, classification, pre-processing, and data optimization, as well as decision support analysis. A key part of this analysis utilizes computer algorithms, such as Speeded Up Robust Features (SURF) and Thresholding and Blob detection, to develop a multi-camera image recognition system for freight transport management and logistics in smart cities. The objective is to design a system that optimizes the route planning and time of vehicle passage on selected road sections, ultimately leading to the reduction of emissions. During the study, data obtained from multiple sources were compared, and the analysis uncovered different results for the same assumptions. We discuss the reasons for these variances. Overall, the results obtained in the analysis indicated that it is necessary to correct the predictions of the multi-camera image recognition system with additional methods and algorithms.

Diffractive optical elements for calibration of LIDAR systems: materials and fabrication

The promise of fully autonomous vehicles to replace the judgment of human drivers with real-time algorithmic decision-making based on optoelectronic systems relies fundamentally on the quality of the available data. Limitations imposed by sensor-resolution, available optical power, and achievable signal-to-noise ratios have been well studied in the light detection and ranging (LIDAR) application space. Additionally, the problem of integrating multiple sources of image data as well as the need to establish and maintain the system calibration over life are critically important to system reliability and safety. These latter concerns will receive even greater attention as self-driving vehicles begin to transition toward fully autonomous operation. Because of the importance of calibration to system performance and safety, the process of validating and recalibrating the system will ideally be integrated into the LIDAR system itself with calibration occurring automatically “anywhere and at any time,” without dedicated external infrastructure. Mass-market adoption is also being driven by the systems’ size and weight, as well as reliable manufacturability and resilience to environmental stresses. Due to their extreme stability, manufacturability, and small size, diffractive optical elements (DOEs) are well suited for use as optical calibration references. Current three-dimensional (3D) mapping systems based on structured light illumination already rely on DOEs as precision pattern generators to provide 3D depth sensing in a wide array mobile devices. We examine the potential use of DOEs as calibration elements in multicamera or LIDAR systems, including appropriate choices of materials, designs, and fabrication methods to ensure reliable long-term performance under automotive use conditions. We present simulations of the impact of DOE material properties on the accuracy of the generated dot patterns and consequently on the depth accuracy and lateral distortion of the 3D image. Additionally, we present requirements for DOE manufacture using conventional semiconductor fabrication technologies optimized for creating engineered surface nanostructures capable of transforming the output of a laser or other narrow-band source into a precise reference pattern.

A precise measurement method for tube endpoints based on spatial geometry and perspective projection model

Tubes are widely used in the aerospace field for such things as engines, fuel pipelines, cooling systems, etc. To improve manufacturing efficiency and accuracy, researchers develop multi-camera vision methods for measuring tube geometry parameters. However, these methods cannot measure the exact position of the endpoints of the tube, because the projection points of the tube endpoints have no obvious texture features in the image. To solve the above problem, a precise measurement method for tube endpoints based on spatial geometry and perspective projection model is proposed in this paper. First, we establish a Plücker coordinate system, which treats the end of the tube as a cylinder. Then, several line equations parallel to the center line are calculated on the surface of the cylinder and their intersections with the end face are obtained on these lines, which refers to the optical center of the camera, the radius of the tube, and etc. These intersections are projected onto the centerline and optimized to get the endpoint. Finally, we build a multi-camera vision measurement system consisting of ten industrial cameras to verify the effectiveness of the proposed method. Extensive experiments have shown that the average error of this method is within 0.1 mm, and the standard deviation is within 0.08 mm, which proves the superiority of our proposed method.

Inline Quality Monitoring of Reverse Extruded Aluminum Parts with Cathodic Dip-Paint Coating (KTL).

Perfectly coated surfaces are an essential quality feature in the automotive and consumer goods industries. They are the result of an optimized, controlled coating process. Because entire assemblies could be rejected if Out-of-Specification (OOS) parts are installed, this has a severe economic impact. This paper presents a novel, line-integrated multi-camera system with intelligent algorithms for anomaly detection on small KTL-coated aluminum parts. The system also aims to automatize the previously used human inspection to a sophisticated and automated vision system that efficiently detects defects and anomalies on coated parts.

RAPID AND LOW-COST PHOTOGRAMMETRIC SURVEY OF HAZARDOUS SITES, FROM MEASUREMENTS TO VR DISSEMINATION

Abstract. In the field of geomatics, the development of 3D surveying techniques that can produce dense point clouds and high-resolution three-dimensional models has opened the possibility of using these techniques to generate digital replicas of a given asset to be used in contexts of cultural promotion or entertainment.These non-traditional applications have been made easier by the spreading of low-cost but fast surveying techniques, allowing them to be used in contexts or projects with fewer resources. Close-range photogrammetry is nowadays the cheapest, most flexible, and fastest technique that can be used to produce high-resolution and high-quality digital copies of reality. In this paper, we present an application of the Ant3D multi-camera system, a novel photogrammetric device designed for the rapid 3D surveying of sites characterized by poor accessibility and narrow spaces. The paper describes the three-dimensional acquisition of underground tunnels in a hazardous abandoned mining site. The digitalisation work aims to create 3D models to be experienced safely in a virtual- or augmented- reality immersive navigation of the site for cultural promotion. A workflow is proposed to turn raw reality-based 3D models into refined and efficient 3D assets that can be exploited in a game engine. Applications for mobile devices, desktop, and VR headsets, using the produced 3D assets are described.

Calibration of Ring Multicamera System With Transparent Target for Panoramic Measurement

The calibration of a multicamera system is a prerequisite for high-accuracy measurement. In this study, a method for calibrating a ring multicamera system with a transparent target combined with digital image correlation (DIC) technology is proposed to achieve panoramic dynamic and static measurement of the object. The transparent target is placed in the middle of the ring camera array, and the target can be driven by the turntable to calibrate multiple cameras simultaneously. A complete refraction correction model is built on the transparent side of the target, and the corrected calibration data are integrated into the multicamera DIC system. The global optimization further improves the overall calibration accuracy of the system. Special markers are attached to the target surface to automatically identify the refractive and nonrefractive surfaces of the target. This type of camera can establish a coordinate relationship with the cameras on the other side of the target although no overlapping field of view exists between adjacent cameras on the same side of the target. The calibration data are analyzed, and dynamic and static experiments are implemented to verify the measurement accuracy of the system. All experimental results show that the calibration and measurement accuracies of the proposed method are high, and the system can be fully used for high-precision panoramic dynamic and static measurements.

Dynamic flame temperature tomography utilizing multi-camera common optical path

In the single projection direction, we propose a multi-camera common optical path system for three-dimensional radiation thermometry of dynamic flame, which is realized in the form of prism cascade. Different cameras can focus on certain sections of the flame along the main optical axis synchronously and independently. The image recorded by each camera linearly consists of a focused flame section and other remaining sections with defocused state. On the basis of point spread function, the spatial object-image mapping relationship is established by convolution, and the tomography of flame sections is performed by deconvolution. Then, the flame temperature is reconstructed by use of Planck's Law. The effectiveness of proposed method is verified by experiments, which could characterize the spatiotemporal evolution of flame temperature. Furthermore, it is expected to be able to achieve compatible measurements with high spatial and high temporal resolution.

Online position correction approach of an industrial robot by using a new photogrammetric measurement system

Six-axis industrial robots are widely used, mainly because of their versatility. However, they have one drawback: they are not as accurate as conventional machine tools. There are currently a number of approaches to increase the accuracy of these robots. This paper reports on the online correction of an ABB IRB 120 industrial robot using a novel photogrammetric measurement device, the Multi Aperture Positioning System (MAPS). As described by Luhmann, multi-camera solutions were previously required for the online measurement of the six degrees of freedom (6DOF = position and orientation in 3D space) of the robot TCP. MAPS offers the advantages of a single-camera system and at the same time the features and accuracy of a multi-camera system. To increase the accuracy of the robot, a correction algorithm is developed. It uses the measured values from MAPS to calculate the deviation of the robot Tool Center Point (TCP) position from the planned position and corrects it online if necessary. In this paper, three experiments are presented and their results discussed. Two to determine the position and trajectory accuracy of the robot and one to determine its z-stability on a trajectory in the xy plane. In each of these experiments, the robot end effector is tracked with the MAPS measurement device. After the actual state of the robot was recorded, all experiments were performed again, this time with the online correction. In all experiments, a significant increase in the accuracy of the robot can be observed. For example, the mean absolute error (MAE) of the position accuracy improved by a factor of 7 from 0.0207 to 0.0029 mm and the path accuracy (MAE) from 0.1140 to 0.0151 mm.

Deep Learning-Based Traffic Behavior Analysis under Multiple Camera Environment

In a video surveillance system, tracking multiple moving objects using a single camera feed is having numerous challenges. A multi-camera system increases the output image quality in both overlapping and non-overlapping environment. Traffic behavior analysis is an intensified demand in a recent topic of research. Due to increasing traffic in intercity roads, interstate, and national highways. Automated traffic visual surveillance applications with the multi-camera are a topic of research in computer vision. This paper, present a multi-camera system study for the overlapping area of the road for traffic analysis in three sections. The second section represents the thorough literature survey on the multi-camera system. Here, the third section is our proposed system using a dual-camera experimental setup with their coordination. A deep neural network is used in the experiments for traffic behavior analysis. The emphasis of this paper is on the physical arrangement of the multi-camera system, calibration, and advantages- disadvantages. On a conclusion note, future development and advancement in traffic analysis using a multi-camera system is discussed.

Minimal Solvers for Relative Pose Estimation of Multi-Camera Systems using Affine Correspondences

We propose three novel solvers for estimating the relative pose of a multi-camera system from affine correspondences (ACs). A new constraint is derived interpreting the relationship of ACs and the generalized camera model. Using the constraint, we demonstrate efficient solvers for two types of motions. Considering that the cameras undergo planar motion, we propose a minimal solution using a single AC and a solver with two ACs to overcome the degenerate case. Also, we propose a minimal solution using two ACs (a minimal number of one AC and one point correspondence) with known vertical direction, e.g., from an IMU. Since the proposed methods require significantly fewer correspondences than state-of-the-art algorithms, they can be efficiently used within RANSAC for outlier removal and initial motion estimation. The solvers are tested both on synthetic data and on three real-world scenes. It is shown that the accuracy of the estimated poses is superior to the state-of-the-art techniques. Source code is released at https://github.com/jizhaox/relative_pose_gcam_affine.

Supporting workspace awareness in remote assistance through a flexible multi-camera system and Augmented Reality awareness cues

Workspace awareness is critical for remote assistance with physical tasks, yet it remains difficult to facilitate. For example, if the remote helper is limited to the single viewpoint provided by the worker’s hand-held or head-mounted camera, she lacks the ability to gain an overview of the workspace. This may be addressed by granting the helper view-independence, e.g., through a multi-camera system. However, it can be cumbersome to set up and calibrate multiple cameras, and it can be challenging for the local worker to identify the current viewpoint of the remote helper. We present CueCam, a multi-camera remote assistance system that supports mutual workspace awareness through a flexible ad-hoc camera calibration and various Augmented Reality cues that communicate the helper’s viewpoint and focus. In particular, we propose visual cues presented through a head-mounted Augmented Reality display (Virtual Hand, Color Cue), and sound cues emitted from the cameras’ physical locations (Spatial Sound). Findings from a lab study indicate that all proposed cues effectively support the worker’s awareness of helper’s location and focus, while the Color Cue demonstrated superiority in task performance and preference ratings during a search task.

Development and experimental characterization of a robotic butterfly with a mass shifter mechanism

The development of biomimetic aerial robots has emerged as a new solution for studying the flight mechanisms of flying creatures. This study designs a biomimetic robotic butterfly steered via a mass shift mechanism named USTButterfly-II and investigates its flight characteristics using an optical tracking facility. First, a planar four-bar linkage was used to drive the flapping of the designed butterfly-like artificial wings. Next, an innovative tailless steering control method was proposed based on a mass shift mechanism. Finally, the wing kinematics and motion trajectory of the USTButterfly-II were measured using a multi-camera motion capture system, and some difficult-to-measure flapping aerodynamic parameters, such as the instantaneous net lift and thrust coefficients, were determined. These findings present a novel experimental framework that not only provides effective data support for the design and improvement of the robotic butterfly but also benefits the study of biological butterfly flight mechanisms.

Portable RGB-D Camera-Based System for Assessing Gait Impairment Progression in ATTRv Amyloidosis

Hereditary Amyloidosis associated with variant Transthyretin (ATTRv Amyloidosis) is a progressive and highly disabling neurological disorder that affects gait. Quantitative motion analysis is useful for assessing motor function, including gait, in diseases affecting movement. A single markerless RGB-D camera enables 3D full-body motion capture in a less expensive and intrusive, and more portable way than multi-camera marker-based systems. In this study, we examine whether a gait analysis system based on an RGB-D camera can be used to detect significant changes in the gait of ATTRv amyloidosis patients over time, when compared with a 12-camera system. We acquired 3D data provided by both systems from six ATTRv amyloidosis patients, while performing a simple gait task, once (T0) and 18 months later (T1). A direct comparison of systems has already been conducted. In this work, however, for each patient, we investigated if the RGB-D camera system detects statistically significant differences between the two different acquisitions in a similar way to the reference system, and whether it is reliable to use during patients’ follow-up. The obtained results show that the differences detected between T0 and T1 for both systems follow the same tendency for 65% of the spatiotemporal gait parameters, and for 38% of the kinematic parameters (38%). The most reliable parameters were: stride duration/length, gait speed (and its variability), and arm/foot swing velocity, all with an almost perfect strength of agreement.

Improving Worst Case Visual Localization Coverage via Place-Specific Sub-Selection in Multi-Camera Systems

6-DoF visual localization systems utilize principled approaches rooted in 3D geometry to perform accurate camera pose estimation of images to a map. Current techniques use hierarchical pipelines and learned 2D feature extractors to improve scalability and increase performance. However, despite gains in typical recall <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$@0.25\,{\rm{m}}$</tex-math></inline-formula> type metrics, these systems still have limited utility for real-world applications like autonomous vehicles because of their <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">worst</i> areas of performance - the locations where they provide insufficient recall at a certain required error tolerance. Here we investigate the utility of using <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">place specific configurations</i> , where a map is segmented into a number of places, each with its own configuration for modulating the pose estimation step, in this case selecting a camera within a multi-camera system. On the Ford AV benchmark dataset, we demonstrate substantially improved worst-case localization performance compared to using off-the-shelf pipelines - minimizing the percentage of the dataset which has low recall at a certain error tolerance, as well as improved overall localization performance. Our proposed approach is particularly applicable to the crowdsharing model of autonomous vehicle deployment, where a fleet of AVs are regularly traversing a known route.

Kinematic analysis of ankle joint during gait in drop foot patients wearing passive Ankle-Foot Orthosis *

The term ‘drop-foot’ refers to an uncommon presentation of degenerative diseases, derived from neurological, muscular or anatomical complications, resulting in difficulties in performing foot dorsiflexion. This causes an abnormal gait pattern, characterized by two complications: the ‘foot slap’ at the initial contact with the ground, and the ‘toe-drag’ during the swing phase. In order to support gait, a common non-surgical treatment is the use of lightweight ‘L’ shaped ankle-foot orthosis (AFO). In this study we combine gait analysis and biomechanical modeling tools to analyse the kinematics of ankle joint in gait with and without the support of a passive AFO, namely the ‘Codivilla’ spring. The study was conducted on a population of eighteen adult subjects suffering from unilateral drop-foot (7 females, age 58.1 ± 14.5 years, BMI 23.4 ± 4.60). The infrared multicamera system SMART DX by BTS Bioengineering (Milan, Italy), was used to acquire data relating the 3D position of 22 markers placed on patient’s body, following the Davis protocol, during walking trials performed in two conditions: barefoot and with the AFO mounted on the affected limb. Data were then imported in OpenSim (SimTK software) and used to solve an inverse kinematic problem in order to reproduce walking and compute the evolution of body joints angles. Ankle dorsiflexion and subtalar inversion signals were analysed using MATLAB (2022a) to compute the following metrics for each angle: Range of Motion (RoM), angle at Heel Strike (HS) and angle at Toe Off (TO). The two-way ANOVA statistical test was used to analyse the effects of two factors: walking condition (with versus without AFO) and limb (affected versus contralateral). The interaction factor (condition*limb) was also analysed. Fig. 1 shows the evolution over the gait cycle of the ankle and subtalar angles in the two walking conditions for each foot. Solid lines are obtained as the mean value of subjects’ averaged step. Shaded areas represent the variability among subjects. Table 1 reports the results of the two-way ANOVA test in terms of p-value. Results show that the AFO ‘Codivilla’ spring acts on the kinematics of ankle by reducing the RoM during walking, as it limits the plantar flexion of drop foot. However, it does not significantly affect the ankle angle at HS and TO, although the first is lower in the affected foot. Results also show that the evolution of subtalar angle in the affected foot is altered, not in the RoM, rather in values, in particular at the initial phases of gait cycle. However, the considered orthosis does not produce improvements in the kinematics of subtalar angle. This is probably due to the L-shape of the device, which does not limit ankle inversion/eversion. In conclusion our study confirms the alterations of the drop foot in the kinematics of the ankle joint, particularly at Heel Strike in which both the ankle and the subtalar angle are altered compared to the contralateral foot. The use of Codivilla spring results in a more physiological dorsiflexion of the ankle but has no significant effect on the kinematics of the subtalar joint.

Tracking the Body, Wing, and Leg Kinematics of Moving Mosquitoes.

In this protocol, we discuss general techniques for tracking the three-dimensional (3D) locations of the mosquito body, wings, legs, or other features of interest using videos. Tracking data must be acquired to produce detailed kinematics of moving mosquitoes. The software of focus for this protocol, DLTdv, was chosen for its widespread use and excellent support and because it is open-source. In addition, DLTdv allows both manual and automatic tracking. The automatic tracking can be done using a classic machine vision or machine-learning algorithm. The software supports both single-camera analysis and multicamera systems and can take advantage of sophisticated calibration algorithms, both for intrinsic lens distortion correction and for 3D DLT-based reconstruction. For this protocol, we assume all kinematic data is acquired post hoc through video analysis.

Designing a Generic Videography Experiment for Studying Mosquito Behavior.

In this protocol, we describe the basic design considerations and general method to set up a videography system to study mosquito behavior. A basic videography system to study mosquito behavior requires one or more cameras with an optical lens, camera lighting, a calibration setup, and a system to record the video data or otherwise control the camera. Here, we define two types of systems: (1) a real-time videography-based tracking system for determining the position of multiple moving (flying) mosquitoes, and (2) a high-fidelity videography system that can track the detailed movements of body, wings, and legs of a single mosquito at high spatial and temporal resolutions. These high-fidelity trackers are divided into single-camera systems for studying two-dimensional (2D) movements, and multicamera systems that can reconstruct three-dimensional (3D) movements of the mosquito.

IMPLEMENTACIÓN DE UN SISTEMA MULTICÁMARA DE ALTA PRECISIÓN PARA LA CORRECCIÓN DEL POSICIONAMIENTO DE UN BRAZO ROBÓTICO UTILIZADO EN OPERACIONES QUIRÚRGICAS

Photogrammetry is a technique that obtains information from objects and their environment through the processing of a set of images, thus obtaining a measurement. It computes the spatial reconstruction of a scene or object through a series of taken images from different points of view. Through the processing of these images, measurements or reconstructions of the observed object can be achieved. This paper describes the design and implementation of a multi-camera system that, using photogrammetry technique, is able to guide arm robotic movements to achieve higher degree of accuracy in its positioning regardless of the robot used. It is proposed to use a robotic arm, guided by the cameras, to perform brain retraction tasks in surgical procedures. In this procedure, it is necessary to manipulate the brain with pinpoint precision in order to access internal parts. Keywords: Multi-camera system, brain retraction, surgery, photogrammetry, robotics, precision engineering

Non-Invasive Data Acquisition and IoT Solution for Human Vital Signs Monitoring: Applications, Limitations and Future Prospects.

The rapid development of technology has brought about a revolution in healthcare stimulating a wide range of smart and autonomous applications in homes, clinics, surgeries and hospitals. Smart healthcare opens the opportunity for a qualitative advance in the relations between healthcare providers and end-users for the provision of healthcare such as enabling doctors to diagnose remotely while optimizing the accuracy of the diagnosis and maximizing the benefits of treatment by enabling close patient monitoring. This paper presents a comprehensive review of non-invasive vital data acquisition and the Internet of Things in healthcare informatics and thus reports the challenges in healthcare informatics and suggests future work that would lead to solutions to address the open challenges in IoT and non-invasive vital data acquisition. In particular, the conducted review has revealed that there has been a daunting challenge in the development of multi-frequency vital IoT systems, and addressing this issue will help enable the vital IoT node to be reachable by the broker in multiple area ranges. Furthermore, the utilization of multi-camera systems has proven its high potential to increase the accuracy of vital data acquisition, but the implementation of such systems has not been fully developed with unfilled gaps to be bridged. Moreover, the application of deep learning to the real-time analysis of vital data on the node/edge side will enable optimal, instant offline decision making. Finally, the synergistic integration of reliable power management and energy harvesting systems into non-invasive data acquisition has been omitted so far, and the successful implementation of such systems will lead to a smart, robust, sustainable and self-powered healthcare system.

Research on automatic positioning technology for magnetic field measurement of multipole magnet harmonic coil in heavy ion accelerator

In the magnetic field measurement of accelerator magnets, efficient and accurate positioning of the magnetic field measurement system and the magnet is a critical process. By analyzing the mechanical structure and working principle of harmonic coil magnetic field measurement system, an automatic positioning technology is proposed in this research for the high-precision and fast positioning of multipole magnet and harmonic coil magnetic field measurement system. Through the special design and comprehensive application of multi-camera photogrammetry system, control system and six-degree-of-freedom (6DoF) attitude automatic adjustment system, fast non-contact measurement and automatic attitude adjustment of harmonic coil magnetic field measurement system and multipole magnet can be realized. The proposed automatic positioning system and operation scheme can not only save labor costs, but also improve the reliability of harmonic coils and the positioning efficiency of magnetic field measurement system. The proposed research can also monitor the position changes of the harmonic coil and the multipole magnet in the magnetic field measurement process in real time through the multi-camera photogrammetry system, and further control the accuracy of the magnetic field measurement of the harmonic coil.