Camera System Research Articles

Real Time Traffic Sign Recognition Using Integrated Camera Sensors and YOLOV8 Algorithm

Traffic signs are an important element in maintaining smoothness and safety on the road. However, there are still many drivers who violate them, causing various negative impacts. Traffic Sign Recognition (TSR) is a technology that has the ability to detect and identify various types of traffic signs by utilizing artificial intelligence in the computer vision domain. TSR has been applied in various vehicle applications, such as Advance Driver Assistance System (ADAS) and Autonomous Driving System (ADS). This system is made by integrating camera sensors with the YOLOv8 algorithm which has high accuracy and fast data processing. The data used were 2093 images and annotated through Roboflow. Then the data is trained through Google Collaboratory with a mAP evaluation result of 95.5% which shows that the system can detect objects accurately. The level of precision and success of the model in detecting objects is 93.5% and the success is 93.3%. The results of system testing can use images, videos, and through cameras in real time.

Targeting ankle proprioception: A novel, valid, and reliable approach to measuring ankle proprioception for clinical implementation: Suggested short title: Targeting ankle proprioception

Ankle proprioception is vital to maintain balance but proprioceptive measurement is challenging in clinic. A new clinical measure of ankle proprioception, measuring joint position error (JPE), was designed and tested for validity and reliability. A total of 20 young healthy adults attended 2 visits. Subjects performed a repositioning task with a laser affixed to the foot, projected to a wall target, for all four planes of ankle motion. Laser JPE was calculated and validated against a 3D motion capture camera system. Average ankle laser JPE was 4.25° (±1.20) across all motions. Laser JPE was strongly associated with camera JPE, with over 95 % agreement. Laser JPE also showed excellent agreement within person and across days (ICC≥0.91). These strong results, combined with ease of setup and procedures, suggest our new measure of ankle proprioception is clinically feasible and appropriate.

Online LiDAR-camera extrinsic parameters self-checking and recalibration

Abstract During production, smart cars are equipped with calibrated LiDARs and cameras. However, due to the vibrations that inevitably occur during driving, the sensors’ extrinsic parameters may change slightly over time. It is a significant challenge to ensure the ongoing security of these systems throughout the car’s lifetime. To address this issue, we propose a self-checking and recalibration algorithm that can continuously detect the sensor data of intelligent vehicles. If the sensor’s miscalibration is detected, the data can be repaired promptly to ensure the vehicle’s reliability. Our self-checking algorithm extracts features from the point cloud and image and performs pixel-wise comparisons. To improve feature quality, we utilize the patch-wise transformer to enhance local information exchange, which also benefits the subsequent extrinsic recalibration. To facilitate the study, we generate two customized datasets from the KITTI dataset and the Waymo Open Dataset. The experiments conducted on these datasets demonstrate the effectiveness of our proposed method in accurately calibrating the LiDAR and camera systems throughout the car’s lifetime. This study is the first to highlight the importance of continually checking the calibrated extrinsic parameters for autonomous driving. Our findings contribute to the broader goal of improving safety and reliability in autonomous driving systems. The dataset and code are available at https://github.com/OpenCalib/LiDAR2camera_self-check.

Using joint species distribution modelling to predict distributions of seafloor taxa and identify vulnerable marine ecosystems in New Zealand waters

Effective ecosystem-based management of bottom-contacting fisheries requires understanding of how disturbances from fishing affect seafloor fauna over a wide range of spatial and temporal scales. Spatial predictions of abundance for 67 taxa were developed, using an extensive dataset of faunal abundances collected using a towed camera system and spatially explicit predictor variables including bottom-trawl fishing effort, using a Joint Species Distribution Model (JSDM). The model fit metrics varied by taxon: the mean tenfold cross-validated AUC score was 0.70 ± 0.1 (standard deviation) for presence–absence and an R2 of 0.11 ± 0.1 (standard deviation) for abundance models. Spatial predictions of probability of occurrence and abundance (individuals per km2) varied by taxon, but there were key areas of overlap, with highest predicted taxon richness in areas of the continental shelf break and slope. The resulting joint predictions represent significant advances on previous predictions because they are of abundance, allow the exploration of co-occurrence patterns and provide credible estimates of taxon richness (including for rare species that are often not included in more commonly used single-species distribution modelling). Habitat-forming taxa considered to be Vulnerable Marine Ecosystem (VME) indicators (those taxa that are physically or functionally fragile to anthropogenic impacts) were identified in the dataset. Spatial estimates of likely VME distribution (as well as associated estimates of uncertainty) were predicted for the study area. Identifying areas most likely to represent a VME (rather than simply VME indicator taxa) provides much needed quantitative estimates of vulnerable habitats, and facilitates an evidence-based approach to managing potential impacts of bottom-trawling.

Velocimetry of coarse particles in pipeline flow based on GMM model and flow direction constraints

Coarse particle vertical pipeline hydraulic transport has been widely used in deep sea mining, petrochemical industry and shaft slag removal. The precise measurement of the velocity of coarse particles within a solid–liquid two-phase flow in a vertical pipe constitutes a pivotal and challenging aspect when investigating particle motion characteristics. We introduce a method for measuring coarse particle velocity that combines a GMM motion detection model with flow direction constraint matching. In this approach, a high-speed camera system captures the flow image of solid–liquid two-phase flow within a pipeline, and the GMM model analyzes the image to detect the motion particles. Subsequently, a rapid particle matching algorithm, incorporating flow direction constraints, is formulated to establish the particle matching relationships. Simulation and real experiments substantiate the efficacy of this method. Moreover, the method achieves high-precision matching with an accuracy of up to 99%. For videos captured at 30 frames per second (fps), the method enables real-time speed measurement, with an overall measurement error of less than 5%. In summary, this is an effective method for measuring coarse particle velocity field which is important to the study of the motion characteristics of solid–liquid two-phase flow.

Fixed-Wing UAV Pose Estimation Using a Self-Organizing Map and Deep Learning

In many Unmanned Aerial Vehicle (UAV) operations, accurately estimating the UAV’s position and orientation over time is crucial for controlling its trajectory. This is especially important when considering the landing maneuver, where a ground-based camera system can estimate the UAV’s 3D position and orientation. A Red, Green, and Blue (RGB) ground-based monocular approach can be used for this purpose, allowing for more complex algorithms and higher processing power. The proposed method uses a hybrid Artificial Neural Network (ANN) model, incorporating a Kohonen Neural Network (KNN) or Self-Organizing Map (SOM) to identify feature points representing a cluster obtained from a binary image containing the UAV. A Deep Neural Network (DNN) architecture is then used to estimate the actual UAV pose based on a single frame, including translation and orientation. Utilizing the UAV Computer-Aided Design (CAD) model, the network structure can be easily trained using a synthetic dataset, and then fine-tuning can be done to perform transfer learning to deal with real data. The experimental results demonstrate that the system achieves high accuracy, characterized by low errors in UAV pose estimation. This implementation paves the way for automating operational tasks like autonomous landing, which is especially hazardous and prone to failure.

Detection of Growth Stages of Chilli Plants in a Hydroponic Grower Using Machine Vision and YOLOv8 Deep Learning Algorithms

Vertical indoor farming (VIF) with hydroponics offers a promising perspective for sustainable food production. Intelligent control of VIF system components plays a key role in reducing operating costs and increasing crop yields. Modern machine vision (MV) systems use deep learning (DL) in combination with camera systems for various tasks in agriculture, such as disease and nutrient deficiency detection, and flower and fruit identification and classification for pollination and harvesting. This study presents the applicability of MV technology with DL modelling to detect the growth stages of chilli plants using YOLOv8 networks. The influence of different bird’s-eye view and side view datasets and different YOLOv8 architectures was analysed. To generate the image data for training and testing the YOLO models, chilli plants were grown in a hydroponic environment and imaged throughout their life cycle using four camera systems. The growth stages were divided into growing, flowering, and fruiting classes. All the trained YOLOv8 models showed reliable identification of growth stages with high accuracy. The results indicate that models trained with data from both views show better generalisation. YOLO’s middle architecture achieved the best performance.

On-site calibration method for an underwater stereoscopic camera system based on Scheimpflug principle

On-site calibration method for an underwater stereoscopic camera system based on Scheimpflug principle

Study on the contribution matrix for EAST radial neutron camera system

The EAST (Experiment Advanced Superconductor Tokamak) Radial Neutron Camera (RNC) is designed to measure the temporal and spatial distribution of fast neutrons. The system primarily consists of detector components, shielding collimators, support structure, and data acquisition system. We have studied on the neutronic transport through MCNP with EAST reference D-D neutron source (EAST #77160), and calculated the plasma neutron source chord integral along the observation sightlines and scattering neutrons from structural facilities. Contribution matrixes, incorporating both uncollided and scattered neutrons determines the response from the neutron source to every detector unit. We evaluate the RNC primarily through the variation of the RNC sightlines to improve detector performance by reducing interference particles, thereby laying the foundation for subsequent upgrades. Benefiting from these measurements, we can better study the detector signals in relation to corresponding plasma configurations in the future.

5G as Communication Platform for Solar Tower Plants

Wiring of heliostat fields for solar tower plants is a cost factor that becomes more important as the overall cost target is decreasing. Wireless heliostats with radio communication and autarchic energy supply have therefore been proposed in the past. But none of the communication solutions investigated so far could realistically scale to commercial size plants with tens of thousands of heliostats. Moreover, the digitalization of CSP plants with numerous mobile and stationary sensor systems requires a suitable data communication, too. The new generation of mobile radio communication (5G) is capable of handling the heterogenous communication profile portfolio comprising large numbers of units with low data rates – like heliostats - and few units with very large data volume – like e.g. drone-based camera systems. The communication requirements of a typical solar tower installation are assessed in this work and a data traffic model is created for the most relevant communication channels. The various existing 5G implementations are assessed to find the most suitable solution. Different operator models for 5G are considered and their applicability in CSP target countries is discussed. A simulation test case is presented that models the radio communication traffic of a heliostat field control during a cloud passage. Finally, the experimental 5G campus network is introduced that is currently installed at the Solar Tower Jülich research plant and will be operated in the upcoming months to demonstrate the technical feasibility of 5G radio for communication in solar fields.

The Importance of an Open Camera System Demonstrated with Wide-ranging Applications of MerlinEM, Hybrid Pixel Direct Electron Detector for Scanning Transmission Electron Microscopy

The Importance of an Open Camera System Demonstrated with Wide-ranging Applications of MerlinEM, Hybrid Pixel Direct Electron Detector for Scanning Transmission Electron Microscopy

A Review of Patient Bed Sensors for Monitoring of Vital Signs.

The analysis of biomedical signals is a very challenging task. This review paper is focused on the presentation of various methods where biomedical data, in particular vital signs, could be monitored using sensors mounted to beds. The presented methods to monitor vital signs include those combined with optical fibers, camera systems, pressure sensors, or other sensors, which may provide more efficient patient bed monitoring results. This work also covers the aspects of interference occurrence in the above-mentioned signals and sleep quality monitoring, which play a very important role in the analysis of biomedical signals and the choice of appropriate signal-processing methods. The provided information will help various researchers to understand the importance of vital sign monitoring and will be a thorough and up-to-date summary of these methods. It will also be a foundation for further enhancement of these methods.

Retinal Image Dataset of Infants and Retinopathy of Prematurity

Retinopathy of prematurity (ROP) represents a vasoproliferative disease, especially in newborns and infants, which can potentially affect and damage the vision. Despite recent advances in neonatal care and medical guidelines, ROP still remains one of the leading causes of worldwide childhood blindness. The paper presents a unique dataset of 6,004 retinal images of 188 newborns, most of whom are premature infants. The dataset is accompanied by the anonymized patients’ information from the ROP screening acquired at the University Hospital Ostrava, Czech Republic. Three digital retinal imaging camera systems are used in the study: Clarity RetCam 3, Natus RetCam Envision, and Phoenix ICON. The study is enriched by the software tool ReLeSeT which is aimed at automatic retinal lesion segmentation and extraction from retinal images. Consequently, this tool enables computing geometric and intensity features of retinal lesions. Also, we publish a set of pre-processing tools for feature boosting of retinal lesions and retinal blood vessels for building classification and segmentation models in ROP analysis.

An Adaptive Learning Control for MIMO Nonlinear System with Nonuniform Trial Lengths and Invertible Control Gain Matrix

In the traditional iterative learning control (ILC) method, the operational time interval is conventionally fixed to facilitate a seamless learning process along the iteration axis. However, this condition may frequently be contravened in real-time applications owing to unknown uncertainties and unpredictable factors. In essence, replicating a control system at a consistent time interval proves challenging in practical scenarios. This paper proposes an adaptive iterative learning control (AILC) method for the multi-input–multi-output (MIMO) nonlinear system with nonuniform trial lengths and an invertible control gain matrix. Compared to the existing AILC research that features nonuniform trial lengths, the control gain matrix of the system in this paper is assumed to be invertible. Hence, the general requirement in the conventional AILC method that the control gain matrix of the system is positive-definite (or negative-definite) or even known is relaxed. Moreover, the tracking reference allows it to be iteration-varying. Finally, to prove the convergence of the system, the composite energy function is introduced and to verify the validity of the AILC method, a robot movement imitation with an uncalibrated camera system is used. The simulation results show that the actual output can track the desired reference trajectory well, and the tracking error converges to zero after 30 iterations.

Method of Identification of Combat Vehicles Based on Yolo

A method for recognizing contours of objects in a video data stream is proposed. Data will be uploaded using a video camera in real time and object recognition will be performed. We will use the YOLO network – a method of identifying and recognizing objects in real time. Recognized objects will be recorded in a video sequence showing the contours of the objects. The approach proposed in the project reasonably synthesizes methods of artificial intelligence, theories of computer vision on the one hand, and pattern recognition on the other; it makes it possible to obtain control influences and mathematical functions for decision-making at every moment of time with the possibility of analyzing the influence of external factors and forecasting the flow of processes, and refers to the fundamental problems of mathematical modeling of real processes. The installation of the neural network is shown in detail. The characteristics of the neural network are shown and its capabilities are substantiated. Approaches to computer vision for object extraction are shown. Well-known methods are methods of expanding areas, methods based on clustering, contour selection, and methods using a histogram. The work envisages building a system for rapid identification of combat vehicles based on the latest image filtering methods developed using deep learning methods. The time spent on identifying the machine will be 10 –20 % shorter, thanks to the developed new information technology for detecting objects in conditions of rapidly changing information.

Definition of multispectral camera system parameters to model the asteroid 2001 SN263

In 2012, Brazil began the studies to send its first deep space exploration mission, ASTER, which would be the first mission to orbit a triple asteroid system, 2001 SN263. We aim to contribute to the ASTER mission by defining the parameters of a multispectral camera system that will be used to study the asteroid system 2001 SN263, through software simulations that should help planning the data collection. We inserted the shape model of the objects in the software POV-Ray and modeled two cameras, a Wide Angle (WAC) and a Narrow Angle (NAC). We inserted the asteroid's parameters and simulated the satellite position. We created various scenes so we could obtain a good view of the asteroid. Alpha is entirely visible only in the WAC images, while the NAC is expected to reveal surface details. Beta seems relatively small in the WAC images, whereas we obtain a broad view from the NAC at 100 km distance. Gamma, smaller than Beta, should provide more detailed images through the NAC, whereas the WAC images should be able to show its inclined orbit around Alpha. To see Gamma behind Alpha in its revolution movement, we would have to elevate the camera's orbit. The method employed to simulate images generated by satellite cameras can be applied to other scenarios where the target requires imaging, extending beyond the field of planetary geology.

Automatic welding seam tracking and real-world coordinates identification with machine learning method

In the automatic welding process, the precision of the welding position significantly influences welding quality, relying heavily on the accurate identification of the welding seam. However, in complex and unstructured welding environments, such as those with strong arc lights, welding splashes, and thermal-induced deformations, various disturbances present a significant challenge to identify the welding seam. This paper introduces a cost-effective welding seam tracking model constructed using a laser line and a fixed viewpoint single RGB camera. The model's primary objective is to detect a chain of welding-seam points in both captured images and real-time camera feeds. The paper presents the techniques for camera calibration, laser detection, and the use of artificial intelligence to establish a real-world coordinate frame. The process begins with camera calibration using the OpenCV library to filter noise and undistort images. Subsequently, the thresholding technique is employed for color image segmentation, isolating the laser sensor light on the workpiece. An algorithm is then introduced to detect welding spots. Finally, the image coordinates are converted to world coordinates using two machine learning methods: Random Forest Regression and Gradient Boosting Regression. Experimental demonstrations were conducted at different positions on the object to assess algorithm precision. The results indicate average deviations of 0.83 mm, 0.74 mm, and 0.77 mm for X, Y, and Z coordinates, respectively, which fall below the expected standard deviation of 3 mm. Despite the overall consistency of the program, slight noise from environmental lighting may still exist. In conclusion, the proposed welding seam tracking method and coordinate conversion program have proven effective and cost-efficient, with potential for future improvements.

Trinocular 360-degree stereo for accurate all-round 3D reconstruction considering uncertainty

This research proposes a method for accurate all-round 3D reconstruction of an indoor environment in one-shot using a system of trinocular 360-degree cameras. Binocular 360-degree stereo is unable to reconstruct in all directions due to lack of disparity along epipolar directions. Thus, a third camera along a perpendicular epipolar direction is introduced to cover for this, making the system trinocular. However, previous works with trinocular stereo did not adequately take into account the uncertainty of disparity estimation and geometric constraints around 3D reconstruction. Therefore, we propose a geometric optimization scheme considering disparity estimation uncertainty and show that this results in both higher accuracy and lesser outliers along epipolar directions, in both simulated and real environments.

Experimental study on the elastic response characteristics of a rotating body under an oblique water impact load

In this study, an integrated experimental model was designed and a data acquisition system was established to capture the structural response of an elastic rotating body throughout the oblique water impact process. This established experimental methodology offers a dependable approach for gathering structural response data during the water impact process. The axial impact loads and effective stress on the rotating body throughout water impact were measured using acceleration sensor and strain gauge rosettes installed in the water impact zone of the structure, while a high-speed camera system captured the evolution of the flow field. Results from the arbitrary Lagrangian-Eulerian (ALE) numerical model were compared with those from repeated experiments, revealing good agreements. The article conducted water impact tests at different impact initial conditions, analyzing the response of the body to the impact load during water entry process across different water impact velocities and angles. Observations during the oblique water impact process revealed that the stress response at the center of the rotating body impact surface is approximately equal to that of the first water impact point, with this effect becoming more pronounced at higher velocities. This study contributes valuable experimental data for numerical investigations into the water impact of elastic structures.

Airside Surveillance by Computer Vision in Low-Visibility and Low-Fidelity Environment

Low visibility at airports can significantly impact airside capacity, leading to ground delays and runway/taxiway incursions. Digital tower technology, enabled by live camera feeds, leverages computer vision to enhance airside surveillance and operational efficiency. However, technical challenges in digital camera systems can introduce low-fidelity transmission effects such as blurring, pixelation, or JPEG compression. Additionally, adverse weather conditions like rain and fog can further reduce visibility for tower controllers, whether from digital video or out-of-tower views. This paper proposes a computer vision framework and deep learning algorithms to detect and track aircraft in low-visibility (due to bad weather) and low-fidelity (due to technical issues) environments to enhance visibility using digital video input. The framework employs a convolutional neural network for aircraft detection and Kalman filters for tracking, especially in low-visibility conditions. Performance enhancements come from pre- and postprocessing algorithms like object filtering, corrupted image detection, and image enhancement. It proves effective on an airport video dataset from Houston Airport, enhancing visibility under adverse weather conditions.