External Camera Research Articles

A New Deep Learning Methodology for Alarm Supervision in Marine Power Stations.

Marine engineering officers operate and maintain the ship's machinery during normal navigation. Most accidents on board are related to human factors which, at the same time, are associated with the workload of the crew members and the working environment. The number of alarms is so high that, most of the time, instead of helping to prevent accidents, it causes more stress for crew members, which can result in accidents. Convolutional Neural Networks (CNNs) are being employed in the recognition of images, which depends on the quality of the images, the image recognition algorithm, and the very complex configuration of the neural network. This research study aims to develop a user-friendly image recognition tool that may act as a visual sensor of alarms adjusted to the particular needs of the ship operator. To achieve this, a marine engineering simulator was employed to develop an image recognition tool that advises marine engineering officers when they are conducting their maintenance activities, with the aim to reduce their stress as a work risk prevention tool. Results showed adequate accuracy for three-layer Convolutional Neural Networks and balanced data, and the use of external cameras stands out for user-friendly applications.

Highly Photoresponsive Vertically Stacked Silicon Nanowire Photodetector with Biphasic Current Stimulator IC for Retinal Prostheses

This paper presents an integrated approach for a retinal prosthesis that overcomes the scalability challenges and limitations of conventional systems that use external cameras. Silicon nanowires (SiNWs) are utilized as photonic sensors due to their nanoscale dimensions and high surface-to-volume ratio. To enhance these properties and achieve high photoresponsivity, our research team developed a vertically stacked SiNW structure using a fabrication method entirely based on dry etching. The fabricated SiNW photodetector demonstrated excellent electrical and optical characteristics, including linear I–V characteristics that confirmed ohmic contact formation and high photoresponsivity exceeding 105 A/W across the 400–800 nm wavelength range. The SiNW photodetector, following its integration with a switched capacitor stimulator circuit, exhibited a proportional increase in stimulation current in response to higher light intensity and increased SiNW density. In vitro experiments confirmed the efficacy of the integrated system in inducing neural responses from retinal cells, as indicated by an increased number of neural spikes observed at higher light intensities and SiNW densities. This study contributes to sensor technology by demonstrating an approach to integrating nanostructures and electronic components, which enhances control and functionality.

Development of multi-view 3D reconstruction system for bubble flow measurement

This work introduces the development of bubble measurement method utilizing a three-dimensional (3D) reconstruction technique from multi-view images. Multiple synchronized cameras were positioned around a water container, and calibration was performed to obtain external and internal camera parameters. Images of bubbles emerging from a nozzle were captured, and a machine learning technique was used to extract bubble silhouettes as foreground probability distributions, enabling the extraction of bubbles from images obtained with a simple lighting setup. These distributions were then projected onto a 3D voxel space using the visual hull method. It was confirmed that the method can successfully capture bubble generation, detachment, and rise behaviors, offering insights into understanding bubble dynamics and potential applications in 3D computational fluid dynamics validation.

Automatic Setup And Calibration Of A Robotic-PIV System Using Fiducial Markers

The use of industrial robots allows the precise manipulation of all components necessary for setting up a large-scale particle image velocimetry (PIV) system. The known internal calibration matrix of the cameras in combination with the actual pose of the industrial robots and the calculated transform from the fiducial markers to camera coordinates allow the precise positioning of the individual PIV components according to the measurement demands. In addition, the complete calibration procedure for generating the external camera matrix and the mapping functions for e.g. dewarping the stereo images can be automatically determined without further user interaction and thus the degree of automation can be extended to nearly 100%. This increased degree of automation expands the applications range of PIV systems, in particular for measurement tasks with severe time constraints.

Head tracking using an optical soft tactile sensing surface.

This research proposes a sensor for tracking the motion of a human head via optical tactile sensing. It implements the use of a fibrescope a non-metal alternative to a webcam. Previous works have included robotics grippers to mimic the sensory features of human skin, that used monochrome cameras and depth cameras. Tactile sensing has shown advantages in feedback-based interactions between robots and their environment. The methodology in this paper is utilised to track motion of objects in physical contact with these sensors to replace external camera based motion capture systems. Our immediate application is related to detection of human head motion during radiotherapy procedures. The motion was analysed in two degrees of freedom, respective to the tactile sensor (translational in z-axis, and rotational around y-axis), to produce repeatable and accurate results. The movements were stimulated by a robot arm, which also provided ground truth values from its end-effector. The fibrescope was implemented to ensure the device's compatibility with electromagnetic waves. The cameras and the ground truth values were time synchronised using robotics operating systems tools. Image processing methods were compared between grayscale and binary image sequences, followed by motion tracking estimation using deterministic approaches. These included Lukas-Kanade Optical Flow and Simple Blob Detection, by OpenCV. The results showed that the grayscale image processing along with the Lukas-Kanade algorithm for motion tracking can produce better tracking abilities, although further exploration to improve the accuracy is still required.

Design of a Mobile Industrial Robot Platform based on Fixed Cameras

This study presents a design for an autonomous mobile robot system that utilizes environmental cameras for navigation and obstacle avoidance. Operating within a known environment, the robot employs these cameras to facilitate path planning and to circumvent obstacles. The hardware setup includes fixed cameras strategically placed in the environment to pinpoint the robot’s location and to identify obstacles amidst varying conditions. A computer system is essential for processing the data captured by the cameras, and a wireless transmission system communicates operational instructions to the robot.For navigation, the study explores various algorithms for path planning, selecting the most efficient one through comparative analysis. Once the path is established, the robot’s instructions are computed under the designed control rate, employing the vehicle’s PD control method. This method is informed by the external camera’s simulation of the vehicle, ensuring accurate and responsive navigation.The project’s goal is to create a mobile robot platform that leverages an external camera as its primary sensor. This design promises efficient autonomous navigation and the capability to avoid static obstacles. The robot’s design is conceptualized using SolidWorks and brought to life through implementation in the Python and MATLAB programming languages, demonstrating the system’s practicality and adaptability in real-world scenarios. Additionally, the system’s design considers ease of integration and scalability to accommodate future technological advancements and new application requirements.

Study on automated guided vehicle navigation method with external computer vision

Automated guided vehicle (AGV) navigation is extensively used in industrial manufacturing. Existing AGV navigation methods have high accuracy but usually require expensive positioning sensors. This paper proposes a novel method for AGV navigation based on external computer vision (NECV). No matter how many AGVs are in the workshop, the proposed NECV method uses only an external camera mounted on the top of the roof to detect and track AGVs, and all the AGVs don’t need to be equipped with any positioning sensors. Because there is no need to equip positioning sensors on AGVs, and also don’t need to arrange positioning signs, NECV significantly reduces the positioning cost of navigation. YOLOv8 was selected as the detector for NECV, and the training was completed using a prepared dataset. We improved the structure of the StrongSORT algorithm and used it as the tracker. The improved StrongSORT algorithm is the core of NECV. The imaging coordinates of the AGVs are detected by the detector, transformed into global coordinates through inverse perspective mapping, and passed to the master console. Experimental results indicated that the NECV detection deviation q of the AGV and the experimental accuracy metrics of the NECV after compensating q were considerably improved, close to those of the popular Quick Response (QR) code navigation method. Statistically, NECV can reduce the cost of AGV positioning detection by 90%.

Assessing the Acceptance of a Mid-Air Gesture Syntax for Smart Space Interaction: An Empirical Study

Mid-gesture interfaces have become popular for specific scenarios, such as interactions with augmented reality via head-mounted displays, specific controls over smartphones, or gaming platforms. This article explores the use of a location-aware mid-air gesture-based command triplet syntax to interact with a smart space. The syntax, inspired by human language, is built as a vocative case with an imperative structure. In a sentence like “Light, please switch on!”, the object being activated is invoked via making a gesture that mimics its initial letter/acronym (vocative, coincident with the sentence’s elliptical subject). A geometrical or directional gesture then identifies the action (imperative verb) and may include an object feature or a second object with which to network (complement), which also represented by the initial or acronym letter. Technically, an interpreter relying on a trainable multidevice gesture recognition layer makes the pair/triplet syntax decoding possible. The recognition layer works on acceleration and position input signals from graspable (smartphone) and free-hand devices (smartwatch and external depth cameras), as well as a specific compiler. On a specific deployment at a Living Lab facility, the syntax has been instantiated via the use of a lexicon derived from English (with respect to the initial letters and acronyms). A within-subject analysis with twelve users has enabled the analysis of the syntax acceptance (in terms of usability, gesture agreement for actions over objects, and social acceptance) and technology preference of the gesture syntax within its three device implementations (graspable, wearable, and device-free ones). Participants express consensus regarding the simplicity of learning the syntax and its potential effectiveness in managing smart resources. Socially, participants favoured the Watch for outdoor activities and the Phone for home and work settings, underscoring the importance of social context in technology design. The Phone emerged as the preferred option for gesture recognition due to its efficiency and familiarity. The system, which can be adapted to different sensing technologies, addresses the scalability concerns (as it can be easily extended for new objects and actions) and allows for personalised interaction.

Adaptive adjustment of brightness and blur of the camera for high precision internal parameter calibration

Camera calibration is a prerequisite for many computer vision tasks, such as visual measurement and visual localization. This paper proposes a new method for high-precision calibration of camera intrinsic parameters. By dynamically adjusting image brightness and blur to cope with changes in external lighting conditions and camera working distance, high-precision internal parameter measurement is achieved. This method solves the problems of the traditional camera intrinsic parameter calibration process being cumbersome and susceptible to changes in the placement of calibration boards and fluctuations in external lighting conditions. Firstly, an adaptive exposure adjustment algorithm is proposed to adjust the camera exposure value by statistically analyzing the average gray value of randomly selected rectangular regions on the calibration board, thereby solving the problem of the camera calibration accuracy being affected by overly bright or dark environments. Subsequently, an evaluation criterion and adaptive adjustment strategy for adjusting image blur are proposed. By extracting edge points of feature circles and obtaining the slope of the fitted curve, the relationship between edge sharpness and slope is utilized to adjust the image blur. Finally, the proposed method achieves high-precision semi-automatic calibration. Experimental results demonstrate that the proposed method has significant advantages in terms of correctness, robustness, and flexibility, with an average reprojection error of 0.0167.

Research on Image Stitching Technology for Road Surface Cracks

Image stitching refers to the combination of multiple images with a certain over lappingarea rate acquired by the image acquisition device into one image, which has a full picture of each image. Image stitching technology is an important part in the field of digital image processing. With the increasing requirements for image quality and the limited resolution of ordinary cameras, it is impossible to obtain large angle of view and clear image at the same time. In this paper, the key technologies involved in pavement crack image stitching are intensively studied, including image feature point detection and matching, internal and external camera parameter estimation and optimization, and image fusion. The work done in this paper and the main innovations include: (1)Two main feature detection and description algorithms, SIFT algorithm and ORBalgorithm, are studied. In view of the fact that the SIFT algorithm has redundancy when the Gaussian pyramid is established, this paper appropriately reduces the number of groups and maintains the number and stability of feature point detection. Combined with the advantages and disadvantages of the improved SIFT algorithm and ORB algorithm, and the performance of the two algorithms in the detection of pavement cracks, the improvedSIF'T algorithm is used as the feature point detection algorithm, and rBRlEF algorithm for describing the feature points in the ORB algorithm is used as the feature point description algorithm. The description algorithm is called the BSIF'T algorithm. Experiments show that the robustness of BSIFT algorithm is similar to that of SIFT algorithm, which is better than ORB algorithm and faster than SIF'T algorithm. (2)Image fiusion algorithms are studied, including best seam-line search and image fusion. Aiming at the ghost phenomenon that may occur in overlapping regions, this paper proposes a best seam-line search algorithm based on distance transform.

Improving the Temporal Accuracy of Eye Gaze Tracking for the da Vinci Surgical System through Automatic Detection of Decalibration Events and Recalibration

Robust and accurate eye gaze tracking can advance medical telerobotics by providing complementary data for surgical training, interactive instrument control, and augmented human–robot interactions. However, current gaze tracking solutions for systems such as the da Vinci Surgical System (dVSS) are limited to complex hardware installations. Additionally, existing methods do not account for operator head movement inside the surgeon console, invalidating the original calibration. This work provides an initial solution to these challenges that can seamlessly integrate into console devices beyond the dVSS. Our approach relies on simple and unobtrusive wearable eye tracking glasses and provides calibration routines that can contend with operator-head movements. An external camera measures movement of the glasses through trackers mounted on the glasses to detect invalidation of the prior calibration from head movement and slippage. Movements beyond a threshold of 5 cm or 9ˆ∘ prompt another calibration sequence. In a study where users moved freely in the surgeon console after an initial calibration procedure, we show that our system tracks the eye tracking glasses to initiate recalibration procedures. Recalibration can reduce the mean tracking error up to 89% compared to the current prevailing approach which relies on the initial calibration only. This work is an important first step towards incorporating user movement into gaze-based applications for the dVSS.

A feasibility study using an array of LaBr3(Ce) scintillation detectors as a Compton camera for prompt gamma imaging during BNCT

Boron Neutron Capture Therapy (BNCT) is a binary cancer therapy where a low energy neutron beam is incident upon a patient who has been administered a tumour-seeking 10B loaded compound. The neutron capture reaction on 10B results in the production of two short range particles, 7Li and 4He, that deposit all of their energies within the targeted cell. However, accurate, online dosimetry during BNCT is challenging as it requires knowledge of both the neutron fluence and 10B concentration in cells. An additional product in the neutron capture reaction on 10B is a 478 keV prompt gamma ray, and if the production vertices of these gamma rays could be imaged by an external camera, the density of the vertices could be used to infer the dose delivered to the patient. In this study, the feasibility of using an array of LaBr3 scintillators as a modified Compton camera for prompt gamma imaging during BNCT was investigated using Geant4 simulations. These simulations demonstrated that a phantom containing a 3 cm diameter region of 400 ppm 10B could be reconstructed using clinically relevant neutron fluences. This result opens up more possibilities for future research to improve dosimetry during BNCT.

High-precision fruit localization using active laser-camera scanning: Robust laser line extraction for 2D-3D transformation

Recent advancements in deep learning-based approaches have led to remarkable progress in fruit detection, enabling robust fruit identification in complex environments. However, much less progress has been made on fruit 3D localization, which is equally crucial for robotic harvesting. Complex fruit shape/orientation, fruit clustering, varying lighting conditions, and occlusions by leaves and branches have greatly restricted existing sensors from achieving accurate fruit localization in the natural orchard environment. In this paper, we report on the design of a novel localization technique, called Active Laser-Camera Scanning (ALACS), to achieve accurate and robust fruit 3D localization. The ALACS hardware setup comprises a red line laser, an RGB color camera, a linear motion slide, and an external RGB-D camera. Leveraging the principles of dynamic-targeting laser-triangulation, ALACS enables precise transformation of the projected 2D laser line from the surface of apples to the 3D positions. To facilitate laser pattern acquisitions, a Laser Line Extraction (LLE) method is proposed for robust and high-precision feature extraction on apples. Comprehensive evaluations of LLE demonstrated its ability to extract precise patterns under variable lighting and occlusion conditions. The ALACS system achieved average apple localization accuracies of 6.9 - 11.2 mm at distances ranging from 1.0 m to 1.6 m, compared to 21.5 mm by a commercial RealSense RGB-D camera, in an indoor experiment. Orchard evaluations demonstrated that ALACS has achieved a 95% fruit detachment rate versus a 71% rate by the RealSense camera. By overcoming the challenges of apple 3D localization, this research contributes to the advancement of robotic fruit harvesting technology.

Analysis of thermophysical parameters of solar water desalination plant with an external camera

In this work, an analysis of the thermophysical processes taking place inside a specially designed chamber with a geometry different from other works was carried out. This process is designed in COMSOL Multiphysics software. Boundary conditions were investigated for ambient temperature of 293.15 K and solar radiation of 1000 W/m2. The process was taken as natural convection. In this case, the flow of air with high humidity inside the solar water heater was analyzed. It can be seen that the air temperature rises to 450 K. At the same time, the speed of moist air inside the chamber, heat flow and other thermophysical quantities were determined and analyzed.

Investigation of Freestyle Performance in Swimmers with Different Equipments

The present study sought to examine the freestyle proficiency of juvenile swimmers deploying various accessories, namely fins, kickboard, and pull-buoy. Twenty-three participants, comprising 14 females and 9 males aged between 10 and 11 years, who reside in Istanbul, pursue activities in exclusive clubs, have held an athlete license for no less than three years, and expressed a keen interest in swimming opted to take part in the study. The research was implemented voluntarily in a private club's indoor swimming pool (25 m short lane). First, the swimmers' body weight, height, and arm length were measured. They then demonstrated their freestyle performance at distances of 25 m, 50 m, 75 m, and 100 m using different equipment: maximal, fins, pull-buoy, and kickboard-only flutter kick. Swimming performances with each piece of equipment were performed two days apart, and all swimmers' performances were recorded with an SJCAM 4k external camera. After conducting the test, the captured images were transferred to the computer and processed using the Kinovea 0.9.5 program in MP4 format. Individual calculations were made for each swimmer's finish time, lap times and velocity for each distance and equipment. The data were evaluated using the IBM SPSS 24.0 analysis program. It was ensured that the data followed a normal distribution ("±1.5"). The comparison of finish times, lap times and velocity of swimmers using different equipment was analysed using repeated measures variance. There was a statistical difference between the fins, kickboard and pull-buoy in finish time, lap times and velocity (p

Robot Operating System 2 (ROS2)-Based Frameworks for Increasing Robot Autonomy: A Survey

Future challenges in manufacturing will require automation systems with robots that are increasingly autonomous, flexible, and hopefully equipped with learning capabilities. The flexibility of production processes can be increased by using a combination of a flexible human worker and intelligent automation systems. The adoption of middleware software such as ROS2, the second generation of the Robot Operating System, can enable robots, automation systems, and humans to work together on tasks that require greater autonomy and flexibility. This paper has a twofold objective. Firstly, it provides an extensive review of existing literature on the features and tools currently provided by ROS2 and its main fields of application, in order to highlight the enabling aspects for the implementation of modular architectures to increase autonomy in industrial operations. Secondly, it shows how this is currently potentially feasible in ROS2 by proposing a possible high-level and modular architecture to increase autonomy in industrial operations. A proof of concept is also provided, where the ROS2-based framework is used to enable a cobot equipped with an external depth camera to perform a flexible pick-and-place task.

Leveraging machine learning for predicting and monitoring clogging in laser cladding processes: An exploration of neural sensors

This study addresses the development of smart neural sensors to predict the powder mass flow and track clogging in real time during laser cladding. The challenges posed by powder granulometry and challenging environmental conditions that can lead to delivery failures are considered. An extensive experimental setup was conducted that included manipulation of key factors, such as laser power, travel speed, Z-step, N-layers, nozzle-to-substrate distance, and two types of process patterns. The mass flow rate of the powder was used as an independent variable to evaluate the predictive ability of the neural sensor with respect to the mass flow rate. Several models were trained and evaluated with different datasets and images of the cladding equipment. The model that integrated all data and images showed the best accuracy and precision also showed a strong predictive power for real-time estimation of the powder mass flow rate. Considering two practical rules—an error detection time of no more than one second and a confidence interval of less than 1.8 g/min—two strategies were proposed to meet these criteria. The first recommends the use of the comprehensive “all-features” model, while the second proposes a simplified model (with Z-step, N-slices, and the external camera as inputs) for efficient real-time error detection. The study provides an understanding of powder clogging prediction in laser cladding and suggests strategies for leaders in the field. Future research should validate these results and test these models in different environments to predict complex cladding properties and support the development of stand-alone laser cladding systems.

Ultrathin Pyroelectric Photodetector with Integrated Polarization-Sensing Metasurface.

An abundance of metallic metasurfaces have been realized with miniscule, intricate features capable of tailored scattering, reflection, and absorption; however, high losses through heat limit their use in optoelectronics. Here, codesign of a detector and a polarization-sensing metasurface overcomes this challenge by utilizing the heat generation for integrated pyroelectric detection of the incoming light polarization. Using a nanogap metasurface with asymmetric metallic elements, polarization-sensitive photodetection exhibits high extinction ratios up to 19 for orthogonally polarized light and allows extraction of Stokes parameters with <12% deviation from theoretical values. This polarization-sensitive photodetector is ultrathin, consisting of active layers of only 290 nm, and exhibits fast response times of ∼2 ns. The structure is fully integrated, requiring no external cameras, detectors, or power sources, and points toward the creation of layered, multifunctional devices that utilize exotic metasurface properties for novel and compact sensing and imaging.

Electro-localization method using a muscle conductive phantom for needle position detection towards medical training

Simulation in healthcare can help train, improve, and evaluate medical personnel’s skills. In the case of needle insertion/manipulation inside the muscle during an nEMG examination, a training simulator Requires estimating the position of the needle to output the electrical muscle activity in real time according to the training plan. External cameras can be used to estimate the needle location; however, different error sources can make its implementation difficult and new medical sensing technologies are needed. This study introduces and demonstrates the feasibility of a conductive phantom that serves as the medium for needle insertion and senses the 3D needle position based on a technique named electro-localization for the first time. The proposed conductive phantom is designed so that different voltage distributions are generated in the phantom using electrodes placed on its borders. The needle is inserted in the phantom, and the recorded voltages are mapped to spatial coordinates using a finite element method (FEM)-based computational model of the conductive phantom to estimate the 3D needle tip position. Experimental and simulation results of phantom voltage distributions agreed. In 2D mapping (no depth consideration), the needle position error was 1.7 mm, which was marginally reduced if only the central area of the phantom was used (1.5 mm). In 3D mapping, the error was 4 mm. This study showed the feasibility of using a conductive muscle phantom as a new embedded sensor that estimates needle position for medical training of nEMG without relying on external sensors.

In-Situ Young’s Modulus Characterization of Elastomeric Microfluidic Chips With Micropillars

This study presents a novel in-situ Young’s modulus measurement technique of polydimethylsiloxane (PDMS) microfluidic chips that are designed to measure fluid viscosities. In such chips, viscosity can be accurately determined once the Young’s modulus of PDMS is also known. Yet, the Young’s modulus values of elastomers are highly dependent on their chemical composition and curing temperature. The presented approach involves integrating a piezoelectric sheet underneath the chip, oscillating the chip through sweeping the drive frequency, and monitoring the blur and speckle contrast of the pillar via an external camera. A local maximum in the blur and speckle contrast is achieved at the fundamental resonance of the pillars, from which its Young’s modulus is predicted. The experimental results validate the relationship between the Young’s modulus of PDMS and base polymer:curing agent ratio, curing time, and temperature, demonstrating that it could be measured optically. Six different PDMS microfluidic chips prepared by curing elastomer at different temperatures and using different base polymer to curing agent ratios, 5:1, 10:1, and 20:1, were utilized. The Young’s modulus of the prepared sample microfluidic chips varied within the range of 0.72 to 3.64 MPa. The successful measurement of the Young’s modulus of PDMS using the proposed techniques demonstrates its potential in determining the mechanical properties of fluidic chips in-situ, with a low-cost and simple setup.