Vision Monitoring Research Articles

A computer vision and point cloud-based monitoring approach for automated construction tasks using full-scale robotized mobile cranes

A computer vision and point cloud-based monitoring approach for automated construction tasks using full-scale robotized mobile cranes

Multirate UKF Damage Identification Based on Computer Vision Monitoring of Ship–Bridge Collisions

Multirate UKF Damage Identification Based on Computer Vision Monitoring of Ship–Bridge Collisions

Cr3+-Doped LiAlO2 NIR-I Emitting Phosphors with Superior Resistance to Thermal Quenching for Night Vision Monitoring and Bioimaging.

Near-infrared phosphor-converted light-emitting diodes (NIR pc-LEDs) as a new NIR light source has outstanding application potential in the fields of NIR spectroscopy analysis, sensing, imaging, and pattern recognition. Therefore, the development of NIR phosphors with good NIR luminescence thermal stability has attracted much attention. To this end, we developed LiAlO2:Cr3+ garnet-type NIR phosphors by a high-temperature solid-state reaction method. Under 405 nm excitation, the Cr3+ ions located in tetrahedrons of LiAlO2 emit NIR emission in a broadband NIR emission that covers 650-900 nm mixed with several sharp narrowband R-line emissions, which showed excellent luminescence thermal stability with integrated intensity of emission measured at 573 K is ∼90.86% of that measured at 303 K caused by good structural rigidity and low thermal expansion coefficient of the matrix material. An NIR pc-LED device assembled with the optimized LiAlO2:Cr3+ and a commercially available purple LED chip emitted NIR output power of ∼98.172 mW at a driving current of 300 mA and demonstrated an electro-optical conversion efficiency of ∼9.09%, while demonstrated it has excellent application potential in the fields of night vision monitoring and biomedical imaging.

Recent Advancements in Nanomaterials for Near‐Infrared to Long‐Wave Infrared Photodetectors

AbstractNanomaterials have superior electronic, optical, and mechanical properties making them highly suitable for a range of applications in optoelectronics, biomedical fields, and photonics. Nanomaterials‐based IR detectors are rapidly growing due to enhanced sensitivity, wide spectral range, and device miniaturization compared to commercial photodetectors. This review paper focuses on the significant role of nanomaterials in infrared detection, an area critical for enhancing night vision and health monitoring technologies. The latest advancements in IR photodetectors that employ various nanomaterials and their hybrids are discussed. The manuscript covers the operational mechanisms, device designing, performance optimization strategies, and material challenges. This review aims to provide a comprehensive overview of the current developments in nanomaterial‐based IR photodetectors and to identify key directions for future research and technological advancements.

Advances in Structural Health Monitoring: Bio-Inspired Optimization Techniques and Vision-Based Monitoring System for Damage Detection Using Natural Frequency

This paper introduces the improvements in natural-frequency-based SHM by applying bio-inspired optimization methods and a vision-based monitoring system for effective damage detection. This paper proposes a natural frequency extraction method using a motion magnification-based vision monitoring system with bio-inspired optimization techniques to estimate the damage location and depth in a cantilever beam. The proposed optimization techniques are inspired by natural processes and biological evolution including genetic algorithms, particle swarm optimization, sea lion optimization, and coral reefs optimization. To verify the performance of each bio-inspired optimization method, the eigenvalues of a two-bay truss structure are used for estimating the damaged elements. Then, using the proposed video motion magnification method, the natural frequency for each undamaged and damaged cantilever beam is extracted and compared with the LDV sensor to verify the proposed vision-based monitoring system. The performance of each bio-inspired optimizer in damage detection is compared. As a result, coral reefs optimization shows the lowest average error, around 1%, in damage detection using the natural frequency.

Digital technologies for signal radio vision and radio monitoring

Objectives. Radiophysical processes involving the electrodynamic formation of signal radio images diffusely scattered by the signature of small-sized objects or induced by the near field of radio devices are relevant for identifying radiogenomic (cumulant) features of objects in the microwave range in the development of neuroimaging ultra-short pulse (USP) signal radio vision systems, telemonitoring, and near-radio detection. The paper sets out to develop methods and algorithms for vector analysis of radio wave deformation of nonstationary fields forming a signal radio image based on radiophysical and topological characteristics of small-sized objects; to develop software and hardware for registration and neural network recognition of signal radio images, including methods for the synthesis and extraction of signal radiogenomes using digital twins of objects obtained through vector electrodynamic modeling; and to analyze signal radio images induced by elements of printed topology of electronic devices.Methods. The study is based on statistical radiophysics methods, time-frequency approaches for wavelet transformation of USP radio images, numerical electrodynamic methods for creating digital twins of small-sized objects, as well as neural network authentication algorithms based on the cumulant theory of pole-genetic and resonant physically unclonable functions used in identifying signal radio images.Results. The results of fundamental research on electrodynamic effects of vector-wave deformation of nonstationary fields of sub-nanosecond configuration are presented as a means of identifying and authenticating signal radio images. Neural network techniques for cumulant formation of radio genomes of signal radio images are proposed on the basis of pole-genetic and resonant functions.Conclusions. A radiogenome, representing the unique authenticator of a radio image, is shown to be formed on the basis of physically unclonable functions determined by the structure and set of radiophysical parameters of the image. Cumulant features of signal radio images identified on the basis of pole-genetic and physically unclonable resonant functions of small-sized objects are revealed.

Optic nerve elongation during fronto-facial surgery for Crouzon syndrome: 3D quantification and clinical implications.

Fronto-facial monobloc advancement with internal distraction (FFMBA) is a key procedure in the management of syndromic craniosynostoses. FFMBA involves circumferential dissection and linear enlargement of the orbit, potentially leading to mechanical stress on the optic nerve (ON). Several reports of transient vision loss during the distraction process led us to investigate ON shape modifications during facial advancement, with the aim to potentially refine current clinical guidelines on postoperative management and the distraction schedule. Twenty-six patients with Crouzon syndrome were included in this study. ONs were segmented on pre- and postoperative CT scans. Distraction amplitudes, linear and curved lengths, and cross-section diameters of the ON were assessed along the main axis of the nerve. A two-level hierarchical multivariate linear model was used to screen for factors associated with ON morphology. The mean age at FFMBA was 4.4 ± 3.8 years. Two patients presented with transient impaired vision during distraction. The final mean fronto-orbital and temporo-zygomatic distraction amplitudes were 18 ± 4 mm and 18 ± 6 mm, respectively. At the end of distraction, ONs were elongated (+1.8 mm for curved lengths, p = 0.013), and their mean cross-section was reduced (-1.9 mm2, p < 0.001) in the proximal intraorbital portion (first 15 mm). In the 2 patients with visual symptoms, functional impairment was associated with ON area reduction (OR 0.487, p < 0.001) and increased temporo-zygomatic distraction amplitude (OR 2.240, p < 0.001). ON was elongated during FFMBA, with proximal diameter reduction. Transient visual impairment with normal fundus examination during distraction seemed to have a morphological basis, based on 2 cases. These results suggest the importance of vision monitoring associated with fundus examination during distraction, and advocate for early extubation after FFMBA to allow clinical follow-up.

Data-fusion for in-situ monitoring and molten state identification during LPBF of NiCoCr medium-entropy alloy

Laser powder bed fusion (LPBF) is an additive manufacturing technology with high practical value. In order to improve the quality of the fabricated parts, process monitoring has become a crucial solution, offering the potential to ensure manufacturing stability and repeatability. However, a cardinal challenge involves discerning a precise correlation between process characteristics and potential defects. This paper elucidates the integration of an off-axis vision monitoring mechanism via a high-speed camera focused on capturing the single-track melting phenomenon. An innovative image processing method was devised to segment the plume and spatters, while Kalman filter was employed for multi-object tracking of the spatters. The features of both the plume and spatters were extracted, and their relationship with molten states was investigated. Finally, the PSO-XGBoost algorithm was utilized to identify five molten states, achieving an accuracy of 92.16%. The novelty of this approach resides in its unique combination of plume characteristics, spatter features, and computationally efficient machine learning models, which collectively address the challenge of limited field of view prevalent in real production scenarios, thereby enhancing process monitoring efficacy. Relative to existing methodologies, the proposed PSO-XGBoost approach offers heightened accuracy, convenience, and appropriateness for the monitoring of the LPBF process. This work provides an effective and novel approach to monitor the LPBF process and evaluate the part fabrication quality for complex and changeable working conditions.

Achromatic and Coma-Corrected Hybrid Meta-Optics for High-Performance Thermal Imaging.

Long-wave infrared (LWIR) imaging, or thermal imaging, is widely applied in night vision and security monitoring. However, the widespread use of LWIR imagers is impeded by their bulky size, considerable weight, and high cost. While flat meta-optics present a potential solution to these limitations, existing pure LWIR meta-optics face constraints such as severe chromatic or coma aberrations. Here, we introduce an approach utilizing large-scale hybrid meta-optics to address these challenges and demonstrate the achromatic, coma-corrected, and polarization-insensitive thermal imaging. The hybrid metalens doublet is composed of a metasurface corrector and a refractive lens, featuring a full field-of-view angle surpassing 20° within the 8-12 μm wavelength range. Employing this hybrid metalens doublet, we showcase high-performance thermal imaging capabilities both indoors and outdoors, effectively capturing ambient thermal radiation. The proposed hybrid metalens doublet holds considerable promise for advancing miniaturized, lightweight, and cost-effective LWIR optical imaging systems.

Research on electronic optical monitoring technology and system design for vacuum electron beam welding process

Electron beam welding technology is increasingly used in industrial manufacturing due to its high power density and the purity of welded joints. In vacuum environments, traditional optical and machine vision monitoring methods are susceptible to contamination by metal vapors, which hinders the monitoring effect. Due to the limitations of optical monitoring methods, electronic optical imaging technology is a new alternative monitoring method that has been introduced into the electron beam processing in recent years, but the complexity of the industrial electron beam welding environment makes it relatively less studied in the field of electron beam welding. In this study, a YAG scintillator detector is proposed to collect the backscattered electron information generated by the interaction of electrons with the sample, and an FPGA real-time processing system is used to realize the weld quality monitoring in the industrial electron beam welding processes. The distribution law of backscattered electrons in space is simulated by Monte Carlo method, and the results show that the number of distribution in the 40°∼60° angle region is more, the backscattering coefficient increases with the increase of the incident angle of the electron beam, and the backscattering coefficient of the samples with large atomic numbers is relatively high. The field monitoring experiments show that the designed system exhibits good sensitivity to the characteristic morphology, and the imaging resolution reaches more than 200 μm, and the weld hole defects are clearly visible. A good linear correlation between the current change curve during scanning and the actual shape profile is shown, while the image resolution is better when the spot current is 1 mA.

Improved broadband luminescence in Gd2GaSb1-xTaxO7:Cr3+,Yb3+ pyrochlore phosphors: Near-infrared spectroscopic applications and dual-mode optical thermometry

The growing demand for spectroscopic applications in the fields of night vision monitoring, non-destructive analysis and bioimaging has led to extensive research into near infrared (NIR) light sources. Thus, the design of high-performance broadband Cr3+-doped NIR phosphors is of urgent and critical importance. Herein, novel Gd2GaSb1-xTaxO7:Cr3+,Yb3+ NIR-emitting pyrochlore phosphors were successfully explored. Via partly substituting Sb5+ by Ta5+ and tuning Cr3+ amount, a significant enhancement, broadening and redshift of Cr3+ NIR luminescence are exhibited under the radiation of 460 nm light. Moreover, higher thermal stability and wider NIR luminescence are realized by the proper introduction of Yb3+. A satisfactory NIR output power of 46.51 mW @ 300 mA and photoelectric efficiency of 10.82 % @ 10 mA of the fabricated NIR phosphor-converted light-emitting diodes (NIR pc-LEDs) based on Gd1.96Ga0.97Sb0.9Ta0.1O7:3%Cr3+,4%Yb3+ phosphor enable the demonstration in night vision and bioimaging applications. Besides, superior dual-mode optical thermometry was successfully constructed on fluorescence intensity ratio (Cr3+/Yb3+) and fluorescence lifetime (4T2) of Cr3+ in Gd2GaSb1-xTaxO7:Cr3+,Yb3+. These results confirm that Gd2GaSb1-xTaxO7:Cr3+,Yb3+ can serve as inspiring luminescent converters employed in NIR pc-LEDs and non-contact temperature sensing.

Displacement measurement of underground prefabricated metro stations by computer vision technology

The increasing applications and structural vulnerability of prefabricated metro stations have resulted in a growing demand for monitoring their displacements during the construction and post-construction phases. This paper proposes a dual-camera computer vision monitoring system consisting of a dual-camera base station, a series of monitoring prisms and two reference prisms. The displacement and rotation of the camera station are considered to reduce measurement errors caused by camera motion. The system has been applied in the displacement monitoring of the 164 m assembly section of an underground station in Shenzhen during the construction and post-construction periods. A movable target equipped with a spiral micrometer is used for field calibration experiments, and the results show that the average measurement deviation is 0.01 mm and the standard deviation is 0.068 mm. During the 5-month monitoring, displacement caused by backfilling, rainfall, and other construction conditions are precisely reflected, which shows a potential correlation between uneven displacement and some minor structural defects. At the post-construction period, settlement variance is relatively small, and the temperature effect causes horizontal convergence to fluctuate.

A physics-informed deep learning liquid crystal camera with data-driven diffractive guidance

Whether in the realms of computer vision, robotics, or environmental monitoring, the ability to monitor and follow specific targets amidst intricate surroundings is essential for numerous applications. However, achieving rapid and efficient target tracking remains a challenge. Here we propose an optical implementation for rapid tracking with negligible digital post-processing, leveraging an all-optical information processing. This work combines a diffractive-based optical nerual network with a layered liquid crystal electrical addressing architecture, synergizing the parallel processing capabilities inherent in light propagation with liquid crystal dynamic adaptation mechanism. Through a one-time effort training, the trained network enable accurate prediction of the desired arrangement of liquid crystal molecules as confirmed through numerical blind testing. Then we establish an experimental camera architecture that synergistically combines an electrically-tuned functioned liquid crystal layer with materialized optical neural network. With integrating the architecture into optical imaging path of a detector plane, this optical computing camera offers a data-driven diffractive guidance, enabling the identification of target within complex backgrounds, highlighting its high-level vision task implementation and problem-solving capabilities.

Investigation and analysis of the status of myopia and related behavior in children and adolescents in Northeast Sichuan.

In this study, we examined the current status of myopia among primary and secondary school children in northeastern Sichuan to analyze the factors connected to myopia and provide data support and a theoretical foundation for the prevention and control of myopia. Using a cross-sectional study and a comprehensive sampling survey, 34,138 students aged 5-19 years were screened for refraction in 22 primary and secondary schools in Langzhong, and 4000 behavioral questionnaires were delivered at random. After evaluation and rational problem-solving, a total of 3764 valid questionnaires were obtained. SPSS 23.0 statistical software was used for data analysis. The percentage of myopia among primary and secondary school students in Langzhong was 65.61%, with female students having a higher rate than male students ( P < 0.05); 52.81% of primary school students, 86.26% of secondary school students, and 88.17% of high school students had myopia. The incidence of myopia detection increased with school age ( P < 0.001), indicating a correlation between age and myopia prevalence. The prevalence of myopia was mainly low (40.53%) and moderate myopia (19.89%). The prevalence of high myopia (5.19%) was relatively high. The prevalence of myopia among female students (5.54%) was greater than that in male students ( P < 0.05) and increased with age ( P < 0.001). The proportion of students who wore eyeglasses was 24.36%, with a larger proportion of female students (25.93%) than male students (22.61%) ( P < 0.001). In addition, the rate of eyeglass use increased with school age ( P < 0.001). A logistic regression analysis revealed that higher grade point averages, female gender, and long-term usage of electronic items were risk factors for myopia. The results of the questionnaire survey revealed that students in this region were under immense pressure to perform well academically, spent a lot of time engaged in near-work activities, and had a low rate of myopia awareness; 24.43% of the students had not had a vision examination in the previous year, indicating that parents did not pay sufficient attention to eye health. The incidence of myopia among children and teenagers is high in Northeast Sichuan, and the outlook for addressing the problem is bleak. Therefore, it is critical to improve vision monitoring and eye health education.

College student management based on machine vision and intelligent monitoring system

College student management based on machine vision and intelligent monitoring system

Tunable near-infrared phosphor LiSc2SbO6:Cr3+ by Mg2+ substitution and Yb3+ co-doping for phosphor-converted light-emitting diodes

The NIR pc-LED prepared with LMSS:Cr3+,Yb3+ performed well in night vision, nondestructive testing, multiple anti-counterfeiting and information encryption technologies.

Simple Non‐Destructive and 3D Multi‐Layer Visual Hull Reconstruction with an Ultrabroadband Carbon Nanotubes Photo‐Imager

AbstractWhile millimeter‐wave (MMW)–infrared (IR) broad photo‐imaging exhibits significant potential as an inspection technique, photo‐thermoelectric (PTE) detector operations employing carbon nanotubes (CNTs) are highly suitable because of their efficient absorption characteristics in the above bands. Such an imaging device design and the associated material coordination allow the aggregation of multiple wavelength‐specific optical information for the constituent identification of target objects in a non‐destructive manner with high sensitivity. On the other side, computer vision techniques facilitate 3D structure reconstruction as approaches from inspection methodologies, in addition to the aforementioned imaging device design. Although non‐destructive inspections require collectively satisfying both constituent identification and structural reconstruction, investigations combining MMW–IR broad photo imaging and computer vision monitoring are still insufficient. This work demonstrates computer vision‐driven simple 3D composite multi‐layer reconstructions via non‐destructive broadband multiple‐wavelength monitoring with the CNT film PTE imager. Visual hull evaluation, which is one of the representative computer vision techniques, converts 2D transmissive PTE images with different viewpoints into 3D reconstruction models. The subsequential graphical superposition of these wavelength‐specific models results in non‐destructive entire reconstructions of 3D composite multi‐layer target objects. Therefore, the present computer vision‐driven PTE broadband 3D reconstruction bridges the gap between methodologies and device design strategies toward non‐destructive inspection applications.

Tunable efficient visible and near-infrared multi-emission in Ho3+ doped Cs2NaBiCl6 double perovskite via Ag+ alloying

Herein, we successfully obtained Ho3+ and Ag+ -codoped Cs2NaBiCl6 halide double perovskites (DPs) via a hydrothermal method, which exhibit orange-red broadband self-trapped excitons (STE) emission and visible (Vis) and near-infrared (NIR) narrow band emission attributed to Ho3+f-f transition, under 360 nm excitation. Compared to the single Ho3+-doped sample, the Vis PL intensity of the codoped sample was increased by nearly 13.8 times, and an enhanced photoluminescence quantum yield (PLQY) of 87.45 % was achieved. The PL enhancement is attributed to the partial substituting of Na+ with Ag+, leading to an efficient STE emission, and the more energy transition from STE to Ho3+. Moreover, the synthesized phosphor exhibits good stability in thermal and atmospheric environments. White light-emitting diode (WLED) is fabricated by employing Cs2Na0·997Ag0·003Bi0.935Ho0.065Cl6 on near-ultraviolet (n-UV) LED chips, achieving a color rendering index (CRI) of 95.0 and correlated color temperature (CCT) of 3814 K. Anti-counterfeit patterns and the application for night vision monitoring were also discussed. This study inspires the designing of high-quality phosphors of rare earth-doped DPs and expands the multi-domain luminescence applications of halide DPs.

Anomaly Detection Based on a 3D Convolutional Neural Network Combining Convolutional Block Attention Module Using Merged Frames.

With the recent rise in violent crime, the real-time situation analysis capabilities of the prevalent closed-circuit television have been employed for the deterrence and resolution of criminal activities. Anomaly detection can identify abnormal instances such as violence within the patterns of a specified dataset; however, it faces challenges in that the dataset for abnormal situations is smaller than that for normal situations. Herein, using datasets such as UBI-Fights, RWF-2000, and UCSD Ped1 and Ped2, anomaly detection was approached as a binary classification problem. Frames extracted from each video with annotation were reconstructed into a limited number of images of 3×3, 4×3, 4×4, 5×3 sizes using the method proposed in this paper, forming an input data structure similar to a light field and patch of vision transformer. The model was constructed by applying a convolutional block attention module that included channel and spatial attention modules to a residual neural network with depths of 10, 18, 34, and 50 in the form of a three-dimensional convolution. The proposed model performed better than existing models in detecting abnormal behavior such as violent acts in videos. For instance, with the undersampled UBI-Fights dataset, our network achieved an accuracy of 0.9933, a loss value of 0.0010, an area under the curve of 0.9973, and an equal error rate of 0.0027. These results may contribute significantly to solve real-world issues such as the detection of violent behavior in artificial intelligence systems using computer vision and real-time video monitoring.

Ni2+-doped MgTa2O6 phosphors capable of near-infrared II and III emission under blue-light excitation

Presently, Ni2+-activated near-infrared (NIR) phosphors still have disadvantages such as relatively short emission wavelength and narrow half-peak width, mismatched excitation wavelength to that of commercially available high-efficiency blue chips. Here, Ni2+-doped tetragonal MgTa2O6 NIR phosphor was successfully prepared by high-temperature solid phase method. The matrix material provided an octahedral lattice for Ni2+ to replace Ta5+ in the [TaO6] octahedron of the crystal lattice. Under the excitation of 470 nm blue light, Ni2+ was excited through 3T2(F)→3A2(F) spin-allowed transition and emitted an ultra-broadband NIR light that was 1300–2000 nm in wavelength, peaked ∼ 1620 nm, covered the NIR-II and NIR-III regions, had a full width at half maximum reached 297 nm, and a PLQY of 25.62 %. The unique ultra-broadband and long wavelength NIR emission by this phosphor was derived from Ni2+ which is situated in a weak crystal field environment due to space asymmetric distortion caused by a high charge polarization surrounding the center of the [TaO6] octahedron in a MgTa2O6 matrix. A NIR pc-LED was constructed using Ni2+-doped near-infrared phosphors packaged with a commercial high-efficiency blue LED chip (@ 470 nm), and its application prospect was demonstrated by NIR night vision monitoring and nondestructive imaging of venous in human fingers.