Visual Research Articles

Comparative analysis of personal protective equipment nonadherence detection: computer vision versus human observers.

Human monitoring of personal protective equipment (PPE) adherence among healthcare providers has several limitations, including the need for additional personnel during staff shortages and decreased vigilance during prolonged tasks. To address these challenges, we developed an automated computer vision system for monitoring PPE adherence in healthcare settings. We assessed the system performance against human observers detecting nonadherence in a video surveillance experiment. The automated system was trained to detect 15 classes of eyewear, masks, gloves, and gowns using an object detector and tracker. To assess how the system performs compared to human observers in detecting nonadherence, we designed a video surveillance experiment under 2 conditions: variations in video durations (20, 40, and 60 seconds) and the number of individuals in the videos (3 versus 6). Twelve nurses participated as human observers. Performance was assessed based on the number of detections of nonadherence. Human observers detected fewer instances of nonadherence than the system (parameter estimate -0.3, 95% CI -0.4 to -0.2, P < .001). Human observers detected more nonadherence during longer video durations (parameter estimate 0.7, 95% CI 0.4-1.0, P < .001). The system achieved a sensitivity of 0.86, specificity of 1, and Matthew's correlation coefficient of 0.82 for detecting PPE nonadherence. An automated system simultaneously tracks multiple objects and individuals. The system performance is also independent of observation duration, an improvement over human monitoring. The automated system presents a potential solution for scalable monitoring of hospital-wide infection control practices and improving PPE usage in healthcare settings.

Intelligent Vision System with Pruning and Web Interface for Real-Time Defect Detection on African Plum Surfaces

Agriculture stands as the cornerstone of Africa’s economy, supporting over 60% of the continent’s labor force. Despite its significance, the quality assessment of agricultural products remains a challenging task, particularly at a large scale, consuming valuable time and resources. The African plum is an agricultural fruit that is widely consumed across West and Central Africa but remains underrepresented in AI research. In this paper, we collected a dataset of 2892 African plum samples from fields in Cameroon representing the first dataset of its kind for training AI models. The dataset contains images of plums annotated with quality grades. We then trained and evaluated various state-of-the-art object detection and image classification models, including YOLOv5, YOLOv8, YOLOv9, Fast R-CNN, Mask R-CNN, VGG-16, DenseNet-121, MobileNet, and ResNet, on this African plum dataset. Our experimentation resulted in mean average precision scores ranging from 88.2% to 89.9% and accuracies between 86% and 91% for the object detection models and the classification models, respectively. We then performed model pruning to reduce model sizes while preserving performance, achieving up to 93.6% mean average precision and 99.09% accuracy after pruning YOLOv5, YOLOv8 and ResNet by 10–30%. We deployed the high-performing YOLOv8 system in a web application, offering an accessible AI-based quality assessment tool tailored for African plums. To the best of our knowledge, this represents the first such solution for assessing this underrepresented fruit, empowering farmers with efficient tools. Our approach integrates agriculture and AI to fill a key gap.

Demyelination and neurodegeneration early in experimental autoimmune encephalomyelitis contribute to functional deficits in the anterior visual pathway

Impaired visual function is a prevalent feature of optic neuritis (ON) in multiple sclerosis (MS). Abnormal visual evoked potential (VEP) findings of increased latencies, reduced amplitudes and abnormal waveforms as well as decreased retinal nerve fiber layer (RNFL) assessed by optical coherence tomography (OCT) are hallmarks of ON-induced visual dysfunction. Here we utilized the experimental autoimmune encephalomyelitis (EAE) mouse model of MS to investigate the functional and pathological progression during early (before any clinical symptoms), peak (initial maximal clinical symptoms), and late (chronic disease for > 3 weeks) disease stages. Demyelination and initial stages of axon damage were observed in early EAE. Significant demyelination, inflammation, increased axon damage and impaired P1/N2 amplitudes and latencies by VEP were seen in middle and late EAE groups. A decrease in RNFL thickness by OCT was observed only during late EAE. NanoString analysis of optic nerves from late EAE indicated elevated inflammation-related genes, reduced myelin-related genes, and changes in axon degeneration-related genes. Early inflammatory demyelination and functional deficits of the visual pathway, if untreated, may lead to severe irrecoverable axon damage in EAE. These studies potentially help explain the progression of visual dysfunction during MS.

Distinct cortical spatial representations learned along disparate visual pathways.

The authors declare no competing financial interests.

Regional tuning of photoreceptor adaptation in the primate retina

Adaptation in cone photoreceptors allows our visual system to effectively operate over an enormous range of light intensities. However, little is known about the properties of cone adaptation in the specialized region of the primate central retina called the fovea, which is densely packed with cones and mediates high-acuity central vision. Here we show that macaque foveal cones exhibit weaker and slower luminance adaptation compared to cones in the peripheral retina. We find that this difference in adaptive properties between foveal and peripheral cones is due to differences in the magnitude of a hyperpolarization-activated current, Ih. This Ih current regulates the strength and time course of luminance adaptation in peripheral cones where it is more prominent than in foveal cones. A weaker and slower adaptation in foveal cones helps maintain a higher sensitivity for a longer duration which may be well-suited for maximizing the collection of high-acuity information at the fovea during gaze fixation between rapid eye movements.

Longitudinal In Vivo Imaging of Retinal Cells with Tailored Adeno-Associated Virus Serotypes via Confocal Scanning Laser Ophthalmoscopy.

The dynamic nature of retinal cellular processes necessitates advancements in gene delivery and live monitoring techniques to enhance the understanding and treatment of ocular diseases. This study introduces an optimized adeno-associated virus (AAV) approach, utilizing specific serotypes and promoters to achieve optimal transfection efficiency in targeted retinal cells, including retinal ganglion cells (RGCs) and Müller glia. Leveraging the precision of confocal scanning laser ophthalmoscopy (CSLO), this work presents a non-invasive method for in vivo imaging that captures the longitudinal expression of AAV-mediated green fluorescent protein (GFP). This approach eliminates the need for terminal procedures, preserving the continuity of observation and the well-being of the subject. Furthermore, the GFP signal can be traced in AAV-infected RGCs along the visual pathway to the superior colliculus (SC) and lateral geniculate nucleus (LGN), enabling the potential for direct visual pathway mapping. These findings provide a detailed protocol and demonstrate the application of this powerful tool for real-time studies of retinal cell behavior, disease pathogenesis, and the efficacy of gene therapy interventions, offering valuable insights into the living retina and its connections.

Seeing on the fly: Physiological and behavioral evidence show that space-to-space representation and processing enable fast and efficient performance by the visual system.

When we view the world, our eyes saccade quickly between points of interest. Even when fixating a target our eyes are not completely at rest but execute small fixational eye movements (FEMs). That vision is not blurred despite this ever-present jitter has seemingly motivated an increasingly popular theory denying the reliance of the visual system on pure spatial processing in favor of a space-to-time mechanism generated by the eye drifting across the image. Accordingly, FEMs are not detrimental but rather essential to good visibility. However, the space-to-time theory is incompatible with physiological data showing that all information is conveyed by the short neural volleys generated when the eyes land on a target, and with our faithful perception of briefly displayed objects, during which time FEMs have no effect. Another difficulty in rejecting the idea of image representation by the locations and nature of responding cells in favor of a timecode, is that somewhere, somehow, this code must be decoded into a parallel spatial one when reaching perception. Thus, in addition to the implausibility of generating meaningful responses during retinal drift, the space-to-time hypothesis calls for replacing efficient point-to-point parallel transmission with a cumbersome, delayed, space-to-time-to-space process. A novel physiological framework is presented here wherein the ability of the visual system to quickly process information is mediated by the short, powerful neural volleys generated by the landing saccades. These volleys are necessary and sufficient for normal perception without FEMs contribution. This mechanism enables our excellent perception of brief stimuli and explains that visibility is not blurred by FEMs because they do not generate useful information.

Sky-GVIO: Enhanced GNSS/INS/Vision Navigation with FCN-Based Sky Segmentation in Urban Canyon

Accurate, continuous, and reliable positioning is critical to achieving autonomous driving. However, in complex urban canyon environments, the vulnerability of stand-alone sensors and non-line-of-sight (NLOS) caused by high buildings, trees, and elevated structures seriously affect positioning results. To address these challenges, a sky-view image segmentation algorithm based on a fully convolutional network (FCN) is proposed for NLOS detection in global navigation satellite systems (GNSSs). Building upon this, a novel NLOS detection and mitigation algorithm (named S−NDM) uses a tightly coupled GNSS, inertial measurement units (IMUs), and a visual feature system called Sky−GVIO with the aim of achieving continuous and accurate positioning in urban canyon environments. Furthermore, the system combines single-point positioning (SPP) with real-time kinematic (RTK) methodologies to bolster its operational versatility and resilience. In urban canyon environments, the positioning performance of the S−NDM algorithm proposed in this paper is evaluated under different tightly coupled SPP−related and RTK−related models. The results exhibit that the Sky−GVIO system achieves meter-level accuracy under the SPP mode and sub-decimeter precision with RTK positioning, surpassing the performance of GNSS/INS/Vision frameworks devoid of S−NDM. Additionally, the sky-view image dataset, inclusive of training and evaluation subsets, has been made publicly accessible for scholarly exploration.

Learning Internal Representations of 3D Transformations From 2D Projected Inputs.

We describe a computational model for inferring 3D structure from the motion of projected 2D points in an image, with the aim of understanding how biological vision systems learn and internally represent 3D transformations from the statistics of their input. The model uses manifold transport operators to describe the action of 3D points in a scene as they undergo transformation. We show that the model can learn the generator of the Lie group for these transformations from purely 2D input, providing a proof-of-concept demonstration for how biological systems could adapt their internal representations based on sensory input. Focusing on a rotational model, we evaluate the ability of the model to infer depth from moving 2D projected points and to learn rotational transformations from 2D training stimuli. Finally, we compare the model performance to psychophysical performance on structure-from-motion tasks.

Magnetic Railway Sleeper Detector

In an ever expanding railway network all around the world, the need for track maintenance grows steadily. Traditionally, one major part of track maintenance is ramming large vibrating steel picks into the gravel between and under railway sleepers to compress the gravel and generate a safe substructure. Even today, maintenance personnel still have to manually locate the sleepers if they cannot be detected by computer vision systems or visually by the operator. Here we developed a first of its kind magnetic sleeper detector, even able to find sleepers, buried in gravel, undetectable by vision based systems. Our approach of magnetic detection is based on a DC magnetic field excitation and a detector moving with respect to the rail system, including the sleepers and fasteners for mounting the rails. Due to railway application constraints a large air gap between the sensor and the sleeper structure is required, which significantly complicates the magnetic sensing task for robust sleeper detection. The design and optimization of the magnetic circuit was based on extensive 3D simulation studies to ensure highest possible variation in magnetic flux density at the sensor locations for absence and presence of a sleeper. Furthermore, a low noise and high sensitivity electronic circuit has been realized to cope with sensor signal offsets from unknown or changing sensor orientations with respect to the earth’s magnetic field, or magnetic interferences from other trains potentially passing by during active measurements. Since we only want to detect sleepers in close vicinity of the moving sensor system, digital signal processing of the acquired signals can easily compensate for disturbing slowly changing or static field components within real world application scenarios. We demonstrate that magnetic detection of even buried sleepers on railway tracks is possible for distances of up to 172 mm between the sensor and the sleeper. This enables an even higher level of railway maintenance automation previously impossible in certain scenarios.

MiR-210 is essential to retinal homeostasis in fruit flies and mice

BackgroundmiR-210 is one of the most evolutionarily conserved microRNAs. It is known to be involved in several physiological and pathological processes, including response to hypoxia, angiogenesis, cardiovascular diseases and cancer. Recently, new roles of this microRNA are emerging in the context of eye and visual system homeostasis. Recent studies in Drosophila melanogaster unveiled that the absence of miR-210 leads to a progressive retinal degeneration characterized by the accumulation of lipid droplets and disruptions in lipid metabolism. However, the possible conservation of miR-210 knock-out effect in the mammalian retina has yet to be explored.ResultsWe further investigated lipid anabolism and catabolism in miR-210 knock-out (KO) flies, uncovering significant alterations in gene expression within these pathways. Additionally, we characterized the retinal morphology of flies overexpressing (OE) miR-210, which was not affected by the increased levels of the microRNA. For the first time, we also characterized the retinal morphology of miR-210 KO and OE mice. Similar to flies, miR-210 OE did not affect retinal homeostasis, whereas miR-210 KO mice exhibited photoreceptor degeneration. To explore other potential parallels between miR-210 KO models in flies and mice, we examined lipid metabolism, circadian behaviour, and retinal transcriptome in mice, but found no similarities. Specifically, RNA-seq confirmed the lack of involvement of lipid metabolism in the mice’s pathological phenotype, revealing that the differentially expressed genes were predominantly associated with chloride channel activity and extracellular matrix homeostasis. Simultaneously, transcriptome analysis of miR-210 KO fly brains indicated that the observed alterations extend beyond the eye and may be linked to neuronal deficiencies in signal detection and transduction.ConclusionsWe provide the first morphological characterization of the retina of miR-210 KO and OE mice, investigating the role of this microRNA in mammalian retinal physiology and exploring potential parallels with phenotypes observed in fly models. Although the lack of similarities in lipid metabolism, circadian behaviour, and retinal transcriptome in mice suggests divergent mechanisms of retinal degeneration between the two species, transcriptome analysis of miR-210 KO fly brains indicates the potential existence of a shared upstream mechanism contributing to retinal degeneration in both flies and mammals.

Measurements of chromatic adaptation and luminous efficiency while wearing colored filters.

The visual system adapts dynamically to stabilize perception over widely varying illuminations. Such adaptation allows the colors of objects to appear constant despite changes in spectral illumination. Similarly, the wearing of colored filters also alters spectral content, but this alteration can be more extreme than typically encountered in nature, presenting a unique challenge to color constancy mechanisms. While it is known that chromatic adaptation is affected by surrounding spatial context, a recent study reported a gradual temporal adaptation effect to colored filters such that colors initially appear strongly shifted but over hours of wear are perceived as closer to an unfiltered appearance. Presently, it is not clear whether the luminance system adapts spatially and temporally like the chromatic system. To address this, spatial and temporal adaptation effects to a colored filter were measured using tasks that assess chromatic and luminance adaptation separately. Prior to and for 1 hour after putting on a pair of colored filters, participants made achromatic and heterochromatic flicker photometry (HFP) settings to measure chromatic and luminance adaptation, respectively. Results showed significant chromatic adaptation with achromatic settings moving closer to baseline settings over 1 hour of wearing the filters and greater adaptation with spatial context. Conversely, there was no significant luminance adaptation and HFP matches fell close to what was predicted photometrically. The results are discussed in the context of prior studies of chromatic and luminance adaptation.

Adaptive spatial down-sampling method based on object occupancy distribution for video coding for machines

As the performance of machine vision continues to improve, it is being used in various industrial fields to analyze and generate massive amounts of video data. Although the demand for and consumption of video data by machines has increased significantly, video coding for machines needs to be improved. It is therefore necessary to consider a new codec that differs from conventional codecs based on the human visual system (HVS). Spatial down-sampling plays a critical role in video coding for machines because it reduces the volume of the video data to be processed while maintaining the shape of the data’s features that are important for the machine to reference when processing the video. An effective method of determining the intensity of spatial down-sampling as an efficient coding tool for machines is still in the early stages. Here, we propose a method of determining an optimal scale factor for spatial down-sampling by collecting and analyzing information on the number of objects and the ratio of the area occupied by the object within a picture. We compare the data reduction ratio to the machine accuracy error ratio (DRAER) to evaluate the performance of the proposed method. By applying the proposed method, the DRAER was found to be a maximum of 21.40 dB and a minimum of 11.94 dB. This shows that video coding gain for the machines could be achieved through the proposed method while maintaining the accuracy of machine vision tasks.

Deep-sea fish reveal alternative pathway for vertebrate visual development.

Vertebrate vision is accomplished by two phenotypically distinct types of photoreceptors in the retina: the saturation-resistant cones for the detection of bright light and the highly sensitive rods for dim light conditions [1]. The current dogma is that, during development, all vertebrates initially feature a cone-dominated retina, and rods are added later [2, 3]. By studying the ontogeny of vision in three species of deep-sea fishes, we show that their larvae express cone-specific genes in photoreceptors with rod-like morphologies. Through development, these fishes either retain this rod-like cone retina ( Maurolicus mucronatus ) or switch to a retina with true rod photoreceptors with expression of rod-specific genes and transcription factors ( Vinciguerria mabahiss and Benthosema pterotum ). In contrast to the larvae of most marine fishes, which inhabit the bright upper layer of the open ocean, the larvae of deep-sea fishes occur deeper, exposing them to a dimmer light environment [4-7]. Spectral maxima predictions from molecular dynamics simulations and environmental light estimations suggest that using transmuted photoreceptors that combine the characteristics of both cones and rods maximises visual performance in these dimmer light conditions. Our findings provide molecular, morphological, and functional evidence for the evolution of an alternative developmental pathway for vertebrate vision.

A Near-Infrared Retinomorphic Device with High Dimensionality Reservoir Expression.

Physical reservoir-based reservoir computing (RC) systems for intelligent perception have recently gained attention because they require fewer computing resources. However, the system remains limited in infrared (IR) machine vision, including materials and physical reservoir expression power. Inspired by biological visual perception systems, the study proposes a near-infrared (NIR) retinomorphic device that simultaneously perceives and encodes narrow IR spectral information (at ≈980nm). The proposed device, featuring core-shell upconversion nanoparticle/poly (3-hexylthiophene) (P3HT) nanocomposite channels, enables the absorption and conversion of NIR into high-energy photons to excite more photo carriers in P3HT. The photon-electron-coupled dynamics under the synergy of photovoltaic and photogating effects influence the nonlinearity and high dimensionality of the RC system under narrow-band NIR irradiation. The device also exhibits multilevel data storage capability (≥8 levels), excellent stability (≥2000 s), and durability (≥100 cycles). The system accurately identifies NIR static and dynamic handwritten digit images, achieving recognition accuracies of 91.13% and 90.07%, respectively. Thus, the device tackles intricate computations like solving second-order nonlinear dynamic equations with minimal errors (normalized mean squared errorof 1.06 × 10⁻3 during prediction).

Artificial general intelligence for the upstream geoenergy industry: A review

Artificial General Intelligence (AGI) is set to profoundly impact the traditional upstream geoenergy industry (i.e., geothermal energy, oil and gas industry) by introducing unprecedented efficiencies and innovations. This paper explores AGI's foundational principles and its transformative applications, particularly focusing on the advancements brought about by large language models (LLMs) and extensive computer vision systems in the upstream sectors of the industry. The integration of Artificial Intelligence (AI) has already begun reshaping the upstream geoenergy landscape, offering enhancements in production optimization, downtime reduction, safety improvements, and advancements in exploration and drilling techniques. These technologies streamline logistics, minimize maintenance costs, automate monotonous tasks, refine decision-making processes, foster team collaboration, and amplify profitability through error reduction and actionable insights extraction. Despite these advancements, the deployment of AI technologies faces challenges, including the necessity for skilled professionals for implementation and the limitations of model training on constrained datasets, which affects the models' adaptability across different contexts. The advent of generative AI, exemplified by innovations like ChatGPT and the Segment Anything Model (SAM), heralds a new era of high-density innovation. These developments highlight a shift towards natural language interfaces and domain-knowledge-driven AI, promising more accessible and tailored solutions for the upstream geoenergy industry. This review articulates the vast potential AGI holds for tackling complex operational challenges within the upstream geoenergy industry, requiring near-human levels of intelligence. We discussed the promising applications, the hurdles of large-scale AGI model deployment, and the necessity for domain-specific knowledge in maximizing the benefits of these technologies.

A Review of the Application of the Laser-Light Backscattering Imaging Technique to Agricultural Products

Growing concerns about food safety and waste have increased consumer demand for high-quality agricultural products, particularly at the postharvest stage. This demand has prompted the development of non-destructive methods to assess or inspect the internal quality of fruits and vegetables. The backscattering imaging technique, also known as diffuse reflectance imaging, is considered a highly promising approach. Numerous studies have focused on practical applications, using laser light at selected wavelengths to develop quick multispectral methods. Due to the rapid interaction of photons with biological tissue, together with the highly computational performance of machine vision, backscattering imaging can offer a valuable alternative to traditional methods. Its primary benefits include quick measurements without chemical sample preparation, easy integration with high-throughput automatic quality control, and reduced waste, since this non-destructive technique does not damage samples. This review presents a comprehensive overview of backscattering imaging, including the measurement geometry, data analysis, and design considerations for vision systems. Additionally, it explores this technique’s advantages, challenges, and accuracy, as demonstrated using various case studies.

Bilateral differences in structural connectivity of the afferent visual pathways of children with perinatal stroke

Bilateral differences in structural connectivity of the afferent visual pathways of children with perinatal stroke

SCIENTIFIC AND METHODOLOGICAL APPROACH TO DETERMINING THE DEGREE OF DANGER OF AN AIR TARGET BY CREW OF A COMBAT VEHICLE WITH A TURRETED MACHINE GUN

In the course of hostilities in eastern Ukraine, the main threat to critical infrastructure is posed by airborne attack weapons, in particular, the “Shahed-136” kamikaze drone and its russian counterpart, the “Geranium-2”. A combat vehicle with a turreted machine gun mount, which is part of mobile fire groups, has a limited number of means that provide high-quality visual observation, detection and recognition of air targets. Therefore, the task of determining the degree of danger of a detected and recognized drone is relevant in order to make an adequate decision on its destruction. The paper considers methods for detecting and recognizing an aerial target, determining its angle and direction of movement by the human visual system using optical and electronic devices and optical means of visual observation. The article proposes a scientific and methodological approach that allows detecting and recognizing an aerial target such as a kamikaze drone and determining the degree of its danger, which minimizes the time for the commander of a combat vehicle equipped with turreted machine guns to make a quick and adequate decision on destruction.

Visual placebo and nocebo effects.

Placebo and nocebo effects modulate symptom perception through expectations and learning processes in various domains. Predominantly, their impact has been investigated on pain and physical performance. However, the influence of placebos and nocebos on visual system functionality has yet to be explored. The present study aimed to test whether placebo and nocebo effects can intervene in altering participants' performance outcomes during a novel visual accuracy task and to examine the underlying neural mechanisms through EEG. After performing a baseline session, visual accuracy was said to be enhanced or disrupted by a sham transcranial electrical stimulation over the occipital lobe. Behavioural results showed a significant increase in visual accuracy for the placebo group, from the baseline session to the test session, whereas the nocebo group showed a decrease in visual accuracy. EEG analyses on the event-related potential P300 component, conducted on both a centro-parietal electrode patch and a parieto-occipital, one displayed an increase in the amplitude of P300 for the placebo group, and a decrease in the nocebo group. These findings suggest for the first time that placebo and nocebo effects can influence visual perception and attentional processes linked to it. Overall, the present study contributes to understanding how expectations affect sensory perception beyond pain and the motor system, paving the way for investigating these phenomena in other sensory modalities such as auditory or olfactory perception. KEY POINTS: Placebo and nocebo effects have been studied predominantly in pain and motor performance fields. In a novel visual task, the impact of placebo and nocebo effects on the visual system has been evaluated, in both early components (stimuli-related) and late components (attention-related). The placebo group showed an increase in visual accuracy and EEG-evoked potential amplitudes, whereas the nocebo group showed a decrease in both. This study shows how expectations and the related placebo and nocebo effects can shape basic stimuli sensory perception in the visual domain.