Real-time Image Data Research Articles

Enhancing Lung and Breast Cancer Screening with Advanced AI and Image Processing Techniques

This research investigates the application of CNNs for diagnostics improvements in lung and breast cancers based on AI image classification approaches. Using the datasets with 15,000 images describing lung cancer and 10,000 images describing cases of breast cancer, the models showed high performance: 90% for lung cancer and 99% for breast cancer classification. The descriptive analysis pointed out different features in imaging, such as dense tissue structure and irregular cell patterns; the models successfully identified these. The findings underlined the vital role that AI could play in assisting radiologists by delivering preliminary analysis, triaging high-risk cases, and leading to early cancer detection. Essential challenges were highlighted: ethical considerations concerning patients' privacy and AI algorithms' transparency. The limitation of the dataset diversity resulted in the conclusion that only broader data can ensure good generalization in various clinical settings. They recommended integrating the AI tool with clinical workflow and also called for training radiologists for effectiveness. Future research directions include real-time imaging and patient data integration for comprehensive diagnostic support and multi-modal approaches that combine imaging with genomics for more precise predictions. This leads to a more personalized cancer diagnosis and treatment plan, thus ultimately improving the results of the patients. This research, therefore, underlines the transformative capabilities of AI and image processing in modernizing cancer screening and diagnostics toward more accurate and efficient healthcare practices.

Plant Diseases Prediction Using Machine Learning

The early detection and accurate prediction of plant diseases are crucial for improving crop health and maximizing agricultural productivity. Traditional methods of disease detection, which rely heavily on manual observation, are often timeconsuming, labor-intensive, and prone to human error. Recent advancements in machine learning (ML) have opened new possibilities for developing efficient, automated systems that can predict plant diseases with high accuracy. This paper explores various machine learning techniques, including supervised and deep learning models, for plant disease prediction. By analyzing features such as leaf texture, color, and environmental data, these models can identify patterns indicative of specific diseases. Several datasets, including real-time image data and environmental metrics (such as humidity, temperature, and soil quality), are utilized to train and evaluate the models. The study focuses on key ML algorithms like Support Vector Machines (SVM), Decision Trees, Random Forests, Convolutional Neural Networks (CNN), and Transfer Learning to understand their efficacy in disease prediction. These models are evaluated based on their accuracy, precision, recall, and computational efficiency. The results demonstrate that deep learning models, particularly CNNs, outperform traditional methods in image-based plant disease identification. In addition, environmental data integration improves the predictive power of ML models, enabling proactive disease management strategies

Neural mechanisms underlying placebo and nocebo effects in tonic muscle pain

Pain is a highly subjective and multidimensional experience, significantly influenced by various psychological factors. Placebo analgesia and nocebo hyperalgesia exemplify this influence, where inert treatments result in pain relief or exacerbation, respectively. While extensive research has elucidated the psychological and neural mechanisms behind these effects, most studies have focused on transient pain stimuli. To explore these mechanisms in the context of tonic pain, we conducted a study using a 15-minute tonic muscle pain induction procedure, where hypertonic saline was infused into the left masseter of healthy participants. We collected real-time Visual Analogue Scale (VAS) scores and functional magnetic resonance imaging (fMRI) data during the induction of placebo analgesia and nocebo hyperalgesia via conditioned learning. Our findings revealed that placebo analgesia was more pronounced and lasted longer than nocebo hyperalgesia. Real-time pain ratings correlated significantly with neural activity in several brain regions. Notably, the putamen was implicated in both effects, while the caudate and other regions were differentially involved in placebo and nocebo effects. These findings confirm that the tonic muscle pain paradigm can be used to investigate the mechanisms of placebo and nocebo effects and indicate that placebo analgesia and nocebo hyperalgesia may have more distinct than common neural bases.

(Invited) Aiot-Integrated Digital Imaging Sensor for Molecular-Fingerprint Profiling of Extracellular Vesicles in Liquid Biopsy

The advancement of liquid biopsy techniques for non-invasive tumor analysis holds immense potential in continuously tracking tumor status—recurrence, progression, and treatment response—in real-time. To comprehensively assess tumor evolution, cellular diversity, and drug resistance mechanisms, a pivotal step involves scrutinizing proteins and nucleic acids within extracellular vesicles (EVs) present in biofluids. Our focus lies in crafting an advanced digital bead-based sensor system that seamlessly evaluates the molecular profile of EVs in glioblastoma cases. Our innovative approach enriches tumor-relevant genetic data within EVs using antibody-specific magnetic beads, subsequently amplifying it via reverse transcription recombinase polymerase amplification system across forty thousand droplets. These results are scrutinized using a compact imaging device. The core strengths of our methodology include: (a) Streamlined Sample Processing: Our system facilitates ultra-sensitive miRNA detection in EVs within a rapid ninety-minute window, owing to optimized sample preparation and a compact detection mechanism; (b) Enhanced Signal Clarity: Employing a straightforward isothermal assay significantly boosts the fluorescent signal-to-noise ratio within each 70-µm droplet, achieving over 104 turnovers of fluorescent reporters on a chip; (c) Real-Time Data Analysis: Our system seamlessly captures real-time imaging data for the edge computing, utilizing trained YOLOv5 models for classification, and presents genetic results on the smartphone. Its user-friendly design enables utilization by healthcare workers with varying skill levels. This pioneering approach has the potential to offer affordable, rapid, and informative diagnostics particularly beneficial for underserved regions and healthcare systems facing financial constraints. Furthermore, its adaptability for diverse cancers could revolutionize the pace of diagnosis and treatment decision-making through liquid biopsy, potentially establishing its widespread application in global healthcare.

A deep learning model to assist visually impaired in pothole detection using computer vision

Visually impaired individuals encounter numerous impediments when traveling, such as navigating unfamiliar routes, accessing information, and transportation, which can limit their mobility and restrict their access to opportunities. However, assistive technologies and infrastructure solutions such as tactile paving, audio cues, voice announcements, and smartphone applications have been developed to mitigate these challenges. Visually impaired individuals also face difficulties when encountering potholes while traveling. Potholes can pose a significant safety hazard, as they can cause individuals to trip and fall, potentially leading to injury. For visually impaired individuals, identifying and avoiding potholes can be particularly challenging. The solutions ensure that all individuals can travel safely and independently, regardless of their visual abilities. An innovative approach that leverages the You Only Look Once (YOLO) algorithm to detect potholes and provide auditory or haptic feedback to visually impaired individuals has been proposed in this paper. The dataset of pothole images was trained and integrated into an application for detecting potholes in real-time image data using a camera. The app provides feedback to the user, allowing them to navigate potholes and increasing their mobility and safety. This approach highlights the potential of YOLO for pothole detection and provides a valuable tool for visually impaired individuals. According to the testing, the model achieved 82.7% image accuracy and 30 Frames Per Second (FPS) accuracy in live video. The model is trained to detect potholes close to the user, but it may be hard to detect potholes far away from the user. The current model is only trained to detect potholes, but visually impaired people face other challenges. The proposed technology is a portable option for visually impaired people.

SPATIAL TEMPORAL ANALYSIS OF LAND SURFACE TEMPERATURE CHANGES IN AMBON ISLAND FROM LANDSAT 8 IMAGE DATA USING GEOGLE EARTH ENGINE

This study aims to analyze land surface temperature changes on Ambon Island using Landsat 8 imagery data and Google Earth Engine. The research method involves the use of Ambon Island boundary data from the Geospatial Information Agency (BIG) as well as Landsat 8 Collection 2 Tier 1 and Real-Time satellite image data that have been calibrated to reflect the reflectance of the Earth's surface to the atmosphere. The analysis steps include temperature conversion from Kelvin to Celsius, land surface temperature classification, and data export to Arc GIS software. The results showed an increase in land surface temperature associated with urban development in Ambon City, highlighting the importance of sustainable urban planning and good resource management to mitigate the negative impacts of land surface temperature increase and support adaptation to climate change. Keywords: Ambon Island, Geogle Earth Engine, Land Surface Temperature

SFMnet: Edge detection of HABs based on spatial feature mapping encoder-decoder network

Achieving high-accuracy real-time recognition of marine red tide algae images is crucial in implementing red tide algae early warning systems. However, due to uneven lighting and the deposition of unknown suspended particles in seawater, real-time collected harmful algal blooms (HABs) microscopic images in marine engineering suffer from poor clarity and blurred cell contours, leading to lower accuracy of current edge detection models for microscopic images. Firstly, to enhance the quality of degraded images in the original HABs dataset, we design an image enhancement algorithm based on underwater image restoration models, which involves precise depth estimation of seawater scenes and fusion of transmission maps. This algorithm effectively eliminates chromatic aberration and enhances image edge details and contrast. Secondly, we propose a deep-learning model based on the VGG16 network. This model integrates multi-scale features through the Spatial Feature Mapping module to improve edge detection accuracy. Experimental results using real-time HABs microscopic image data show that the proposed enhancement method achieves the best results regarding contrast, image enhancement metrics, and clarity gradient. Moreover, the edge detection method achieves an ODS of 0.636 and an OIS of 0.701. The results demonstrate that performing edge detection after the image enhancement preprocess significantly improves detection accuracy.

Deep RegNet-150 architecture for single image super resolution of real-time unpaired image data

Single Image Super-Resolution (SISR) is a fundamental computer vision task aimed at enhancing the spatial resolution and quality of low-resolution images. In recent years, deep learning techniques have revolutionized the field of SISR, enabling remarkable advancements in image enhancement. Typically, the Convolutional Neural Networks (CNNs) and Generative Adversarial Networks (GANs) based deep learning techniques are employed for the SISR. However, the existing CNN and GAN architectures lag in providing maximum performance in terms of SSIM and PSNR. In this paper, a regression deep learning architecture is designed for the SISR. The proposed Deep Regression Network with 150 (Deep RegNet-150) is designed as a U-net. The proposed Deep RegNet-150 includes up-sampling and down-sampling portions with Residual Channel Attention Block (RCAB) for SISR. The RCAB acts as the building block of image super resolution (SR) by including ReLU and Conv layer. The performance of the proposed technique is evaluated on different image datasets, including a text image dataset and an unpaired dataset. The performance in terms of SSIM and PSNR for 4X and 8X SR is evaluated. The proposed Deep RegNet-150 attained maximum PSNR / SSIM values of 43.99 / 0.989 and 42.73 / 0.987 for 4X and 8X SR of the benchmark dataset, respectively. The 4X and 8X SR of unpaired datasets attained 46.13 / 0.991 and 45.97 / 0.990 of PSNR /SSIM, respectively. Ultimately, it proves that the Deep RegNet-150 is an effective deep learning architecture for the SISR.

MOUSE AND KEYBOARD MOTION SENSING USING HAND SIGNS

The project presents a way for Human- Computer Interaction (HCI) that employs a web camera or an integrated camera to control the movement of the indicator instead to the current methods, which typically involve manually entering buttons or altering the positions of a physical mouse. Instead, it can perform the functions that a physical computer mouse does, and it does it through camera and technology for computer vision to manage different mouse events. The Virtual Mouse color recognition application constantly acquires real-time image data and processes it using several filtering and conversion phases. The application utilizes an image processing technique to obtain the coordinates of the desired colors that have emerged within the frames holding a list of color combinations, each of them correlates to a different mouse function. The program will run the mouse function and transform it to an actual mouse function which will be able to applied on the user’s PC if the current color combination passes. The main aim of the project is to virtually interface with the system in virtually every manner imaginable. Keywords: Gesture Recognition, Human Computer Interactions, Graphical User Interface, Open-Source Computer Vision.

Development of an IoT-based firefighting drone for enhanced safety and efficiency in fire suppression

An uncontrolled fire poses severe threats to both humans and the environment, making firefighting a perilous and complex task. Traditional fire suppression methods are inefficient, costly, and without thorough testing, leading to delays in gaining control over fire outbreaks. This paper presents a novel firefighting drone aimed at mitigating risks to firefighters by extinguishing fires and providing real-time imaging, gas concentration and fire location data monitoring. The proposed intelligent quadcopter utilizes the Pixhawk PX4 microcontroller for precise control and the Pixhawk Telemetry system for data processing. The proposed device is constructed from an ultra-strength S500 Quadcopter frame, NodeMCU, Arduino Nano, various gas sensors, a servo motor to extinguish the fire and a camera to detect fire events in real time. Equipped with an FPV camera and a video transmitter, it transmits live video feed to the ground, enabling efficient navigation using the Flysky I6X controller. The intended position and height of the drone are controlled using an adaptive optimization technique known as fuzzy-based backstepping control. This article demonstrates the effectiveness of the device by collecting and analyzing gas emissions data from controlled burns of various materials. The drone successfully measured concentrations of CO, CO2, O3, SO2, and NO2 in affected areas, providing valuable insights for firefighting operations. Different levels of gases have been measured depending on the concentration from burning alcohol, clothes, plastic materials, paper, leaves, and so on. The novelty of this work lies in the development and comprehensive analysis of an IoT-based firefighting drone conducting extensive real-time experiments.

Mathematics for 2D face recognition from real time image data set using deep learning techniques

The recognition of human faces poses a complex challenge within the domains of computer vision and artificial intelligence. Emotions play a pivotal role in human interaction, serving as a primary means of communication. This manuscript aims to develop a robust recommendation system capable of identifying individual faces from rasterized images, encompassing features such as eyes, nose, cheeks, lips, forehead, and chin. Human faces exhibit a wide array of emotions, with some emotions, including anger, sadness, happiness, surprise, fear, disgust, and neutrality, being universally recognizable. To achieve this objective, deep learning techniques are leveraged to detect objects containing human faces. Every human face exhibits common characteristics known as Haar features, which are employed to extract feature values from images containing multiple elements. The process is executed through three distinct stages, starting with the initial image and involving calculations. Real-time images from popular social media platforms like Facebook are employed as the dataset for this endeavor. The utilization of deep learning techniques offers superior results, owing to their computational demands and intricate design when compared to classical computer vision methods using OpenCV. The implementation of deep learning is carried out using PyTorch, further enhancing the precision and efficiency of face recognition.

A Privacy-Preserving Approach to Effectively Utilize Distributed Data for Malaria Image Detection.

Malaria is one of the life-threatening diseases caused by the parasite known as Plasmodium falciparum, affecting the human red blood cells. Therefore, it is an important to have an effective computer-aided system in place for early detection and treatment. The visual heterogeneity of the malaria dataset is highly complex and dynamic, therefore higher number of images are needed to train the machine learning (ML) models effectively. However, hospitals as well as medical institutions do not share the medical image data for collaboration due to general data protection regulations (GDPR) and the data protection act (DPA). To overcome this collaborative challenge, our research utilised real-time medical image data in the framework of federated learning (FL). We have used state-of-the-art ML models that include the ResNet-50 and DenseNet in a federated learning framework. We have experimented both models in different settings on a malaria dataset constituting 27,560 publicly available images and our preliminary results showed that the DenseNet model performed better in accuracy (75%) in contrast to ResNet-50 (72%) while considering eight clients, while the trend was observed as common in four clients with the similar accuracy of 94%, and six clients showed that the DenseNet model performed quite well with the accuracy of 92%, while ResNet-50 achieved only 72%. The federated learning framework enhances the accuracy due to its decentralised nature, continuous learning, and effective communication among clients, as well as the efficient local adaptation. The use of federated learning architecture among the distinct clients for ensuring the data privacy and following GDPR is the contribution of this research work.

Abstract 3248: Metabolic rewiring shifts engineered TCR T cell bioenergetics to enhance cytotoxicity

Abstract Background: Adoptive T cell-based transfer has shown tremendous success in hematological cancers. However, an important obstacle to using cell and gene therapy for solid tumors is the aggressive tumor microenvironment (TME). To significantly improve prognosis of subjects undergoing immunotherapy, it is essential to incorporate unique methods to drive T cell energy production and metabolism towards empowering T cells to dismantle the suppressive TME. This can promote anti-tumor effects and persistence of the transfused engineered T cells. In the present study we present a simple metabolic preconditioning process that improves T cell performance and bioenergetics. Methods: Using the Agilent xCELLigence RTCA eSight, and Seahorse XF Analyzer we assessed the killing efficiency, and bioenergetics of engineered T-cells after pretreating with Arginine. Briefly, engineered TCR T-cells against MART-1 were generated by transducing CD3+ T-cells (Hemacare, Seattle, WA) with retrovirus SAMEN-DMF5 with a CD34 marker gene, against MART-1. The TCR T cells were pre-conditioned in 6mM Arginine for 7 days, followed by a killing assay using MART-1 expressing melanoma cell line as target cells (624.38) at the E:T ratio of 5:1. Target melanoma cells, 624.38 cells, were engineered to express a red-fluorescent nuclear protein. The comparison was made with transduced T-cells grown in RPMI (no added amino acid supplementation, and non-transduced T cells. The cytotoxicity assay was assessed using real-time impedance and live cell imaging data collected on xCELLigence RTCA eSight. Concurrently, the metabolic profiling of the engineered T-cells was examined by the XF96 Analyzer. Results: Supplementing the growth medium with 6 mM Arginine significantly increased the potency of MART-1 engineered T cells (up to 6-fold) and bioenergetic studies revealed increased spare respiratory capacity, basal respiration, and ATP production via mitochondrial respiration. Conclusions: Arginine pre-conditioning improved TCR T cell potency through metabolic rewiring favoring mitochondrial respiration. For Research Use Only. Not for use in diagnostic procedures RA45236.3483333333 Citation Format: Rashmi Pillai, Xiaoyu Zhang, Yama Abassi. Metabolic rewiring shifts engineered TCR T cell bioenergetics to enhance cytotoxicity [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 3248.

Real-time analysis of large-scale neuronal imaging enables closed-loop investigation of neural dynamics.

Large-scale imaging of neuronal activities is crucial for understanding brain functions. However, it is challenging to analyze large-scale imaging data in real time, preventing closed-loop investigation of neural circuitry. Here we develop a real-time analysis system with a field programmable gate array-graphics processing unit design for an up to 500-megabyte-per-second image stream. Adapted to whole-brain imaging of awake larval zebrafish, the system timely extracts activity from up to 100,000 neurons and enables closed-loop perturbations of neural dynamics.

An Improved ResNet50 for Environment Image Classification

With the drastic changes in the ecological environment, the issue of blue-green algae blooms has become increasingly common, significantly affecting human habitats. Real-time image data extraction and classification via deep learning models play a critical role in detecting blue-green algae in water bodies and facilitating subsequent alerts and interventions. However, current research in this field faces several challenges. Firstly, traditional convolutional methods struggle to effectively extract features such as textures and outlines from images. Secondly, the target and background areas in captured images often share similar environments, leading to classification errors. Addressing these issues, this paper initially enhances the feature representation capabilities of the ResNet50 architecture by integrating a detail feature extraction module. Subsequently, a binary mask image fusion method is proposed to direct the model’s focus towards learning features of water regions. Our constructed model shows improved classification performance on real-time camera data compared to other classification models.

Chinese expert consensus on multidisciplinary discussion of interstitial lung disease

Interstitial lung disease (ILD) is a group of heterogeneous diseases characterized by interstitial lung inflammation and fibrosis. Its causes are diverse, symptoms can be non-specific, and the associated imaging and pathologic patterns can vary widely. The diagnosis and management of these conditions often require input from multiple medical disciplines. Over the past few decades, the importance of multidisciplinary discussion (MDD) in the management of ILD has grown internationally, and MDD has gradually become a part of the gold standard for ILD diagnosis and an essential reference for disease management. Recognizing this, the Chinese Medical Association-Chinese Thoracic Society and the Chinese Association of Chest Physicians have collaborated with experts to standardize the ILD-MDD process in China.Key recommendations:1. Formation of teams: ILD centers should establish multidisciplinary teams (MDT) and integrate MDD into the ILD patient care process. (strong recommendation)2. Patient preparation: Prior to ILD-MDD, patients should undergo as thorough an ILD assessment as possible, including medical history, diagnostic tests, and functional evaluations. (strong recommendation)3. MDD frequency & logistics:(1) The frequency, duration and number of cases to be discussed in ILD-MDD should be based on the clinical needs of each center, but should be held at least monthly. (2) Consistent timing and location of ILD-MDD is recommended to improve attendance. (medium recommendation)4. Team composition: The MDT should include experts from a range of disciplines with experience of ILD. These should include specialists in Pulmonary and Critical Care Medicine, Radiology, and Pathology. Where appropriate, experts in Rheumatology, Hematology, Thoracic Surgery, and Respiratory Care and Rehabilitation should also be involved. (strong recommendation)5. Case presentation & goals:(1) The physician in charge should present clinical information, preferably using standardized slides. Real-time imaging and pathology data are encouraged. (2) The primary goals of ILD-MDD discussions should be disease diagnosis and management. (strong recommendation)6. Operational leadership: Assign a dedicated professional to manage the ILD-MDD operations and maintain the information discussed. (strong recommendation)7. Specific conditions:(1) Suspected idiopathic pulmonary fibrosis should be diagnosed by a standardized MDD process. (2) Suspected idiopathic interstitial pneumonia (IIP) should be classified by MDD. If IIP is indicated by imaging or pathology, prioritize identifying potential underlying causes such as connective tissue diseases (CTDs), certain medications, or aspiration.(3) Suspected hypersensitivity pneumonitis should also be diagnosed by MDD. (4) If an ILD patient shows signs suggestive of CTDs, the diagnosis should be made by ILD-MDD with the involvement of rheumatologists. (strong recommendation)8. Unclassifiable ILD: For ILD patients who cannot be clearly classified, create a tailored management plan based on the patient's specific disease features. (strong recommendation)9. Progressive pulmonary fibrosis: Use ILD-MDD to determine whether an ILD patient meets the criteria for progressive pulmonary fibrosis and to discuss a personalized care plan. (strong recommendation).

An Advanced Software Platform and Algorithmic Framework for Mobile DBH Data Acquisition

Rapid and precise tree Diameter at Breast Height (DBH) measurement is pivotal in forest inventories. While the recent advancements in LiDAR and Structure from Motion (SFM) technologies have paved the way for automated DBH measurements, the significant equipment costs and the complexity of operational procedures continue to constrain the ubiquitous adoption of these technologies for real-time DBH assessments. In this research, we introduce KAN-Forest, a real-time DBH measurement and key point localization algorithm utilizing RGB-D (Red, Green, Blue-Depth) imaging technology. Firstly, we improved the YOLOv5-seg segmentation module with a Channel and Spatial Attention (CBAM) module, augmenting its efficiency in extracting the tree’s edge features in intricate forest scenarios. Subsequently, we devised an image processing algorithm for real-time key point localization and DBH measurement, leveraging historical data to fine-tune current frame assessments. This system facilitates real-time image data upload via wireless LAN for immediate host computer processing. We validated our approach on seven sample plots, achieving bbAP50 and segAP50 scores of: 90.0%(+3.0%), 90.9%(+0.9%), respectively with the improved YOLOv5-seg model. The method exhibited a DBH estimation RMSE of 17.61∼54.96 mm (R2=0.937), and secured 78% valid DBH samples at a 59 FPS. Our system stands as a cost-effective, portable, and user-friendly alternative to conventional forest survey techniques, maintaining accuracy in real-time measurements compared to SFM- and LiDAR-based algorithms. The integration of WLAN and its inherent scalability facilitates deployment on Unmanned Ground Vehicles (UGVs) to improve the efficiency of forest inventory. We have shared the algorithms and datasets on Github for peer evaluations.

Multi-scale context feature and cross-attention network-enabled system and software-based for pavement crack detection

Pavement crack detection continues to be a stubborn problem given the interference of various factors in the actual pavement and the complex topological structure of asphalt pavement. Among all the obstacles, the bottleneck of pavement crack detection lies in the difficulty of segmenting the cracks in the pavement images whose edges are blurred. This paper proposes a multi-scale context feature and cross-attention based on convolutional neural network for accurate and robust pavement crack segmentation. The multi-scale context feature module is built in different deep networks to extract rich crack feature information. Subsequently, in order to effectively promote the seamless integration of features at different levels, we deploy cross-attention modules to each branch. After that, we add deep supervision to each branch to accelerate training. Finally, we integrate the outputs of each branch to obtain the final output diagram. The comparative experiments on various pavement datasets show that the method has better robustness. At the same time, this paper designs a complete system of pavement crack detection (PCD) and develops corresponding engineering application software. The PCD system can record the real-time pavement image data to the edge server, and the client can also monitor the real-time pavement images from the edge server through the HTTP protocol.

Towards Visual Inspection of Distributed and Irregular Structures: A Unified Autonomy Approach

This paper highlights the significance of maintaining and enhancing situational awareness in Urban Search and Rescue (USAR) missions. It focuses specifically on investigating the capabilities of Unmanned Aerial Vehicles (UAV) equipped with limited sensing capabilities and onboard computational resources to perform visual inspections of apriori unknown fractured and collapsed structures in unfamiliar environments. The proposed approach, referred to as First Look Inspect-Explore (FLIE), employs a flexible bifurcated behavior tree that leverages real-time RGB image and depth cloud data. By employing a recursive and reactive synthesis of safe view pose within the inspection module, FLIE incorporates a novel active visual guidance scheme for identifying previously inspected surfaces. Furthermore, the integration of a tiered hierarchical exploration module with the visual guidance system enables the UAV to navigate towards new and unexplored structures without relying on a map. This decoupling reduces memory overhead and computational effort by eliminating the need to plan based on an incrementally built, error-prone global map. The proposed autonomy is extensively evaluated through simulation and experimental verification under various scenarios and compared against state-of-art approaches, demonstrating its performance and effectiveness.

Convolutional Neural Network Combined With Stochastic Parallel Gradient Descent to Decompose Fiber Modes Based on Far-Field Measurements

Modal decomposition (MD) of fiber modes based on direct far-field measurement combining the convolutional neural network (CNN) with a stochastic parallel gradient descent (SPGD) algorithm is investigated both numerically and experimentally. For obtaining the modal coefficients of fiber modes guided in a large-mode-area fiber, the fiber modes are decomposed into a finite number of Hermite gaussian modes, the initial conditions of the modal coefficients are obtained through the CNN, and further optimization of them are carried out through the SPGD. The ambiguity problem that may happen in the CNN owing to the existence of the pair-beam field is resolved by properly labelling the phase differences with a single-valued parameter set in consideration of the mode-order indices. The feasibility and effectiveness of the proposed MD method is verified both numerical simulations and experimental demonstrations with both recorded image data and online real-time image data. The correlation error incurred by the proposed method is below 6.6×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-4</sup> and 8.7×10 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-3</sup> in the numerical simulations and the experimental demonstrations, respectively. The online real-time operation of the proposed method is also experimentally demonstrated at a decomposing rate of ∼2 Hz.