Augmented System Research Articles

Investigating the accuracy of machine vision and augmented reality in percutaneous computed tomography-guided interventions: A phantom study.

This study aimed to evaluate the accuracy of percutaneous computed tomography (CT)-guided puncture based on machine vision and augmented reality in a phantom. The surgical space coordinate system was established, and accurate registration was ensured using the hierarchical optimization framework. Machine vision tracking and augmented reality display technologies were used for puncture navigation. CT was performed on a phantom, and puncture paths with three different lengths were planned from the surface of the phantom to the metal ball. Puncture accuracy was evaluated by measuring the target positioning error (TPE), lateral error (LE), angular error (AE), and first success rate (FSR) based on the obtained CT images. A highly qualified attending interventional physician performed a total of 30 punctures using puncture navigation. For the short distance (4.5-5.5 cm), the TPE, LE, AE, and FSR were 1.90 ± 0.62 mm, 1.23 ± 0.70 mm, 1.39 ± 0.86°, and 60%, respectively. For the medium distance (9.5-10.5 cm), the TPE, LE, AE, and FSR were 2.35 ± 0.95 mm, 2.00 ± 1.07 mm, 1.20 ± 0.62°, and 40%, respectively. For the long distance (14.5-15.5 cm), the TPE, LE, AE, and FSR were 2.81 ± 1.17 mm, 2.33 ± 1.34 mm, 0.99 ± 0.55°, and 30%, respectively. The augmented reality and machine vision-based CT-guided puncture navigation system allows for precise punctures in a phantom. Further studies are needed to explore its clinical applicability.

3D Gamut Morphing for Non-Rectangular Multi-Projector Displays.

In a spatially augmented reality system, multiple projectors are tiled on a complex shaped surface to create a seamless display on it. This has several applications in visualization, gaming, education and entertainment. The main challenges in creating seamless and undistorted imagery on such complex shaped surfaces are geometric registration and color correction. Prior methods that provide solutions for the spatial color variation in multi-projector displays assume rectangular overlap regions across the projectors that is possible only on flat surfaces with extremely constrained projector placement. In this article, we present a novel and fully automated method for removing color variations in a multi-projector display on arbitrary shaped smooth surfaces using a general color gamut morphing algorithm that can handle any arbitrarily shaped overlap between the projectors and assures imperceptible color variations across the display surface.

Achieving new excellence in an augmented reality dental education system

Background/purposeSimEx-Plus (EPED. Inc) was already a mature augmented reality (AR) dental training simulator that allowed students to have a high quality dental education practice. Now the EPCAD software has been further developed into a comprehensive computer-aided design software. It helps relevant professionals learn how to create digital fixed prosthesis in a virtual environment. This study used the newly developed EPCAD software that was integrated within the SimEx-Plus to design dental restorations then digital output milling restorations. We hope that the SimEx-Plus AR dental training simulator can achieve the multi-purpose function in one machine. Materials and methodsA right maxillary central incisor was prepared in the standard dental model. The restorations were designed by EPCAD software integrated with the SimEx-Plus and milled by a 4-axis dental milling machine (N4+, vhf camfacture AG). The internal gap of the restoration was evaluated. ResultsThe internal gaps of the crowns set on the right maxillary central incisor were among 68.27–89.80 μm. The different setting for digital output did not influence the internal gap (P 0.05, One-Way ANOVA). However, the internal gap on the palatal side was larger than that on the labial, mesial, distal, and incisal surfaces (P < 0.05, One-Way ANOVA). ConclusionThe new software EPCAD that was integrated within the SimEx-Plus AR dental training simulator could design restorations, and the designed restorations had good internal gap for clinical use. The professionals can learn tooth preparation and digital restoration fabrication by the SimEx-Plus AR dental training system.

Generalized and novel iterative scheme for best approximate solution of large and sparse augmented linear systems

This article presents a generalized novel iterative technique for obtaining the approximate solution of systems of linear equations(ABBT0)(xy)=(pq). Parameters involved in the scheme play a vital role to obtain the rapid and best approximate solution. The technique is applied to some important problems and the results are presented in detail. The article compares the effectiveness of the new presented iterative scheme with modified SOR-like method, generalized SOR and SOR-like method for solving large systems of linear equations. The convergence criteria for the iterative scheme are also analysed and it can be observed in the article that the theoretical derivation is supported by numerical calculations and graphical exhibition. Overall, the article provides a new and effective method with a detailed analysis of its performance and convergence criteria for solving systems of linear equations of the form presented. The newly derived technique has potential applications in various fields of science and engineering where systems of linear equations arise and need to be solved.

Development of ARenal Android Application: A Supplementary Tool in Learning the Human Urinary System

Augmented reality (AR) is an interactive three-dimensional (3D) experience that uses computers to overlay virtual information in the real world. We aimed to develop an online 3D AR of human urinary system (HUS) as a supplementary tool to aid in the teaching and learning process whether via online or traditional teaching methods, particularly for undergraduate medical students. Augmented reality human urinary system (ARenal) is an Android application that implements AR for better comprehension of the HUS. It is an original work of 3D models, diagrams, pictures, notes, self-recording videos, quizzes and a manual. The AR elements will improve learning effectiveness and attention span, enabling students to comprehend anatomical principles, basic physiology, and common disorders of HUS. Instead of 3D on paper, students can access educational resources and AR tools via a detailed 3D kidney model on Android devices. The user acceptability testing (UAT) (n = 10) revealed that 60% of participants indicated it was their first experience with AR, and 60% strongly agreed that ARenal was operating well. They also strongly agreed that it is user-friendly (50%). Most participants expressed a higher level of acceptability for ARenal and recommended adding more activities, games, quizzes, and other features to the 3D model for better interaction. In conclusion, ARenal has the potential to be employed as an additional aid for learning the HUS.

A Network Meta-Analysis Comparing the Efficacy and Safety of Pedicle Screw Placement Techniques Using Intraoperative Conventional, Navigation, Robot-Assisted, and Augmented Reality Guiding Systems.

Studies were reviewed and collected to compare different image guidance systems for pedicle screw placement (PSP) regarding accuracy and safety outcomes. Included were conventional, navigation, robot-assisted, and recent technology such as augmented reality (AR) guiding systems. This network meta-analysis obtained human comparative studies and randomized controlled trials (RCTs) regarding PSP found in 3 databases (Cochrane, PubMed, and Scopus). Data extraction for accuracy, safety, and clinical outcomes were collected. The network meta-analysis was analyzed, and a surface under the cumulative ranking curve (SUCRA) was used to rank the treatment for all outcomes. The final 61 studies, including 13 RCTs and 48 non-RCTs, were included in the meta-analysis. These studies included a total of 17,023 patients and 35,451 pedicle screws. The surface under the cumulative ranking curve ranking demonstrated the supremacy of robotics in almost all accuracy outcomes except for the facet joint violation. Regarding perfect placement, the risk difference for AR was 19.1 (95% CI: 8.1-30.1), which was significantly higher than the conventional method. The robot-assisted and navigation systems had improved outcomes but were not significantly different in accuracy vs the conventional technique. There was no statistically significant difference concerning safety or clinical outcomes. The accuracy of PSP achieved by robot-assisted technology was the highest, whereas the safety and clinical outcomes of the different methods were comparable. The recent AR technique provided better accuracy compared with navigation and conventional methods.

Mittag-Leffler function based security control for fractional-order complex network system subject to deception attacks via Observer-based AETS and its applications

The goal of this paper is to investigate the security control for uncertain fractional-order delayed complex network systems under deception attacks using the Mittag-Leffler function and observer-based adaptive event-triggered scheme (AETS) with the fractional commensurate order in q ∈ (0, 1). The adaptive event-triggering scheme is used during the data transmission process from the sensors to the observer, where the triggering threshold can be dynamically modified to reduce resource waste. We make a novel model for the estimation error system that takes into account both the effects of the adaptive event-triggered scheme and the effects of deception attacks. A sufficient condition is obtained to guarantee the stochastic mean-square stability of the augmented error system using the Mittag-Leffler (M-L) functions and the Lyapunov functional method and by using the singular value decomposition (SVD) and linear matrix inequality (LMI) techniques, the co-design problem of desired observer and controller gains is found, and it is shown that the solution ensures the stability of a closed-loop uncertain fractional-order complex networked system. At the end of this study, two numerical examples and diesel engine system model are given to show that the above findings are correct.

Code-Based Differential GNSS Ranging for Lunar Orbiters: Theoretical Review and Application to the NaviMoon Observables

In the near future, international space agencies have planned to achieve significant milestones in investigating the utilization of Global Navigation Satellite Systems (GNSS) within and beyond the current space service volume up to their application to lunar missions. These initiatives aim to demonstrate the feasibility of GNSS navigation at lunar altitudes. Based on the outcomes of such demonstrations, dozens of lunar missions will likely be equipped with a GNSS receiver to support autonomous navigation in the lunar proximity. Relying on non-invasive, consolidated differential techniques, GNSS will enable baseline estimation, thus supporting a number of potential applications to lunar orbiters such as collaborative navigation, formation flight, orbital manoeuvers, remote sensing, augmentation systems and beyond. Unfortunately, the large dynamics and the geometry of such differential GNSS scenarios set them apart from current terrestrial and low-earth orbit use cases. These characteristics result in an increased sensitivity to measurements time misalignment among orbiters. Hence, this paper offers a review of baseline estimation methods and characterizes the divergences and limitations w.r.t. to terrestrial applications. The study showcases the estimation of the baseline length between a lunar CubeSat mission, VMMO, and the communication relay Lunar Pathfinder mission. Notably, real GNSS measurements generated by an Engineering Model of the NaviMoon receiver in the European Space Agency (ESA/ESTEC) Radio Navigation Laboratory are utilized. A radio-frequency constellation simulator is used to generate the GNSS signals in these hardware-in-the-loop tests. The performed analyses showed the invalidity of common terrestrial differential GNSS ranging techniques for space scenarios due to the introduction of significant biases. Improved ranging algorithms were proposed and their potential to cancel ranging errors common to both receivers involved was confirmed.

Q-learning based tracking control with novel finite-horizon performance index

A data-driven method is designed to realize the model-free finite-horizon optimal tracking control (FHOTC) of unknown linear discrete-time systems based on Q-learning in this paper. First, a novel finite-horizon performance index (FHPI) that only depends on the next-step tracking error is introduced. Then, an augmented system is formulated, which incorporates with the system model and the trajectory model. Based on the novel FHPI, a derivation of the augmented time-varying Riccati equation (ATVRE) is provided. We present a data-driven FHOTC method that uses Q-learning to optimize the defined time-varying Q-function. This allows us to estimate the solutions of the ATVRE without the system dynamics. Finally, the validity and features of the proposed Q-learning-based FHOTC method are demonstrated by means of conducting comparative simulation studies.

Preliminary assessment of augmentative and alternative communication‐related stereotypy

AbstractImproving social communication and reducing challenging forms of restricted and repetitive behaviors (i.e., stereotypy) are often the focus of behavioral interventions for individuals with autism and other developmental disabilities. Interventions incorporating augmentative and alternative communication (AAC) systems are utilized to improve independent communication. However, using electronic AAC devices (e.g., iPads) may increase stereotypic behaviors (i.e., pressing the same icon on an AAC device two or more times within 2 s), which may impact the development and maintenance of communication skills. There is a shortage of research focusing on AAC‐related stereotypy, with only one study that includes a systematic assessment and treatment evaluation (Cook et al., 2017). The purpose of the current investigation was to conduct a functional analysis screening procedure that only included play and no‐interaction conditions to adequately assess the function of AAC‐related stereotypy. We sought to ask, how client‐specific modifications to functional assessment procedures affect the outcomes of functional assessments of AAC‐related stereotypy. Initial findings suggest that AAC‐related stereotypy is often automatically reinforced, but its occurrence can be sensitive to the presence/absence of specific environmental stimuli.

Sampled‐data cooperative semi‐global robust practical output regulation for nonlinear multi‐agent systems with an uncertain leader

AbstractThis article explores the problem of sampled‐data cooperative robust practical output regulation for nonlinear multi‐agent systems whose leader system is unknown. In this study, the nonlinearities of the follower systems are not constrained by the linear growth condition. Based on the robust control technology and the adaptive control technology, a distributed output‐based sampled‐data control protocol is developed to deal with the uncertainty of the exosystem which can belong to any large known compact set. We first perform the coordinate transformation of closed‐loop system to obtain the so‐called augmented system, and then perform the stability analysis for this system. As a result, it is proven that the steady‐state tracking errors are in any small given neighborhood of the origin. Finally, the performance of the proposed controller is validated by a numerical example.

Adaptive Internal Model Backstepping Control for a Class of Second-Order Electromagnetic Micromirror with Output Performance Constraints and Anomaly Control.

This paper investigates the asymptotic tracking problem for a class of second-order electromagnetic micromirror model with output performance constraints and anomaly control, which is subject to model parameter uncertainties and external disturbances. Specifically, this paper formulates the trajectory tracking control problem of an electromagnetic micromirror as a closed-loop control trajectory tracking problem based on the general solution framework of output regulation. Moreover, the extended internal model is introduced to reformulate the closed-loop control problem into a state stabilization problem of the augmented system. Based on the augmented system, an internal model backstepping controller is proposed by integrating the barrier Lyapunov Functions (BLF) and the Nussbaum gain function with the backstepping structure.This controller not only satisfies the output performance constraints of the micromirror, but also maintains the control performance in anomalous control situations. The final performance simulation demonstrates the efficacy of the proposed controller.

Augmented reality–assisted intraoperative navigation increases precision of glenoid inclination in reverse shoulder arthroplasty

BackgroundReverse Total Shoulder Arthroplasty (RTSA) is the standard of care for patients with glenohumeral osteoarthritis and rotator cuff deficiency. Preoperative RTSA planning based on medical images and patient-specific instruments has been established over the last decade. This study aims to determine the effects of using augmented reality assisted intraoperative navigation (ARIN) for baseplate positioning in RTSA compared to preoperative planning. It is hypothesized that ARIN will decrease deviation between preoperative planning and postoperative baseplate positioning. Moreover, ARIN will decrease deviation between the (senior) more (> 50 RTSA/year) and less experienced (junior) surgeon (5-10 RTSA/year). MethodsPreoperative CT scans of sixteen fresh-frozen cadaveric shoulders were obtained. Baseplate placements were planned using a validated software. The data was then converted and uploaded to the augmented reality system (NextAR; Medacta International, Castel San Pietro, Switzerland). Each of the eight RTSAs were implanted by a senior and a junior surgeon, with four RTSAs utilizing ARIN and four without. A postoperative CT scan was performed in all cases. The scanned scapulae were segmented, and the preoperative scan was laid over the postoperative scapula by nearest iterative point cloud analysis. The deviation from the planned entry point and trajectory was calculated regarding the inclination, retroversion, medialization (reaming depth)/lateralization, antero-posterior position and superior/inferior position of the baseplate. Data are reported as mean ± standard deviation (SD) or mean and 95% confidence interval (CI). P values <0.05 were considered statistically significant. ResultsThe use of ARIN yielded a reduction in the absolute difference between planned and obtained inclination from 9° (SD: 4°) to 3° (SD°:2) (p=0.011). Mean difference in planned-obtained inclination between surgeons was 3° in free-hand surgeries (95%CI: -4,10, p=0.578), while this difference reduced to 1° (95%CI: -6, 7, p=0.996) using ARIN. Retroversion, medialization (reaming depth)/lateralization, antero-posterior position and superior/inferior position of the baseplate were not affected by using ARIN. Surgical duration was increased using ARIN for both the senior (10 mins) and junior (18 mins) surgeon. ConclusionsThe implementation of augmented reality assisted intraoperative navigation (ARIN) leads to greater accuracy of glenoid component placement, specifically with respect to inclination. Further studies have to verify if this increased accuracy is clinical important. Furthermore, ARIN allows less experienced surgeons to achieve a similar level of accuracy in component placement comparable to more experienced surgeons. However, the potential advantages of ARIN in RTSA are counterbalanced by an increase in operative time.

Shared Book Reading Experiences for Young Children Who Use Augmentative and Alternative Communication Systems

Shared Book Reading Experiences for Young Children Who Use Augmentative and Alternative Communication Systems

Observer‐based optimal fault‐tolerant tracking control for input‐constrained interconnected nonlinear systems with mismatched disturbances

SummaryThis paper investigates an observer‐based optimal fault‐tolerant tracking control problem for interconnected nonlinear systems with input constraints and mismatched disturbances via adaptive dynamic programming (ADP). Firstly, an augmented system consisting of tracking error and the reference trajectory is constructed, and then the original optimal tracking control problem is transformed into the optimal regulation control problem of the augmented system. An integral sliding‐mode‐based optimal fault‐tolerant control scheme is developed to eliminate the effect of actuator faults and guarantee the optimal control performance. Furthermore, a neural network‐based observer is designed to identify the completely unknown system dynamics based on the input‐output data, thereby relaxing the restriction on system dynamics. Subsequently, the modified Hamilton‐Jacobi‐Bellman equations are solved by using the ADP algorithm under a critic network framework. According to the Lyapunov approach, all signals in the closed‐loop augmented system are uniformly ultimately bounded. Finally, the effectiveness of the developed control scheme is demonstrated via simulation results.

Ligament augmentation and reconstruction system (LARS) synthetic grafts are safe and effective for medial collateral ligament and posterolateral corner reconstructions in elite athletes.

This study documents the efficacy and safety of using a Ligament Augmentation and Reconstruction System (LARS) ligament graft to augment extra-articular knee ligament reconstructions in elite athletes by reporting return-to-play(RTP) rates and levels, career longevityand complications. A consecutive series of all extra-articular knee ligament reconstructions augmented by LARS ligaments in elite athletes undertaken by three specialist sports knee surgeons between 2013 and 2020 were reviewed. Seventy-six elite athletes, aged over 16 years old, and more than 2 years postsurgery were included. RTPwas defined as competing at professional levelor national/international level in amateur sport. There were 64 medial collateral ligament (MCL) and 12 posterolateral corner (PLC) reconstructions. Fifty-two (68.4%) underwent concomitant autograft cruciate ligament(s) reconstruction. The mean age was 25.1 years (SD ± 4.5). Most were football (35, 46.1%) or rugby players (35, 46.1%). Sixty-seven athletes (88.2%) RTP with 65 (97.0%) of these playing at the same or higher Tegner level. Fifty-six (83.6%) of the athletes that RTP were still playing at 2 years postsurgery. Twenty (57.1%) of those who had RTP and were more than 5 years postsurgery were still playing at 5 years. Six (7.9%) players required further surgery relating to the LARS/metalwork. One case had soft tissue inflammation adjacent to the proximal end of the synthetic graft, but it is unknown if this was mechanical irritation or a biological reaction. One MCL reruptured 4 years after RTP. Utilising LARS to augment extra-articular knee ligament reconstructions allows 88.2% of athletes with a variety of knee ligament injuries to return to elite sport. The low morbidity rates coupled with 57% of athletes still playing 5 years postsurgery demonstrates that the LARS is safe and effective in these cases. Although there are reports of LARS ligaments being used in MCL and PLC reconstructions, there is very little evidence investigating if they are safe and effective. This study demonstrates that LARS synthetic grafts can be safely used for MCL and PLC reconstructions in elite athletes and they permit a high RTP with a low risk of complications. Level IV.

Assessment of communicative competence in adult patients with minimum response in intensive care units: A scoping review.

Few formal instruments exist to assess the communicative competence of patients hospitalized in intensive care units (ICUs). This can limit interventions by health professionals. To map the categories and instruments for assessing the communicative competence of adult patients with minimal response in ICUs. A scoping review was carried out following the Joanna Briggs Institute protocol between February and March 2022 and using the MEDLINE (PubMed), Scopus, Scielo, Business Source Complete (via EBSCOhost), Academic Search Complete (via EBSCOhost) and Web of Science databases, in Portuguese, English and Spanish. Eight studies met the inclusion criteria. The different communication and pain assessment protocols covered awareness, cognition, sensory capacity, motor capacity, language, speech and literacy. The present review offers a starting point for the construction of a formal assessment instrument to enable clinicians to implement an augmentative or alternative communication system (AACS) for the voluntary, independent and active participation of patients. What is already known on the subject Patients in ICUs are subject to various forms of treatment and continuous and intensive monitoring, compromising their capacity to communicate and actively participate (e.g., sharing symptoms and making decisions). Although there is some awareness of their disadvantage in such a regard, few protocols of assessment of communicative competence have been adapted to patients with a minimum response. What this paper adds to the existing knowledge The present review highlights different protocols for the assessment of communication and pain. They include the following categories: awareness, sensory capacity, auditory and visual acuity, positioning and motor capacity, language, speech, and literacy. The review offers a starting point for the construction of a formal assessment instrument encompassing the aforementioned categories, along with duly validated guidelines for its application. What are the potential or actual clinical implications of this work? Our formal assessment instrument takes into account the need to adapt to different patient profiles. It is hoped that it will provide speech therapists and other health professionals with the information required to implement an AACS in which patients participate actively.

Exploration of the application of augmented reality technology for teaching spinal tumor's anatomy and surgical techniques.

Augmented reality (AR) technology is gradually being applied in surgical teaching as an innovative teaching method. Developing innovative teaching methods to replicate clinical theory and practical teaching scenarios, simulate preoperative planning and training for bone tumor surgery, and offer enhanced training opportunities for young physicians to acquire and apply clinical knowledge is a crucial concern that impacts the advancement of the discipline and the educational standards for young orthopedic physicians. This study explores the application effect of augmented reality technology in anatomy teaching and surgical clinical teaching for spinal tumor. The method utilizes virtual reality and augmented reality technology to present a spinal tumor model and the surgical process of percutaneous vertebroplasty. We conducted a random selection of 12 students forming into the augmented reality teaching group and 13 students forming into the traditional teaching group among the 8-year medical students from Peking Union Medical College and Tsinghua University, ensuring that the age and learning stage of the students in both groups were similar. Two groups of students were taught using traditional teaching methods and augmented reality technology-assisted teaching methods, respectively. A questionnaire survey was conducted after class to assess the quality of course instruction, student motivation in learning, their proficiency in anatomical structures, their comprehension of spinal tumor growth and metastasis, and their understanding and proficiency in percutaneous vertebroplasty. This study was the first to apply augmented reality technology in teaching, using spinal tumors and percutaneous vertebroplasty as examples, a head-mounted augmented reality device was used to create learning scenarios, presenting the complex three-dimensional spatial structure intuitively. The two groups of students differ significantly in their rating of teaching quality, enthusiasm for learning, knowledge of anatomical features, understanding of spinal trabecular structure, and understanding of steps in percutaneous vertebroplasty. The augmented reality technology-assisted teaching system demonstrates outstanding advantages. Augmented reality technology has great potential and broad prospects in teaching bone tumors, which can help improve the visualization, interactivity, and three-dimensional spatial sense of medical teaching in spinal tumor. The application and development prospects of using augmented reality technology for anatomy instruction, surgical teaching, and simulation training are extensive.

Event-triggered [formula omitted]-fault-tolerant consistency control for multi-agent systems

This paper investigates the issue of active fault-tolerant control for multi-agent systems using an event-triggered mechanism, aiming to achieve state-consistency control under a leader–follower framework. The proposed control protocol consists of two main components: leader input estimation and fault tolerant controller design under an event-triggered mechanism. Firstly, a leader–follower error system is constructed, where the leader’s input is considered as a supplementary vector for the system state, resulting in an augmented error system. Then, a distributed state observer is designed for this augmented error system to transform the leader’s input estimation problem into an augmented state estimation issue. Furthermore, an event-triggered mechanism is designed to optimize network communication resources by updating the control law only when specific triggered conditions are met. Next, taking into account the errors introduced by the distributed observer and event-triggered mechanism, the problem of determining the state error feedback gain matrix is transformed into a linear matrix inequality problem using H∞ robustness design and Lyapunov stability analysis methods. Finally, the effectiveness of this theoretical framework is validated through a practical example involving several VTOLs.

Augmented reality navigation method based on image segmentation and sensor tracking registration technology

With the rapid development of modern science and technology, navigation technology provides great convenience for people's life, but the problem of inaccurate localization in complex environments has always been a challenge that navigation technology needs to be solved urgently. To address this challenge, this paper proposes an augmented reality navigation method that combines image segmentation and multi-sensor fusion tracking registration. The method optimizes the image processing process through the GA-OTSU-Canny algorithm and combines high-precision multi-sensor information in order to achieve accurate tracking of positioning and guidance in complex environments. Experimental results show that the GA-OTSU-Canny algorithm has a faster image edge segmentation rate, and the fastest start speed is only 1.8 s, and the fastest intersection selection time is 1.2 s. The navigation system combining the image segmentation and sensor tracking and registration techniques has a highly efficient performance in real-world navigation, and its building recognition rates are all above 99%. The augmented reality navigation system not only improves the navigation accuracy in high-rise and urban canyon environments, but also significantly outperforms traditional navigation solutions in terms of navigation startup time and target building recognition accuracy. In summary, this research not only provides a new framework for the theoretical integration of image processing and multi-sensor data, but also brings innovative technical solutions for the development and application of practical navigation systems.