Visual Mapping Research Articles

Indoor fixed-point hovering control for UAVs based on visual inertial SLAM

Purpose Autonomous flight of unmanned aerial vehicles (UAVs) in global position system (GPS)-denied environments has become an increasing research hotspot. This paper aims to realize the indoor fixed-point hovering control and autonomous flight for UAVs based on visual inertial simultaneous localization and mapping (SLAM) and sensor fusion algorithm based on extended Kalman filter. Design/methodology/approach The fundamental of the proposed method is using visual inertial SLAM to estimate the position information of the UAV and position-speed double-loop controller to control the UAV. The motion and observation models of the UAV and the fusion algorithm are given. Finally, experiments are performed to test the proposed algorithms. Findings A position-speed double-loop controller is proposed, by fusing the position information obtained by visual inertial SLAM with the data of airborne sensors. The experiment results of the indoor fixed-points hovering show that UAV flight control can be realized based on visual inertial SLAM in the absence of GPS. Originality/value A position-speed double-loop controller for UAV is designed and tested, which provides a more stable position estimation and enabled UAV to fly autonomously and hover in GPS-denied environment.

Intelligent diagnostic model for pterygium by combining attention mechanism and MobileNetV2.

To evaluate the application of an intelligent diagnostic model for pterygium. For intelligent diagnosis of pterygium, the attention mechanisms-SENet, ECANet, CBAM, and Self-Attention-were fused with the lightweight MobileNetV2 model structure to construct a tri-classification model. The study used 1220 images of three types of anterior ocular segments of the pterygium provided by the Eye Hospital of Nanjing Medical University. Conventional classification models-VGG16, ResNet50, MobileNetV2, and EfficientNetB7-were trained on the same dataset for comparison. To evaluate model performance in terms of accuracy, Kappa value, test time, sensitivity, specificity, the area under curve (AUC), and visual heat map, 470 test images of the anterior segment of the pterygium were used. The accuracy of the MobileNetV2+Self-Attention model with 281 MB in model size was 92.77%, and the Kappa value of the model was 88.92%. The testing time using the model was 9ms/image in the server and 138ms/image in the local computer. The sensitivity, specificity, and AUC for the diagnosis of pterygium using normal anterior segment images were 99.47%, 100%, and 100%, respectively; using anterior segment images in the observation period were 88.30%, 95.32%, and 96.70%, respectively; and using the anterior segment images in the surgery period were 88.18%, 94.44%, and 97.30%, respectively. The developed model is lightweight and can be used not only for detection but also for assessing the severity of pterygium.

Tourism digital detox and digital-free tourism: What do we know? What do we not know? Where should we be heading?

PurposeWhile past research has begun exploring digital-free tourism, tourism digital detox and their benefits, no study to date has comprehensively mapped trends, findings and limitations across this growing body of literature. This study aims to conduct the first bibliometric analysis and systematic literature review to address this gap.Design/methodology/approachThis study utilized a mixed methodology of bibliometric analysis and systematic literature review. Structured search strings were applied to databases to identify relevant papers, which were screened according to inclusion criteria. Bibliometric analysis of included papers was performed using Bibliometrix, an R package enabling network visualization, statistical tests and science mapping. This allowed the identification of significant topics, theories, methods, citations and publication trends over time.FindingsThe results clearly show that factors previously lacking attention in past tourism research, such as the interplay between online and offline experiences during travel, are emerging as important determinants of travelers' well-being. This study outlines the current state of scholarship on managing technology's impacts on travelers' psychological and social needs. Specifically, we found limited research integrating how digital detox tools shape pre-trip planning, on-site activities and post-trip sharing of travel experiences.Originality/valueThis is the first study to comprehensively map trends and findings in digital-free tourism and tourism digital detox research using a blended bibliometric analysis and systematic literature review methodology. It offers vital direction toward strengthening theoretical understanding and supporting balanced connectivity and fulfillment for all tourists going forward. By addressing limitations, this research approach helps develop this area of scholarship in a unified manner.

Visual Mapping of Operating Theater Team Dynamics and Communication for Reflexive Feedback and Surgical Practice Optimization.

Effective operating theater (OT) communication and teamwork are essential to optimal surgical outcomes. We mapped the OT team dynamics and infection control practices using visual methods to guide reflexive feedback and optimize perioperative practices. Data were gathered from adult gastrointestinal surgical teams at a tertiary hospital in India using observations, sociograms (communication mapping tool), and focus group discussions (FGDs). Our methods aimed to map team communication, roles and responsibilities in infection-related practices, and door openings. Qualitative data were thematically analyzed. Quantitative data were analyzed using descriptive statistics. Data were gathered from 10 surgical procedures (over 51 hours) using 16 sociograms, 15 traffic flow maps, and 3 FGDs. Senior surgeons directly influence team hierarchies, dynamics, and communication. While the surgeons, anesthetic residents, and technicians lead most tasks during procedures, the scrub nurse acts as a mediator coordinating activity among role players across hierarchies. Failing to provide the scrub nurse with complete details of the planned surgery leads to multiple door openings to fetch equipment and disposables. Traffic flow observed in 15-minute intervals corresponds to a mean frequency of 56 door openings per hour (min: 16; max: 108), with implications for infection control. Implementing the World Health Organization surgical safety checklist was inconsistent across pathways and does not match reported compliance data. Human factors research is important in optimizing surgical teamwork. Using visual methods to provide feedback to perioperative teams on their communication patterns and behaviors, provided an opportunity for contextualized enhancement of infection prevention and control practices.

Knowledge Mapping for the University: Analysis of Approaches and an Example of Its Use

The purpose of the article is to analyze the current research, development and application of knowledge maps for visualizing heterogeneous, disparate, and often hidden knowledge in universities, and discuss the features of the development of maps in higher education institutions.Materials and methods. The market for intellectual services is actively developing in Russia and worldwide, and the methods of express assessment of human capital are extremely relevant. The intellectual capital of universities is one of the most important assets in the dynamic market of educational and consulting services. Research institutes and universities are modern centers of knowledge and technology, where lecturers and researchers accumulate, transfer and produce knowledge, multiplying and replenishing explicit and implicit intellectual assets. Knowledge management helps educational and research organizations to solve the problems of managing intellectual capital, gain a competitive advantage, improve management efficiency, and reduce risks caused by the concentration of knowledge among a small number of experts. The article is devoted to the problem of analyzing new opportunities for promoting and monetizing the intellectual capital of universities through visualization and knowledge mapping. Knowledge maps of university lecturers are a relevant tool to improve the quality of research and marketing communications, as well as positioning in the market of consulting and intellectual services. Knowledge maps of educational institutions and research teams are of particular research interest, since there is no characteristic business desire to protect knowledge from transfer and reproduction in this field.The result of the study is a typology of knowledge maps, which helps to identify patterns, structures and elements necessary for the development of knowledge bases in universities. The authors based on the results of a pilot study give examples of specific maps. Based on an analysis of existing approaches to the typology of knowledge maps and Bloom’s taxonomy, a hierarchical model reflecting different levels of understanding when interpreting and using knowledge maps was developed. The main research methodology relies on fairly well developed and widely used in the world practice knowledge engineering methods – knowledge extraction, structuring and formalisation. A separate analysis of potential consumers (or stakeholders) of knowledge maps was carried out based on stakeholder theory.Conclusions. The article discusses the methodological aspects of the development and features of the practical application of knowledge maps in higher educational institutions. Knowledge analysis makes it possible to develop the organizational “memory” of a university, collecting strategic knowledge and identifying critical knowledge, and generally enriches the knowledge structure. Knowledge maps as a category of special visual tools used to analyze the knowledge and competencies of university lecturers can improve the process of managing employee competencies, management decision making, and thereby reducing cognitive load. Such maps increase the image and marketing performance of a university in the educational services market, and make the professional landscape of knowledge more transparent.

RETRACTION: Knowledge map visualization of technology hotspots and development trends in China's textile manufacturing industry

RETRACTION: R. Huang, P. Yan, X. Yang, “Knowledge Map Visualization of Technology Hotspots and Development Trends in China's Textile Manufacturing Industry,” IET Collaborative Intelligent Manufacturing 3, no. 3 (2021): 243–251, https://doi.org/10.1049/cim2.12024.The above article, published online on 27 March 2021 in Wiley Online Library (wileyonlinelibrary.com) has been retracted by agreement between the journal's Editors‐in‐Chief, Liang Gao and Weiming Shen; the Institution of Engineering and Technology; and John Wiley & Sons Ltd.The retraction has been agreed on after concerns about this manuscript were raised by a third party. An investigation revealed substantial flaws in the literature analysis presented. The methodical details are described insufficiently. Accordingly, the literature analysis results cannot be reproduced, and the conclusions are considered invalid.The authors have been informed of the decision to retract.

Evaluation of regional comprehensive development efficiency under low-carbon policy: based on optimized DDF-GML combined with unsupervised clustering method

In the study of urban development, it is very important to evaluate the influence of production factors reasonably and efficiently for the region to achieve efficient development. The principal aim of this investigation is to amalgamate the conventional measurement model characterized by robust interpretability with the non-parametric model characterized by limited interpretability, thereby enhancing the precision of research outcomes. Towards this objective, the study employs an optimized directional distance function integrated with a global Malmquist–Luenberger index to formulate a comprehensive total factor productivity measurement framework. In elucidating the homogeneous attributes of regions, departing from prior methodologies reliant on manual or direct algorithmic partitioning, this paper employs the K-means clustering algorithm for index discernment, abstracting the concept of K-means clustering centroids to encapsulate regional homogeneity, thereby delineating results through the visualization of regional development potential maps and the evolution of centroid-based clustering trend maps. The findings of the investigation illuminate common patterns of change across disparate regions, proposing a strategy for leveraging regional resource endowments towards a cohesive framework, thereby transcending constraints imposed by production efficiency limitations. Amidst the backdrop of the COVID-19 pandemic, this study draws upon provincial-level data spanning from 2000 to 2018 in China. The conclusive analytical outcomes underscore the pivotal role of energy factors in regional development efficiency, particularly within high-potential development regions, followed by the capital and labor factors. Concurrently, the study discerns a discernible hierarchical pattern among areas of development potential, which exhibits correlation with factor mobility dynamics.

A visual knowledge map analysis of coal mine intensity based on CiteSpace software and web of science core database

ABSTRACT The research on coal mining intensity provides a basis for rational coal resource extraction and mine ecological environment source protection. This study uses the 815 publications and based on CiteSpace’s visualisation software, takes the authors, institutions, and keywords as the nodes, and constructs the related knowledge graph through keyword co-occurrence and other methods. China ranks first in the world in this field; the current research mainly focuses on coal and gas protrusion, carbon dioxide emission, etc.; future research on coal mining intensity should focus on high-intensity mining, factors affecting coal mining intensity, and mining disturbance.

RNE-DSNet: A Re-parameterization Neighborhood Enhancement-based Dual-Stream Network for CT image recognition

Deep learning models based on Transformer and CNN are current research hotspots. However, there are more time complexity and space complexity in extracting global–local features, the feature extraction ability is insufficient for small lesions in CT images. A Re-parameterization Neighborhood Enhancement-based Dual-Stream Network (RNE-DSNet) is proposed for CT image recognition in this paper. This method improves the ability to recognize small lesions while reducing the Params number and FLOPs, which gives a trade-off between performance and cost. The main works are as following: Firstly, a Re-parameterization Neighborhood Enhancement-based Dual-Stream Network is designed to improve the global–local feature extraction ability. Secondly, the Re-parameterization Group Residual Block is designed in the CNN branch, which improves the local feature extraction ability through multi-scale and multi-branch structures, and the number of the Params and FLOPs are reduced effectively. The Neighborhood Enhancement Transformer Block is designed in the Transformer branch, in which the global attention is calculated by parallel Hadamard product and Dot product, and the global attention maps are enhanced by deep convolution operation to embed local features in global features. Thirdly, the Adaptive-random Mask Cropping method is designed to improve the features extraction ability for small lesions in CT images through heat map and random Mask techniques. RNE-DSNet is compared with other SOTA models. The Accuracy (ACC), Precision (PRE), Recall (REC), F1 score (F1), and Area Under Curve (AUC) values of RNE-DSNet are reached 99.22%, 99.23%, 99.08%, 99.23% and 99.38%, respectively, which verified the effectiveness of the RNE-DSNet model. The number Params and FLOPs are reduced by 62.78% and 84.20%, respectively. The effectiveness of RNE-DSNet is verified by heat map visualization. The performance of RNE-DSNet is generally better than the others methods. There is positive significance for computer aided diagnosis.

Stereo-RIVO: Stereo-Robust Indirect Visual Odometry

Mobile robots and autonomous systems rely on advanced guidance modules which often incorporate cameras to enable key functionalities. These modules are equipped with visual odometry (VO) and visual simultaneous localization and mapping (VSLAM) algorithms that work by analyzing changes between successive frames captured by cameras. VO/VSLAM-based systems are critical backbones for autonomous vehicles, virtual reality, structure from motion, and other robotic operations. VO/VSLAM systems encounter difficulties when implementing real-time applications in outdoor environments with restricted hardware and software platforms. While many VO systems target achieving high accuracy and speed, they often exhibit high degree of complexity and limited robustness. To overcome these challenges, this paper aims to propose a new VO system called Stereo-RIVO that balances accuracy, speed, and computational cost. Furthermore, this algorithm is based on a new data association module which consists of two primary components: a scene-matching process that achieves exceptional precision without feature extraction and a key-frame detection technique based on a model of scene movement. The performance of this proposed VO system has been tested extensively for all sequences of KITTI and UTIAS datasets for analyzing efficiency for outdoor dynamic and indoor static environments, respectively. The results of these tests indicate that the proposed Stereo-RIVO outperforms other state-of-the-art methods in terms of robustness, accuracy, and speed. Our implementation code of stereo-RIVO is available at: https://github.com/salehierfan/Stereo-RIVO.

Mothers in doctoral education: Who cares? An ecological-systems analysis of support for PhD mums

PurposeIn light of a largely negative discourse, this study aims to identify the various ways in which PhD mums have been supported in a range of contexts to develop a comprehensive typology of positive support, as well as to identify patterns that transcend institutional, national and disciplinary borders.Design/methodology/approachThe study is guided by ecological systems theory which allows for the investigation of the various interrelated systems that influence (in this case) doctoral researchers. A mixed-methods survey elicited the first-hand experiences from recent and current PhD mums across the world.FindingsThe authors have identified a range of potential supports for PhD mums, but note a careful balance is needed to ensure that PhD mums are supported in their roles as both mother and doctoral researcher, where support in one domain does not contradict nor ignore support for the other.Originality/valueThis study complements the existing knowledge body, which consists mainly of localised studies, by providing a birds-eye view of issues that transcend national, geographic and disciplinary borders. A topography provides a visual map of the various sources of potential support and the complicated relationships between them.

Study on botulinum toxin in dermatology from 2000 to 2023: A CiteSpace-based bibliometric analyses.

Using bibliometric methods, this study analyzed and summarized the current situation and development of the global application of botulinum toxin in dermatology. Literature published in the Web of Science Core Collection database from January 1, 2000 to October 28, 2023 was searched for topics such as "Botulinum toxin," "Dermatology," and so forth. The number of publications, countries, institutions, journals, authors, cited literature, keywords, and so forth, were analyzed and a visual knowledge map was created using scientometric tools such as CiteSpace, VOSviewer, and Scimago Graphica. A total of 2039 documents were retrieved and 1877 documents were included after de-duplication and transformation. The country with the highest number of published periodical articles was the United States; the main research institution was Yonsei University; the author with the highest number of published periodical articles was Kim, Hee Jin; and the high-frequency keywords mainly related to indications, combination therapy, and safety optimization. The results of this study provide information on the current status and trends in clinical studies of botulinum toxin in dermatology, which will help researchers identify hotspots and explore new research directions in this field.

An evaluation of CNN models and data augmentation techniques in hierarchical localization of mobile robots

This work presents an evaluation of CNN models and data augmentation to carry out the hierarchical localization of a mobile robot by using omnidirectional images. In this sense, an ablation study of different state-of-the-art CNN models used as backbone is presented and a variety of data augmentation visual effects are proposed for addressing the visual localization of the robot. The proposed method is based on the adaption and re-training of a CNN with a dual purpose: (1) to perform a rough localization step in which the model is used to predict the room from which an image was captured, and (2) to address the fine localization step, which consists in retrieving the most similar image of the visual map among those contained in the previously predicted room by means of a pairwise comparison between descriptors obtained from an intermediate layer of the CNN. In this sense, we evaluate the impact of different state-of-the-art CNN models such as ConvNeXt for addressing the proposed localization. Finally, a variety of data augmentation visual effects are separately employed for training the model and their impact is assessed. The performance of the resulting CNNs is evaluated under real operation conditions, including changes in the lighting conditions. Our code is publicly available on the project website https://github.com/juanjo-cabrera/IndoorLocalizationSingleCNN.git.

RNAscape: geometric mapping and customizable visualization of RNA structure.

Analyzing and visualizing the tertiary structure and complex interactions of RNA is essential for being able to mechanistically decipher their molecular functions in vivo. Secondary structure visualization software can portray many aspects of RNA; however, these layouts are often unable to preserve topological correspondence since they do not consider tertiary interactions between different regions of an RNA molecule. Likewise, quaternary interactions between two or more interacting RNA molecules are not considered in secondary structure visualization tools. The RNAscape webserver produces visualizations that can preserve topological correspondence while remaining both visually intuitive and structurally insightful. RNAscape achieves this by designing a mathematical structural mapping algorithm which prioritizes the helical segments, reflecting their tertiary organization. Non-helical segments are mapped in a way that minimizes structural clutter. RNAscape runs a plotting script that is designed to generate publication-quality images. RNAscape natively supports non-standard nucleotides, multiple base-pairing annotation stylesand requires no programming experience. RNAscape can also be used to analyze RNA/DNA hybrid structures and DNA topologies, including G-quadruplexes. Users can upload their own three-dimensional structures or enter a Protein Data Bank (PDB) ID of an existing structure. The RNAscape webserver allows users to customize visualizations through various settings as desired. URL: https://rnascape.usc.edu/.

Semihierarchical reconstruction and weak‐area revisiting for robotic visual seafloor mapping

AbstractDespite impressive results achieved by many on‐land visual mapping algorithms in the recent decades, transferring these methods from land to the deep sea remains a challenge due to harsh environmental conditions. Images captured by autonomous underwater vehicles, equipped with high‐resolution cameras and artificial illumination systems, often suffer from heterogeneous illumination and quality degradation caused by attenuation and scattering, on top of refraction of light rays. These challenges often result in the failure of on‐land Simultaneous Localization and Mapping (SLAM) approaches when applied underwater or cause Structure‐from‐Motion (SfM) approaches to exhibit drifting or omit challenging images. Consequently, this leads to gaps, jumps, or weakly reconstructed areas. In this work, we present a navigation‐aided hierarchical reconstruction approach to facilitate the automated robotic three‐dimensional reconstruction of hectares of seafloor. Our hierarchical approach combines the advantages of SLAM and global SfM that are much more efficient than incremental SfM, while ensuring the completeness and consistency of the global map. This is achieved through identifying and revisiting problematic or weakly reconstructed areas, avoiding to omit images and making better use of limited dive time. The proposed system has been extensively tested and evaluated during several research cruises, demonstrating its robustness and practicality in real‐world conditions.

Knowledge Mapping of the Development Trend of Smart Fisheries in China: A Bibliometric Analysis

In recent years, smart fisheries, as an emerging model for fishery development, have become a research hotspot in the fishery and aquaculture industries of many countries. Smart fisheries can be thought of as a system that combines techniques for raising, catching, or selling aquatic products to improve production and sustainable development. Smart fisheries are crucial to improving fishery and aquaculture management. In this study, a comprehensive analysis was conducted using bibliometric analysis, the results of which are presented through visual mapping and data charts. This study collected data from the China National Knowledge Infrastructure (CNKI) database and compared it with the WoS database. A total of 949 articles were retrieved on topics related to smart fisheries, including 579 articles from WoS and 370 articles from CNKI. The results present the visualization and analysis of annual publications, author collaboration maps, research collaboration institutions, keywords, etc. The development of smart fisheries in China is obviously different from that in foreign countries. China attaches great importance to technology and production, while foreign countries focus on environmental issues. Therefore, this study helps us to understand the current research status, research hotspots, and future development directions of smart fisheries, providing certain references for future management.

Mitigating stuttering self-stigma: How do we start and where do we go? Using a Participative Concept Mapping Approach to develop a local framework of principles

PurposeThis study aims to create a stigma reduction framework for stuttering in the local context of Québec, Canada using the Participative Concept Mapping Approach (PCMA), focusing on both self and societal stigma. MethodEmploying a mixed-methods approach, this study engaged 17 experts—people who stutter, clinicians and health innovation specialists—in PCMA workshops. Via diverse steps, including generation, sorting and rating of ideas in response to the focus prompt, "To effectively address stuttering (self-)stigma, an intervention should…" these sessions led to a framework depicted in visual maps, then refined into actionable principles through qualitative analysis. Mixed-methods data analysis used the open-source R-CMap software to generate visual maps illustrating the relationships among ideas as well as importance and feasibility ratings. ResultsThe collaborative workshops identified 95 ideas in response to the focus prompt, reunited in 7 clusters, evolving into 16 principles to mitigate stuttering stigma and self-stigma. At the therapy level, these principles emphasize personalized therapy, thorough assessments, stigma-free therapeutic environment, empowerment, and the importance of group inclusivity and educating the relational circles. Societally, they advocate for initiatives such as improved educational outreach, empathy enhancement, and better representation. This dual approach targets individual experiences and societal views on stuttering, stressing the need for an all-encompassing intervention framework. ConclusionThe findings demonstrate PCMA's usefulness in crafting local, culturally sensitive, tailored interventions for stigma reduction. The study emphasizes the necessity of holistic approaches that address individual experiences and societal perceptions, offering a model to conduct similar exercises in diverse local settings.

Comparison between NDVI and InSAR data coherence in deformation analysis caused by the 2022 Pasaman earthquake

Abstract One of the destructive earthquakes occurred in Pasaman Barat on February 25, 2022, with a magnitude of 6.1 Mw that caused 27 deaths, 19,221 people evacuated, and an estimated 6,627 houses destroyed. Currently, InSAR is one of the fastest methods to analyze the deformation impacted by an earthquake from processing Sentinel-1 Single Look Complex images. Analyzing the deformation of the 2022 Pasaman Earthquake using InSAR could be challenging because the area consists of high vegetation. It can cause decorrelation due to time difference which increases the possibility of the low coherence value of this data. Excluding the low-coherence result is essential for a better interpretation. This study will analyze the correlation between NDVI and interferogram coherence and eliminate low-coherence data for the displacement deformation map. The NDVI value was calculated using band 4 (Red) and band 5 (NIR) from Landsat 8 images. The InSAR was processed using SNAP, starting from interferogram formation until the conversion phase into displacements, and the map visualization was designed using QGIS. This application of the InSAR method (ascending mode) from Sentinel-1 SAR images can show the deformation, but the coherence level is relatively low. Deformations around the earthquake are shown as uplift and subsidence near the epicenter of the 2022 Pasaman Earthquakes. The high value of vegetation density in the study area causes low coherence between images, so the fringes that identify deformation due to the 2022 Pasaman Earthquake were not seen significantly for the entire study area.

Deep Learning for Breast Cancer Risk Prediction: Application to a Large Representative UK Screening Cohort.

Purpose To develop an artificial intelligence (AI) deep learning tool capable of predicting future breast cancer risk from a current negative screening mammographic examination and to evaluate the model on data from the UK National Health Service Breast Screening Program. Materials and Methods The OPTIMAM Mammography Imaging Database contains screening data, including mammograms and information on interval cancers, for more than 300 000 female patients who attended screening at three different sites in the United Kingdom from 2012 onward. Cancer-free screening examinations from women aged 50-70 years were performed and classified as risk-positive or risk-negative based on the occurrence of cancer within 3 years of the original examination. Examinations with confirmed cancer and images containing implants were excluded. From the resulting 5264 risk-positive and 191 488 risk-negative examinations, training (n = 89 285), validation (n = 2106), and test (n = 39 351) datasets were produced for model development and evaluation. The AI model was trained to predict future cancer occurrence based on screening mammograms and patient age. Performance was evaluated on the test dataset using the area under the receiver operating characteristic curve (AUC) and compared across subpopulations to assess potential biases. Interpretability of the model was explored, including with saliency maps. Results On the hold-out test set, the AI model achieved an overall AUC of 0.70 (95% CI: 0.69, 0.72). There was no evidence of a difference in performance across the three sites, between patient ethnicities, or across age groups. Visualization of saliency maps and sample images provided insights into the mammographic features associated with AI-predicted cancer risk. Conclusion The developed AI tool showed good performance on a multisite, United Kingdom-specific dataset. Keywords: Deep Learning, Artificial Intelligence, Breast Cancer, Screening, Risk Prediction Supplemental material is available for this article. ©RSNA, 2024.

Automated construction of cognitive maps with visual predictive coding

Humans construct internal cognitive maps of their environment directly from sensory inputs without access to a system of explicit coordinates or distance measurements. Although machine learning algorithms like simultaneous localization and mapping utilize specialized inference procedures to identify visual features and construct spatial maps from visual and odometry data, the general nature of cognitive maps in the brain suggests a unified mapping algorithmic strategy that can generalize to auditory, tactile and linguistic inputs. Here we demonstrate that predictive coding provides a natural and versatile neural network algorithm for constructing spatial maps using sensory data. We introduce a framework in which an agent navigates a virtual environment while engaging in visual predictive coding using a self-attention-equipped convolutional neural network. While learning a next-image prediction task, the agent automatically constructs an internal representation of the environment that quantitatively reflects spatial distances. The internal map enables the agent to pinpoint its location relative to landmarks using only visual information.The predictive coding network generates a vectorized encoding of the environment that supports vector navigation, where individual latent space units delineate localized, overlapping neighbourhoods in the environment. Broadly, our work introduces predictive coding as a unified algorithmic framework for constructing cognitive maps that can naturally extend to the mapping of auditory, sensorimotor and linguistic inputs.