Details Of Scene Research Articles

IGAF: Incremental Guided Attention Fusion for Depth Super-Resolution.

Accurate depth estimation is crucial for many fields, including robotics, navigation, and medical imaging. However, conventional depth sensors often produce low-resolution (LR) depth maps, making detailed scene perception challenging. To address this, enhancing LR depth maps to high-resolution (HR) ones has become essential, guided by HR-structured inputs like RGB or grayscale images. We propose a novel sensor fusion methodology for guided depth super-resolution (GDSR), a technique that combines LR depth maps with HR images to estimate detailed HR depth maps. Our key contribution is the Incremental guided attention fusion (IGAF) module, which effectively learns to fuse features from RGB images and LR depth maps, producing accurate HR depth maps. Using IGAF, we build a robust super-resolution model and evaluate it on multiple benchmark datasets. Our model achieves state-of-the-art results compared to all baseline models on the NYU v2 dataset for ×4, ×8, and ×16 upsampling. It also outperforms all baselines in a zero-shot setting on the Middlebury, Lu, and RGB-D-D datasets. Code, environments, and models are available on GitHub.

Practicing community and sociality in public: scenes from amateur football at the Hackney Marshes, London

ABSTRACT This article explores the social life of amateur football at the Hackney Marshes in London. Based on a season-long period of participant observation, the article looks at the different registers of social and cultural life achieved through the practice of playing amateur Sunday League football. Studying this everyday amateur sporting scene in detail highlights how communities of practice are made, and how focusing on shared activities can shed new light on how the public life of cities can be more than a community of strangers. The article also contributes to the emerging literatures on social infrastructures showing how spaces like sports fields can support the development and maintenance of social connection. This case study highlights how the social life being observed cannot be separated from the practice of playing football, and that if we want to facilitate the realization of more abstract values in urban environments – e.g. community, cooperation, kinship – then it is necessary to provide the kinds of social infrastructures that make it easier to take part in activities together.

Influence of Narrative Specificity and Voice Quality When Listening to Audio Descriptions:

Audio description (AD) serves as a vital means to make visual media accessible to non-sighted and visually impaired audiences. This study systematically investigates the impact of narrative specificity and voice quality on imageability and comprehension in both sighted and non-sighted populations. Twenty non-sighted participants, including congenitally blind individuals and those who lost their sight early in life, were compared with a group of 20 sighted participants, matched for verbal working memory capabilities. Participants listened to 50 short event descriptions, describing spatiotemporal relations with varying levels of narrative specificity, presented in both typical and dysphonic voices. After each event description, participants rated their ability to imagine the content, overall comprehension, listening effort, and listening enjoyment. Results indicate that high narrative specificity enhanced imageability in non-sighted individuals, especially for scenarios involving changes in motion, and, to some extent, for visuospatial relations, irrespective of sightedness. Additionally, dysphonic voices increased listening effort and reduced enjoyment for non-sighted participants only. These findings underscore the importance of considering voice quality and narrative specificity in AD for non-sighted users and have implications for both professional audio describers and the development of automated AD systems. Lay summary Imagine you're watching a movie but instead of seeing the scenes, you're listening to someone describe them to you. This is what audio description (AD) does - it makes movies, TV shows, and other visual media accessible for people who can't see or have trouble seeing. But not all descriptions are created equal. Think about the difference between someone telling you "a person walks into a room" versus "a tall, anxious man in a red shirt bursts into a sunlit, cluttered room, glancing over his shoulder." The second description paints a much clearer picture in your mind, doesn't it? This study looked at how specific these descriptions are (like our detailed scene above) and the quality of the voice telling the story, to see how they affect people's ability to imagine and understand what's being described. We worked with 40 people - half of whom have never been able to see or lost their sight when they were very young, and the other half who can see. Everyone listened to 50 short stories about different events. These stories varied in how detailed they were and were told in either a clear voice or a voice that was hard to listen to (a hoarse voice). After hearing each story, participants judged how well they could picture what was described, how well they understood it, how hard they had to work to listen, and how much they enjoyed listening. The results were interesting. When the stories were really detailed, people who couldn't see were better able to "see" the action in their minds, especially if the story involved movement or where things were located. This was true for everyone, regardless of whether they could see or not. But, when the voice telling the story was hard to listen to, it made it tougher for people who couldn't see to follow along and enjoy the story. What this tells us is that for audio descriptions to be really helpful and enjoyable for everyone, especially those who rely on them, it's important to not only choose the right words but also the right voice. This insight is valuable not just for people who create audio descriptions but also for developing technology that can automatically generate them. Making movies and TV shows more enjoyable and accessible for everyone is the goal, and we hope that this study help getting us there. Declaration of use of AI: ChatGPT4 was used to construct this lay-text based on the abstract from the original manuscript

From Detection to Action: A Multimodal AI Framework for Traffic Incident Response

With the rising incidence of traffic accidents and growing environmental concerns, the demand for advanced systems to ensure traffic and environmental safety has become increasingly urgent. This paper introduces an automated highway safety management framework that integrates computer vision and natural language processing for real-time monitoring, analysis, and reporting of traffic incidents. The system not only identifies accidents but also aids in coordinating emergency responses, such as dispatching ambulances, fire services, and police, while simultaneously managing traffic flow. The approach begins with the creation of a diverse highway accident dataset, combining public datasets with drone and CCTV footage. YOLOv11s is retrained on this dataset to enable real-time detection of critical traffic elements and anomalies, such as collisions and fires. A vision–language model (VLM), Moondream2, is employed to generate detailed scene descriptions, which are further refined by a large language model (LLM), GPT 4-Turbo, to produce concise incident reports and actionable suggestions. These reports are automatically sent to relevant authorities, ensuring prompt and effective response. The system’s effectiveness is validated through the analysis of diverse accident videos and zero-shot simulation testing within the Webots environment. The results highlight the potential of combining drone and CCTV imagery with AI-driven methodologies to improve traffic management and enhance public safety. Future work will include refining detection models, expanding dataset diversity, and deploying the framework in real-world scenarios using live drone and CCTV feeds. This study lays the groundwork for scalable and reliable solutions to address critical traffic safety challenges.

Spontaneous associative thought may facilitate scene-gist memory via implicit scene-labeling.

Spontaneous associative processes (e.g., mind wandering, spontaneous memory recollection) are prevalent in everyday life, yet their influence on perceptual scene memory is under debate. Given that scene perception involves extraction of contextual associations, we hypothesized that associative thought would enhance scene memory by promoting encoding of contextual associations. In an online experiment (N = 75), participants viewed scenes, and following each scene either generated chained-free associations (associative processing), or, as control, listed words that begin with a specific letter (phonological processing). Scene memory was tested after an intermediate creativity task, which is also shown to rely on associative processes. Results revealed that associative thought, regardless of its conceptual (semantic) distances between responses, enhanced scene-gist memory, but hampered memory of scene details, implying that associative thought facilitates contextual encoding. In a follow-up experiment (N = 74), we found that the effect of associative thought on scene-gist memory was mediated by scene labeling. When participants were asked to explicitly label the scene before completing an associative processing or a phonological processing task, scene-gist memory was prioritized at the expense of scene details, eliminating the memory differences between tasks. These findings imply that labeling past perceived scenes, whether explicitly or implicitly during associative thought, facilitates scene-gist memory. Lastly, in both experiments, creativity was not correlated with scene memory but was positively correlated with the semantic distances between scene-based associations, extending past findings that link creativity with the breadth of associative processes. Together, these findings highlight the likely effect of post-perceptual associative processes on higher-order cognitive functions, such as memory consolidation and creative thought.

SeBIR: Semantic-guided burst image restoration

Burst image restoration methods offer the possibility of recovering faithful scene details from multiple low-quality snapshots captured by hand-held devices in adverse scenarios, thereby attracting increasing attention in recent years. However, individual frames in a burst typically suffer from inter-frame misalignments, leading to ghosting artifacts. Besides, existing methods indiscriminately handle all burst frames, struggling to seamlessly remove the corrupted information due to the neglect of multi-frame spatio-temporal varying degradation. To alleviate these limitations, we propose a general semantic-guided model named SeBIR for burst image restoration incorporating the semantic prior knowledge of Segment Anything Model (SAM) to enable adaptive recovery. Specifically, instead of relying solely on a single aligning scheme, we develop a joint implicit and explicit strategy that sufficiently leverages semantic knowledge as guidance to achieve inter-frame alignment. To further adaptively modulate and aggregate aligned features with spatio-temporal disparity, we elaborate a semantic-guided fusion module using the intermediate semantic features of SAM as an explicit guide to weaken the inherent degradation and strengthen the valuable complementary information across frames. Additionally, a semantic-guided local loss is designed to boost local consistency and image quality. Extensive experiments on synthetic and real-world datasets demonstrate the superiority of our method in both quantitative and qualitative evaluations for burst super-resolution, burst denoising, and burst low-light image enhancement tasks.

Research on Target Image Classification in Low-Light Night Vision.

In extremely dark conditions, low-light imaging may offer spectators a rich visual experience, which is important for both military and civic applications. However, the images taken in ultra-micro light environments usually have inherent defects such as extremely low brightness and contrast, a high noise level, and serious loss of scene details and colors, which leads to great challenges in the research of low-light image and object detection and classification. The low-light night vision image used as the study object in this work has an excessively dim overall picture and very little information about the screen's features. Three algorithms, HE, AHE, and CLAHE, were used to enhance and highlight the image. The effectiveness of these image enhancement methods is evaluated using metrics such as the peak signal-to-noise ratio and mean square error, and CLAHE was selected after comparison. The target image includes vehicles, people, license plates, and objects. The gray-level co-occurrence matrix (GLCM) was used to extract the texture features of the enhanced images, and the extracted image texture features were used as input to construct a backpropagation (BP) neural network classification model. Then, low-light image classification models were developed based on VGG16 and ResNet50 convolutional neural networks combined with low-light image enhancement algorithms. The experimental results show that the overall classification accuracy of the VGG16 convolutional neural network model is 92.1%. Compared with the BP and ResNet50 neural network models, the classification accuracy was increased by 4.5% and 2.3%, respectively, demonstrating its effectiveness in classifying low-light night vision targets.

NeRF-OR: neural radiance fields for operating room scene reconstruction from sparse-view RGB-D videos.

RGB-D cameras in the operating room (OR) provide synchronized views of complex surgical scenes. Assimilation of this multi-view data into a unified representation allows for downstream tasks such as object detection and tracking, pose estimation, and action recognition. Neural radiance fields (NeRFs) can provide continuous representations of complex scenes with limited memory footprint. However, existing NeRF methods perform poorly in real-world OR settings, where a small set of cameras capture the room from entirely different vantage points. In this work, we propose NeRF-OR, a method for 3D reconstruction of dynamic surgical scenes in the OR. Where other methods for sparse-view datasets use either time-of-flight sensor depth or dense depth estimated from color images, NeRF-OR uses a combination of both. The depth estimations mitigate the missing values that occur in sensor depth images due to reflective materials and object boundaries. We propose to supervise with surface normals calculated from the estimated depths, because these are largely scale invariant. We fit NeRF-OR to static surgical scenes in the 4D-OR dataset and show that its representations are geometrically accurate, where state of the art collapses to sub-optimal solutions. Compared to earlier work, NeRF-OR grasps fine scene details while training 30 faster. Additionally, NeRF-OR can capture whole-surgery videos while synthesizing views at intermediate time values with an average PSNR of 24.86dB. Last, we find that our approach has merit in sparse-view settings beyond those in the OR, by benchmarking on the NVS-RGBD dataset that contains as few as three training views. NeRF-OR synthesizes images with a PSNR of 26.72dB, a 1.7% improvement over state of the art. Our results show that NeRF-OR allows for novel view synthesis with videos captured by a small number of cameras with entirely different vantage points, which is the typical camera setting in the OR. Code is available via: github.com/Beerend/NeRF-OR .

GLFuse: A Global and Local Four-Branch Feature Extraction Network for Infrared and Visible Image Fusion

Infrared and visible image fusion integrates complementary information from different modalities into a single image, providing sufficient imaging information for scene interpretation and downstream target recognition tasks. However, existing fusion methods often focus only on highlighting salient targets or preserving scene details, failing to effectively combine entire features from different modalities during the fusion process, resulting in underutilized features and poor overall fusion effects. To address these challenges, a global and local four-branch feature extraction image fusion network (GLFuse) is proposed. On one hand, the Super Token Transformer (STT) block, which is capable of rapidly sampling and predicting super tokens, is utilized to capture global features in the scene. On the other hand, a Detail Extraction Block (DEB) is developed to extract local features in the scene. Additionally, two feature fusion modules, namely the Attention-based Feature Selection Fusion Module (ASFM) and the Dual Attention Fusion Module (DAFM), are designed to facilitate selective fusion of features from different modalities. Of more importance, the various perceptual information of feature maps learned from different modality images at the different layers of a network is investigated to design a perceptual loss function to better restore scene detail information and highlight salient targets by treating the perceptual information separately. Extensive experiments confirm that GLFuse exhibits excellent performance in both subjective and objective evaluations. It deserves note that GLFuse effectively improves downstream target detection performance on a unified benchmark.

Semantic Segmentation Network Based on Adaptive Attention and Deep Fusion Utilizing a Multi-Scale Dilated Convolutional Pyramid.

Deep learning has recently made significant progress in semantic segmentation. However, the current methods face critical challenges. The segmentation process often lacks sufficient contextual information and attention mechanisms, low-level features lack semantic richness, and high-level features suffer from poor resolution. These limitations reduce the model's ability to accurately understand and process scene details, particularly in complex scenarios, leading to segmentation outputs that may have inaccuracies in boundary delineation, misclassification of regions, and poor handling of small or overlapping objects. To address these challenges, this paper proposes a Semantic Segmentation Network Based on Adaptive Attention and Deep Fusion with the Multi-Scale Dilated Convolutional Pyramid (SDAMNet). Specifically, the Dilated Convolutional Atrous Spatial Pyramid Pooling (DCASPP) module is developed to enhance contextual information in semantic segmentation. Additionally, a Semantic Channel Space Details Module (SCSDM) is devised to improve the extraction of significant features through multi-scale feature fusion and adaptive feature selection, enhancing the model's perceptual capability for key regions and optimizing semantic understanding and segmentation performance. Furthermore, a Semantic Features Fusion Module (SFFM) is constructed to address the semantic deficiency in low-level features and the low resolution in high-level features. The effectiveness of SDAMNet is demonstrated on two datasets, revealing significant improvements in Mean Intersection over Union (MIOU) by 2.89% and 2.13%, respectively, compared to the Deeplabv3+ network.

VE-LIOM: A Versatile and Efficient LiDAR-Inertial Odometry and Mapping System

LiDAR has emerged as one of the most pivotal sensors in the field of navigation, owing to its expansive measurement range, high resolution, and adeptness in capturing intricate scene details. This significance is particularly pronounced in challenging navigation scenarios where GNSS signals encounter interference, such as within urban canyons and indoor environments. However, the copious volume of point cloud data poses a challenge, rendering traditional iterative closest point (ICP) methods inadequate in meeting real-time odometry requirements. Consequently, many algorithms have turned to feature extraction approaches. Nonetheless, with the advent of diverse scanning mode LiDARs, there arises a necessity to devise unique methods tailored to these sensors to facilitate algorithm migration. To address this challenge, we propose a weighted point-to-plane matching strategy that focuses on local details without relying on feature extraction. This improved approach mitigates the impact of imperfect plane fitting on localization accuracy. Moreover, we present a classification optimization method based on the normal vectors of planes to further refine algorithmic efficiency. Finally, we devise a tightly coupled LiDAR-inertial odometry system founded upon optimization schemes. Notably, we pioneer the derivation of an online gravity estimation method from the perspective of S2 manifold optimization, effectively minimizing the influence of gravity estimation errors introduced during the initialization phase on localization accuracy. The efficacy of the proposed method was validated through experimentation employing various LiDAR sensors. The outcomes of indoor and outdoor experiments substantiate its capability to furnish real-time and precise localization and mapping results.

Lightweight object detection in low light: Pixel-wise depth refinement and TensorRT optimization

Images captured in low-light conditions present significant challenges for accurate object detection due to factors such as high noise, poor illumination, and low contrast. In this study, we propose an innovative approach that combines pixel-wise depth refinement and TensorRT optimization to improve object detection performance and inference speed under challenging lighting scenarios. The Pixel-wise depth refinement utilizes a deep learning model to estimate the depth curve for each pixel in the low-light image, providing detailed scene depth information. We used the Lightweight YOLOv8 object detection model using TensorRT. This will reduce the model's computational complexity and memory footprint, enabling faster inference and real-time performance on resource-constrained devices. By combining these techniques, our approach aims to accomplish enhanced entity recognition precision in reduced-illumination environments while maintaining efficiency and real-time performance. We evaluate the effectiveness of our proposed modifications to the depth curve estimation block and the overall approach on benchmark datasets and demonstrate its potential for enhancing object detection capabilities in challenging low-light conditions. Furthermore, the experimental results show an improvement in the precision, recall, and mean average precisions (mAP), and also by using the TensorRT the inference speed is four times faster than the existing one.

VR Scene Detail Enhancement Method Based on Depth Reinforcement Learning Algorithm

In virtual reality, due to factors such as light sources and surface materials of objects, the details of the scene exhibit extremely complex changes, making it difficult to capture environmental modeling relationships and reducing the quality of scene details. Therefore, a VR scene detail enhancement method based on deep reinforcement learning algorithm is proposed. Using the Total Variation (TV) denoising algorithm to decompose the VR scene, the VR scene is divided into two parts: noisy and non-noisy, to complete the noise removal. Based on the denoised VR scene, a Hessian matrix is constructed to extract VR scene features using the SURF algorithm. Using deep reinforcement learning algorithms to train and process the extracted VR visual features, introducing meta-learning within the gradient descent method, updating the parameters of the deep reinforcement learning Expose framework, and accelerating the training speed of the deep reinforcement learning Expose framework. By designing L1 loss, structural similarity loss, content perception loss, and sharpness loss functions, the enhancement effect of VR visual details can be improved. The experimental results show that the proposed method enhances the gray-scale values and distribution of VR scene detail images to be higher and more uniform. When the magnification is different, the studied method can effectively enhance the signal-to-noise ratio of VR scenes. Therefore, it indicates that the VR scene detail enhancement effect of the method proposed in this article is good.

Songs tell a story: The Arc of narrative for music.

Research suggests that a core lexical structure characterized by words that define plot staging, plot progression, and cognitive tension underlies written narratives. Here, we investigate the extent to which song lyrics follow this underlying narrative structure. Using a text analytic approach and two publicly available datasets of song lyrics including a larger dataset (N = 12,280) and a smaller dataset of greatest hits (N = 2,823), we find that music lyrics tend to exhibit a core Arc of Narrative structure: setting the stage at the beginning, progressing the plot steadily until the end of the song, and peaking in cognitive tension in the middle. We also observe differences in narrative structure based on musical genre, suggesting different genres set the scene in greater detail (Country, Rap) or progress the plot faster and have a higher rate of internal conflict (Pop). These findings add to the evidence that storytelling exhibits predictable language patterns and that storytelling is evident in music lyrics.

Leveraging Deep Learning for Automated Reconstruction of Indoor Unstructured Elements in Scan-to-BIM

Abstract. Achieving automatic 3D reconstruction for indoor scenes is extremely useful in the field of scene understanding. Building information modeling (BIM) models are essential for lowering project costs, assisting in building planning and renovations, as well as improving building management efficiency. However, nearly all current available scan-to-BIM approaches employ manual or semi-automatic methods. These approaches concentrate solely on significant structured objects, neglecting other unstructured elements such as furniture. The limitation arises from challenges in modeling incomplete point clouds of obstructed objects and capturing indoor scene details. Therefore, this research introduces an innovative and effective reconstruction framework based on deep learning semantic segmentation and model-driven techniques to address these limitations. The proposed framework utilizes wall segment recognition, feature extraction, opening detection, and automatic modeling to reconstruct 3D structured models of point clouds with different room layouts in both Manhattan and non-Manhattan architectures. Moreover, it provides 3D BIM models of actual unstructured elements by detecting objects, completing point clouds, establishing bounding boxes, determining type and orientation, and automatically generating 3D BIM models with a parametric algorithm implemented into the Revit software. We evaluated this framework using publicly available and locally generated point cloud datasets with varying furniture combinations and layout complexity. The results demonstrate the proposed framework's efficiency in reconstructing structured indoor elements, exhibiting completeness and geometric accuracy, and achieving precision and recall values greater than 98%. Furthermore, the generated unstructured 3D BIM models keep essential real-scene characteristics such as geometry, spatial locations, numerical aspects, various shapes, and orientations compared to literature methods.

TRIPS: Trilinear Point Splatting for Real‐Time Radiance Field Rendering

AbstractPoint‐based radiance field rendering has demonstrated impressive results for novel view synthesis, offering a compelling blend of rendering quality and computational efficiency. However, also latest approaches in this domain are not without their shortcomings. 3D Gaussian Splatting [KKLD23] struggles when tasked with rendering highly detailed scenes, due to blurring and cloudy artifacts. On the other hand, ADOP [RFS22] can accommodate crisper images, but the neural reconstruction network decreases performance, it grapples with temporal instability and it is unable to effectively address large gaps in the point cloud. In this paper, we present TRIPS (Trilinear Point Splatting), an approach that combines ideas from both Gaussian Splatting and ADOP. The fundamental concept behind our novel technique involves rasterizing points into a screen‐space image pyramid, with the selection of the pyramid layer determined by the projected point size. This approach allows rendering arbitrarily large points using a single trilinear write. A lightweight neural network is then used to reconstruct a hole‐free image including detail beyond splat resolution. Importantly, our render pipeline is entirely differentiable, allowing for automatic optimization of both point sizes and positions.Our evaluation demonstrate that TRIPS surpasses existing state‐of‐the‐art methods in terms of rendering quality while maintaining a real‐time frame rate of 60 frames per second on readily available hardware. This performance extends to challenging scenarios, such as scenes featuring intricate geometry, expansive landscapes, and auto‐exposed footage. The project page is located at:https://lfranke.github.io/trips

Aquavision Alchemy Illuminating Underwater Imagery Through Medium Transmission

Restoring underwater scenes often involves addressing interference caused by the underwater environment. Many existing methods overlook the scale-related characteristics inherent in these scenes. To tackle this, we propose a synergistic multi-scale detail refinement method for enhancing underwater scene details. This approach comprises multiple stages, starting with a low-degradation stage that enriches original images with multi-scale details using the Adaptive Selective Intrinsic Supervised Feature module. ASISF, through intrinsic supervision, precisely manages feature transmission across different degradation stages, refining multi-scale details while minimizing irrelevant information. Within the framework, we introduce the Bifocal Intrinsic-Context Attention Module, which efficiently utilizes multi-scale scene information by leveraging spatial contextual relationships. Throughout training, a multi-degradation loss function enhances the network's ability to extract information across various scales. The proposed method consistently outperforms state-of-the-art methods when evaluated against them. Key Words: Interference mitigation, Multi-scale detail refinement, Intrinsic supervision, Spatial contextual relationships.

GSN: Generalisable Segmentation in Neural Radiance Field

Traditional Radiance Field (RF) representations capture details of a specific scene and must be trained afresh on each scene. Semantic feature fields have been added to RFs to facilitate several segmentation tasks. Generalised RF representations learn the principles of view interpolation. A generalised RF can render new views of an unknown and untrained scene, given a few views. We present a way to distil feature fields into the generalised GNT representation. Our GSN representation generates new views of unseen scenes on the fly along with consistent, per-pixel semantic features. This enables multi-view segmentation of arbitrary new scenes. We show different semantic features being distilled into generalised RFs. Our multi-view segmentation results are on par with methods that use traditional RFs. GSN closes the gap between standard and generalisable RF methods significantly. Project Page: https://vinayak-vg.github.io/GSN/

SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation

Recently, self-supervised monocular depth estimation has gained popularity with numerous applications in autonomous driving and robotics. However, existing solutions primarily seek to estimate depth from immediate visual features, and struggle to recover fine-grained scene details. In this paper, we introduce SQLdepth, a novel approach that can effectively learn fine-grained scene structure priors from ego-motion. In SQLdepth, we propose a novel Self Query Layer (SQL) to build a self-cost volume and infer depth from it, rather than inferring depth from feature maps. We show that, the self-cost volume is an effective inductive bias for geometry learning, which implicitly models the single-frame scene geometry, with each slice of it indicating a relative distance map between points and objects in a latent space. Experimental results on KITTI and Cityscapes show that our method attains remarkable state-of-the-art performance, and showcases computational efficiency, reduced training complexity, and the ability to recover fine-grained scene details. Moreover, the self-matching-oriented relative distance querying in SQL improves the robustness and zero-shot generalization capability of SQLdepth. Code is available at https://github.com/hisfog/SfMNeXt-Impl.

Synergistic Multiscale Detail Refinement via Intrinsic Supervision for Underwater Image Enhancement

Visually restoring underwater scenes primarily involves mitigating interference from underwater media. Existing methods ignore the inherent scale-related characteristics in underwater scenes. Therefore, we present the synergistic multi-scale detail refinement via intrinsic supervision (SMDR-IS) for enhancing underwater scene details, which contain multi-stages. The low-degradation stage from the original images furnishes the original stage with multi-scale details, achieved through feature propagation using the Adaptive Selective Intrinsic Supervised Feature (ASISF) module. By using intrinsic supervision, the ASISF module can precisely control and guide feature transmission across multi-degradation stages, enhancing multi-scale detail refinement and minimizing the interference from irrelevant information in the low-degradation stage. In multi-degradation encoder-decoder framework of SMDR-IS, we introduce the Bifocal Intrinsic-Context Attention Module (BICA). Based on the intrinsic supervision principles, BICA efficiently exploits multi-scale scene information in images. BICA directs higher-resolution spaces by tapping into the insights of lower-resolution ones, underscoring the pivotal role of spatial contextual relationships in underwater image restoration. Throughout training, the inclusion of a multi-degradation loss function can enhance the network, allowing it to adeptly extract information across diverse scales. When benchmarked against state-of-the-art methods, SMDR-IS consistently showcases superior performance. Our code is available at https://github.com/zhoujingchun03/SMDR-IS