Low-resolution Representation Research Articles

Few-shot classification with Fork Attention Adapter

Few-shot learning aims to transfer the knowledge learned from seen categories to unseen categories with a few references. It is also an essential challenge to bridge the gap between humans and deep learning models in real-world applications. Despite extensive previous efforts to tackle this problem by finding an appropriate similarity function, we emphasize that most existing methods have merely considered a single low-resolution representation pair utilized in similarity calculations between support and query samples. Such representational limitations could induce the instability of category predictions. To better achieve metric learning stabilities, we present a novel method dubbed Fork Attention Adapter (FA-adapter), which can seamlessly establish the dense feature similarity with the newly generated nuanced features. The utility of the proposed method is more performant and efficient via the two-stage training phase. Extensive experiments demonstrate consistent and substantial accuracy gains on the fine-grained CUB, Aircraft, non-fine-grained mini-ImageNet, and tiered-ImageNet benchmarks. By comprehensively studying and visualizing the learned knowledge from different source domains, we further present an extension version termed FA-adapter++ to boost the performance in fine-grained scenarios.

CFNet: Cross-scale fusion network for medical image segmentation

Learning multi-scale feature representations is essential for medical image segmentation. Most existing frameworks are based on U-shape architecture in which the high-resolution representation is recovered progressively by connecting different levels of the decoder with the low-resolution representation from the encoder. However, intrinsic defects in complementary feature fusion inhibit the U-shape from aggregating efficient global and discriminative features along object boundaries. While Transformer can help model the global features, their computation complexity limits the application in real-time medical scenarios. To address these issues, we propose a Cross-scale Fusion Network (CFNet), combining a cross-scale attention module and pyramidal module to fuse multi-stage/global context information. Specifically, we first utilize large kernel convolution to design the basic building block capable of extracting global and local information. Then, we propose a Bidirectional Atrous Spatial Pyramid Pooling (BiASPP), which employs atrous convolution in the bidirectional paths to capture various shapes and sizes of brain tumors. Furthermore, we introduce a cross-stage attention mechanism to reduce redundant information when merging features from two stages with different semantics. Extensive evaluation was performed on five medical image segmentation datasets: a 3D volumetric dataset, namely Brats benchmarks. CFNet-L achieves 85.74% and 90.98% dice score for Enhanced Tumor and Whole Tumor on Brats2018, respectively. Furthermore, our largest model CFNet-L outperformed other methods on 2D medical image. It achieved 71.95%, 82.79%, and 80.79% SE for STARE, DRIVE, and CHASEDB1, respectively. The code will be available at https://github.com/aminabenabid/CFNet

A multi-scale and multi-physical coupling method for the transient characteristics of space nuclear reactor

In the future of deep space exploration, space nuclear reactors (SNRs) are essential because they can provide a robust, sustainable energy supply independent of the environment. SNRs have complex multi-physics coupling effects, and their systems involve many modules. The traditional method uses the lumped parameter model, or single-filed, which cannot accurately describe the transient characteristics and has low spatial resolution and poor data representation. Thus, developing a multi-dimensional, multi-physics coupled method to analyze and design SNRs is necessary. This work proposes a multi-dimensional, multi-physics coupled method for the SNRs. A transient coupling code for the space thermionic reactor TOPAZ-II based on OpenFOAM was developed. The reactor neutron physics model, multi-region heat transfer model, thermoelectric conversion model, electromagnetic pump model, and heat pipe radiator model are included. Different scale models are imported by the code, including a three-dimensional reactor model and a two-dimensional heat pipe. Furthermore, the experimental data from the V-71 test unit were used to verify the developed code. The shutdown and startup of the improved TOPAZ-II system are simulated using the developed code. The results show that the calculated results agree well with the experimental data. During the startup and shutdown processes, the material temperatures remained within acceptable limits, so it can be considered that the developed 2D/3D system analysis for SNRs has successfully achieved a high-dimensional simulation of transient characteristics, accurately capturing the spatial distribution of SNRs at different scales.

EXCOGITO, an Extensible Coarse-Graining Toolbox for the Investigation of Biomolecules by Means of Low-Resolution Representations.

Bottom-up coarse-grained (CG) models proved to be essential to complement and sometimes even replace all-atom representations of soft matter systems and biological macromolecules. The development of low-resolution models takes the moves from the reduction of the degrees of freedom employed, that is, the definition of a mapping between a system's high-resolution description and its simplified counterpart. Even in the absence of an explicit parametrization and simulation of a CG model, the observation of the atomistic system in simpler terms can be informative: this idea is leveraged by the mapping entropy, a measure of the information loss inherent to the process of coarsening. Mapping entropy lies at the heart of the extensible coarse-graining toolbox, EXCOGITO, developed to perform a number of operations and analyses on molecular systems pivoting around the properties of mappings. EXCOGITO can process an all-atom trajectory to compute the mapping entropy, identify the mapping that minimizes it, and establish quantitative relations between a low-resolution representation and the geometrical, structural, and energetic features of the system. Here, the software, which is available free of charge under an open-source license, is presented and showcased to introduce potential users to its capabilities and usage.

HR-YOLOv8: A Crop Growth Status Object Detection Method Based on YOLOv8

Crop growth status detection is significant in agriculture and is vital in planting planning, crop yield, and reducing the consumption of fertilizers and workforce. However, little attention has been paid to detecting the growth status of each crop. Accuracy remains a challenging problem due to the small size of individual targets in the image. This paper proposes an object detection model, HR-YOLOv8, where HR means High-Resolution, based on a self-attention mechanism to alleviate the above problem. First, we add a new dual self-attention mechanism to the backbone network of YOLOv8 to improve the model’s attention to small targets. Second, we use InnerShape(IS)-IoU as the bounding box regression loss, computed by focusing on the shape and size of the bounding box itself. Finally, we modify the feature fusion part by connecting the convolution streams from high resolution to low resolution in parallel instead of in series. As a result, our method can maintain a high resolution in the feature fusion part rather than recovering high resolution from low resolution, and the learned representation is more spatially accurate. Repeated multiresolution fusion improves the high-resolution representation with the help of the low-resolution representation. Our proposed HR-YOLOv8 model improves the detection performance on crop growth states. The experimental results show that on the oilpalmuav dataset and strawberry ripeness dataset, our model has fewer parameters compared to the baseline model, and the average detection accuracy is 5.2% and 0.6% higher than the baseline model, respectively. Our model’s overall performance is much better than other mainstream models. The proposed method effectively improves the ability to detect small objects.

Surveying the energy landscape of coarse-grained mappings.

Simulations of soft materials often adopt low-resolution coarse-grained (CG) models. However, the CG representation is not unique and its impact upon simulated properties is poorly understood. In this work, we investigate the space of CG representations for ubiquitin, which is a typical globular protein with 72 amino acids. We employ Monte Carlo methods to ergodically sample this space and to characterize its landscape. By adopting the Gaussian network model as an analytically tractable atomistic model for equilibrium fluctuations, we exactly assess the intrinsic quality of each CG representation without introducing any approximations in sampling configurations or in modeling interactions. We focus on two metrics, the spectral quality and the information content, that quantify the extent to which the CG representation preserves low-frequency, large-amplitude motions and configurational information, respectively. The spectral quality and information content are weakly correlated among high-resolution representations but become strongly anticorrelated among low-resolution representations. Representations with maximal spectral quality appear consistent with physical intuition, while low-resolution representations with maximal information content do not. Interestingly, quenching studies indicate that the energy landscape of mapping space is very smooth and highly connected. Moreover, our study suggests a critical resolution below which a "phase transition" qualitatively distinguishes good and bad representations.

Feature Pyramid Fusion Network for Hyperspectral Pansharpening.

Hyperspectral (HS) pansharpening aims at fusing an observed HS image with a panchromatic (PAN) image, to produce an image with the high spectral resolution of the former and the high spatial resolution of the latter. Most of the existing convolutional neural networks (CNNs)-based pansharpening methods reconstruct the desired high-resolution image from the encoded low-resolution (LR) representation. However, the encoded LR representation captures semantic information of the image and is inadequate in reconstructing fine details. How to effectively extract high-resolution and LR representations for high-resolution image reconstruction is the main objective of this article. In this article, we propose a feature pyramid fusion network (FPFNet) for pansharpening, which permits the network to extract multiresolution representations from PAN and HS images in two branches. The PAN branch starts from the high-resolution stream that maintains the spatial resolution of the PAN image and gradually adds LR streams in parallel. The structure of the HS branch remains highly consistent with that of the PAN branch, but starts with the LR stream and gradually adds high-resolution streams. The representations with corresponding resolutions of PAN and HS branches are fused and gradually upsampled in a coarse to fine manner to reconstruct the high-resolution HS image. Experimental results on three datasets demonstrate the significant superiority of the proposed FPFNet over the state-of-the-art methods in terms of both qualitative and quantitative comparisons.

Expediting Large-Scale Vision Transformer for Dense Prediction Without Fine-Tuning.

In a wide range of dense prediction tasks, large-scale Vision Transformers have achieved state-of-the-art performance while requiring expensive computation. In contrast to most existing approaches accelerating Vision Transformers for image classification, we focus on accelerating Vision Transformers for dense prediction without any fine-tuning. We present two non-parametric operators specialized for dense prediction tasks, a token clustering layer to decrease the number of tokens for expediting and a token reconstruction layer to increase the number of tokens for recovering high-resolution. To accomplish this, the following steps are taken: i) token clustering layer is employed to cluster the neighboring tokens and yield low-resolution representations with spatial structures; ii) the following transformer layers are performed only to these clustered low-resolution tokens; and iii) reconstruction of high-resolution representations from refined low-resolution representations is accomplished using token reconstruction layer. The proposed approach shows promising results consistently on 6 dense prediction tasks, including object detection, semantic segmentation, panoptic segmentation, instance segmentation, depth estimation, and video instance segmentation. Additionally, we validate the effectiveness of the proposed approach on the very recent state-of-the-art open-vocabulary recognition methods. Furthermore, a number of recent representative approaches are benchmarked and compared on dense prediction tasks.

Top-down generation of low-resolution representations improves visual perception and imagination

Perception or imagination requires top-down signals from high-level cortex to primary visual cortex (V1) to reconstruct or simulate the representations bottom-up stimulated by the seen images. Interestingly, top-down signals in V1 have lower spatial resolution than bottom-up representations. It is unclear why the brain uses low-resolution signals to reconstruct or simulate high-resolution representations. By modeling the top-down pathway of the visual system using the decoder of a variational auto-encoder (VAE), we reveal that low-resolution top-down signals can better reconstruct or simulate the information contained in the sparse activities of V1 simple cells, which facilitates perception and imagination. This advantage of low-resolution generation is related to facilitating high-level cortex to form geometry-respecting representations observed in experiments. Furthermore, we present two findings regarding this phenomenon in the context of AI-generated sketches, a style of drawings made of lines. First, we found that the quality of the generated sketches critically depends on the thickness of the lines in the sketches: thin-line sketches are harder to generate than thick-line sketches. Second, we propose a technique to generate high-quality thin-line sketches: instead of directly using original thin-line sketches, we use blurred sketches to train VAE or GAN (generative adversarial network), and then infer the thin-line sketches from the VAE- or GAN-generated blurred sketches. Collectively, our work suggests that low-resolution top-down generation is a strategy the brain uses to improve visual perception and imagination, which inspires new sketch-generation AI techniques.

Low Light Image Enhancement using MIRNet

With the goal of recovering high-quality image content from its degraded version, image restoration enjoys numerous applications, such as in surveillance, computational photography, medical imaging, and remote sensing. Recently, convolutional neural networks (CNNs) have achieved dramatic improvements over conventional approaches for image restoration task. Existing CNN- based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatially precise but contextually less robust results are achieved, while in the latter case, semantically reliable but spatially less accurate outputs are generated. In this paper, we present a novel architecture with the collective goals of maintaining spatiallyprecise high- resolution representations through the entire network, and receiving strong contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing several key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) spatial and channel attention mechanisms for capturing contextual information, and (d) attention based multi- scale feature aggregation. In a nutshell, our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on five real image benchmark datasets demonstrate that our method, named as MIRNet, achieves state-of-the-art results for a variety of image processing tasks, including image denoising, super-resolution and image enhancement.

High-Resolution Iterative Feedback Network for Camouflaged Object Detection

Spotting camouflaged objects that are visually assimilated into the background is tricky for both object detection algorithms and humans who are usually confused or cheated by the perfectly intrinsic similarities between the foreground objects and the background surroundings. To tackle this challenge, we aim to extract the high-resolution texture details to avoid the detail degradation that causes blurred vision in edges and boundaries. We introduce a novel HitNet to refine the low-resolution representations by high-resolution features in an iterative feedback manner, essentially a global loop-based connection among the multi-scale resolutions. To design better feedback feature flow and avoid the feature corruption caused by recurrent path, an iterative feedback strategy is proposed to impose more constraints on each feedback connection. Extensive experiments on four challenging datasets demonstrate that our HitNet breaks the performance bottleneck and achieves significant improvements compared with 29 state-of-the-art methods. In addition, to address the data scarcity in camouflaged scenarios, we provide an application example to convert the salient objects to camouflaged objects, thereby generating more camouflaged training samples from the diverse salient object datasets. Code will be made publicly available.

High-Resolution Swin Transformer for Automatic Medical Image Segmentation.

The resolution of feature maps is a critical factor for accurate medical image segmentation. Most of the existing Transformer-based networks for medical image segmentation adopt a U-Net-like architecture, which contains an encoder that converts the high-resolution input image into low-resolution feature maps using a sequence of Transformer blocks and a decoder that gradually generates high-resolution representations from low-resolution feature maps. However, the procedure of recovering high-resolution representations from low-resolution representations may harm the spatial precision of the generated segmentation masks. Unlike previous studies, in this study, we utilized the high-resolution network (HRNet) design style by replacing the convolutional layers with Transformer blocks, continuously exchanging feature map information with different resolutions generated by the Transformer blocks. The proposed Transformer-based network is named the high-resolution Swin Transformer network (HRSTNet). Extensive experiments demonstrated that the HRSTNet can achieve performance comparable with that of the state-of-the-art Transformer-based U-Net-like architecture on the 2021 Brain Tumor Segmentation dataset, the Medical Segmentation Decathlon's liver dataset, and the BTCV multi-organ segmentation dataset.

Learning Enriched Features for Fast Image Restoration and Enhancement.

Given a degraded input image, image restoration aims to recover the missing high-quality image content. Numerous applications demand effective image restoration, e.g., computational photography, surveillance, autonomous vehicles, and remote sensing. Significant advances in image restoration have been made in recent years, dominated by convolutional neural networks (CNNs). The widely-used CNN-based methods typically operate either on full-resolution or on progressively low-resolution representations. In the former case, spatial details are preserved but the contextual information cannot be precisely encoded. In the latter case, generated outputs are semantically reliable but spatially less accurate. This paper presents a new architecture with a holistic goal of maintaining spatially-precise high-resolution representations through the entire network, and receiving complementary contextual information from the low-resolution representations. The core of our approach is a multi-scale residual block containing the following key elements: (a) parallel multi-resolution convolution streams for extracting multi-scale features, (b) information exchange across the multi-resolution streams, (c) non-local attention mechanism for capturing contextual information, and (d) attention based multi-scale feature aggregation. Our approach learns an enriched set of features that combines contextual information from multiple scales, while simultaneously preserving the high-resolution spatial details. Extensive experiments on six real image benchmark datasets demonstrate that our method, named as MIRNet-v2, achieves state-of-the-art results for a variety of image processing tasks, including defocus deblurring, image denoising, super-resolution, and image enhancement. The source code and pre-trained models are available at https://github.com/swz30/MIRNetv2.

DKTNet: Dual-Key Transformer Network for small object detection

SummaryObject detection is a fundamental computer vision task that plays a crucial role in a wide range of real-world applications. However, it is still a challenging task to detect the small size objects in the complex scene, due to the low resolution and noisy representation appearance caused by occlusion, distant depth view, etc. To tackle this issue, a novel transformer architecture, Dual-Key Transformer Network (DKTNet), is proposed in this paper. To improve the feature attention ability, the coherence of linear layer outputs Q and V are enhanced by a dual-K integrated from K1 and K2, which are computed along Q and V, respectively. Instead of spatial-wise attention, channel-wise self-attention mechanism is adopted to promote the important feature channels and suppress the confusing ones. Moreover, 2D and 1D convolution computations for Q, K and V are proposed. Compared with the fully-connected computation in conventional transformer architectures, the 2D convolution can better capture local details and global contextual information, and the 1D convolution can reduce network complexity significantly. Experimental evaluation is conducted on both general and small object detection datasets. The superiority of the aforementioned features in our proposed approach is demonstrated with the comparison against the state-of-the-art approaches.

Deep learning-based small object detection: A survey.

Small object detection (SOD) is significant for many real-world applications, including criminal investigation, autonomous driving and remote sensing images. SOD has been one of the most challenging tasks in computer vision due to its low resolution and noise representation. With the development of deep learning, it has been introduced to boost the performance of SOD. In this paper, focusing on the difficulties of SOD, we analyze the deep learning-based SOD research papers from four perspectives, including boosting the resolution of input features, scale-aware training, incorporating contextual information and data augmentation. We also review the literature on crucial SOD tasks, including small face detection, small pedestrian detection and aerial image object detection. In addition, we conduct a thorough performance evaluation of generic SOD algorithms and methods for crucial SOD tasks on four well-known small object datasets. Our experimental results show that network configuring to boost the resolution of input features can enable significant performance gains on WIDER FACE and Tiny Person. Finally, several potential directions for future research in the area of SOD are provided.

A Fast and Accurate Human Pose Estimation Method Based on Multi-Scale Feature Fusion Grid Structure

Human pose estimation (HPE) is a research hotspot in the field of computer vision. Most of the existing approaches first generate low-resolution representation from high-resolution representation through continuous serial downsampling, and then reconstruct high-resolution results from low-resolution features through continuous serial upsampling, which loses a lot of effective feature information and leads to slow model inference. In this paper, the Fast Accuracy Network (FANet), a framework that enables fast and high-accuracy HPE, is proposed. The innovation lies in that, first of all, a grid structure is proposed and adopted, which can be regarded as a set of deep paths and shallow paths. The structure uses multiple high-resolution and low-resolution branch pairs to perform skip-level connections at different scale-space levels so that the information can be exchanged between different resolution representations for many times. The feature information fusion of multi-scale space is realized to obtain more abundant feature information. Second, an improved bottleneck block is proposed to extract effective feature information with fewer parameters, ensuring that the computational burden is reduced without sacrificing accuracy performance. The experimental results show that, compared with other current models, FANet has faster inference speed on the premise of a slight improvement in accuracy performance.

Liens entre le déclin sensoriel et cognitif dans le vieillissement normal : revue critique de la littérature et apports de l’approche incarnée et située de la cognition

Biological aging impacts many organs including sensory's ones and the brain, and thus, cognition. Research has shown that the sensory and cognitive decline are positively correlated. The main data from this literature are firstly reviewed in the present article. Then, the four main hypotheses that are generally proposed to explain these associations are presented. According to them, sensory decline is supposed to cause cognitive decline, or vice-versa, or these concomitant changes result from the general alteration of the nervous system. However, none of them seems able to account for all of the existing data. Moreover, the precise mechanisms that may drive these associations remain to be clarified. The rest of the article is then dedicated to the embodied and situated cognition approach as it provides a particularly interesting and adequate framework to account for these links. Indeed, according to this approach, cognitive representations are grounded in their sensorimotor properties. In other words, cognitive functioning is not conceived as detached from sensory functioning, but instead directly dependent of it. Representations are thus thought to emerge from the sensorimotor simulation of the properties involved in these representations. Therefore, sensory decline should directly impact cognitive performance. It is then hypothesized that older adults have low-resolution representations decreasing the signal on noise ratio of traces, increasing sensorimotor interferences and thus decreasing cognitive performance. This approach leads to consider low and high levels of sensory and perceptual functioning, which are both impaired in aging. Therefore, the repercussion of the sensory-perceptual decline is not only valid for present processing, but also extend to all past knowledge. Several predictions are then proposed on (1) the link between high-level perceptual functioning and cognitive functioning in older adults; (2) the possible interaction in young adults between sensory and high-level perceptual functioning as a function of the level of interference of the material involved; (3) the possible link between the motor and cognitive functioning in older adults. The clinical consequences in terms of cognitive stimulation of such a perspective will close the article. It is proposed to develop a stimulation program based on pattern separation mechanism to better process sensory interference in order to train older adults to improve cognitive precision and thus performance.

Few-shot object detection via high-and-low resolution representation

This paper proposes a new Few-Shot Object Detection (FSOD) method, which can be trained with only a few samples, allowing the model to recognize new instances. Unlike previous FSOD methods, this work emphasizes the significance of high-and-low resolution features for the detection of object center points in a few samples and proposes a new architecture for few-shot object detection via the combination of high-and-low resolution features. The high-resolution representation module can effectively extract detailed features of the object. Meanwhile, the low-resolution representation module can provide effective semantic information for the object center points, which is more convenient for the network to locate the new object's center and improve the detection accuracy of the network. Several experiments show that our algorithm can reach the most advanced level in the FSOD field, demonstrating its effectiveness.

High-Resolution Representation Learning and Recurrent Neural Network for Singing Voice Separation

Music source separation has traditionally followed the encoder-decoder paradigm (e.g., hourglass, U-Net, DeconvNet, SegNet) to isolate individual music components from mixtures. Such networks, however, result in a loss of location-sensitivity, as low-resolution representation drops the useful harmonic patterns over the temporal dimension. We overcame this problem by performing singing voice separation using a high-resolution representation learning (HRNet) system coupled with a long short-term memory (LSTM) module to retain high-resolution feature map and capture the temporal behavior of the acoustic signal. We called this joint combination of HRNet and LSTM as HR-LSTM. The predicted spectrograms produced by this system are close to ground truth and successfully separate music sources, achieving results superior to those realized by past methods. The proposed network was tested using four datasets (DSD100, MIR-1K, Korean Pansori, and Nepal Idol singing voice). Our experiments confirmed that the proposed HR-LSTM outperforms state-of-the-art networks at singing voice separation when the DSD100 dataset is used, performs comparably to alternative methods when the MIR-1K dataset is used, and separates the voice and accompaniment components well when the Pansori and NISVS datasets are used. In addition to proposing and validating our network, we also developed and shared our Nepal Idol dataset.

Enhanced Automatic Identification of Urban Community Green Space Based on Semantic Segmentation

At the neighborhood scale, recognizing urban community green space (UCGS) is important for residential living condition assessment and urban planning. However, current studies have embodied two key issues. Firstly, existing studies have focused on large geographic scales, mixing urban and rural areas, neglecting the accuracy of green space contours at fine geographic scales. Secondly, the green spaces covered by shadows often suffer misclassification. To address these issues, we created a neighborhood-scale urban community green space (UCGS) dataset and proposed a segmentation decoder for HRNet backbone with two auxiliary decoders. Our proposed model adds two additional branches to the low-resolution representations to improve their discriminative ability, thus enhancing the overall performance when the high- and low-resolution representations are fused. To evaluate the performance of the model, we tested it on a dataset that includes satellite images of Shanghai, China. The model outperformed the other nine models in UCGS extraction, with a precision of 83.01, recall of 85.69, IoU of 72.91, F1-score of 84.33, and OA of 89.31. Our model also improved the integrity of the identification of shaded green spaces over HRNetV2. The proposed method could offer a useful tool for efficient UCGS detection and mapping in urban planning.