Multiresolution Representation Research Articles

A joint feature refining and multi-resolution feature learning approach for bandwidth extension of speech coded by EVS codec

The bandwidth of the speech signals is often limited in speech communications due to the specifications of the standardized codecs or insufficient bitrates. So, we present a bandwidth extension (BWE) framework of speech signals coded by the enhanced voice services (EVS) codec. Previous studies on speech bandwidth extension based on convolutional neural network (CNN) which exist channel and spatial redundancy feature information. This work proposes an end-to-end architecture which combines novel channel and spatial reconstruction module. Specifically, the spatial reconstruction module utilizes a mask to distinguish high-frequency and low-frequency features. Group convolutions is then employed to enhance high-frequency features and suppress low-frequency features. The channel reconstruction module is introduced to reduce unnecessary feature information. Additionally, we introduce a novel time–frequency loss function, incorporating time-domain loss and frequency-domain loss based on Mel-spectrum and multi-resolution representation, to optimize the network. To assess the model and loss function’s performance, we conducted experiments on subdatasets encoded at rates of 6.6kbps, 7.2kbps, and 8kbps, respectively. The experimental results demonstrated that our proposed model surpassed other baseline models in terms of LSD, SNR, PESQ, and MOS scores.

DTDNet: Dynamic Target Driven Network for pedestrian trajectory prediction.

Predicting the trajectories of pedestrians is an important and difficult task for many applications, such as robot navigation and autonomous driving. Most of the existing methods believe that an accurate prediction of the pedestrian intention can improve the prediction quality. These works tend to predict a fixed destination coordinate as the agent intention and predict the future trajectory accordingly. However, in the process of moving, the intention of a pedestrian could be a definite location or a general direction and area, and may change dynamically with the changes of surrounding. Thus, regarding the agent intention as a fixed 2-d coordinate is insufficient to improve the future trajectory prediction. To address this problem, we propose Dynamic Target Driven Network for pedestrian trajectory prediction (DTDNet), which employs a multi-precision pedestrian intention analysis module to capture this dynamic. To ensure that this extracted feature contains comprehensive intention information, we design three sub-tasks: predicting coarse-precision endpoint coordinate, predicting fine-precision endpoint coordinate and scoring scene sub-regions. In addition, we propose a original multi-precision trajectory data extraction method to achieve multi-resolution representation of future intention and make it easier to extract local scene information. We compare our model with previous methods on two publicly available datasets (ETH-UCY and Stanford Drone Dataset). The experimental results show that our DTDNet achieves better trajectory prediction performance, and conducts better pedestrian intention feature representation.

Contrast source inversion of sparse targets through multi-resolution Bayesian compressive sensing

The retrieval of non-Born scatterers is addressed within the contrast source inversion (CSI) framework by means of a novel multi-step inverse scattering method that jointly exploits prior information on the class of targets under investigation and progressively-acquired knowledge on the domain under investigation. The multi-resolution (MR) representation of the unknown contrast sources is iteratively retrieved by applying a Bayesian compressive sensing (BCS) sparsity-promoting approach based on a constrained relevance vector machine solver. Representative examples of inversions from synthetic and experimental data are reported to give some indications on the reliability and the robustness of the proposed MR-BCS-CSI method. Comparisons with recent and competitive state-of-the-art alternatives are reported, as well.

Neural Physical Simulation with Multi-Resolution Hash Grid Encoding

We explore the generalization of the implicit representation in the physical simulation task. Traditional time-dependent partial differential equations (PDEs) solvers for physical simulation often adopt the grid or mesh for spatial discretization, which is memory-consuming for high resolution and lack of adaptivity. Many implicit representations like local extreme machine or Siren are proposed but they are still too compact to suffer from limited accuracy in handling local details and a long time of convergence. We contribute a neural simulation framework based on multi-resolution hash grid representation to introduce hierarchical consideration of global and local information, simultaneously. Furthermore, we propose two key strategies: 1) a numerical gradient method for computing high-order derivatives with boundary conditions; 2) a range analysis sample method for fast neural geometry boundary sampling with dynamic topologies. Our method shows much higher accuracy and strong flexibility for various simulation problems: e.g., large elastic deformations, complex fluid dynamics, and multi-scale phenomena which remain challenging for existing neural physical solvers.

Measurable multiresolution systems, endomorphisms, and representations of Cuntz relations

Measurable multiresolution systems, endomorphisms, and representations of Cuntz relations

Multi-Scale Image- and Feature-Level Alignment for Cross-Resolution Person Re-Identification

Cross-Resolution Person Re-Identification (re-ID) aims to match images with disparate resolutions arising from variations in camera hardware and shooting distances. Most conventional works utilize Super-Resolution (SR) models to recover Low Resolution (LR) images to High Resolution (HR) images. However, because the SR models cannot completely compensate for the missing information in the LR images, there is still a large gap between the HR image recovered from the LR images and the real HR images. To tackle this challenge, we propose a novel Multi-Scale Image- and Feature-Level Alignment (MSIFLA) framework to align the images on multiple resolution scales at both the image and feature level. Specifically, (i) we design a Cascaded Multi-Scale Resolution Reconstruction (CMSR2) module, which is composed of three cascaded Image Reconstruction (IR) networks, and can continuously reconstruct multiple variables of different resolution scales from low to high for each image, regardless of image resolution. The reconstructed images with specific resolution scales are of similar distribution; therefore, the images are aligned on multiple resolution scales at the image level. (ii) We propose a Multi-Resolution Representation Learning (MR2L) module which consists of three-person re-ID networks to encourage the IR models to preserve the ID-discriminative information during training separately. Each re-ID network focuses on mining discriminative information from a specific scale without the disturbance from various resolutions. By matching the extracted features on three resolution scales, the images with different resolutions are also aligned at the feature-level. We conduct extensive experiments on multiple public cross-resolution person re-ID datasets to demonstrate the superiority of the proposed method. In addition, the generalization of MSIFLA in handling cross-resolution retrieval tasks is verified on the UAV vehicle dataset.

A Deep Multiresolution Representation Framework for Pansharpening

A Deep Multiresolution Representation Framework for Pansharpening

Automatic Detection of Spine Region Using Multiple Pseudo 3D U-Net Models with Weighted Average Voting and Attention Mechanisms

The field of CT imaging has been witnessing significant advancements. However, extracting precise information from complex image data remains a challenging task. This study focuses on automating the extraction of CT images. In our study, we adopt the U-Net architecture, a multi-scale blurring technique on data, to obtain a multi-resolution representation. This method is specifically designed to capture information at various granularities, from more detailed information to broader structures. After applying this multi-step blur, we calculate the difference between adjacent images to take advantage of the change in situation between different resolutions. Although feeding the blurred results Directly into the U-Net model may yield satisfactory results, our approach to computing differences between blurred images focuses on the nuances of these changes. To further enhance accuracy, we focused on ensemble learning, leveraging the weights from the training processes of multiple models to average their output during prediction. The results demonstrated that by adopting our approach, we achieved a Dice accuracy of 96.8% and improved the accuracy of CT image extraction.

Feature Pyramid Fusion Network for Hyperspectral Pansharpening.

Hyperspectral (HS) pansharpening aims at fusing an observed HS image with a panchromatic (PAN) image, to produce an image with the high spectral resolution of the former and the high spatial resolution of the latter. Most of the existing convolutional neural networks (CNNs)-based pansharpening methods reconstruct the desired high-resolution image from the encoded low-resolution (LR) representation. However, the encoded LR representation captures semantic information of the image and is inadequate in reconstructing fine details. How to effectively extract high-resolution and LR representations for high-resolution image reconstruction is the main objective of this article. In this article, we propose a feature pyramid fusion network (FPFNet) for pansharpening, which permits the network to extract multiresolution representations from PAN and HS images in two branches. The PAN branch starts from the high-resolution stream that maintains the spatial resolution of the PAN image and gradually adds LR streams in parallel. The structure of the HS branch remains highly consistent with that of the PAN branch, but starts with the LR stream and gradually adds high-resolution streams. The representations with corresponding resolutions of PAN and HS branches are fused and gradually upsampled in a coarse to fine manner to reconstruct the high-resolution HS image. Experimental results on three datasets demonstrate the significant superiority of the proposed FPFNet over the state-of-the-art methods in terms of both qualitative and quantitative comparisons.

In-depth exploration of digital image watermarking with discrete cosine transform and discrete wavelet transform

Digital image watermarking is a crucial technique used to protect the integrity and ownership of digital images by embedding imperceptible watermarks into the image content. This review concentrates on the utilization of discrete cosine transform (DCT) and discrete wavelet transform (DWT) in digital image watermarking schemes. DCT, widely used in image compression like JPEG, is an attractive choice for watermarking, modifying DCT coefficients with minimal impact on image quality. On the other hand, DWT offers multiresolution representation, enabling better localization and robustness against attacks. DWT-based methods use wavelet coefficients to embed watermarks in specific frequency bands or image regions. The review examines the strengths and weaknesses of DCT and DWT in digital image watermarking, exploring algorithms and approaches proposed in the literature. It also addresses challenges like attacks, synchronization, and robustness to image processing. Additionally, a comparative analysis of DCT and DWT-based methods considers imperceptibility, robustness, capacity, and computational complexity. By offering valuable insights, this review aids researchers and practitioners in implementing secure and efficient digital image watermarking solutions.

Multiresolution Representation of Geoelectric Model and Parameter Operation in Wavelet Domain

In this paper, an algorithm for wavelet domain geoelectric model representation and parameter manipulation is introduced, which can be utilized for multiresolution inversions of geophysical data based on tree structure. With proper wavelet choice, the model can be represented by much less parameters than traditional model representation, expecting to reduce the computational cost for inversion remarkably. The use of tree structure to represent the wavelet coefficients makes the parameter operations (e.g., perturbating model parameter and adding more details in a model) in inversion much simpler. Introducing detail in a model or removing detail from a model can be simply realized by adding nodes to or deleting nodes from the existing tree structure. The model parameter value can be perturbated by changing the tree node values. Our algorithm is general and can be applied to develop any geophysical inversions in wavelet domain with trivial additional work once the forward modeling and inversion algorithm is available.

Automatic Mesh and Shader Level of Detail.

The level of detail (LOD) technique has been widely exploited as a key rendering optimization in many graphics applications. Numerous approaches have been proposed to automatically generate different kinds of LODs, such as geometric LOD or shader LOD. However, none of them have considered simplifying the geometry and shader at the same time. In this paper, we explore the observation that simplifications of geometric and shading details can be combined to provide a greater variety of tradeoffs between performance and quality. We present a new discrete multiresolution representation of objects, which consists of mesh and shader LODs. Each level of the representation could contain both simplified representations of shader and mesh. To create such LODs, we propose two automatic algorithms that pursue the best simplifications of meshes and shaders at adaptively selected distances. The results show that our mesh and shader LOD achieves better performance-quality tradeoffs than prior LOD representations, such as those that only consider simplified meshes or shaders.

New Proposed Mixed Transforms: CAW and FAW and Their Application in Medical Image Classification

The transformation model plays a vital role in medical image processing. This paper proposed new two Mixed Transforms models that are the hybrid combination of linear and nonlinear Transformations techniques. The first mixed transform is computed in three steps: calculate 2D discrete cosine transform (DCT) of the image, and applying Arnold Transform (AT) on the DCT coefficients, and applying the discrete Wavelet Transform (DWT) on the result to get which was abbreviated as (CAW). The second mixed transform consists of firstly computing the discrete Fourier transform (DFT), net applying the Arnold Transform (AT), and finally, the computation of discrete Wavelet Transform (DWT) which was abbreviated as (FAW). These transforms have superior directional representations as compared to other multiresolution representations such as DWT or DCT and work as non-adaptive mixed transformations for multi-scale object analysis. Due to their relationship to the wavelet idea, they are finding increasing use in areas like image processing and scientific computing. These transforms are tested in medical image classification task and their performances are compared with that of the traditional transforms. CAW and FAW transforms are used in the feature extraction stage of a classification VGG16 deep learning (DNN) task of Tumor MRI medical image. The numerical findings favor CAW and FAW over the wavelet transform for estimating and classifying pictures. From the results obtained it was shown that the CAW and FAW transform gave e much higher classification rate than that achieved with the traditional transforms, namely DCT, DFT and DWT. Furthermore, this combination leads to a family of directional and multi-transformation bases for image processing.

Multi-Temporal Hyperspectral Classification of Grassland Using Transformer Network.

In recent years, grassland monitoring has shifted from traditional field surveys to remote-sensing-based methods, but the desired level of accuracy has not yet been obtained. Multi-temporal hyperspectral data contain valuable information about species and growth season differences, making it a promising tool for grassland classification. Transformer networks can directly extract long-sequence features, which is superior to other commonly used analysis methods. This study aims to explore the transformer network's potential in the field of multi-temporal hyperspectral data by fine-tuning it and introducing it into high-powered grassland detection tasks. Subsequently, the multi-temporal hyperspectral classification of grassland samples using the transformer network (MHCgT) is proposed. To begin, a total of 16,800 multi-temporal hyperspectral data were collected from grassland samples at different growth stages over several years using a hyperspectral imager in the wavelength range of 400-1000 nm. Second, the MHCgT network was established, with a hierarchical architecture, which generates a multi-resolution representation that is beneficial for grass hyperspectral time series' classification. The MHCgT employs a multi-head self-attention mechanism to extract features, avoiding information loss. Finally, an ablation study of MHCgT and comparative experiments with state-of-the-art methods were conducted. The results showed that the proposed framework achieved a high accuracy rate of 98.51% in identifying grassland multi-temporal hyperspectral which outperformed CNN, LSTM-RNN, SVM, RF, and DT by 6.42-26.23%. Moreover, the average classification accuracy of each species was above 95%, and the August mature period was easier to identify than the June growth stage. Overall, the proposed MHCgT framework shows great potential for precisely identifying multi-temporal hyperspectral species and has significant applications in sustainable grassland management and species diversity assessment.

Generalizing Downsampling from Regular Data to Graphs

Downsampling produces coarsened, multi-resolution representations of data and it is used, for example, to produce lossy compression and visualization of large images, reduce computational costs, and boost deep neural representation learning. Unfortunately, due to their lack of a regular structure, there is still no consensus on how downsampling should apply to graphs and linked data. Indeed reductions in graph data are still needed for the goals described above, but reduction mechanisms do not have the same focus on preserving topological structures and properties, while allowing for resolution-tuning, as is the case in regular data downsampling. In this paper, we take a step in this direction, introducing a unifying interpretation of downsampling in regular and graph data. In particular, we define a graph coarsening mechanism which is a graph-structured counterpart of controllable equispaced coarsening mechanisms in regular data. We prove theoretical guarantees for distortion bounds on path lengths, as well as the ability to preserve key topological properties in the coarsened graphs. We leverage these concepts to define a graph pooling mechanism that we empirically assess in graph classification tasks, providing a greedy algorithm that allows efficient parallel implementation on GPUs, and showing that it compares favorably against pooling methods in literature.

MathNet: Haar-like wavelet multiresolution analysis for graph representation learning

Graph neural networks (GNNs) have recently attracted great attention and achieved significant progress in graph-level applications. In this paper, we propose a framework for graph neural networks with multiresolution Haar-like wavelets, or MathNet, with interrelated convolution and pooling strategies. The rendering method takes graphs in different structures as input and assembles consistent graph representations for readout layers, which then accomplishes label prediction. To achieve this, multiresolution graph representations are first constructed and fed into graph convolutional layers for processing. The hierarchical graph pooling layers are then involved to downsample graph resolution while simultaneously removing redundancy of graph signals. The whole workflow could be formed with a multilevel graph analysis, which not only helps embed the intrinsic topological information of each graph into the GNN, but also supports fast computation of forward and adjoint graph transforms. Extensive experiments present notable accuracy gains of the proposed MathNet on graph classification and regression tasks with performance stability. MathNet outperforms various existing GNN models, especially on big datasets.

MRET: Multi-resolution transformer for video quality assessment

No-reference video quality assessment (NR-VQA) for user generated content (UGC) is crucial for understanding and improving visual experience. Unlike video recognition tasks, VQA tasks are sensitive to changes in input resolution. Since large amounts of UGC videos nowadays are 720p or above, the fixed and relatively small input used in conventional NR-VQA methods results in missing high-frequency details for many videos. In this paper, we propose a novel Transformer-based NR-VQA framework that preserves the high-resolution quality information. With the multi-resolution input representation and a novel multi-resolution patch sampling mechanism, our method enables a comprehensive view of both the global video composition and local high-resolution details. The proposed approach can effectively aggregate quality information across different granularities in spatial and temporal dimensions, making the model robust to input resolution variations. Our method achieves state-of-the-art performance on large-scale UGC VQA datasets LSVQ and LSVQ-1080p, and on KoNViD-1k and LIVE-VQC without fine-tuning.

End-to-end deep speaker embedding learning using multi-scale attentional fusion and graph neural networks

As an attractive research in biometric authentication, Text Independent Speaker Verification (TI-SV) problem aims to specify whether two given unconstrained utterances come from the same speaker or not. As state-of-the-art solutions, end-to-end approaches using deep neural networks seek to learn a highly discriminative speaker embedding space.In this paper, we propose a novel end-to-end approach for speaker embedding learning by focusing on two crucial factors: speaker embedder architecture and objective function. The proposed module in the speaker embedder is composed of an Efficient Multi-resolution feature Representation (EMR) block followed by a Multi-scale Channel Attention Fusion (MCAF) block. The EMR effectively addresses the issue of fixed resolution convolutional kernels which commonly used in most embedder architectures. Moreover, the MCAF significantly improves the simple summation-based feature fusion used in residual embedder networks. Regarding the objective function, we conduct the speaker embedding space towards learning the embedding-to-embedding relations, in addition to only embedding-to-training class relations employed by most previous methods. So, we propose to employ a dynamic graph attention network, on top of the proposed embedder to learn all informative relations between embeddings, and then learn both embedder and graph-based networks in an end-to-end manner.We conduct various experiments on a large-scale benchmark dataset called VoxCeleb1&2. The effectiveness of all proposed components is verified through an ablation study. We show the superior or competitive performances of the proposed approach compared to seven well-known embedding architectures and 32 SV systems, regarding two evaluation metrics, EER and minDCF, as well as the number of embedder parameters.

A fast grid generation algorithm for local irregular parts of hexagonal discrete global grid systems

ABSTRACT Discrete Global Grid Systems (DGGS) provide a multi-resolution discrete representation of the Earth and are preferable for the organization, integration, and analysis of large and multi-source geospatial datasets. Generating grids for the area of interest is usually the premise and basis for DGGS applications. Owing to incongruent hierarchies that restrict the multi-resolution applications of hexagonal DGGS, current grid generation of hexagonal DGGS for local areas mainly depends on inefficient single-resolution traversal methods by judging the spatial relationship between each cell and the area. This study designs a fast generation algorithm for local parts of hexagonal DGGS based on the hierarchical properties of DGGS. A partition structure at intervals of multiple levels is first designed to ensure the coverage relevance between parent and children cells of different levels. Based on this structure, the algorithm begins with coarser resolution grids and recursively decomposes them into the target resolution, with multiple decomposition patterns used and a unique condition proposed to make the generated grids without gaps or overlaps. Efficient integer coordinate operations are used to generate the vast majority of cells. Experimental results show that the proposed algorithm achieves a significant improvement in efficiency. In the aperture 4 hexagonal DGGS, the efficiency ratio of the proposed and traversal algorithms increases from six times in level 14 to approximately 339 times in level 18. This study provides a solid foundation for subsequent data quantization and multi-resolution applications in hexagonal DGGS and has broad prospects.

BalanceHRNet: An effective network for bottom-up human pose estimation

In the study of human pose estimation, which is widely used in safety and sports scenes, the performance of deep learning methods is greatly reduced in high overlap rate and crowded scenes. Therefore, we propose a bottom-up model, called BalanceHRNet, which is based on balanced high-resolution module and a new branch attention module. BalanceHRNet draws on the multi-branch structure and fusion method of a popular model HigherHRNet. And our model overcomes the shortcoming of HigherHRNet that cannot obtain a large enough receptive field. Specifically, through the connecting structure in balanced high-resolution module, we can connect almost all convolutional layers and obtain a sufficiently large receptive field. At the same time, the multi-resolution representation can be maintained due to the use of balanced high-resolution module, which enable our model to recognize objects with richer scales and obtain more complex semantics information. And for branch fusion method, we design branch attention to obtain the importance of different branches at different stages. Finally, our model improves the accuracy while ensuring a smaller amount of computation than HigherHRNet. The CrowdPose dataset is used as test dataset, and HigherHRNet, AlphaPose, OpenPose and so on are taken as comparison models. The AP measured by BalanceHRNet is 63.0%, increased by 3.1% compared to best model — HigherHRNet. We also demonstrate the effectiveness of our network through the COCO(2017) keypoint detection dataset. Compared with HigherHRNet-w32, the AP of our model is improved by 1.6%.