Loss Of Information Research Articles

Single-Document Abstractive Text Summarization: A Systematic Literature Review

Abstractive text summarization is a task in natural language processing that automatically generates the summary from the source document in a human-written form with minimal loss of information. Research in text summarization has shifted towards abstractive text summarization due to its challenging aspects. This study provides a broad systematic literature review of abstractive text summarization on single-document summarization to gain insights into the challenges, widely used datasets, evaluation metrics, approaches, and methods. This study reviews research articles published between 2011 and 2023 from popular electronic databases. In total, 226 journal and conference publications were included in this review. The in-depth analysis of these papers helps researchers understand the challenges, widely used datasets, evaluation metrics, approaches, and methods. This paper identifies and discusses potential opportunities and directions, along with a generic conceptual framework and guidelines on abstractive summarization models and techniques for research in abstractive text summarization.

An Information Gradient Approach to Optimizing Traffic Sensor Placement in Statewide Networks

Traffic sensors are vital to the development and operation of Intelligent Transportation Systems, providing essential data for traffic monitoring, management, and transportation infrastructure planning. However, optimizing the placement of these sensors, particularly across large and complex statewide highway networks, remains a challenging task. In this research, we presented a novel search algorithm designed to address this challenge by leveraging information gradients from K-nearest neighbors within an embedding space. Our method enabled more informed and strategic sensor placement under budget and resource constraints, enhancing overall network coverage and data quality. Additionally, we incorporated spatial kriging analysis, harnessing spatial correlations of existing sensors to refine and reduce the search space. Our proposed approach was tested against the widely used Genetic Algorithm, demonstrating superior efficiency in terms of convergence time and producing more effective solutions with reduced information loss.

UCC: A unified cascade compression framework for vision transformer models

In recent years, Vision Transformer (ViT) and its variants have dominated many computer vision tasks. However, the high computational consumption and training data requirements of ViT make it challenging to be deployed directly on resource-constrained devices and environments. Model compression is an effective approach to accelerate deep learning networks, but existing methods for compressing ViT models are limited in their scopes and struggle to strike a balance between performance and computational cost. In this paper, we propose a novel Unified Cascaded Compression Framework (UCC) to compress ViT in a more precise and efficient manner. Specifically, we first analyze the frequency information within tokens and prune them based on a joint score of their both spatial and spectral characteristics. Subsequently, we propose a similarity-based token aggregation scheme that combines the abundant contextual information contained in all pruned tokens with the host tokens according to their weights. Additionally, we introduce a novel cumulative cascaded pruning strategy that performs bottom-up cascaded pruning of tokens based on cumulative scores, avoiding information loss caused by individual idiosyncrasies of blocks. Finally, we design a novel two-level distillation strategy, incorporating imitation and exploration, to ensure the diversity of knowledge and better performance recovery. Extensive experiments demonstrate that our unified cascaded compression framework outperforms most existing state-of-the-art approaches, compresses the floating-point operations of ViT-Base as well as DeiT-Base models by 22 % and 54.1 %, and improves the recognition accuracy of the models by 3.74 % and 1.89 %, respectively, significantly reducing model computational consumption while enhancing performance, which enables efficient end-to-end training of compact ViT models.

Network Coding Schemes and Cooperative Communication Systems Amplify and Forward in Outdoor Environments

Fading is a phenomenon of decreasing signal strength during transmission, caused by changes in environmental conditions or physical obstacles in the propagation path. Fading can cause interference in signal reception, such as distortion or loss of information, especially in Non-Line of Sight (NLoS) paths, where signals must pass through many paths or obstacles such as buildings, trees, or other objects. To eliminate fading interference, a cooperative communication system is created to reduce the effects of fading in wireless networks. The cooperative communication system works by utilizing other nodes in sending signals to the destination. The relay protocol used in this study is amplify and forward, where the signal to be sent will be amplified and forwarded to the destination by the relay node. A network coding scheme is added to the cooperative communication system to improve network performance. This scheme encodes each piece of information with the XOR technique. The parameters used in measuring the network coding scheme in the cooperative communication system are BER and throughput. Viewed from the BER value, the conventional cooperative communication system has better performance compared to the cooperative communication system with network coding. Cooperative communication system scheme without NC BER value is 0 when the transmit power value is -19 dBm or Gain 32. While for the cooperative communication system scheme with network coding BER value is 0 when transmit power of -9.25 dBm. Throughput system with network coding has increased by 17% compared to the conventional system.

Local inconsistency detection using the Kullback–Leibler divergence measure

BackgroundThe standard approach to local inconsistency assessment typically relies on testing the conflict between the direct and indirect evidence in selected treatment comparisons. However, statistical tests for inconsistency have low power and are subject to misinterpreting a p-value above the significance threshold as evidence of consistency.MethodsWe propose a simple framework to interpret local inconsistency based on the average Kullback–Leibler divergence (KLD) from approximating the direct with the corresponding indirect estimate and vice versa. Our framework uses directly the mean and standard error (or posterior mean and standard deviation) of the direct and indirect estimates obtained from a local inconsistency method to calculate the average KLD measure for selected comparisons. The average KLD values are compared with a semi-objective threshold to judge the inconsistency as acceptably low or material. We exemplify our novel interpretation approach using three networks with multiple treatments and multi-arm studies.ResultsAlmost all selected comparisons in the networks were not associated with statistically significant inconsistency at a significance level of 5%. The proposed interpretation framework indicated 14%, 66%, and 75% of the selected comparisons with an acceptably low inconsistency in the corresponding networks. Overall, information loss was more notable when approximating the posterior density of the indirect estimates with that of the direct estimates, attributed to indirect estimates being more imprecise.ConclusionsUsing the concept of information loss between two distributions alongside a semi-objectively defined threshold helped distinguish target comparisons with acceptably low inconsistency from those with material inconsistency when statistical tests for inconsistency were inconclusive.

DMCCT: Dual-Branch Multi-Granularity Convolutional Cross-Substitution Transformer for Hyperspectral Image Classification

In the field of hyperspectral image classification, deep learning technology, especially convolutional neural networks, has achieved remarkable progress. However, convolutional neural network models encounter challenges in hyperspectral image classification due to limitations in their receptive fields. Conversely, the global modeling capability of Transformers has garnered attention in hyperspectral image classification. Nevertheless, the high computational cost and inadequate local feature extraction hinder its widespread application. In this study, we propose a novel fusion model of convolutional neural networks and Transformers to enhance performance in hyperspectral image classification, namely the dual-branch multi-granularity convolutional cross-substitution Transformer (DMCCT). The proposed model adopts a dual-branch structure to separately extract spatial and spectral features, thereby mitigating mutual interference and information loss between spectral and spatial data during feature extraction. Moreover, a multi-granularity embedding module is introduced to facilitate multi-scale and multi-level local feature extraction for spatial and spectral information. In particular, the improved convolutional cross-substitution Transformer module effectively integrates convolution and Transformer, reducing the complexity of attention operations and enhancing the accuracy of hyperspectral image classification tasks. Subsequently, the proposed method is evaluated against existing approaches using three classical datasets, namely Pavia University, Kennedy Space Center, and Indian Pines. Experimental results demonstrate the efficacy of the proposed method, achieving significant classification results on these datasets with overall classification accuracies of 98.57%, 97.96%, and 96.59%, respectively. These results establish the superiority of the proposed method in the context of hyperspectral image classification under similar experimental conditions.

FV-DLV3+: a light-weight flooded vegetation extraction method with attention-based DeepLabv3+

ABSTRACT Climate change and human activities have led to a decline in wetland vegetation and sharp reduction of wetland area, which has caused considerable damage to the hydrological cycle and ecological functions of many wetlands. Flooded vegetation (FV) is an important wetland landscape, and strengthening the research on methods of the extracting FV using remote sensing or aerial photos plays an important role in monitoring changes to wetlands. This paper used optical satellite and synthetic aperture radar satellite images as data sources, construct a set of feature variable sets suitable for FV extraction; in addition, a deep learning method based on the semantic segmentation model DeepLabv3+, FV-DLV3+, is proposed for FV extraction. The method adopts the lightweight MobileNetV2 model as the backbone feature extraction network, introduces the SegFormer network to make up for the feature information loss caused by MobileNetV2, and further captures more effective features and suppresses the interference of the background through an attention mechanism to enhance the ability to extract feature boundaries, thereby improving the accuracy of FV extraction. Taking the wetland of Poyang Lake as the experimental area, Landsat 8 and Sentinel-1A satellite data were selected as the data sources to produce a deep learning dataset, and U-Net, DeepLabv3+, Swin-Unet, Swin Transformer, and CVTNet were selected for comparison when conducting model training and validating experiments. The results show that FV-DLV3+ has an excellent effect on FV extraction of the Poyang Lake wetland, which effectively reduces the computational complexity of the process while improving extraction accuracy; the mean intersection over union of this paper’s method was higher than the five above-mentioned methods, which was 12.93% and 5.85% higher than that of DeepLabv3+ and CVTNet, respectively.

Knowledge abstraction and filtering based federated learning over heterogeneous data views in healthcare

Robust data privacy regulations hinder the exchange of healthcare data among institutions, crucial for global insights and developing generalised clinical models. Federated learning (FL) is ideal for training global models using datasets from different institutions without compromising privacy. However, disparities in electronic healthcare records (EHRs) lead to inconsistencies in ML-ready data views, making FL challenging without extensive preprocessing and information loss. These differences arise from variations in services, care standards, and record-keeping practices. This paper addresses data view heterogeneity by introducing a knowledge abstraction and filtering-based FL framework that allows FL over heterogeneous data views without manual alignment or information loss. The knowledge abstraction and filtering mechanism maps raw input representations to a unified, semantically rich shared space for effective global model training. Experiments on three healthcare datasets demonstrate the framework’s effectiveness in overcoming data view heterogeneity and facilitating information sharing in a federated setup.

Materializing Digital Archaeology: Managing 3D Models

The archaeological record is fragmented yet the artifacts and data that comprise it are used to make vast claims about human nature and behavior, and about past and contemporary cultures and societies (Huggett, 2012; Huvila, 2019). The digital archaeological record is further fragmented by the rarity of digitization (due to resource and time constraints) and the loss of information through the digitization process Huvila & Huggett, 2018; Schroer & Mudge, 2017). Due to the lack of connection between the physical artifacts and their digital instantiations (as 3D models, metadata records, scanned images, etc.) it is difficult to understand what contextual information has been lost and/or altered from the physical data. This results in knowledge and information claims based on limited and biased data which can lead to an exacerbation of the silences and gaps in the archaeological record (Huggett, 2020) which 1) reduces the accuracy of archaeological research and 2) can differentially harm marginalized communities because, like archival records, the archaeological record can be used to assert “master” or “grand” narratives (Bastian, 2009; Carbajal & Caswell, 2021). This Work-in-Progress poster is focused on information loss during the 3D modeling of archaeological artifacts. I posit that instead of understanding 3D models of artifacts as singular sources of information, they should be treated as one component of a complex resource. However, it has been difficult to find my way through the required reorientation of concepts like materiality from the digital humanities (Drucker, 2013; Kirschenbaum, 2012) and to implement this reorientation in practice.

An Integrated Spatial Fuzzy‐Based Site Suitability Assessment Framework for Agricultural BMP Placement

ABSTRACTAssigning crisp class boundaries to landscape features can result in the loss of vital information for land evaluation objectives, especially when these boundaries lack clear definitions. This challenge becomes particularly pronounced when land suitability is assessed for implementing agricultural best management practices (BMPs)—conservation measures aimed at reducing the environmental risks of farming activities to aquatic ecosystems while simultaneously achieving water quality and economic objectives. To address the limitations associated with Boolean suitability assessment frameworks, we have introduced an integrated spatial, fuzzy‐based land evaluation framework that considers a range of hydrological and economic determinants for BMP placement. By employing data‐driven fuzzy membership functions and overlay operators, this framework generates a joint suitability index for BMP placement across agricultural watersheds. The application of the proposed framework to the Thames River Watershed in southwestern Ontario, Canada, produced the first joint suitability index of the watershed. Further analysis of the average farm‐level joint suitability scores identified statistically significant clusters of highly suitable and unsuitable lands for BMP placement, with 85% of highly suitable lands being situated in the upper basin areas. The proposed framework is adaptable to various agricultural production geographies, especially in data‐limited environments, allowing for strategic BMP placement to mitigate the global impacts of anthropogenic nutrient loadings on aquatic ecosystems. For optimal results, context‐specific applications should prioritize research on locally relevant fuzzy membership functions and BMP implementation drivers.

Erratum: Surface gravity and the information loss problem [Phys. Rev. D 105 , 124032 (2022)

Erratum: Surface gravity and the information loss problem [Phys. Rev. D <b>105</b> , 124032 (2022)

A Review of Hypergraph Neural Networks

In recent years, Graph Neural Networks (GNNs) have seen notable success in fields such as recommendation systems and natural language processing, largely due to the availability of vast amounts of data and powerful computational resources. GNNs are primarily designed to work with graph data that involve pairwise relationships. However, in many real-world networks, the relationships between entities are complex and go beyond simple pairwise connections, as seen in scientific collaboration networks, protein networks, and similar domains. If these complex relationships are directly represented as pairwise relationships using graph structures, it can lead to information loss. A hypergraph, as a special kind of graph-structured data, can represent higher-order relationships that cannot be fully captured by graphs, thereby addressing the limitations of graphs. In light of this, researchers have begun to focus on how to design neural networks on hypergraphs, leading to the proposal of hypergraph neural network (HGNN) models for downstream tasks. Therefore, this paper reviews the existing hypergraph neural network models. The review is conducted from two perspectives: spectral analysis methods and neural network methods on hypergraphs, discussing both unfolded and non-unfolded methods, and further subdividing them based on their algorithm characteristics and application scenarios. Subsequently, the design concepts of various algorithms are analyzed and compared, and the advantages and disadvantages of each type of algorithm are summarized based on experimental results. Finally, potential future research directions in hypergraph learning are discussed.

On the Modelling and Prediction of High-Dimensional Functional Time Series

We propose a two-step procedure to model and predict high-dimensional functional time series, where the number of function-valued time series p is large in relation to the length of time series n. Our first step performs an eigenanalysis of a positive definite matrix, which leads to a one-to-one linear transformation for the original high-dimensional functional time series, and the transformed curve series can be segmented into several groups such that any two subseries from any two different groups are uncorrelated both contemporaneously and serially. Consequently in our second step those groups are handled separately without the information loss on the overall linear dynamic structure. The second step is devoted to establishing a finite-dimensional dynamical structure for all the transformed functional time series within each group. Furthermore the finite-dimensional structure is represented by that of a vector time series. Modelling and forecasting for the original high-dimensional functional time series are realized via those for the vector time series in all the groups. We investigate the theoretical properties of our proposed methods, and illustrate the finite-sample performance through both extensive simulation and two real datasets.

Attention-Linear Trajectory Prediction

Recently, a large number of Transformer-based solutions have emerged for the trajectory prediction task, but there are shortcomings in the effectiveness of Transformers in trajectory prediction. Specifically, while position encoding preserves some of the ordering information, the self-attention mechanism at the core of the Transformer has its alignment invariance that leads to the loss of temporal information, which is crucial for trajectory prediction. For this reason, we design a simple and efficient strategy for temporal information extraction and prediction of trajectory sequences using the self-attention mechanism and linear layers. The experimental results show that the strategy can improve the average accuracy by 15.31%, effectively combining the advantages of the linear layer and the self-attention mechanism, while compensating for the shortcomings of the Transformer. Additionally, we conducted an empirical study to explore the effectiveness of the linear layer and sparse self-attention mechanisms in trajectory prediction.

Time-lapse difference seismic inversion based on scattering theory

Time-lapse seismic monitoring technology is an effective tool for quantitatively estimating subsurface medium variations and plays a crucial role in resource and engineering explorations. Among existing time-lapse inversion methods, the difference inversion technique offers higher accuracy but requires linear inversion operators. During linearization, the background is often approximated as a homogeneous medium, resulting in the loss of effective information. Additionally, merely using reflection wave information is not conducive to improving inversion accuracy. However, the true structure of the subsurface is often complex and inhomogeneous, which can easily generate complex wavefields, such as scattering waves, thereby hindering the resolution of differences between time-lapse models. Therefore, incorporating more comprehensive wavefield information is essential for enhancing the accuracy of time-lapse seismic inversion. To address this issue, a wave-equation-based time-lapse difference inversion method is proposed to retrieve the production-induced property perturbation. In the proposed method, the forward operator is formulated through the scattering wave equation, the operator is linearized through the Born approximation, and the WKBJ method is used to approximate Green's function with high accuracy for the inhomogeneous background. In this study, time-lapse seismic parallel inversion and difference inversion are conducted using the proposed forward operator. Both numerical experiments and application of field seismic data indicate that the proposed time-lapse difference inversion method can realize a more accurate quantitative characterization of time-lapse variations.

AMVFNet: Attentive Multi-View Fusion Network for 3D Object Detection

Pillar-based method is significant in the field of LiDAR-based 3D object detection which could directly make use of efficient 2D backbone and save computational resources during reference. Existing methods usually sequentially project the original point clouds into the cylindrical view or the bird-eye view for feature extraction. However, the former suffers from obscured problems and the scales of instances vary greatly with distance, while the latter leads to considerable confusion problems due to the loss of semantic information caused by the sparsity of the projected point cloud. In this paper, we present a novel and efficient two-stage point-pillar hybrid architecture named Attentive Multi-View Fusion Network (AMVFNet), in which we abstract features from all cylindrical view, bird-eye view, and raw point clouds. Rather than designing more complex modules to solve the problems inherent in the single-view approach, our multi-view fusion architecture effectively combines the strengths of multiple perspectives to improve performance at a more fundamental level. Besides, to compensate for quantization distortion caused by projection operations, we propose attentive feature enhancement layers to further improve the capability of contextual information capturing. Extensive experiments on the KITTI detection benchmark illustrate that our proposed AMVFNet achieves competitive performance compared with other SOTA 3D object detectors.

A method for extracting Martian terrain features based on unsupervised contrastive learning

Intelligent recognition of Martian surface terrain is of great significance to autonomous exploration of Mars rovers. Feature extraction methods for Martian terrain images are currently mainly divided into two categories: traditional shallow visual feature extraction and deep feature extraction based on supervised learning. Loss of image information and the need for a large amount of labeled data are key issues to be addressed. A method for Martian terrain feature recognition based on unsupervised contrastive learning is proposed. By establishing an image dictionary dataset, two sets of neural networks, "query" and "encoding", are used to compare a single image with other images in the "dictionary" dataset, and the network is trained using the similarity functional as the loss function, thereby realizing feature recognition of Martian terrain images. The proposed method also has the ability to recognize new types of terrain images outside the training dataset, and has outstanding superiority in subsequent recognition and classification. Computer simulation verification shows that the recognition accuracy of the proposed method is and the 85.4%recognition accuracy of new types of terrain images is 84.5%.

Dehazing network based on residual context attention and cross-layer fusion

Current image dehazing algorithms based on deep learning usually use traditional convolutional layers when extracting features, which easily causes the loss of image details and edge information, ignores the position information of the image when extracting features, and ignores the original information of the image when fusing features, and cannot restore high-quality haze-free images with complete structure and clarity. To address this problem, a dehazing algorithm based on residual context attention and cross-layer feature fusion is proposed. Firstly, the residual group structure is obtained by serializing the proposed residual context blocks, and the features of the first two layers of the network, i.e., the shallow layers, are extracted to obtain rich context information of the shallow layers; secondly, coordinate attention is introduced to establish an attention map with position information, and it is applied to the residual context feature extraction and placed in the third layer of the network, i.e., the deep layer, to extract deeper semantic information; then, in the middle layer of the network, feature information from different resolution streams is fused across layers to enhance the information exchange between the shallow and deep layers to achieve the purpose of feature enhancement; finally, the features with rich semantic information obtained by the network are aggregated with the original input, thereby improving the restoration effect. Experimental results on the RESIDE dataset and the Haze4K dataset show that the proposed algorithm achieves good results in both visual effects and objective indicators.

EMA fault diagnosis method based on multi-information fusion of GRU and improved attention mechanism

Aiming at the problems of time series feature loss and fault information loss in traditional electromechanical servo system (EMA) fault diagnosis methods based on machine learning and deep learning, a multi-information fusion EMA fault diagnosis method based on gated recurrent unit (GRU) and improved attention mechanism is proposed. Firstly, the collected different sensor signals are divided into different channels, and the time series features of each channel signal are extracted by GRU. Then, the self-attention mechanism is introduced to further distinguish the important relationship between different time points of the signal. The multi-channel attention mechanism is further introduced to adaptively fuse the features of different channels, and finally the fault diagnosis is realized through the classifier. The experimental results based on the test bench data set show that: compared with the single sensor model, the diagnosis rate is improved 10%; compared with the model without the introduction of the attention mechanism, the diagnosis rate is improved 5.2%; compared with the classic machine learning, deep learning and the improved algorithm based on deep learning in the past two years, the diagnosis rate of the diagnosis model designed in this paper is 98.5%above, and the diagnosis effect is the best.

Should the multi-layer transportation network structure be reduced?

ABSTRACT The increasing diversity of transportation modes and the rapid expansion of transportation networks present significant challenges for modeling multi-layer comprehensive transportation networks. It is crucial to determine whether aggregating certain layers is a viable option for balancing complexity reduction and information preservation. This decision defines the layered structures and informs subsequent analyses of these networks. Two-dimensional factors, namely topological structures and transportation attributes, are considered to enhance understanding of the similarities among network layers. The relative entropy and the Gini index are employed as metrics to assess information gain or loss resulting from layer aggregation or segregation, guiding decisions on network reduction. Furthermore, an integrated similarity measure, based on the quantum Jensen-Shannon divergence and the Gower distance, is utilized to identify the optimal aggregation sequences. Two real-world transportation networks serve as case studies. Results demonstrate that these transportation networks are more effectively maintained with layer-separated structures, preserving maximum information.