Perspective Of Information Theory Research Articles

Design of Anti-Eavesdropping Scheme for SLT-LT Codes Based on Random Symbol Sets

In recent years, physical layer security technology has provided a new way to solve the problem of secure transmissions in wireless communications from an information theory perspective by using physical layer security coding and the inherent characteristics of the channel. As a nonsystematic code, Luby transform (LT) code has the characteristics of random coding and rateless, therefore, eavesdroppers cannot directly obtain useful information from leaked code symbols. As a shifted LT (SLT) code, the SLT codes can efficiently recover information, with a smaller decoding overhead. Therefore, we propose an SLT-LT joint code anti-eavesdropping scheme based on random symbol sets. Based on the environment of the main channel, the sender selects a set of random symbols to send to legitimate receivers as known symbols and uses these random symbols and message symbols to form source symbols for the SLT-LT concatenated coding to increase the bit error rate (BER) of the eavesdroppers. The experimental results show that, compared with other anti-eavesdropping LT schemes, our proposed scheme only adds a small amount of decoding overhead but has a better security performance.

Ensemble Entropy Metric for Hyperspectral Anomaly Detection

In hyperspectral anomaly detection, anomalies are rare targets that exhibit distinct spectral signatures from the background. Thus, anomalies are with low probabilities of occurrence in hyperspectral images. In this article, we develop a new technique for hyperspectral anomaly detection that adopts a new information theory perspective, to fully utilize the aforementioned concepts. Our goal is to transform system entropy into quantitative metrics of anomaly conspicuousness of pixels. To do so, two tasks are first completed: first, the construction of occurrence probability of pixels based on the density peak clustering algorithm, and second, the valid system definitions for pixels in specific anomaly detection problems with multiviews. Specifically, three types of systems are separately established by pixel pairs to conform to the definitions of three entropy definitions in information theory, i.e., Shannon entropy, joint entropy, and relative entropy. Then, three individual entropy-based metrics that assess the anomaly conspicuousness are defined. In addition, we design a standard deviation-based ensemble strategy for the integrated representation of the three individual metrics, which considers both logic “OR” and “AND” operations to simultaneously improve the detection rate and reduce the false alarm rate. Our experimental results obtained on two publicly available datasets with anomalies of different sizes and shapes demonstrate the superiority of our newly proposed anomaly detection method.

A Low-Complexity and Adaptive Distributed Source Coding Design for Model Aggregation in Distributed Learning

A major bottleneck in distributed learning is the communication overhead of exchanging intermediate model update parameters between the worker nodes and the parameter server. Recently, it is found that local gradients among different worker nodes are correlated. Therefore, distributed source coding (DSC) can be applied to increase communication efficiency by exploiting such correlation. However, it is highly non-trivial to exploite the gradient correlations in distributed learning due to the unknown and time-varying gradient correlation. In this paper, we first propose a DSC framework, named successive Wyner-Ziv coding, for distributed learning based on quantization and Slepian-Wolf (SW) coding. We prove that the proposed framework can achieve the theoretically minimum communication cost from an information theory perspective. We also propose a low-complexity and adaptive DSC for distributed learning, including a gradient statistics estimator, rate controller, and a log-likelihood ratio (LLR) computer. The gradient statistics estimator estimates the gradient statistics online based only on the quantized gradients at previous iterations, hence it does not introduce extra communication cost. The computation complexity of the rate controller and the LLR computer is reduced to a linear growth in the number of worker nodes by introducing a semi-analytical Monte Carlo simulation. Finally, we design a DSC-based distributed learning process and find that the extra delay introduced by DSC does not scale with the number of worker nodes.

A novel multimodal fusion network based on a joint-coding model for lane line segmentation

There has recently been growing interest in utilizing multimodal sensors to achieve robust lane line segmentation. In this paper, we introduce a novel multimodal fusion architecture from an information theory perspective, and demonstrate its practical utility using Light Detection and Ranging (LiDAR) camera fusion networks. In particular, we develop a multimodal fusion network as a joint coding model, where each single node, layer, and pipeline is represented as a communication channel, and forward propagation equates to information transmission in the channel. This enables us to qualitatively and quantitatively analyze the effect of different fusion approaches. We argue that an optimal fusion architecture is related to essential capacity and allocation based on the source and channel models. To test this multimodal fusion hypothesis, we progressively formulate a series of multimodal models based on our proposed fusion methods and evaluate them on benchmark KITTI and the A2D2 datasets. Our optimal fusion network achieves 85%+ lane line accuracy and 98.7%+ overall accuracy. We conclude that the performance gap with state-of-the-art models demonstrates the potential of our fusion architecture to serve as a new benchmark resource for the multimodal deep learning community.

Information bounds in determining the 3D orientation of a single emitter or scatterer using point-detector-based division-of-amplitude polarimetry.

Determining the 3D orientation of a single molecule or particle, encoded in its polar and azimuthal angles, is of interest for a variety of fields, being relevant to a range of questions in elementary chemical reactivity, biomolecular motors, and nanorheology. A popular experimental method, known as division-of-amplitude polarimetry, for determining the real-time orientation of a single particle is to split the emitted/scattered light into multiple polarizations and to measure the light intensity using pointdetectors at these polarizations during a time interval Δt. Here, we derive the Cramér-Rao lower bounds for this method from the perspective of information theory in the cases of utilizing a chromophore or a scattering particle as a 3D orientation probe. Such Cramér-Rao lower bounds are new for using this experimental method to measure the full 3D orientation in both the scattering case and the fluorescence case. These results show that, for a scatterer, the information content of one photon is 1.16 deg-2 in the polar and 58.71 deg-2 in the azimuthal angles, respectively. For a chromophore, the information content of one photon is 2.54 deg-2 in the polar and 80.29 deg-2 in the azimuthal angles. In addition, the Cramér-Rao lower bound scales with the square root of the total signal photons. To determine orientation to an uncertainty of one degree requires 7.40 × 104 and 2.34 × 103 photons for the polar and the azimuthal angles, respectively, for fluorescence, whereas it takes 1.62 × 105 and 3.20 × 103 photons for scattering. This work provides experimentalists new guidelines by which future experiments can be designed and interpreted.

A Survey on Energy Harvesting Wireless Networks: Channel Capacity, Scheduling, and Transmission Power Optimization

This paper presents a survey on energy harvesting (EH) wireless communication networks focusing on channel capacity, transmission schemes, and power optimization. While many network researchers focus on energy management policies addressing the intermittency and randomness of the EH processes, but the channel capacity, and transmission power optimization have not been fully explored yet. In this paper, we provide a review and analysis of channel capacity, offline and online transmission schemes, and power optimization from an information theory perspective. By reviewing and analyzing wireless networking literature, we found that EH is a technologically feasible and economically viable paradigm for cost-effectiveness in the design and deployment of next-generation wireless networks. Finally, we identify open research problems and future research directions in the emerging field of EH wireless networks. We expect this study to stimulate more research endeavors to build energy-efficient scalable next-generation wireless network systems.

Online brand communication and building brand trust: social information processing theory perspective

PurposeThe purpose of this paper is to present the mechanism of online customer brand trust building through the lens of Walther’s Computer Mediated Communication (CMC) – Social Information Processing Theory (SIPT).Design/methodology/approachThis paper integrates a range of theoretical and empirical works across branding and marketing, including concept of brand trust, use of online social media in brand communication and customer brand relations and CMC-SIPT theory and relational marketing literature other relevant information which were found useful in the given context were reviewed. The aim was to collect a broad spectrum of ideas, based on their relevance to the research purpose to propose effective online brand trust building framework.FindingsThis paper proposes three different developmental stages in brand trust building on online social media networks. Each stage is guided by the components of CMC-SIPT. Stage I is interaction and compliance of the brand communication clues which suggest that the aggregated level of interaction and compliance in online brand communication on social media determine the movement of the customer to the next level. Stage II suggests information processing at three successive layers by the customer as identification, internalization brand information and bonding, which eventually prompt the customer to the next level of brand trust building. Stage III is related to trust building which is the critical stage as customer internal states of arousal are experienced, which reduces the emotional risk of decision-making and increases confidence and trust of the customers in brands.Research limitations/implicationsThe proposed framework of brand trust building has not been tested empirically. Future research could test and validate the proposed model tracking intrinsic changes in the customer in different brand trust development stages. This research is important for marketers or brands who wish to move beyond the notion of merely satisfying customers, to establishing more powerful emotional bonds between their brands and customers. It fills a gap in brand-trust literature and provides marketers and researchers a means to understand and draw strategies for consumers’ attraction toward brands (Patwardhan and Balasubramanian, 2011). The proposed framework has the capacity to revolutionize the way business and brands engage with society by enhancing and establishing trusting relationship.Originality/valueTo date, this research has not been done specifically from the SIPT perspective. This research is the first to examine brand communication strategies in an effort of building brand trust in the context of online social media network from the CMC-Social information theory perspective. It highlights the peculiarities of online brand communication on social media networks and customer information processing in presenting three stages of customer brand trust development to explain the development and flow of events.

P‐3.6: A training system for the rescue process of acute and severe diseases based on AR

China is one of the countries with the most serious natural disasters in the world. Hidden dangers and risks of all kinds of accidents are intertwined and superimposed, and the factors affecting public safety are increasing. It is urgent to strengthen the construction of emergency medical rescue capacity. During the nineteenth collective learning of the Central Political Bureau, general secretary Xi Jinping stressed that we should strengthen the capacity building of aviation emergency rescue and improve the airspace guarantee mechanism for emergency rescue. As the most important aerial medical evacuation platform, helicopter has the advantages of fast response, strong mobility, wide rescue range, direct arrival at the first line of treatment, etc. it has become an important means of modern rescue, and also the only way to open life treatment in case of land damage. From the international experience, the helicopter emergency medical system established by most developed countries plays an important role in disaster relief and anti‐terrorism. China has a large population, complex terrain and uneven regional development. In case of disasters, the casualties are often more severe and the demand for treatment is more urgent. It is urgent to establish an aviation medical emergency rescue system suitable for the national conditions and with helicopters as the main carrier, and accelerate the research and development of key technical equipment.In recent years, more and more people realize the necessity and benefits of the application of interactive technology for aviation medical simulation training in major events and disasters. Interaction refers to the transmission and exchange of intelligence, data, data and technical knowledge in all aspects of nature and society. From the perspective of information theory, local chronicles are a tangible text information carrier, which collects all kinds of information in a certain region. Interactive technology is to use certain means to achieve the purpose of interaction, and gradually enter the era of multi field application! In thefield of aviation medical rescue, it is in urgent need of penetration and integration.This paper mainly develops an AR emergency rescue process training system. Through the creation of data sets, action recognition based on computer vision and deep learning, the selection of recognition model and action capture, the action is recognized based on AR, and then 22 joint points of human hand and upper arm skeleton are taken to label the data space position. Through the above learning results, we can get the real‐time hand recognition based on depth information Upper arm skeleton output. Neural network is used to recognize human skeleton action. The skeleton information coordinates are imported into RNN neural network for deep learning. Finally, based on the complementarity of CNN and LSTM, combined with the idea of dual stream structure network, the action recognition is carried out.

An Information Theoretic Study of a Duffing Oscillator Array Reservoir Computer

Abstract Typically, nonlinearity is considered to be problematic and sometimes can lead to dire consequences. However, the nonlinearity in a Duffing oscillator array can enhance its ability to be used as a reservoir computer. Machine learning and artificial neural networks, inspired by the biological computing framework, have shown their immense potential, especially in the real-time temporal data processing. Here, the efficacy of a Duffing oscillator array is explored as a reservoir computer by using information theory. To do this, a reservoir computer model is studied numerically, which exploits the dynamics of the array. In this system, the complex dynamics stem from the Duffing term in each of the identical oscillators. The effects of various system parameters of the array on the information processing ability are discussed from the perspective of information theory. By varying these parameters, the information metric was found to be topologically mixed. Additionally, the importance of asynchrony in the oscillator array is also discussed in terms of the information metric. Since such nonlinear oscillators are used to model many different physical systems, this research provides insight into how physical nonlinear oscillatory systems can be used for dynamic computation, without significantly modifying or controlling the underlying dynamical system. To the authors' knowledge, this is the first use of Shannon's information rate for quantifying a reservoir computer of this kind, as well as the first comparison between synchronization phenomena and the computing ability of a reservoir.

Video distillation

Video has become one of the most important data forms. Video distillation explores more compact data forms and information modalities by analyzing the spatial-temporal and semantic features of video data, which is an important task in computer vision and a key technique in artificial intelligence. With the rapid development of video capturing devices and the increasing human requirements, video analysis tasks are facing numbers of opportunities and challenges. In recent years, large amounts of video distillation approaches are proposed. This paper creatively unifies the theoretical basis of video distillation by analyzing the relationship among data, information and knowledge from the perspective of information theory, and argues that the principle of video distillation is to improve the information capacity of video data. Then, we overview existing approaches in the aspects of video data representation, key content summarization, moving object synopsis and text description generation, etc., and relate the development of video summarization, synopsis and captioning, which are typical tasks in video distillation. More importantly, this paper discusses the advantages and drawbacks of existing approaches, and then points out several key scientific problems that have not yet been addressed, and simultaneously analyzes the potential future development in video distillation.

Information Measures of Knowledge Contribution: A New Method to Measure Knowledge Contribution in Collaborative Knowledge Building: An Information Theory Perspective

The measurement of knowledge contribution in collaborative knowledge building is an important research topic in computer-supported collaborative learning. The information measures of knowledge contribution based on information theory are proposed in this study, which includes two measures: amount of information and information gain. Discourse data collected from a collaborative knowledge building activity were analyzed to validate these measures. The results showed that our information measures can complement the traditional behavioral. With the help of the two measures, community-level variation tendency and individual-level knowledge contribution characteristics could be analyzed in collaborative knowledge building activities. A log function was used to fit the community knowledge variation tendency to measure the convergence of knowledge building. Students were clustered into five types according to their behaviors and contributions in collaborative knowledge building. Both teachers and researchers can benefit from these two information measures by using them in practice.

Interpreting rate-distortion of variational autoencoder and using model uncertainty for anomaly detection

Building a scalable machine learning system for unsupervised anomaly detection via representation learning is highly desirable. One of the prevalent methods is using a reconstruction error of variational autoencoder (VAE) by maximizing the evidence lower bound. We revisit VAE from the perspective of information theory to provide some theoretical foundations on using the reconstruction error and finally arrive at a simpler yet effective model for anomaly detection. In addition, to enhance the effectiveness of detecting anomalies, we incorporate a practical model uncertainty measure into the anomaly score. We show empirically the competitive performance of our approach on benchmark data sets.

The Analysis of Influence of BIM Technology on the Performance of China Construction Production Operation and Maintenance System from the Perspective of System Theory and Information Theory

In China, the construction industry is divided into the secondary industry, with the division of "construction industry", and the definition of "architecture". However, for construction production, operation and maintenance, we generally use stages to carry out restricted. After analyzing the concepts of industry, field, and construction, the author regards construction production operation and maintenance as a system, and analyzes project establishment, design, construction, and operation and maintenance as the four elements of the system. Analyze the impact of BIM technology on the performance of the system from the perspective of system theory and information theory, and demonstrate that the application of BIM technology will bring about the optimization of the performance of the building production operation and maintenance system, which is the best way for the construction industry to achieve industrial upgrading. The focus of the development of BIM technology application should be on the reconstruction of the system structure and the cooperation mode of all parties, that is, the production relationship. In the end, the author looks forward to the future development of BIM technology.

Intelligent Reflecting Surface Assisted Secret Key Generation

In secret key generation of physical layer security technology, it is challenging to achieve high key capacity and low bit inconsistency rate. This paper investigates intelligent reflecting surface (IRS)-assisted secret key generation, which aims to maximize the secret key capacity by adjusting the placement of the IRS units. Specifically, we first analyze and deduce the key capacity expression of the IRS-assisted system from the perspective of information theory. Then we investigate how to use the channel state information (CSI) to place the IRS units effectively so as to maximize the secret key capacity. Simulation results show that our scheme could improve the quality of secret key generation significantly.

Secure identity authentication scheme based on PUF for IoUT

The existing identity authentication schemes based on the devices of Internet of Underwater Things (IouT) are all executed at the network layer and higher layers, which require high computing costs and are not suitable for devices with limited computing resources. By applying PUF to key binding, a physical layer-based identity authentication scheme is proposed. The authentication process is realized by the joint typicality coding and decoding technology based on the packing lemma and the covering lemma. The Achievability and Converse propositions prove the feasibility of the scheme, and the confidential capacity domain is delimited from the perspective of information theory. The false authentication analysis theoretically limits the probability of successful deception by the adversary, and reduces it to the lowest limit.

SSSIC: Semantics-to-Signal Scalable Image Coding With Learned Structural Representations.

We address the requirement of image coding for joint human-machine vision, i.e., the decoded image serves both human observation and machine analysis/understanding. Previously, human vision and machine vision have been extensively studied by image (signal) compression and (image) feature compression, respectively. Recently, for joint human-machine vision, several studies have been devoted to joint compression of images and features, but the correlation between images and features is still unclear. We identify the deep network as a powerful toolkit for generating structural image representations. From the perspective of information theory, the deep features of an image naturally form an entropy decreasing series: a scalable bitstream is achieved by compressing the features backward from a deeper layer to a shallower layer until culminating with the image signal. Moreover, we can obtain learned representations by training the deep network for a given semantic analysis task or multiple tasks and acquire deep features that are related to semantics. With the learned structural representations, we propose SSSIC, a framework to obtain an embedded bitstream that can be either partially decoded for semantic analysis or fully decoded for human vision. We implement an exemplar SSSIC scheme using coarse-to-fine image classification as the driven semantic analysis task. We also extend the scheme for object detection and instance segmentation tasks. The experimental results demonstrate the effectiveness of the proposed SSSIC framework and establish that the exemplar scheme achieves higher compression efficiency than separate compression of images and features.

Novel Convolutional Neural Network with Variational Information Bottleneck for P300 Detection.

In the area of brain-computer interfaces (BCI), the detection of P300 is a very important technique and has a lot of applications. Although this problem has been studied for decades, it is still a tough problem in electroencephalography (EEG) signal processing owing to its high dimension features and low signal-to-noise ratio (SNR). Recently, neural networks, like conventional neural networks (CNN), has shown excellent performance on many applications. However, standard convolutional neural networks suffer from performance degradation on dealing with noisy data or data with too many redundant information. In this paper, we proposed a novel convolutional neural network with variational information bottleneck for P300 detection. Wiht the CNN architecture and information bottleneck, the proposed network termed P300-VIB-Net could remove the redundant information in data effectively. The experimental results on BCI competition data sets show that P300-VIB-Net achieves cutting-edge character recognition performance. Furthermore, the proposed model is capable of restricting the flow of irrelevant information adaptively in the network from perspective of information theory. The experimental results show that P300-VIB-Net is a promising tool for P300 detection.

Entanglement and Classical Correlations at the Doping-Driven Mott Transition in the Two-Dimensional Hubbard Model

Tools of quantum information theory offer a new perspective to characterize phases and phase transitions in interacting many-body quantum systems. The Hubbard model is the archetypal model of such systems and can explain rich phenomena of quantum matter with minimal assumptions. Recent measurements of entanglement-related properties of this model using ultracold atoms in optical lattices hint that entanglement could provide the key to understanding open questions of the doped Hubbard model, including the remarkable properties of the pseudogap phase. These experimental findings call for a theoretical framework and new predictions. Here we approach the doped Hubbard model in two dimensions from the perspective of quantum information theory. We study the local entropy and the total mutual information across the doping-driven Mott transition within plaquette cellular dynamical mean-field theory. We find that upon varying doping these two entanglement-related properties detect the Mott insulating phase, the strongly correlated pseudogap phase, and the metallic phase. Imprinted in the entanglement-related properties we also find the pseudogap to correlated metal first-order transition, its finite temperature critical endpoint, and its supercritical crossovers. Through this footprint we reveal an unexpected interplay of quantum and classical correlations. Our work shows that sharp variation in the entanglement-related properties and not broken symmetry phases characterizes the onset of the pseudogap phase at finite temperature.

Soft Compression: An Approach to Shape Coding for Images

In this letter, we propose soft compression, an lossless compression approach to shape coding for images using location index and codebook of designed shapes with various sizes. This method is different from traditional image compression methods, as it aims at finding the basic shape blocks of pictures to improve the compression ratio from the perspective of information theory and frequency. In particular, the triplets including location and codeword are stored when encoding an image instead of the entire array. It is shown that soft compression can greatly reduce the bandwidth and storage space needed in the process of transmitting and storing the same kind of images.

The synergistic impact of ENSO and IOD on Indian summer monsoon rainfall in observations and climate simulations – an information theory perspective

Abstract. The El Niño–Southern Oscillation (ENSO) and Indian Ocean Dipole (IOD) are two well-known temporal oscillations in sea surface temperature (SST), which are both thought to influence the interannual variability of Indian summer monsoon rainfall (ISMR). Until now, there has been no measure to assess the simultaneous information exchange (IE) from both ENSO and IOD to ISMR. This study explores the information exchange from two source variables (ENSO and IOD) to one target (ISMR). First, in order to illustrate the concepts and quantification of two-source IE to a target, we use idealized test cases consisting of linear and nonlinear dynamical systems. Our results show that these systems exhibit net synergy (i.e., the combined influence of two sources on a target is greater than the sum of their individual contributions), even with uncorrelated sources in both the linear and nonlinear systems. We test IE quantification with various estimators (linear, kernel, and Kraskov estimators) for robustness. Next, the two-source IE from ENSO and IOD to ISMR is investigated in observations, reanalysis, three global climate model (GCM) simulations, and three nested higher-resolution simulations using a regional climate model (RCM). This (1) quantifies IE from ENSO and IOD to ISMR in the natural system and (2) applies IE in the evaluation of the GCM and RCM simulations. The results show that both ENSO and IOD contribute to ISMR interannual variability. Interestingly, significant net synergy is noted in the central parts of the Indian subcontinent, which is India's monsoon core region. This indicates that both ENSO and IOD are synergistic predictors in the monsoon core region. But, they share significant net redundant information in the southern part of the Indian subcontinent. The IE patterns in the GCM simulations differ substantially from the patterns derived from observations and reanalyses. Only one nested RCM simulation IE pattern adds value to the corresponding GCM simulation pattern. Only in this case does the GCM simulation show realistic SST patterns and moisture transport during the various ENSO and IOD phases. This confirms, once again, the importance of the choice of GCM in driving a higher-resolution RCM. This study shows that two-source IE is a useful metric that helps in better understanding the climate system and in process-oriented climate model evaluation.