Superfluous Information Research Articles

MVEB: Self-Supervised Learning With Multi-View Entropy Bottleneck.

Self-supervised learning aims to learn representation that can be effectively generalized to downstream tasks. Many self-supervised approaches regard two views of an image as both the input and the self-supervised signals, assuming that either view contains the same task-relevant information and the shared information is (approximately) sufficient for predicting downstream tasks. Recent studies show that discarding superfluous information not shared between the views can improve generalization. Hence, the ideal representation is sufficient for downstream tasks and contains minimal superfluous information, termed minimal sufficient representation. One can learn this representation by maximizing the mutual information between the representation and the supervised view while eliminating superfluous information. Nevertheless, the computation of mutual information is notoriously intractable. In this work, we propose an objective termed multi-view entropy bottleneck (MVEB) to learn minimal sufficient representation effectively. MVEB simplifies the minimal sufficient learning to maximizing both the agreement between the embeddings of two views and the differential entropy of the embedding distribution. Our experiments confirm that MVEB significantly improves performance. For example, it achieves top-1 accuracy of 76.9% on ImageNet with a vanilla ResNet-50 backbone on linear evaluation. To the best of our knowledge, this is the new state-of-the-art result with ResNet-50.

Fault diagnosis of rolling bearing based on adaptive attention network and federated learning

Abstract For rolling bearings in actual industrial scenarios, the problem of low diagnostic accuracy is caused by the large difference in data distribution under different working conditions and the complexity of working conditions with a lot of redundant information, in this paper, a combination of the Federated Adaptive Attention Network (FAAN) based fault diagnostic model is proposed. Firstly, the fault information is randomly divided into multiple sequences, by utilizing dual convolutional layers and adaptive attention mechanism to process abundant vibration data, it enables accurate identification of fault information distribution in the original signal while removing superfluous information nd fusing characteristics to improve diagnostic accuracy. Furthermore, a federated learning model incorporating attention mechanism is proposed. It performs asynchronous updates based on local data distribution, improving the efficiency and accuracy of data analysis uploads, and enhancing the model’s generalization capability. Simulation experiments have been carried out using the datasets from Case Western Reserve University and Jiangnan University, and after comparative analysis, the proposed method has better performance and generalization ability.

DiffRank: Enhancing efficiency in discontinuous frame rate analysis for urban surveillance systems

Urban public safety management relies heavily on video surveillance systems, which provide crucial visual data for resolving a wide range of incidents and controlling unlawful activities. Traditional methods for target detection predominantly employ a two-stage approach, focusing on precision in identifying objects such as pedestrians and vehicles. These objects, typically sparse in large-scale, lower-quality surveillance footage, induce considerable redundant computation during the initial processing stage. This redundancy constrains real-time detection capabilities and escalates processing costs. Furthermore, transmitting raw images and videos laden with superfluous information to centralized back-end systems significantly burdens network communications and fails to capitalize on the computational resources available at diverse surveillance nodes. This study introduces DiffRank, a novel preprocessing method for fixed-angle video imagery in urban surveillance. The method strategically generates candidate regions during preprocessing, thereby reducing redundant object detection and improving the efficiency of the detection algorithm. Drawing upon change detection principles, a background feature learning approach utilizing shallow features has been developed. This approach prioritizes learning the characteristics of fixed-area backgrounds over direct background identification. As a result, alterations in ROI are efficiently discerned using computationally efficient shallow features, markedly accelerating the generation of proposed Regions of Interest (ROIs) and diminishing the computational demands for subsequent object detection and classification. Comparative analysis on various public and private datasets illustrates that DiffRank, while maintaining high accuracy, substantially outperforms existing baselines in terms of speed, particularly with larger image sizes (e.g., an improvement exceeding 300 % at 1920×1080 resolution). Moreover, the method demonstrates enhanced robustness compared to baseline methods, efficiently disregarding static targets like mannequins in display windows. The advancements in candidate area preprocessing enable a balanced approach between detection accuracy and overall detection speed, making the algorithm highly applicable for real-time on-site analysis in edge computing scenarios and cloud-edge collaborative computing environments.

Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations

Zero-shot multi-speaker TTS aims to synthesize speech with the voice of a chosen target speaker without any fine-tuning. Prevailing methods, however, encounter limitations at adapting to new speakers of out-of-domain settings, primarily due to inadequate speaker disentanglement and content leakage. To overcome these constraints, we propose an innovative negation feature learning paradigm that models decoupled speaker attributes as deviations from the complete audio representation by utilizing the subtraction operation. By eliminating superfluous content information from the speaker representation, our negation scheme not only mitigates content leakage, thereby enhancing synthesis robustness, but also improves speaker fidelity. In addition, to facilitate the learning of diverse speaker attributes, we leverage multi-stream Transformers, which retain multiple hypotheses and instigate a training paradigm akin to ensemble learning. To unify these hypotheses and realize the final speaker representation, we employ attention pooling. Finally, in light of the imperative to generate target text utterances in the desired voice, we adopt adaptive layer normalizations to effectively fuse the previously generated speaker representation with the target text representations, as opposed to mere concatenation of the text and audio modalities. Extensive experiments and validations substantiate the efficacy of our proposed approach in preserving and harnessing speaker-specific attributes vis-à-vis alternative baseline models.

Incorporating sufficient physical information into artificial neural networks: A guaranteed improvement via physics-based Rao-Blackwellization

The concept of Rao-Blackwellization is employed to improve predictions of artificial neural networks by physical information. The error norm and the proof of improvement are transferred from the original statistical concept to a deterministic one, using sufficient information on physics-based conditions. The proposed strategy is applied to material modeling and illustrated by examples of the identification of a yield function, the identification of driving forces for quasi-brittle damage and rubber experiments. Sufficient physical information is employed, e.g., in the form of invariants, parameters of a minimization problem, isotropy and differentiability. It is proven how intuitive accretion of information can yield improvement if it is physically sufficient, but also how insufficient or superfluous information can cause impairment. Opportunities for the improvement of artificial neural networks are explored in terms of the training data set, the networks’ structure and output filters. Even crude initial predictions are remarkably improved by reducing noise, overfitting and data requirements.

Effective Surrogate Gradient Learning With High-Order Information Bottleneck for Spike-Based Machine Intelligence.

Brain-inspired computing technique presents a promising approach to prompt the rapid development of artificial general intelligence (AGI). As one of the most critical aspects, spiking neural networks (SNNs) have demonstrated superiority for AGI, such as low power consumption. Effective training of SNNs with high generalization ability, high robustness, and low power consumption simultaneously is a significantly challenging problem for the development and success of applications of spike-based machine intelligence. In this research, we present a novel and flexible learning framework termed high-order spike-based information bottleneck (HOSIB) leveraging the surrogate gradient technique. The presented HOSIB framework, including second-order and third-order formation, i.e., second-order information bottleneck (SOIB) and third-order information bottleneck (TOIB), comprehensively explores the common latent architecture and the spike-based intrinsic information and discards the superfluous information in the data, which improves the generalization capability and robustness of SNN models. Specifically, HOSIB relies on the information bottleneck (IB) principle to prompt the sparse spike-based information representation and flexibly balance its exploitation and loss. Extensive classification experiments are conducted to empirically show the promising generalization ability of HOSIB. Furthermore, we apply the SOIB and TOIB algorithms in deep spiking convolutional networks to demonstrate their improvement in robustness with various categories of noise. The experimental results prove the HOSIB framework, especially TOIB, can achieve better generalization ability, robustness and power efficiency in comparison with the current representative studies.

An Extensive Analysis of Feature Selection in Machine Learning

Abstract: Researchers and engineers working in the disciplines of data mining and machine learning have difficulties while analysing high-dimensional data. A dimension reduction method called feature selection is used to pick characteristics that are pertinent to machine learning tasks. Improving the efficiency of machine learning algorithms, hastening the learning process, and creating basic models all depend critically on reducing the size of the dataset by removing superfluous and useless information. Numerous feature selection techniques have been put forth in the literature to find the pertinent feature or feature subsets in order to accomplish the goals of clustering and classification. The purpose of this study is to review the state of the art for these methods.

Contrastive Attention-guided Multi-level Feature Registration for Reference-based Super-resolution

Given low-quality input and assisted by referential images, reference-based super-resolution (RefSR) strives to enlarge the spatial size with the guarantee of realistic textures, for which sophisticated feature-matching strategies are naturally demanded. However, the miserable transformation gap between inputs and references, e.g., texture rotation and scaling within patches, often yields distorted textures and terrible ghosting artifacts, which seriously hampers the visual senses and their further investigation. To circumvent this challenge, we propose a contrastive attention-guided multi-level feature registration for RefSR, explicitly tapping the potential of interacting between inputs and references. Specifically, we develop a multi-level feature warping scheme, involving patch-level coarse feature swapping and pixel-level deformable alignment, to model generalized spatial transformation correspondences steered by contrastive attention. Notably, a spatial registration module is embedded for further calibration against the potential misalignment issue and inter-feature distribution difference. In addition, aiming at suppressing the impacts of irrelevant or superfluous information on cross-scale features, we incorporate a multi-residual feature fusion module to strive for visually plausible textures. Experimental results on four publicly available datasets demonstrate that our method outperforms most state-of-the-art approaches in terms of both efficiency and perceptual effectiveness.

A Bottleneck Network with Light Attention for Multimodal Clustering

Multimodal Clustering (MMC) aims to boost clustering performance by utilizing complementary information from multimodal data. However, the heterogeneous gap between various modalities and the interference of superfluous information in multimodal data makes it challenging to comprehend the complex dependencies between them. To address these issues, we present a Bottleneck Network with Light Attention for Multimodal Clustering (LABMC) to understand the dependencies between different modalities and explore complementary information while maximally eliminating superfluous information. Specifically, Light Attention (LA) is proposed as a concise multimodal interaction mechanism that introduces a convolutional high-level semantic layer to understand the common high-level semantics among different modalities and explores related complementary information to obtain a multimodal fusion representation. It can efficiently comprehend the latent dependencies between diverse modalities to bridge the heterogeneous gap. Subsequently, we construct a Superfluity Elimination (SE) function to erase superfluous information from the modalities. We divide the mutual information between the fusion representation and multimodal data into superfluous information and complementary information. Under the premise of preserving complementary information with an approximation, we utilize a variational optimization method to minimize the mutual information to safely remove superfluity. The combination of LA and SE naturally forms a bottleneck, which enables the extraction of complementary information while reducing superfluous information. Finally, we measure the quality of the modalities to dynamically assign the optimal weights to SE function, which significantly enhances the rationality of SE. Extensive experiment results on five real-world datasets show that LABMC outperforms the state-of-the-art MMC methods.

Attribute Selection Algorithm with Clustering based Optimization Approach based on Mean and Similarity Distance

With hundreds or thousands of attributes in high-dimensional data, the computational workload is challenging. Attributes that have no meaningful influence on class predictions throughout the classification process increase the computing load. This article's goal is to use attribute selection to reduce the size of high-dimensional data, which will lessen the computational load. Considering selected attribute subsets that cover all attributes. As a result, there are two stages to the process: filtering out superfluous information and settling on a single attribute to stand in for a group of similar but otherwise meaningless characteristics. Numerous studies on attribute selection, including backward and forward selection, have been undertaken. This experiment and the accuracy of the categorization result recommend a k-means based PSO clustering-based attribute selection. It is likely that related attributes are present in the same cluster while irrelevant attributes are not identified in any clusters. Datasets for Credit Approval, Ionosphere, Annealing, Madelon, Isolet, and Multiple Attributes are employed alongside two other high-dimensional datasets. Both databases include the class label for each data point. Our test demonstrates that attribute selection using k-means clustering may be done to offer a subset of characteristics and that doing so produces classification outcomes that are more accurate than 80%.

Multiresolution generative adversarial networks with bidirectional adaptive-stage progressive guided fusion for remote sensing image

ABSTRACT Remote sensing image (RSI) with concurrently high spatial, temporal, and spectral resolutions cannot be produced by a single sensor. Multisource RSI fusion is a convenient technique to realize high spatial resolution multispectral (MS) images (spatial spectral fusion, i.e. SSF) and high temporal and spatial resolution MS images (spatiotemporal fusion, i.e. STF). Currently, deep learning-based fusion models can only implement SSF or STF, lacking models that perform both SSF and STF. Multiresolution generative adversarial networks with bidirectional adaptive-stage progressive guided fusion (BAPGF) for RSI are proposed to implement both SSF and STF, namely BPF-MGAN. A bidirectional adaptive-stage feature extraction architecture in fine-scale-to-coarse-scale and coarse-scale-to-fine-scale modes is introduced. The designed BAPGF introduces a previous fusion result-oriented cross-stage-level dual-residual attention fusion strategy to enhance critical information and suppress superfluous information. Adaptive resolution U-shaped discriminators are implemented to feed multiresolution context into the generator. A generalized multitask loss function unlimited by no-reference images is developed to strengthen the model via constraints on the multiscale feature, structural, and content similarities. The BPF-MGAN model is validated on SSF datasets and STF datasets. Compared with the state-of-the-art SSF and STF models, results demonstrate the superior performance of the proposed BPF-MGAN model in both subjective and objective evaluations.

Enhancing Animal Welfare with Interaction Recognition: A Deep Dive into Pig Interaction Using Xception Architecture and SSPD-PIR Method

The advent of artificial intelligence (AI) in animal husbandry, particularly in pig interaction recognition (PIR), offers a transformative approach to enhancing animal welfare, promoting sustainability, and bolstering climate resilience. This innovative methodology not only mitigates labor costs but also significantly reduces stress levels among domestic pigs, thereby diminishing the necessity for constant human intervention. However, the raw PIR datasets often encompass irrelevant porcine features, which pose a challenge for the accurate interpretation and application of these datasets in real-world scenarios. The majority of these datasets are derived from sequential pig imagery captured from video recordings, and an unregulated shuffling of data often leads to an overlap of data samples between training and testing groups, resulting in skewed experimental evaluations. To circumvent these obstacles, we introduced a groundbreaking solution—the Semi-Shuffle-Pig Detector (SSPD) for PIR datasets. This novel approach ensures a less biased experimental output by maintaining the distinctiveness of testing data samples from the training datasets and systematically discarding superfluous information from raw images. Our optimized method significantly enhances the true performance of classification, providing unbiased experimental evaluations. Remarkably, our approach has led to a substantial improvement in the isolation after feeding (IAF) metric by 20.2% and achieved higher accuracy in segregating IAF and paired after feeding (PAF) classifications exceeding 92%. This methodology, therefore, ensures the preservation of pertinent data within the PIR system and eliminates potential biases in experimental evaluations. As a result, it enhances the accuracy and reliability of real-world PIR applications, contributing to improved animal welfare management, elevated food safety standards, and a more sustainable and climate-resilient livestock industry.

Smart optimization in 802.11p media access control protocol for vehicular ad hoc network

<span lang="EN-US">The innovative idea presented in this research is that advancements in automotive networks and embedded devices can be used to assess the impact of congestion control on throughput and packet delivery ratio (PDR), or so-called multimedia content delivery. Vehicle networking and the distribution of multimedia content have become essential factors in getting packets to their intended recipients due to the availability of bandwidth. Vehicle-to-infrastructure (V2I) communication systems are crucial in vehicular ad hoc networks (VANETs), which permit vehicles to connect by distributing and delivering traffic data and transmission packet schemes. High levels of mobility and changing network topology necessitate dispersed monitoring and execution for congestion control. The amount of traffic congestion for packet transfers could be reduced by enhancing congestion management in terms of throughput and PDR percentages. In a highway setting, the Taguchi approach has been used to optimize the parameters for congestion control. Based on throughput and PDR performance measures, this technique minimizes superfluous traffic information and lowers the likelihood of network congestion. The simulation results have shown that the proposed approach performs better since it increases network performance while effectively utilizing bandwidth. The effectiveness of the suggested technique is evaluated using a typical VANETs scenario for V2I communication while driving on a highway.</span>

Relationship Between GDP and Inflation Rate

In history, inflation has appeared many times. For example, the Recession. Also, GDP inflation and deflation occur from time to time. However, there is little research about the relationship between inflation and GDP. Therefore, this paper wants to investigate how inflation is associated with GDP. However, there is no linear relationship between these two factors. Investment managers must discover a degree of awareness that aids their decision-making without overwhelming them with superfluous information. Discover the implications of inflation and GDP on the marketplace, the business, and the investment. Innovative banking variations across nations, or within a nation and over time, influence how inflation impacts economic growth. Smaller intermediaries costs mean that inflation has a negative production less than in higher-cost nations, and comparable effects apply within the same nation when intermediation prices shift over time. Inflation's impacts on growth have long been debated, both conceptually and experimentally. In contrast to several financial concerns, where the issue is frequently whether a particular impact or no impact exists, there are consistent theories that indicate inflation can have a favorable, unfavorable, or no influence on economic growth. As a result, it appears to be a perfect condition for testing different ideas against evidence.

An Efficient Document Clustering Approach for Devising Semantic Clusters

The rise of superfluous information day by day has made the clustering of information into meaningful sets challenging. We propose an efficient approach for obtaining semantic clusters from a huge volume of documents. The preprocessing based on the lexical ontological information from WordNet helps in reducing the feature space and eliminating synonymy problems among the features. A considerable decrease in computational time is achieved by means of an enhanced k-means clustering algorithm. This algorithm computes the starting centroids using a sorting technique based on Red-Black Tree, thus ensuring efficiency and meaningful clusters. Memoization techniques are utilized in the subsequent stages, to avoid redundant computations. Results indicate that our method produces more meaningful clusters than those employing word embedding models like Word2Vec, FastText, and BERT for feature extraction. The experimental results on the MiniNewsGroup, 20NewsGroup Large dataset, and Reuters-21578 dataset show remarkable achievement in clustering outcomes in terms of purity and execution time. Results on the enormous data collection 20NewsGroup Large dataset show a better NMI (Normalized Mutual Information) score compared to the existing methods.

Self-Supervised Information Bottleneck for Deep Multi-View Subspace Clustering.

In this paper, we explore the problem of deep multi-view subspace clustering framework from an information-theoretic point of view. We extend the traditional information bottleneck principle to learn common information among different views in a self-supervised manner, and accordingly establish a new framework called Self-supervised Information Bottleneck based Multi-view Subspace Clustering (SIB-MSC). Inheriting the advantages from information bottleneck, SIB-MSC can learn a latent space for each view to capture common information among the latent representations of different views by removing superfluous information from the view itself while retaining sufficient information for the latent representations of other views. Actually, the latent representation of each view provides a kind of self-supervised signal for training the latent representations of other views. Moreover, SIB-MSC attempts to disengage the other latent space for each view to capture the view-specific information by introducing mutual information based regularization terms, so as to further improve the performance of multi-view subspace clustering. Extensive experiments on real-world multi-view data demonstrate that our method achieves superior performance over the related state-of-the-art methods.

Tattoos in cinema

In cinema, many visual characteristics of the characters serve as "unspoken" elements that allow the audience to attribute (rightly or wrongly) these characteristics without the need to use superfluous verbal information: clothes, hairstyles, scars, prostheses, physical defects, etc. Tattoos are obviously part of this visual panel. Tattoos may be the central theme of the movie, allow to give one or more attribute to the character, have a real function in the movie or simply has an aesthetic role of beautification. The prevalence of actors and actress with tattoos is unknown. However, some actors may display their tattoos for the purpose of the movie.

Compression and Noise

ABSTRACT Compression is an essential technique used across diverse information systems, one in which supposedly redundant or superfluous information is minimized or eliminated in order to make the storage, transmission, or reception of other information more legible or efficient. Compression is involved in everything from computer data storage (encoding) and efficient computational processes (floating point arithmetic) to the formatting of media (telephony, radio, streaming) or the engineering and circulation of sound and image (dynamic compression of volume, jpeg resolution). Beyond any particular technical implementation, though, compression names the peculiar perceptual regime of late modernity—it is our percepteme, our episteme. As Galloway and LaRivière (2017) have noted—everything is compressed, from the logic of digital computers to our attention spans. Yet, if compression designates the essence of experience today, it is precisely in response to a complementary concept of noise. Noise is the lived affect of our material conditions which cannot be made significant to us: not only the literal acoustic noise of late modernity (the waste-product of technologies which hang over perceptual spaces like smog hangs over cities) but, perhaps more critically, compression emerges to cope with a new, properly “cognitive complexity” embodied in the unprecedented entanglement and mediation of social relations through the technical/computational unfolding of the value-form of capital. Such complexity—lying beyond the grasp of any human individual—is logistically offloaded onto the nootechnical externalizations associated with, for instance, AI algorithms. On the other hand, it is individuated in experience as noise. Because such noise is perceptually intractable, compression becomes the necessary shape of our aesthesis, one that rigorously flattens the available modalities of experience/value—yet giving birth to new forms of abstraction, perception, and thought. While several thinkers—notably, Jason LaRivière and Cécile Malaspina—have brilliantly elaborated each concept on its own, I argue that they can only be properly situated through each other. In order to do this, I combine computational and information theory, philosophy, media and cultural theory, as well as political economy with concrete domains where compression and noise articulate the phenomenological stakes for aesthetics today.

Towards an Effective Orthogonal Dictionary Convolution Strategy

Orthogonality regularization has proven effective in improving the precision, convergence speed and the training stability of CNNs. Here, we propose a novel Orthogonal Dictionary Convolution Strategy (ODCS) on CNNs to improve orthogonality effect by optimizing the network architecture and changing the regularized object. Specifically, we remove the nonlinear layer in typical convolution block “Conv(BN) + Nonlinear + Pointwise Conv(BN)”, and only impose orthogonal regularization on the front Conv. The structure, “Conv(BN) + Pointwise Conv(BN)”, is then equivalent to a pair of dictionary and encoding, defined in sparse dictionary learning. Thanks to the exact and efficient representation of signal with dictionaries in low-dimensional projections, our strategy could reduce the superfluous information in dictionary Conv kernels. Meanwhile, the proposed strategy relieves the too strict orthogonality regularization in training, which makes hyper-parameters tuning of model to be more flexible. In addition, our ODCS can modify the state-of-the-art models easily without any extra consumption in inference phase. We evaluate it on a variety of CNNs in small-scale (CIFAR), large-scale (ImageNet) and fine-grained (CUB-200-2011) image classification tasks, respectively. The experimental results show that our method achieve a stable and superior improvement.

Millimeter-Wave Frequency Modulated Continuous Wave Radar-Based Soft Fall Detection Using Pattern Contour-Confined Doppler-Time Maps

Fall accidents are one of the leading causes of non-natural death of older people. It is necessary to design and implement fall detection systems in older people’s homes. Among all indoor fall detection approaches, radar-based methods are considered one of the optimum solutions. In this paper, a millimetre-wave frequency modulated continuous wave (FMCW) radar-based fall detection method using pattern contour-confined Doppler-time (PCC-DT) maps is proposed. The soft fall motions, which can lead to missed detections, are studied and analyzed. When Doppler-time (DT) maps are used to interpret different motions, the redundant information and outliers can cause detection errors. The proposed PCC-DT map can diminish the superfluous information and remove outlier points to improve detection accuracy. The experimental results show that the proposed method can detect sudden and soft fall motions with high accuracy, sensitivity, and specificity.