Leibler Distance Research Articles

Feature Vector Effectiveness Evaluation for Pattern Selection in Computational Lithography

Pattern selection is crucial for optimizing the calibration process of optical proximity correction (OPC) models in computational lithography. However, it remains a challenge to achieve a balance between representative coverage and computational efficiency. This work presents a comprehensive evaluation of the feature vectors’ (FVs’) effectiveness in pattern selection for OPC model calibration, leveraging key performance indicators (KPIs) based on Kullback–Leibler divergence and distance ranking. Through the construction of autoencoder-based FVs and fast Fourier transform (FFT)-based FVs, we compare their efficacy in capturing critical pattern features. Validation experimental results indicate that autoencoder-based FVs, particularly augmented with the lithography domain knowledge, outperform FFT-based counterparts in identifying anomalies and enhancing lithography model performance. These results also underscore the importance of adaptive pattern representation methods in calibrating the OPC model with evolving complexities.

Symmetric Kullback–Leibler distance based generalized grey target decision method for mixed attributes

PurposeThe reported Kullback–Leibler (K–L) distance-based generalized grey target decision method (GGTDM) for mixed attributes is an asymmetric decision-making basis (DMB) that does not have the symmetric characteristic of distance in common sense, which may affect the decision-making result. To overcome the deficiency of the asymmetric K–L distance, the symmetric K–L distance is investigated to act as the DMB of GGTDM for mixed attributes.Design/methodology/approachThe decision-making steps of the proposed approach are as follows: First, all mixed attribute values are transformed into binary connection numbers, and the target centre indices of all attributes are determined. Second, all the binary connection numbers (including the target centre indices) are divided into deterministic and uncertain terms and converted into two-tuple (determinacy and uncertainty) numbers. Third, the comprehensive weighted symmetric K–L distance can be computed, as can the alternative index of normalized two-tuple (deterministic degree and uncertainty degree) number and that of the target centre. Finally, the decision-making is made by the comprehensive weighted symmetric K–L distance according to the rule that the smaller the value, the better the alternative.FindingsThe case study verifies the proposed approach with its sufficient theoretical basis for decision-making and reflects the preferences of decision-makers to address the uncertainty of an uncertain number.Originality/valueThis work compares the single-direction-based K–L distance to the symmetric one and uses the symmetric K–L distance as the DMB of GGTDM. At the same time, different coefficients are assigned to an uncertain number’s deterministic term and uncertain term in the calculation process, as this reflects the preference of the decision-maker.

Optimized multi-scale framework for image enhancement using spatial information-based histogram equalization

ABSTRACT The histogram equalization method, used to improve images, minimizes the amount of pixel intensities, which results in the loss of detail and an unnatural appearance. This study presents an approach for enhancing low contrast images based on their inherent characteristics. The statistical parameter skewness is derived from the photographs to facilitate the classification process into dark and bright images. Based on the classification, appropriate dark and bright pass filters are applied on the multi-level decomposed images to extract the significant edge details. The level of decomposition is optimized using particle swarm optimization. The extracted edge details are utilized by the two-dimensional histogram equalization technique. It leverages the combined presence of edge information and pixel intensities in the low contrast image. The algorithm's efficacy is assessed on three databases, namely CCID, LOL, and DRESDEN, through the utilization of standard deviation (SD), contrast improvement index (CII), discrete entropy (DE), the natural image quality evaluator (NIQE), and Kullback–Leibler distance (KL). Based on the empirical findings, it can be observed that the suggested methodology exhibits better performance compared to alternative methods, including deep learning architectures, in terms of high CII, SD, DE, and low NIQE, KL values.

METHODOLOGY FOR IMPROVING DEEP LEARNING-BASED CLASSIFICATION FOR CT SCAN COVID-19 IMAGES

Early identification of COVID-19 necessitates a precise interpretation of computed tomography (CT) chest images. Consistent and accurate photographs are critical for the correct diagnosis. In this paper, a novel pre-processing approach (mainly improves the contrast) based on gradient enhancement method (GCE) is proposed for prominent visualization of the diagnostic features from the COVID-19 CT images. This pre-processing stage helps in preserving the diagnostic information in the disease affected area. Edge information in the CT images helps the physicians and classification model for better classification. The edge features are preserved by improving the contrast by using multi-scale dependent dark pass filter. From the edge features and pixel intensities, cumulative distribution function (CDF) is computed. It is then mapped to a uniform distribution, resulting in contrast enhanced COVID-19 CT images. These pre-processed images are fed to the customized deep convolutional neural networks (CNNs) like AlexNet, VGG-19, ResNet-101, DenseNet-201, GoogleNet, MobileNet-v2, SqueezeNet, Inception-v3, Xception, and EfficientNet-b0 for classification. Introducing GCE as a pre-processing stage improves the COVID-19 classification accuracy by nearly 6%. Evaluation of the contrast enhancement by GCE technique is carried out on the CT images by contrast improvement index (CII), discrete entropy (DE), and Kullback–Leibler distance (KL-Distance) measures. The experimental findings reveal that the GCE method produces higher CII and DE values than the other enhancement methods available.

Bi‐directional LSTM based channel estimation in 5G massive MIMO OFDM systems over TDL‐C model with Rayleigh fading distribution

SummaryIn this work, a deep learning (DL)‐based massive multiple‐input multiple‐output (mMIMO) orthogonal frequency division multiplexing (OFDM) system is investigated over the tapped delay line type C (TDL‐C) model with a Rayleigh fading distribution at frequencies ranging from 0.5 to 100 GHz. The proposed bi‐directional long short‐term memory (Bi‐LSTM) channel state information (CSI) estimator uses online learning during training and offline learning during the practical implementation phase. The design of the estimator takes into account situations in which prior knowledge of channel statistics is limited and targets excellent performance, even with limited pilot symbols (PS). Three separate loss functions (mean square logarithmic error [MSLE], Huber, and Kullback–Leibler Distance [KLD]) are assessed in three classification layers. The symbol error rate (SER) and outage probability performance of the proposed estimator are evaluated using a number of optimization techniques, such as stochastic gradient descent (SGD), momentum, and the adaptive gradient (AdaGrad) algorithm. The Bi‐LSTM‐based CSI estimator is trained considering a specific number of PS. It can be readily seen that by incorporating a cyclic prefix (CP), the system becomes more resilient to channel impairments, resulting in a lower SER. Simulations show that the SGD optimization approach and Huber loss function‐trained Bi‐LSTM‐based CSI estimator have the lowest SER and very high estimation accuracy. By using deep neural networks (DNNs), the Bi‐LSTM method for CSI estimation achieves a superior channel capacity (in bps/Hz) at 10 dB than long short‐term memory (LSTM) and other conventional CSI estimators, such as minimum mean square error (MMSE) and least squares (LS). The simulation results validate the analytical results in the study.

Scrutiny of a More Flexible Counterpart of Huang–Kotz FGM’s Distributions in the Perspective of Some Information Measures

In this work, we reveal some distributional traits of concomitants of order statistics (COSs) arising from the extended Farlie–Gumbel–Morgenstern (FGM) bivariate distribution, which was developed and studied in recent work. The joint distribution and product moments of COSs for this family are discussed. Moreover, some useful recurrence relations between single and product moments of concomitants are obtained. In addition, the asymptotic behavior of the concomitant’s rank for order statistics (OSs) is studied. The information measures, differential entropy, Kullback–Leibler (KL) distance, Fisher information number (FIN), and cumulative past inaccuracy (CPI) are theoretically and numerically studied.

Graph-Regularized, Sparsity-Constrained Non-Negative Matrix Factorization with Earth Mover’s Distance Metric

Non-negative matrix factorization (NMF) is widely used as a powerful matrix factorization tool in data representation. However, the traditional NMF, measured by Euclidean distance or Kullback–Leibler distance, does not take into account the internal implied geometric information of the dataset and cannot measure the distance between samples as well as possible. To remedy the defects, in this paper, we propose the NMF method with Earth mover’s distance as a metric, for short GSNMF-EMD. It combines graph regularization and L1/2 smooth constraints. The GSNMF-EMD method takes into account the intrinsic implied geometric information of the dataset and can produce more sparse and stable local solutions. Experiments on two specific image datasets showed that the proposed method outperforms related state-of-the-art methods.

BacterialWmark: telemedicine watermarking technique using bacterial foraging for smart healthcare system

A transform domain digital image watermarking technique is proposed using bacterial foraging optimization (BFO) for telemedicine applications. To make a trade-off between robustness, imperceptibility, and authenticity is a significant focus of our work. The field of telemedicine is one of the critical areas for an intruder to make unwanted changes. The size of medical data is exponentially increasing since the pandemic. So, the security of medical data is also a big concern. One of the most important techniques to safeguard medical data is image watermarking. The digital imaging and communications in medical images of liver ultrasound are taken as input host images. Peak signal-to-noise ratio, Kullback–Leibler distance, and root-mean-square difference are used for BFO. The proposed BacterialWmark technique shows a significant improvement in robustness and imperceptibility compared with other existing techniques. The computational complexity of the proposed work is also less. The authentication of the watermarked image is successfully executed as the features are perfectly matched with the aid of the MinEigen algorithm. So, the BacterialWmark algorithm can be used in the field of telemedicine efficiently.

Structural Nonlinear Damage Identification Method Based on the Kullback–Leibler Distance of Time Domain Model Residuals

Under external load excitation, damage such as breathing cracks and bolt loosening will cause structural time domain acceleration to have nonlinear features. To solve the problem of time domain nonlinear damage identification, a damage identification method based on the Kullback–Leibler (KL) distance of time domain model residuals is proposed in this paper. First, an autoregressive (AR) model order was selected using the autocorrelation function (ACF) and Akaike information criterion (AIC). Then, an AR model was obtained based on the structural acceleration response time series, and the AR model residual was extracted. Finally, the KL distance was used as a damage indicator to judge the structural damage source location. The effectiveness of the proposed method was verified by using a multi-story, multi-span stand model experiment and a simulated eight-story shear structure. The results show that the proposed structural nonlinear damage identification method can effectively distinguish the structural damage location of multi-degree-of-freedom shear structures and complex stand structures, and it is robust enough to detect environmental noise and small damage.

Asymptotic bias of the hazard probability model under model mis-specification

We compare the sensitivity of the estimated effective strip half-width with respect to choice of hazard probability function (Q). This is done by fitting the model under an erroneous assumption about the parametric form of Q, and comparing the estimated and true effective strip half-width. An ‘infinite sample size’ setting is employed, where fitting the model by maximum likelihood amounts to minimising the Kullback Leibler distance between the assumed and true models. The experiment is carried out in a situation that is relevant to minke whale sighting surveys both in the Antarctic and in the northeastern Atlantic. It is found that the hazard probability model is fairly robust with respect to the choice of parametric class for Q. The largest observed bias in the resulting effective strip half-width is less than 10%, while for most situations there is almost no bias.

Detection and Mitigation of IoT-Based Attacks Using SNMP and Moving Target Defense Techniques.

This paper proposes a solution for ensuring the security of IoT devices in the cloud environment by protecting against distributed denial-of-service (DDoS) and false data injection attacks. The proposed solution is based on the integration of simple network management protocol (SNMP), Kullback-Leibler distance (KLD), access control rules (ACL), and moving target defense (MTD) techniques. The SNMP and KLD techniques are used to detect DDoS and false data sharing attacks, while the ACL and MTD techniques are applied to mitigate these attacks by hardening the target and reducing the attack surface. The effectiveness of the proposed framework is validated through experimental simulations on the Amazon Web Service (AWS) platform, which shows a significant reduction in attack probabilities and delays. The integration of IoT and cloud technologies is a powerful combination that can deliver customized and critical solutions to major business vendors. However, ensuring the confidentiality and security of data among IoT devices, storage, and access to the cloud is crucial to maintaining trust among internet users. This paper demonstrates the importance of implementing robust security measures to protect IoT devices in the cloud environment and highlights the potential of the proposed solution in protecting against DDoS and false data injection attacks.

Entropy of sharp restart

Restart has the potential of expediting or impeding the completion times of general random processes. Consequently, the issue of mean-performance takes center stage: quantifying how the application of restart on a process of interest impacts its completion-time’s mean. Going beyond the mean, little is known on how restart affects stochasticity measures of the completion time. This paper is the first in a duo of studies that address this knowledge gap via: a comprehensive analysis that quantifies how sharp restart—a keystone restart protocol—impacts the Shannon entropy of the completion time. The analysis establishes closed-form results for sharp restart with general timers, with fast timers (high-frequency resetting), and with slow timers (low-frequency resetting). These results share a common structure: comparing the completion-time’s hazard rate to a flat benchmark—the constant hazard rate of an exponential distribution whose entropy is equal to the completion-time’s entropy. In addition, using an information-geometric approach based on Kullback–Leibler distances, the analysis establishes results that determine the very existence of timers with which the application of sharp restart decreases or increases the completion-time’s entropy. Our work sheds first light on the intricate interplay between restart and randomness—as gauged by the Shannon entropy.

Generalized cyclic Jensen and information inequalities

We present Levinson’s type generalizations of cyclic refinements of Jensen’s inequality by employing recent class of functions that further characterize and extend the family of 3-convex functions. We get monotonic cyclic Jensen’s inequalities and particularly the renowned Jensen’s inequality for 3-convex functions at a point (f∈κ1c(I)). As an applications in information theory, we first introduce new Csiszár type cyclic divergence functional for 3-convex functions and establish cyclic-Kullback–Leibler and Hellinger distances. We give monotonicity of cyclic divergence functionals which enable us to construct monotonic Shannon, Relative and Zipf–Mandelbrot entropies.

Efficiency of the Moscow Stock Exchange before 2022.

This paper investigates the degree of efficiency for the Moscow Stock Exchange. A market is called efficient if prices of its assets fully reflect all available information. We show that the degree of market efficiency is significantly low for most of the months from 2012 to 2021. We calculate the degree of market efficiency by (i) filtering out regularities in financial data and (ii) computing the Shannon entropy of the filtered return time series. We developed a simple method for estimating volatility and price staleness in empirical data in order to filter out such regularity patterns from return time series. The resulting financial time series of stock returns are then clustered into different groups according to some entropy measures. In particular, we use the Kullback–Leibler distance and a novel entropy metric capturing the co-movements between pairs of stocks. By using Monte Carlo simulations, we are then able to identify the time periods of market inefficiency for a group of 18 stocks. The inefficiency of the Moscow Stock Exchange that we have detected is a signal of the possibility of devising profitable strategies, net of transaction costs. The deviation from the efficient behavior for a stock strongly depends on the industrial sector that it belongs to.

Cross-Lingual Text Reuse Detection at sentence level for English–Urdu language pair

In recent years, the problem of Cross-Lingual Text Reuse Detection (X-TRD) has gained the interest of researchers due to the availability of large digital repositories and automatic translation systems. These systems are promptly available and openly accessible, which makes it easier to reuse text across the languages and hard to detect. In previous studies, different corpora and techniques have been developed for X-TRD at sentence/passage and document level for the English–Urdu language pair. However, there is a lack of large benchmark corpora and standard techniques for X-TRD for the English–Urdu language pair at the sentence level. To overcome this limitation, this study presents a large benchmark sentential cross-lingual (English–Urdu) corpus of 21,669 sentence pairs with simulated cases of X-TR, which are manually annotated at three levels of rewrite (Wholly Derived (WD) = 7,655, Partially Derived (PD) = 6,461, and Non Derived (ND) = 7,553). As a second major contribution, we have applied various state-of-the-art Cross-Lingual Sentence Transformers (CLST), and Translation plus Mono-lingual Analysis (T+MA) including N-gram Overlap (lexical), WordNet based techniques (semantic), mono-lingual word embedding-based techniques, and Kullback–Leibler Distance (KLD) (probabilistic) on our proposed sentential corpus for X-TRD. For the binary classification, the best results are obtained (F1 = 0.94) using a combination of all CLST and T+MA techniques and a combination of all T+MA techniques, whereas, for the ternary classification task, the best results are obtained (F1 = 0.84) using a combination of all CLST and T+MA techniques. The corpus will be publicly available to foster and promote research for X-TRD in an under-resourced language, such as the Urdu language.

Density Regression with Conditional Support Points

Density regression characterizes the conditional density of the response variable given the covariates, and provides much more information than the commonly used conditional mean or quantile regression. However, it is often computationally prohibitive in applications with massive datasets, especially when there are multiple covariates. In this article, we develop a new data reduction approach for the density regression problem using conditional support points. After obtaining the representative data, we exploit the penalized likelihood method as the downstream estimation strategy. Based on the connections among the continuous ranked probability score, the energy distance, the L 2 discrepancy and the symmetrized Kullback–Leibler distance, we investigate the distributional convergence of the representative points and establish the rate of convergence of the density regression estimator. The usefulness of the methodology is illustrated by modeling the conditional distribution of power output given multivariate environmental factors using a large scale wind turbine dataset.

Activity of vehicles in the bus rapid transit system Metrob\xfas in Mexico City

In this paper, we analyze a massive dataset with registers of the movement of vehicles in the bus rapid transit system Metrobús in Mexico City from February 2020 to April 2021. With these records and a division of the system into 214 geographical regions (segments), we characterize the vehicles’ activity through the statistical analysis of speeds in each zone. We use the Kullback–Leibler distance to compare the movement of vehicles in each segment and its evolution. The results for the dynamics in different zones are represented as a network where nodes define segments of the system Metrobús and edges describe similarity in the activity of vehicles. Community detection algorithms in this network allow the identification of patterns considering different levels of similarity in the distribution of speeds providing a framework for unsupervised classification of the movement of vehicles. The methods developed in this research are general and can be implemented to describe the activity of different transportation systems with detailed records of the movement of users or vehicles.

River Channel Extraction in SAR Images Using Level Sets Driven by Symmetric Kullback–Leibler Distance

This paper proposes a novel level set method (LSM) that improves river channel extraction accuracy in synthetic aperture images (SAR) by developing global median image fitting energies. First, we define a new global median fitting image (GMFI) to approximate the input image and use this GMFI to construct the fitting energy based on the symmetric Kullback-Leibler distance (SKL). Second, to exploit more image grayscale features, a squared global median fitting image (SGMFI) is derived and another fitting energy is similarly constructed using this SGMFI based on SKL. Third, we integrate the above two fitting energies and introduce additional regularized energies. The proposed LSM is verified and compared with several state-of-the-art methods on real SAR images. The river channel extraction results indicate that our proposed LSM has a clear advantage in accuracy and is robust to level set initialization.

Model-assisted SCAD calibration for non-probability samples

Increasing costs and non-response rates of probability samples have provoked the extensive use of non-probability samples. However, non-probability samples are subject to selection bias, resulting in difficulty for inference. Calibration is a popular method to reduce selection bias in non-probability samples. When rich covariate information is available, a key problem is how to select covariates and estimate parameters in calibration for non-probability samples. In this paper, the model-assisted SCAD calibration is proposed to make population inference from non-probability samples. A parametric model between the study variable and covariates is first established. SCAD is then used to estimate the model parameters based on non-probability samples. The modified forward Kullback–Leibler distance is lastly explored to conduct calibration for non-probability samples based on the estimated parametric model. The theoretical properties of the model-assisted SCAD calibration estimator are further derived. Results from simulation studies show that the model-assisted SCAD calibration estimator yields the smallest bias and mean square error compared with other estimators. Also, a real data from the 2017 Netizen Social Awareness Survey (NSAS) is used to demonstrate the proposed methodology.

Using Privacy Respecting Sound Analysis to Improve Bluetooth Based Proximity Detection for COVID-19 Exposure Tracing and Social Distancing.

We propose to use ambient sound as a privacy-aware source of information for COVID-19-related social distance monitoring and contact tracing. The aim is to complement currently dominant Bluetooth Low Energy Received Signal Strength Indicator (BLE RSSI) approaches. These often struggle with the complexity of Radio Frequency (RF) signal attenuation, which is strongly influenced by specific surrounding characteristics. This in turn renders the relationship between signal strength and the distance between transmitter and receiver highly non-deterministic. We analyze spatio-temporal variations in what we call “ambient sound fingerprints”. We leverage the fact that ambient sound received by a mobile device is a superposition of sounds from sources at many different locations in the environment. Such a superposition is determined by the relative position of those sources with respect to the receiver. We present a method for using the above general idea to classify proximity between pairs of users based on Kullback–Leibler distance between sound intensity histograms. The method is based on intensity analysis only, and does not require the collection of any privacy sensitive signals. Further, we show how this information can be fused with BLE RSSI features using adaptive weighted voting. We also take into account that sound is not available in all windows. Our approach is evaluated in elaborate experiments in real-world settings. The results show that both Bluetooth and sound can be used to differentiate users within and out of critical distance (1.5 m) with high accuracies of 77% and 80% respectively. Their fusion, however, improves this to 86%, making evident the merit of augmenting BLE RSSI with sound. We conclude by discussing strengths and limitations of our approach and highlighting directions for future work.