Newcomb-Benford Law Research Articles

Oversight of Government Spending on Combating Covid-19

The research aims to analyze the federal government's expenditures in 2020 and 2021 related to combating the COVID-19 pandemic in search of fraud indicators. This period was characterized by a sudden increase in public spending, accompanied by less stringent procurement processes. The Newcomb-Benford Law describes the frequency of leading digits in datasets and is presented as an effective tool for identifying anomalies and fraud in financial records, being widely explored in various fields, particularly in audits and accounting. A comparison was made between the observed and expected frequencies for the first, second, and first two digits of the committed values, segregated by year or procurement method. The analysis was conducted through graphs and statistical tests to verify the adherence of distributions to the Newcomb-Benford Law. Commitments made in 2020 showed better adherence than those in 2021, suggesting a higher probability of manipulations or fraud in the second year of the pandemic. Additionally, expenditures resulting from bidding processes were more compliant when compared to non-bid expenditures, confirming the perception that there is a higher likelihood of manipulation or fraud in non-bid spending. The research reveals indications that can be interpreted as warnings for oversight and control bodies, with the proposed approach serving as a mechanism for auditing government expenditures, assisting in the selection of samples that require detailed analysis by auditors and investigator.

Signal processing analysis for detection of anomalies in numerical series

It might be instinctively assumed that the occurrence of the first digit of a randomly selected number is uniformly distributed among 1 to 9. However, the Newcomb–Benford law (NBL), also known as the first-digit law or Benford’s law, reveals a completely different fact: the distributions follows a logarithm-type law. This bias in the frequency of occurrence of the leading digits of numbers is evident in everything from the lengths of rivers, the areas of countries, stock prices, and population numbers. Omnipresence of NBL emerges in mathematics (the Fibonacci sequence, the distribution of prime numbers, etc.) and also in the field of physics (i.e. the half-lives of unstable nuclei). From the earliest years, a large bibliography has been developed and applied to many fields such as fraud detection in finances, international trade and election results analysis. Usually, the diagnosis of such works is reduced to a qualitative binary assessment (conformity or non-conformity). The present paper aims to establish a supervised methodology to address existing gaps and enhance the reliability of final decisions. These gaps include uncertainty about the minimum data volume required to validate Benford’s Law, the low sensitivity of current metrics (Euclidean distance or Mean Absolute Difference) to detect minor levels of adulteration, and the need to quantify both the level and nature of adulteration. Two new procedures have been established coming from Signal Analysis and Processing theory. The first relates the deconvolution between the empirical leading digit distribution and the Benford one. The second procedure consists of obtaining the generating functions (via Hilbert and Fourier transforms) to increase the contrast between both functions: the empirical under research and Benford’s as the reference. Based on calibrated datasets, we are able to quantify the magnitude of accidental errors or malicious fraud adulteration and make the discrimination between both. Finally, as benchmark, fifteen real-world datasets were studied. This study shows that most of datasets are suspicious of Fraud. Notably, in the field of human health, several datasets demonstrate significant fraudulent deviations, defying conventional patterns of fraud and error. This phenomenon has been termed Berserker Fraud.

Epidemiological anomaly detection in Philippine public health surveillance data through Newcomb-Benford analysis.

Public health surveillance is vital for monitoring and controlling disease spread. In the Philippines, an effective surveillance system is crucial for managing diverse infectious diseases. The Newcomb-Benford Law (NBL) is a statistical tool known for anomaly detection in various datasets, including those in public health. Using Philippine epidemiological data from 2019 to 2023, this study applied NBL analysis. Diseases included acute flaccid paralysis, diphtheria, measles, rubella, neonatal tetanus, pertussis, chikungunya, dengue, leptospirosis and others. The analysis involved Chi-square tests, Mantissa Arc tests, Mean Absolute Deviation (MAD) and Distortion Factor calculations. Most diseases exhibited nonconformity to NBL, except for measles. MAD consistently indicated nonconformity, highlighting potential anomalies. Rabies consistently showed substantial deviations, while leptospirosis exhibited closer alignment, especially in 2021. Annual variations in disease deviations were notable, with acute meningitis encephalitis syndrome in 2019 and influenza-like illness in 2023 having the highest deviations. The study provides practical insights for improving Philippine public health surveillance. Despite some diseases showing conformity, deviations suggest data quality issues. Enhancing the PIDSR, especially in diseases with consistent nonconformity, is crucial for accurate monitoring and response. The NBL's versatility across diverse domains emphasizes its utility for ensuring data integrity and quality assurance.

Comparative Analysis of Dynamic Power Consumption of Parallel Prefix Adder

The Newcomb-Benford law, also known as Benford's law, is the law of anomalous numbers stating that in many real-life numerical datasets, including physical and statistical ones, numbers have a small initial digit. Numbers irregularity observed in nature leads to the question, is the arithmetical-logical unit, responsible for performing calculations in computers, optimal? Are there other architectures, not as regular as commonly used Parallel Prefix Adders, that can perform better, especially when operating on the datasets that are not purely random, but irregular? In this article, structures of a propagate-generate tree are compared including regular and irregular configurations—various structures are examined: regular, irregular, with gray cells only, with both gray and black, and with higher valency cells. Performance is evaluated in terms of energy consumption. The evaluation was performed using the extended power model of static CMOS gates. The model is based on changes of vectors, naturally taking into account spatio-temporal correlations. The energy parameters of the designed cells were calculated on the basis of electrical (Spice) simulation. Designs and simulations were done in the Cadence environment and calculations of the power dissipation were performed in MATLAB. The results clearly show that there are PPA structures that perform much better for a specific type of numerical data. Negligent design can lead to an increase greater than two times of power consumption. The novel architectures of PPA described in this work might find practical applications in specialized adders dealing with numerical datasets, such as, for example, sine functions commonly used in digital signal processing.

A novel approach to evaluate data integrity: evidence from COVID-19 in China

The COVID-19 pandemic has generated an unprecedented amount of epidemiological data. Yet, concerns regarding the validity and reliability of the information reported by health surveillance systems have emerged worldwide. In this paper, we develop a novel approach to evaluating data integrity by combining the Newcomb-Benford Law with outlier methods. We demonstrate the advantages of our framework using a case study from China. To ensure more robust findings, we employ multiple diagnostic procedures, including three conformity estimates, four goodness-of-fit tests, and two distance measures (Cook and Mahalanobis). To promote transparency, we have made all computational scripts publicly available. Our findings indicates a significant deviation in the distribution of new deaths from the theoretical expectations of Benford's Law. Importantly, these results remain accurate even when considering alternative model specifications and conducting various statistical tests. Furthermore, the procedures developed here are easily applicable in other areas of knowledge and can be scaled to assess data quality in both the public and private sectors.

Multiple conformity tests to assess deviations from the Newcomb-Benford Law (NBL): A replication of Koch and Okamura (2020)

In this paper, we critically reevaluate Koch and Okamura’s (2020) conclusions on the conformity of Chinese COVID-19 data with Benford’s Law. Building on Figueiredo et al. (2022), we adopt a framework that combines multiple tests, including Chi-square, Kolmogorov-Smirnov, Euclidean Distance, Mean Absolute Deviation, Distortion Factor, and Mantissa Distribution. The primary rationale behind employing multiple tests is to enhance the robustness of our inference. The main finding of the study indicates that COVID-19 infections in China do not adhere to the distribution expected under Benford’s Law, nor does it align with the figures observed in the U.S. and Italy. The usefulness of deviations from Benford’s Law in detecting misreported or fraudulent data remains controversial. However, addressing this question requires a more careful statistical analysis than what is presented in the Koch and Okamura (2020) paper. By employing a combination of several tests using fully transparent procedures, we establish a more reliable approach to evaluating conformity to the Newcomb-Benford Law in applied research.

Analyzing crop production statistics of the Philippines using the newcomb-benford law

The agricultural sector plays a crucial role in the Philippines, contributing significantly to its food security and economic growth. However, the sector faces various challenges that hinder its productivity and growth. To address these challenges and promote sustainable agricultural practices, accurate and reliable crop production statistics are essential for informed decision-making and resource allocation. The Newcomb-Benford Law (NBL) provides a statistical distribution pattern for leading digits in numerical datasets, offering insights into data reliability. In this research study, we applied the NBL to analyze crop production statistics for major crops (rice, corn, coconut, sugarcane, banana, cassava, and pineapple) in the Philippines. We assessed the conformity of the datasets to NBL expectations and identified significant deviations, indicating potential data accuracy issues, collection method discrepancies, or reporting irregularities. Although these deviations do not conclusively suggest fraud, they underscore the need for meticulous data validation. The first two-digit test further strengthened the findings, providing a comprehensive understanding of dataset conformity. Transparent data collection and validation processes are crucial for trustworthy agricultural statistics, supporting effective policy-making and resource allocation. Future research should investigate the root causes of the deviations, explore data processing errors, and implement stringent data validation procedures. Additionally, expanding the analysis to include other crops and conducting comparative studies between regions and time periods would enhance data integrity and contribute to sustainable agricultural practices and food security in the Philippines.

The limits of conformity analysis under the Newcomb-Benford law and the COVID-19 pandemic in Brazil

This research investigated the possible divergences in results due to the use of the Newcomb-Benford Law to assess the conformity of information regarding infections and deaths caused by the SARS-CoV-2 coronavirus, from transversal clippings made two years after the arrival of the COVID-19 pandemic in Brazil, through the comparative analysis of the results of statistics related to the Z test, the Chi-square test (X2), the Kolmogorov-Smirnov test (KS), and also the mean absolute deviation (MAD). For this, two distinct samples were analyzed: one referring to data accumulated over two years of the pandemic in Brazil, from 02/25/2020 to 02/25/2022; and another exclusively to data from the day the pandemic was completed two years of its arrival in Brazil, therefore exclusively on 02/25/2022. Thus, it was possible to observe that the Brazilian epidemiological information system, governed by the health policies adopted by the Government, did not function properly during the COVID-19 pandemic, and still, there was no improvement in quality, even after two years of the pandemic. Additionally, evidence was observed that the data series relating to the contaminations and deaths caused by the SARS-CoV-2 coronavirus present behavior distinct from the expected frequency distributions according to the Newcomb-Benford Law, either as a result of a natural process or due to particular issues of the Brazilian information system itself, for example, logistics, data distortions, forms of geographic aggregation, errors and/or negligence, among other factors.

Assessing the quality of dengue data in the Philippines using Newcomb-Benford law

Accurate and reliable data are vital for effective disease surveillance and control. This study examined the application of the Newcomb-Benford Law (NBL) as a tool for assessing the quality of dengue cases data in the Philippines. Large-scale datasets from the Epidemiology and Disease Control Surveillance (EDCS) and Philippine Integrated Disease Surveillance and Response (PIDSR) reports were analyzed to determine if the observed leading digit distributions deviate significantly from the expected NBL distribution. The statistical tests employed include the chi-squared test, Mantissa Arc Test, Mean Absolute Deviation (MAD), and distortion factor. The results reveal notable deviations from the expected NBL distribution, particularly in digits 1, 3, 4, 6, and 8, indicating potential irregularities and inconsistencies in the reported data. Factors contributing to these deviations may include data manipulation, measurement errors, and sampling biases. Improving data quality and integrity is crucial to ensure accurate disease surveillance.

COVID-19 data, mitigation policies and Newcomb–Benford law

We study how reliable are available data on COVID-19 cases and deaths in different countries. Our analysis is based on a modification of the law of anomalous numbers, the Newcomb–Benford law, applied to the daily number of deaths and new cases in each country. We first revisit the Newcomb–Benford law and show how to avoid false negative compliance with the data. We then compare the χ2 statistics deviation from this law to a number of social and economic indices for each country by computing the Spearman rank-order correlation between them and the χ2 deviation. We considered the proportion of excess deaths for those countries with sufficient available data for a good estimate and obtained similar results: less democratic, less transparent, and more corrupt countries tend to have data of lesser quality. We also discussed the limitations of the present approach and which countries the results were significant for.

Newcomb–Benford Law as a generic flag for changes in the derivation of long-term solar terrestrial physics timeseries

Abstract The Newcomb–Benford Law (NBL) prescribes the probability distribution of the first digit of variables which explore a broad range under conditions including aggregation. Long-term space weather relevant observations and indices necessarily incorporate changes in the contributing number and types of observing instrumentation over time and we find that this can be detected solely by comparison with the NBL. It detects when upstream solar wind magnetic field high resolution OMNI interplanetary magnetic field incorporated new data from the WIND and Advanced Composition Explorer spacecraft after 1995. NBL comparison can detect underlying changes in the geomagnetic auroral electrojet index (activity-dependent background subtraction) and the SuperMAG electrojet index (different station types) that select individual stations showing the largest deflection, but not where station data are averaged, as in the SuperMAG ring-current index. As composite indices become more widespread across the geosciences, the NBL may provide a generic, data processing-independent flag indicating changes in the constituent raw data, calibration, or sampling method.

“Won’t get fooled again”: statistical fault detection in COVID-19 Latin American data

BackgroundClaims of inconsistency in epidemiological data have emerged for both developed and developing countries during the COVID-19 pandemic.MethodsIn this paper, we apply first-digit Newcomb-Benford Law (NBL) and Kullback-Leibler Divergence (KLD) to evaluate COVID-19 records reliability in all 20 Latin American countries. We replicate country-level aggregate information from Our World in Data.ResultsWe find that official reports do not follow NBL’s theoretical expectations (n = 978; chi-square = 78.95; KS = 4.33, MD = 2.18; mantissa = .54; MAD = .02; DF = 12.75). KLD estimates indicate high divergence among countries, including some outliers.ConclusionsThis paper provides evidence that recorded COVID-19 cases in Latin America do not conform overall to NBL, which is a useful tool for detecting data manipulation. Our study suggests that further investigations should be made into surveillance systems that exhibit higher deviation from the theoretical distribution and divergence from other similar countries.

First Digits’ Shannon Entropy

Related to the letters of an alphabet, entropy means the average number of binary digits required for the transmission of one character. Checking tables of statistical data, one finds that, in the first position of the numbers, the digits 1 to 9 occur with different frequencies. Correspondingly, from these probabilities, a value for the Shannon entropy H can be determined as well. Although in many cases, the Newcomb–Benford Law applies, distributions have been found where the 1 in the first position occurs up to more than 40 times as frequently as the 9. In this case, the probability of the occurrence of a particular first digit can be derived from a power function with a negative exponent p > 1. While the entropy of the first digits following an NB distribution amounts to H = 2.88, for other data distributions (diameters of craters on Venus or the weight of fragments of crushed minerals), entropy values of 2.76 and 2.04 bits per digit have been found.

ANALYSIS OF COVID-19 CONTAMINATION AND DEATHS CASES IN BRAZIL ACCORDING TO THE NEWCOMB-BENFORD

The use of the Newcomb-Benford Law in assessing the quality of health and / orepidemiological information systems can allow relevant decisions to be made to improve these systems. In this context, this research aimed to carry out an assessment of the conformity of theinformation regarding the number of cases of contamination and deaths by COVID-19 in Brazil according to the Newcomb-Benford Law, from the moment of the occurrence of the first case of the disease and from the first death by COVID-19 in the country until the month of September 2020. With the aid of descriptive statistics and the use of metrics related to the Z test and themean absolute deviation it was possible to observe that, both from a national and longitudinal perspective as for the transversal-state perspective, the quantitative data referring to the cases of contamination by the coronavirus and the deaths that occurred as a result of COVID-19 did not present the expected behavior according to the Newcomb-Benford Law. Due to the lack of conformity in relation to the Newcomb-Benford Law, it is suspected that some level of conformity specific to this type of data has occurred, in the Brazilian context, since there are already studies that suggest the existence of proper levels of conformity for certain types of data.

Analyzing Benford’s Law’s Powerful Applications in Image Forensics

The Newcomb–Benford law states that in a set of natural numbers, the leading digit has a probability distribution that decays logarithmically. One of its major applications is the JPEG compression of images, a field of great interest for domains such as image forensics. In this article, we study JPEG compression from the point of view of Benford’s law. The article focuses on ways to detect fraudulent images and JPEG quality factors. Moreover, using the image’s luminance channel and JPEG coefficients, we describe a technique for determining the quality factor with which a JPEG image is compressed. The algorithm’s results are described in considerably more depth in the article’s final sections. Furthermore, the proposed idea is applicable to any procedure that involves the analysis of digital images and in which it is strongly suggested that the image authenticity be verified prior to beginning the analyzing process.

Using the Newcomb\u2013Benford law to study the association between a country\u2019s COVID-19 reporting accuracy and its development

The COVID-19 pandemic has spurred controversies related to whether countries manipulate reported data for political gains. We study the association between accuracy of reported COVID-19 data and developmental indicators. We use the Newcomb–Benford law (NBL) to gauge data accuracy. We run an OLS regression of an index constructed from developmental indicators (democracy level, gross domestic product per capita, healthcare expenditures, and universal healthcare coverage) on goodness-of-fit measures to the NBL. We find that countries with higher values of the developmental index are less likely to deviate from the Newcomb-Benford law. The relationship holds for the cumulative number of reported deaths and total cases but is more pronounced for the death toll. The findings are robust for second-digit tests and for a sub-sample of countries with regional data. The NBL provides a first screening for potential data manipulation during pandemics. Our study indicates that data from autocratic regimes and less developed countries should be treated with more caution. The paper further highlights the importance of independent surveillance data verification projects.

Estimating the epidemic growth dynamics within the first week

Information about the early growth of infectious outbreaks is indispensable to estimate the epidemic spreading. A large number of mathematical tools have been developed to this end, facing as much large number of different dynamic evolutions, ranging from sub-linear to super-exponential growth. Of course, the crucial point is that we do not have enough data during the initial outbreak phase to make reliable inferences. Here we propose a straightforward methodology to estimate the epidemic growth dynamic from the cumulative infected data of just a week, provided a surveillance system is available over the whole territory. The methodology, based on the Newcomb-Benford Law, is applied to the Italian covid 19 case-study. Results show that it is possible to discriminate the epidemic dynamics using the first seven data points collected in fifty Italian cities. Moreover, the most probable approximating function of the growth within a six-week epidemic scenario is identified.

Newcomb-Benford law analysis on COVID-19 daily infection cases and deaths in Indonesia and Malaysia

Each country has been racing to contain the spread of COVID-19. The published data of daily infection and death cases can be used to measure the effectiveness of the control interventions. We focus our study in two Southeast Asia countries: Indonesia and Malaysia during period between March and November 2020. Newcomb-Benford law has been commonly used to analyze the probabilities of the first significant digits in natural occurrences since the late 19th century. It is a prominent statistical tool for its capability to detect frauds in datasets. A chi-squared test was recruited to quantify the closeness of the data and Newcomb-Benford law distributions. The results revealed that the distributions of daily infection and death cases in Indonesia followed Newcomb-Benford law while the opposite results were obtained for Malaysia. We have done the analysis of verifying the daily COVID-19 infection and death cases in Indonesia and Malaysia using Newcomb-Benford law. It can be inferred that, between March and November 2020, the control interventions in Indonesia was less effective compared to Malaysia.

The Newcomb–Benford law: Scale invariance and a simple Markov process based on it

The Newcomb–Benford law, also known as the first-digit law, gives the probability distribution associated with the first digit of a dataset so that, for example, the first significant digit has a probability of 30.1% of being 1 and 4.58% of being 9. This law can be extended to the second and next significant digits. This article presents an introduction to the discovery of the law and its derivation from the scale invariance property as well as some applications and examples. Additionally, a simple model of a Markov process inspired by scale invariance is proposed. Within this model, it is proved that the probability distribution irreversibly converges to the Newcomb–Benford law, in analogy to the irreversible evolution toward equilibrium of physical systems in thermodynamics and statistical mechanics.

Estimating the Epidemic Growth Dynamics within the First Week

Information about the early growth of infectious outbreaks is indispensable to estimate the epidemic spreading. A large number of mathematical tools have been developed to this end, facing as much large number of different dynamic evolutions, ranging from sub-linear to super-exponential growth. Of course, the crucial point is that we do not have enough data during the initial outbreak phase to make reliable inferences. Here we propose a straightforward methodology to estimate the epidemic growth dynamic from the cumulative infected data of just a week, provided a surveillance system is available over the whole territory. The methodology, based on the Newcomb-Benford Law, is applied to the Italian covid 19 case-study. Results show that it is possible to discriminate the epidemic dynamics using the first seven data points collected in fifty Italian cities. Moreover, the most probable approximating function of the growth within a six-week epidemic scenario is identified.