A continuous binning for discrete, sparse and concentrated observations

Rafael Prieto Curiel,Carmen Cabrera Arnau,Mara Torres Pinedo,Humberto González Ramírez,Steven Richard Bishop

doi:10.1016/j.mex.2019.10.020

Abstract

Discrete observations from data which are obtained from sparse, and yet concentrated events are often observed (e.g. road accidents or murders). Traditional methods to compute summary statistics often include placing the data in discrete bins but for this type of data this approach often results in large numbers of empty bins for which no function or summary statistic can be computed.Here, a method for dealing with sparse and concentrated observations is constructed, based on a sequence of non-overlapping bins of varying size, which gives a continuous interpolation of data for computing summary statistics of the values for the data, such as the mean.The method presented here overcomes the problem which sparsity and concentration present when computing functions to represent the data. Implementation of the method presented here is facilitated via open access to the code.•A new method for computing functions over sparse and concentrated data is constructed.•The method allows straightforward functions to be computed over partitions of the data, such as the mean, but also more complicated functions, such as coefficients, ratios, correlations, regressions and others.

Highlights

The method presented here overcomes the problem which sparsity and concentration present when computing functions to represent the data
The cafe wants to investigate how much money customers spend at different times in the day
They have records which give the time and the amount paid by each customer

Summary

Method Article

Rafael Prieto Curiela,*, Carmen Cabrera Arnaub, Mara Torres Pinedoc, Humberto González Ramírezd, Steven Richard Bishopb a Mathematical Institute, University of Oxford, United Kingdom b Mathematics Department, University College London, United Kingdom c Institute for Global Prosperity, University College London, United Kingdom d École Nationale des Travaux Publics de l'État, ENTPE, Universiteé de Lyon 2, France

Introduction

Findings

Method