Use Of Scanner Data Research Articles

The Use of Data Science in a National Statistical Office

Objective statistical information is vital to an open and democratic society. It provides a solid foundation so that informed decisions can be made by our elected representatives, businesses, unions, and non-profit organizations, as well as individual citizens. There is a great shift towards a more virtual and digital economy and society. The traditional official statistical systems are centered on surveys, and must be adapted to this new digital reality. National statistical offices have been increasingly embracing non-survey data sources along with data science methods to better serve society.This paper provides a blueprint for the application of data science in a government organization. It describes how data science enables innovation and the delivery of new high-value, high-quality, relevant, and trusted products that reflect the ever-evolving needs of our society and economy. We discuss practical operational considerations and impactful data science applications that supported the work of Statistics Canada’s analysts and front-line health agencies during the pandemic. We also discuss the innovative use of scanner data in lieu of survey data for large business respondents in the retail industry. We will describe computer vision methodologies, including machine learning models used to detect the start of buildings construction from satellite imagery, greenhouse area and greenhouse production, as well as crop types detection. Data science and machine learning methods have tremendous potential, and their ethical use is of primary importance. We conclude the paper with a forward-facing view of responsible data science use in statistical production.

The Use of Scanner Data for Economics Research

The adoption of barcode scanning technology in the 1970s gave rise to a new form of data: scanner data. Soon afterwards, researchers began using this new resource, and since then a large number of papers have exploited scanner data. The data provide detailed price, quantity, and product characteristic information for completely disaggregate products at high frequency, and they typically track a panel of stores and/or consumers. Their availability has led to advances, inter alia, in the study of consumer demand, the measurement of market power, firms’ strategic interactions and decision making, the evaluation of policy reforms, and the measurement of price dispersion and inflation. In this article we highlight some of the pros and cons of this data source, and we discuss some of the ways its availability to researchers has transformed the economics literature.

Multilateral Methods, Substitution Bias, and Chain Drift: Some Empirical Comparisons

Multilateral methods are increasingly being used in the computation of official price indexes such as the consumer price index (CPI). This reflects the growing use of scanner data by statistical agencies and the fact that fixed‐based or chained price comparisons perform poorly in this context. A range of multilateral approaches have been pursued by different statistical agencies. Yet, it can be shown that some of these methods, at least theoretically, could suffer from substitution bias. We investigate this as well as the drivers of chain drift. We adopt a simulation‐based approach using actual scanner data prices with the corresponding quantities being generated assuming constant elasticity of substitution (CES) preferences. We find that most methods systematically deviate from the exact CES benchmark index. This is even the case for the superlative index methods, which should not exhibit substitution bias. Interestingly, we also find significant chain drift even in the exact CES indexes. We argue this reflects life cycle pricing and particularly run‐out sales at the end of a product's life.

Tests for Price Indices in a Dynamic Item Universe

Abstract There is generally a need to deal with quality change and new goods in the consumer price index due to the underlying dynamic item universe. Traditionally axiomatic tests are defined for a fixed universe. We propose five tests explicitly formulated for a dynamic item universe, and motivate them both from the perspectives of a cost-of-goods index and a cost-of-living index. None of the indices that are currently available for making use of scanner data satisfies all the tests at the same time. The set of tests provides a rigorous diagnostic for whether an index is completely appropriate in a dynamic item universe, as well as pointing towards the directions of possible remedies. We thus outline a large index family that potentially can satisfy all the tests.

The impact of private label brands on customer loyalty and product category profitability

PurposeThe purpose of this study is to investigate the influence of private label resources possessed by a supermarket retailer on the shopping behavior of loyal customers. The study examines whether or not private label products can help in the overall enhancement of product category performance.Design/methodology/approachThe paper examines the performance of a supermarket retailer in the Northeast United States that operates over 100 stores and generates a total yearly sales volume in excess of $3 billion. Data obtained from the Supermarket's point of sale information were used. The paper then developed a research model from the literature review and used structural equation modeling to analyze the data.FindingsThe findings show that overall dollars spent by loyal customers significantly impacted overall profitability.Research limitations/implicationsThe data collected pertained to the supermarket's grocery department that is comprised of center store dry grocery products, frozen food products, and refrigerated dairy products. Perishable departments such as deli, seafood, meat, bakery, floral, general merchandise, health and beauty care, etc. were not researched in this study. Also, data obtained were from one individual supermarket chain.Practical implicationsAlthough private label products may represent increased profitability for retailers, consumers prefer a full assortment of merchandise; an over emphasis on private label brands may result in diminishing category performance.Originality/valueThe paper examines the performance of a supermarket retailer in the Northeast United States that operates over 100 stores and generates a total yearly sales volume in excess of $3 billion. The use of scanner data has value as it measures actual shopping behavior.

Eliminating chain drift in price indexes based on scanner data

The use of scanner data in the CPI makes it possible to compile superlative price indexes at detailed aggregation levels since prices and quantities are available. A potential drawback is the high attrition rate of items. The usual solution to handle this problem, high-frequency chaining, can create drift in the index series due to price and quantity bouncing arising from sales. Ivancic, Diewert and Fox (2009) have recently proposed an approach that provides drift free, superlative-type indexes through adapting multilateral index number theory. In this paper we apply their proposal to seven product groups and find promising results. We compare the results with those obtained by using the Dutch method to deal with supermarket scanner data.

The Use of Scanner Data for Research into Nutrition*

AbstractData from market research firms are increasingly being used by social science researchers. These data provide potentially useful information, including detailed nutritional information and well‐measured prices, and their panel structure is appealing as it permits researchers to control for unobservable time‐invariant household characteristics and to model dynamic aspects of household behaviour. We summarise the information on the nutrients in foods that is contained in one source of market research data. We show that there is a lot of variation in nutrients at the individual product level, even within narrowly defined food categories such as butter. We also show that the duration of time over which data are collected can have important implications for analysis of household‐level nutrient purchases.

Use of scanner data to analyze the table wine demand in the Italian major retailing trade

AbstractWhile the utilization of scanner data for food demand analyses has become increasingly popular in the United States, few food demand studies, and in particular none on table wine, have been conducted using scanner data in Italy. This paper presents a first attempt to estimate a demand system for selected brands of red tetra‐packaged, plastic packaged, and bag‐in‐box table wine using scanner data providing new and useful insights into the marketing of Italian wine. Price and expenditure elasticities of Italian red table wine demand drawn from a linear almost ideal demand system are provided. Results suggest a partially loyal market of table wine, showing a tendency to substitution across brands and a degree of competition among the leading brands. [EconLit citations: Q110, Q130]. © 2006 Wiley Periodicals, Inc. Agribusiness 22: 391–403, 2006.

Using Scanner Data to Estimate Country Price Parities: A Hedonic Regression Approach

This paper uses scanner data from the bar‐code readers of retailers to provide estimates of inter‐country price parities at the level of the basic heading. The use of such data is appealing given its extensive coverage of transactions, information on weights, prices and characteristics of items at a highly detailed level. The study uses dummy variable hedonic and exact/superlative hedonic index number formulations applied to an inter‐country context for both bilateral and multilateral comparisons. Unlike conventional methods, such methods are not confined to matched samples comparisons and thus make use of the entire sample. Their application extends to price survey data using checklists on characteristics. The application is to scanner data on about 1 million transactions for television sets over two months in three countries. It is, to the authors’ knowledge, the first such use of scanner data and application of the above hedonic frameworks in this context.

The measurement of inflation: Aggregation at the basic level

The advent of bar-code, retail scanner data provides an alternative data source for the compilation of consumer price indexes. This paper outlines the nature and merits of such data for this purpose. The data allows aggregation via superlative index number formulae at a very elementary level and provides superior coverage to conventional sources. However, it has been argued that aggregation bias may still arise if the unit values which feed into the superlative indexes are defined for product items, as opposed to product items for given outlets. We explore for six products the nature and extent of unit value bias by outlet type. For three products we found the level of aggregation did not matter. However, we also found it can matter and established the extent of its effects which were always the aggregation by model only, falling less than by model and outlet. This should help provide a basis for an understanding of the issues at stake in the use of scanner data for the compilation of consumer price indexes.

Empirical Analysis Using Scanner Data: Econometric Issues and Policy Implications

Scanner data, data collected at the store or distribution point through electronic means, are now available for food products and offer new opportunities for empirical research. Large retail chains use scanner data to track price and inventory movement and control product movement along their supply chain. In addition, the two major market tracking firms-A.C. Nielsen and Information Resources, Inc. (IRI)-assemble, collect, and provide two types for market analysis: supermarket/store-level data collected from a nationally representative survey of stores, and household panel data collected at the household level (see Cotterill for a review). The three papers make use of both types of data and illustrate new opportunities and challenges faced in using the scanner data. Chung and Kaiser use household panel data to evaluate the issue of aggregation for cross-section analysis in evaluating consumers' response to advertising programs, and whether the heterogeneity of households (captured by the use of scanner data) leads to differences in estimated response to advertising estimated when more aggregated household data are used. Capps and Love explore fundamental aggregation issues related to appropriate demand system model specification by using national (retail market) scanner data. Cotterill and Samson take advantage of the product and brand detail available in market-level data combined

Commercial Use of UPC Scanner Data: Industry and Academic Perspectives

The authors report the findings from an exploratory investigation of the use of UPC scanner data in the consumer packaged goods industry in the U.S. The study examines the practitioner community's view of the use of scanner data and compares these views with academic research. Forty-one executives from ten data suppliers, packaged goods manufacturers, and consulting firms participated in wide-ranging, in-person, interviews conducted by the authors. The interviews sought to uncover key questions practitioners would like to answer with scanner data, how scanner data is applied to these questions, and the industry's perspective regarding the success that the use of scanner data has had in each area. The authors then compare and contrast practitioners' views regarding the resolution of each issue with academic research. This produces a 2 × 2 classification of each question as “resolved” or “unresolved” from the perspectives of industry and academia. Along the diagonal of the 2 × 2, issues viewed as unresolved by both groups are important topics for future research. Issues deemed resolved by both groups are, correspondingly, of lower priority. In the off-diagonal cells, industry and academics disagree. These topics should be given priority for discussion, information exchange, and possible further research. Practitioners reported that scanner data analysis has had the most success and been most widely adopted for decision making in consumer promotions (i.e., coupons), trade promotions, and pricing. For example, logit and regression models applied to scanner data have revealed very low average consumer response to coupons which has directly led to reduced couponing activity. Managers also reported high levels of comfort with and impact from analyses of trade promotions and price elasticities. While industry views most of the issues in these areas to be resolved, academic research raises concerns about a number of practices in common commercial use. These include price threshold analysis and trade promotion evaluation using baseline and incremental sales. In product strategy, advertising, and distribution management, practitioners reported that the use of scanner data has had more limited development, success, and impact. In the case of new product decisions, scanner data use has been slow to develop due to the inherent limitations of historical data for these decisions and a heavy reliance on traditional primary research methods. In advertising, scanner data is widely analyzed with models, but confusion among practitioners is very high due to controversies about methods (e.g., what level of data aggregation is best) and conflicting results. In distribution and retail management, scanner data use has tremendous potential but a mixed track record to date. Thus, practitioners view the use of scanner data as unresolved for most issues in product strategy, advertising, and distribution. This view is largely, though not entirely, consistent with academic research, which has only begun to address many of the key questions raised by practitioners. In light of the large number of unresolved issues and mixed record of scanner data use to date, the authors offer a series of specific recommendations for immediate and long-term research priorities that are likely to have the greatest impact on commercial utilization of UPC scanner data. Topics of immediate priority include price thresholds and gaps, baseline and incremental sales, base price elasticity, competitive reactions, measurement of advertising effects, management of brand equity, rationalization of product assortments, and category management. Long-term priorities include a greater emphasis on profitability versus sales or market share, developing prescriptive models versus descriptive models, and the need for industry standards.

ELEMENTARY AGGREGATES, MICRO‐INDICES AND SCANNER DATA: SOME ISSUES IN THE COMPILATION OF CONSUMER PRICE INDICES

The paper argues for the use of scanner data from EPOS systems for use in the compilation of consumer price indices. A number of methods of calculating micro‐indices from such data are outlined. Scanner data for colour television sets in the U.K. are used as an example. The Tornqvist chained index is used as a benchmark against which alternative formulations, including those based on representative products, can be judged, the errors often being substantial. The paper argues for the use of scanner data, illustrates methods of compiling micro‐indices and points to the potential for serious errors from conventional methods.

Demand for fresh beef products in supermarkets: A trial with scanner data

This article constitutes a trial of the use of scanner data to investigate the demand for fresh beef products (brisket, chuck, ground, loin, rib, round, and all other beef). Own-price elasticities are negative, statistically significant, and in the elastic range. Own-advertisement elasticities are positive and statistically significant but very inelastic. Cross-price effects and cross-advertisement effects are marginal. Seasonality is evident only for purchases of loin and all other beef. This article documents the utility of scanner data in food demand analysis.

Hedonic Prices and Implicit Markets: Product Differentiation in Pure Competition

A class of differentiated products is completely described by a vector of objectively measured characteristics. Observed product prices and the specific amounts of characteristics associated with each good define a set of implicit or "hedonic" prices. A theory of hedonic prices is formulated as a problem in the economics of spatial equilibrium in which the entire set of implicit prices guides both consumer and producer locational decisions in characteristics space. Buyer and seller choices, as well as the meaning and nature of market equilibrium, are analyzed. Empirical implications for hedonic price regressions and index number construction are pointed out.