Traditional Data Sources Research Articles

A Method for Conveying Confidence in iNaturalist Observations: A Case Study Using Non-Native Marine Species.

Concerns and limitations relating to data quality, reliability and accuracy hamper the use of citizen science initiatives in research and conservation. Valued for their cost-effective and large data acquisition potential, citizen science platforms such as iNaturalist have been highlighted as beneficial tools to supplement monitoring using traditional data sources. However, intrinsic uncertainties in unverified observations stem from the nature of species being identified, the quality of uploaded media and georeferencing; these factors can limit the value of the data as they can result in inaccurate records. Verification of data prior to use is critical. This process can, however, be laborious and time-consuming, with bias associated with the individual responsible for the task. To address this challenge this study developed a protocol for assigning confidence in iNaturalist observations, using marine alien and cryptogenic species observations from South Africa as a case study. A positive relationship was found between the accuracy of observations and confidence score. The inherent data quality assessment in iNaturalist, termed quality grade, was found to be an inadequate proxy for accuracy. The results of this study highlight the importance of the expert verification phase when using citizen science data. The confidence score facilitates a streamlined approach to the verification process by reducing the time taken to validate records, while assessing the three levels of uncertainty within observations and reducing researcher bias. It is recommended that this confidence score be used as an essential tool when using citizen science derived data.

Combating COVID-19 health disparities in Black, Indigenous, and People of Colour Communities-A call for critical systems thinking.

The Black, Indigenous, and People of Colour populations in the United States are disproportionately affected by the emerging health threat SARS-CoV-2, which causes COVID-19. This paper aims to demonstrate the usefulness of critical systems thinking by using scenario planning based on epidemiological data and tying epidemiology with soft systems methodology to investigate COVID-19 disparities among disproportionately affected Black, Indigenous, and People of Colour populations. Using a review of the COVID-19 literature and publicly available US COVID-19 data, critical systems thinking is applied in a scenario planning example and a call to link soft systems methodology with epidemiology. According to the four plausible Endgame scenarios, levels of community transmission as well as the current state transmission are based on the driving forces of the scenarios. In addition, soft systems methodology explores the effect on stakeholders and strengthens the picture of disease burden beyond sole reliance on traditional data sources. This analysis underscores employing critical systems thinking to critically assess diverse methods appropriate for the ongoing complexity of global crises. It is argued that critically engaged subjectivity should be given space alongside data-dependent objectivity. COVID-19 disparities are reliant on the social determinants of health's effects as driving forces on disease transmission in Black, Indigenous, and People of Colour populations. It is moreover argued that critical systems thinking is demonstrated by linking epidemiological evidence with scenario planning and soft systems methodology. This in turn supports a critical systems thinking approach to uncover the state of health disparities among minoritized communities under COVID-19.

Community-Engaged Survey Approach to Pandemic Impacts on Marginalized Communities, Massachusetts, 2020-2021.

Objectives. To describe how an innovative, community-engaged survey illuminated previously unmeasured pandemic inequities and informed health equity investments. Methods. The methodological approach of Massachusetts' COVID-19 Community Impact Survey, a cross-sectional online survey, was driven by key health equity principles: prioritizing community engagement, gathering granular and intersectional data, capturing root causes, elevating community voices, expediting analysis for timeliness, and creating data-to-action pathways. Data collection was deployed statewide in 11 languages from 2020 to 2021. Results. The embedded equity principles resulted in a rich data set and enabled analyses of populations previously undescribed. The final sample included 33 800 respondents including unprecedented numbers of populations underrepresented in traditional data sources. Analyses indicated that pandemic impacts related to basic needs, discrimination, health care access, workplace protections, employment, and mental health disproportionately affected these priority populations, which included Asian American/Pacific Islanders and parents. Conclusions. Equity-centered data approaches allow for analyses of populations previously invisible in surveillance data, enable more equitable public health action, and are both possible and necessary to deploy in state health departments. (Am J Public Health. 2024;114(S7):S599-S609. https://doi.org/10.2105/AJPH.2024.307800).

Assessing Timely Migration Trends Through Digital Traces: A Case Study of the UK Before Brexit

Digital trace data presents an opportunity for promptly monitoring shifts in migrant populations. This contribution aims to determine whether the number of European migrants in the United Kingdom (UK) declined between March 2019 and March 2020, using weekly estimates derived from the Facebook Advertising Platform. The collected data is disaggregated according to age, level of education, and country of origin. To examine the fluctuation in the number of migrants, a simple Bayesian trend model is employed, incorporating indicator variables for age, education, and country. The Facebook data indicates a downward trend in the number of European migrants residing in the UK. This result is further confirmed by the data from the Labour Force Survey. Notably, the outcomes reveal that in the run-up to Brexit, the most significant decline occurred among the age group of 20 to 29 years old – the largest migrant group – and the tertiary educated. This analyses could not be implemented with traditional data sources such as the Labour Force Survey, because this level of disaggregation is not provided. However, there are also important limitations associated with digital trace data, such as algorithm changes and representativeness. These limitations need to be addressed by employing sound statistical methodologies. Nevertheless, this research shows the potential of digital trace data in anticipating migration trends at a timely granularity and informing policymakers.

Enhancing credit scoring accuracy with a comprehensive evaluation of alternative data.

This study explores the potential of utilizing alternative data sources to enhance the accuracy of credit scoring models, compared to relying solely on traditional data sources, such as credit bureau data. A comprehensive dataset from the Home Credit Group's home loan portfolio is analysed. The research examines the impact of incorporating alternative predictors that are typically overlooked, such as an applicant's social network default status, regional economic ratings, and local population characteristics. The modelling approach applies the model-X knockoffs framework for systematic variable selection. By including these alternative data sources, the credit scoring models demonstrate improved predictive performance, achieving an area under the curve metric of 0.79360 on the Kaggle Home Credit default risk competition dataset, outperforming models that relied solely on traditional data sources, such as credit bureau data. The findings highlight the significance of leveraging diverse, non-traditional data sources to augment credit risk assessment capabilities and overall model accuracy.

Charting the evolutionary conceptual pathway of analytic network process research: A main path analysis

AbstractThis study delves into the flow and structure of knowledge prevalent in Analytic Network Process (ANP) literature. It traces the evolutionary pathway of ANP research, spotlighting seminal works and unfolding the development of topics over time. Analysing 4583 ANP literature records from 1986 to 2022, this research utilizes two pivotal data sources: Web of Science (WoS) and Dimensions. This main path analysis (MPA) approach is coupled with techniques such as citation analysis, social network analysis, and centrality measures to identify most influential papers, as well as using a search path algorithm (SPC) to provide higher weights to papers that serve as bridges connecting different research paths rather than simply counting all citations equally. Key works emerged as focal points in both datasets, including notable contributions from Lee, Kheybari and Büyüközkan, with the Dimensions dataset uniquely highlighting Saaty as the initial conceptual knowledge source of the ANP. The centrality measures unearthed persistent bridge roles undertaken by key contributors such as Saaty and Sarkis over different time periods. Furthermore, a detailed topic analysis discerned eight distinctive thematic clusters among the two data sources: analytical network process, network process, fuzzy analytic, evaluation laboratory, supply chain, quality function, hierarchy process, and information systems. These clusters emerged, providing a comprehensive map of ANP topics and their evolution. The study offers a comprehensive framework for scholars to navigate the expansive domain of ANP research. Identifying pivotal works and emergent themes provides direction for future research endeavours. It allows practitioners to gain invaluable insights into the foundational and transformative works in the ANP realm, enhancing their understanding and application of the methodology. This paper proposes of a dual methodological strategy: First, by using traditional academic data sources (WoS) along with a more encompassing Dimensions database; and second, by using a SPC algorithm to weight MPA papers based on their connecting role among different research paths. At the discipline level, this study carves out a novel vantage point in understanding the ANP research landscape. It offers an integrated portrayal of the field's conceptual evolution, identifying specific cluster topics as an indispensable resource for scholars, practitioners, and students.

Using human mobility data to detect evacuation patterns in hurricane Ian

ABSTRACT Hurricane Ian in 2022 was a destructive category 4 Atlantic hurricane striking the state of Florida, which caused hundreds of deaths and injuries, catastrophic property damage, and an economic loss of more than $112 billion. Before the landfall of Ian in Florida, the state government issued evacuation orders in high-risk zones to reduce casualties and injuries. However, there is limited data available to monitor the actual evacuation patterns and compliance with the evacuation orders at a large geographic scale. This study utilizes human mobility data (i.e. SafeGraph Weekly Pattern) to analyse the spatial patterns of evacuation during Hurricane Ian in 2022. The objectives of the study include three key aspects: 1) proposing an analytical workflow that utilizes human mobility data to detect mobility patterns in disasters and other emergency events; 2) identifying significant evacuation patterns, and 3) revealing the spatial variations in the compliance with evacuation orders in the affected areas. Using data science and spatial analysis techniques, this study detected notable changes in population movements, both within Florida and nationwide, which are potentially linked to the hurricane-induced population evacuation. The distance decay pattern of population flows from Florida demonstrates a propensity for individuals to relocate to nearby areas during the hurricane. Furthermore, the increase in population outflows from the impacted areas suggests the effectiveness of mandatory evacuation orders. A more pronounced increase in outflows from designated mandatory evacuation areas points to the public awareness of the evacuation zone designation. This study provides large-scale, fine-resolution analysis of evacuation behaviours in natural disasters which cannot be easily detected in traditional data sources. The analytical workflows provide actionable tools for government agencies and policymakers to evaluate the effectiveness of evacuation orders and improve evacuation plans in future disasters.

PANDEM-Source, a tool to collect or generate surveillance indicators for pandemic management: a use case with COVID-19 data.

PANDEM-Source (PS) is a tool to collect and integrate openly available public health-related data from heterogeneous data sources to support the surveillance of infectious diseases for pandemic management. The tool may also be used for pandemic preparedness by generating surveillance data for training purposes. It was developed as part of the EU-funded Horizon 2020 PANDEM-2 project during the COVID-19 pandemic as a result of close collaboration in a consortium of 19 partners, including six European public health agencies, one hospital, and three first responder organizations. This manuscript describes PS's features and design to disseminate its characteristics and capabilities to strengthen pandemic preparedness and response. A requirement-gathering process with EU pandemic managers in the consortium was performed to identify and prioritize a list of variables and indicators useful for surveillance and pandemic management. Using the COVID-19 pandemic as a use case, we developed PS with the purpose of feeding all necessary data to be displayed in the PANDEM-2 dashboard. PS routinely monitors, collects, and standardizes data from open or restricted heterogeneous data sources (users can upload their own data). It supports indicators and health resources related data from traditional data sources reported by national and international agencies, and indicators from non-traditional data sources such as those captured in social and mass media, participatory surveillance, and seroprevalence studies. The tool can also calculate indicators and be used to produce data for training purposes by generating synthetic data from a minimal set of indicators to simulate pandemic scenarios. PS is currently set up for COVID-19 surveillance at the European level but can be adapted to other diseases or threats and regions. With the lessons learnt during the COVID-19 pandemic, it is important to keep building capacity to monitor potential threats and develop tools that can facilitate training in all the necessary aspects to manage future pandemics. PS is open source and its design provides flexibility to collect heterogeneous data from open data sources or to upload end users's own data and customize surveillance indicators. PS is easily adaptable to future threats or different training scenarios. All these features make PS a unique and valuable tool for pandemic management.

Multilateral indices in official price statistics and a new additive splicing method

A substitution of the traditional data sources allows National Statistical Institutes to collect large samples of product items with corresponding prices recorded over more frequent time periods. For some product categories, a frequent replacement of product items requires a dynamic approach to define a product sample that can be achieved by a monthly update of the sample. In a month-on-month comparison of price changes, bilateral indices must be chained that can lead to a chain drift bias. Multilateral methods are designed to reduce the chain drift, albeit not completely eliminate it. A disadvantage of multilateral methods is its ambiguous interpretability and a revision problem. The main objective of this paper is to show an implementation of multilateral methods for official price statistics in practice. The multilateral GEKS method is decomposed into elementary components that identifies the effect of successive price relatives used in its compilation. An importance of the decomposition is shown when interpreting the GEKS index in the context of official price statistics. We demonstrate that it can lead to an inaccurate public dissemination of the consumer price index. Moreover, we propose a new method for the splice extension that is required to avoid the revision of previously published indices. The method uses a Stein-type shrinkage estimator for the splice extension that is applied additively to the last published index. In the empirical study, we use web scraped data, which are obtained from the Slovak market, for comparison of bilateral and multilateral indices.

A Cloud-native Approach for Processing of Crowdsourced GNSS Observations and Machine Learning at Scale: A Case Study from the CAMALIOT Project

The era of modern smartphones, running on Android version 7.0 and higher, facilitates nowadays acquisition of raw dual-frequency multi-constellation GNSS observations. This paves the way for GNSS community data to be potentially exploited for precise positioning, GNSS reflectometry or geoscience applications at large. The continuously expanding global GNSS infrastructure along with the enormous volume of prospective GNSS community data bring, however, major challenges related to data acquisition, its storage, and subsequent processing for deriving various parameters of interest. In addition, such large datasets cannot be managed manually anymore, leading thus to the need for fully automated and sophisticated data processing pipelines. Application of Machine Learning Technology for GNSS IoT data fusion (CAMALIOT) was an ESA NAVISP Element 1 project (NAVISP-EL1-038.2) with activities aiming to address the aforementioned points related to GNSS community data and their exploitation for scientific applications with the use of Machine Learning (ML). This contribution provides an overview of the CAMALIOT project with information on the designed and implemented cloud-native software for GNSS processing and ML at scale, developed Android application for retrieving GNSS observations from the modern generation of smartphones through dedicated crowdsourcing campaigns, related data ingestion and processing, and GNSS analysis concerning both conventional and smartphone GNSS observations. With the use of the developed GNSS engine employing an Extended Kalman Filter, example processing results related to the Zenith Total Delay (ZTD) and Slant Total Electron Content (STEC) are provided based on the analysis of observations collected with geodetic-grade GNSS receivers and from local measurement sessions involving Xiaomi Mi 8 that collected GNSS observations using the developed Android application. For smartphone observations, ZTD is derived in a differential manner based on a single-frequency double-difference approach employing GPS and Galileo observations, whereas satellite-specific STEC time series are obtained through carrier-to-code leveling based on the geometry-free linear combination of observations from both GPS and Galileo constellations. Although the ZTD and STEC time series from smartphones were derived on a demonstration basis, a rather good level of consistency of such estimates with respect to the reference time series was found. For the considered periods, the RMS of differences between the derived smartphone-based time series of differential zenith wet delay and reference values were below 3.1 mm. In terms of satellite-specific STEC time series expressed with respect to the reference STEC time series, RMS of the offset-reduced differences below 1.2 TECU was found. Smartphone-based observations require special attention including additional processing steps and a dedicated parameterization in order to be able to acquire reliable atmospheric estimates. Although with lower measurement quality compared to traditional sources of GNSS data, an augmentation of ground-based networks of fixed high-end GNSS receivers with GNSS-capable smartphones would however, form an interesting source of complementary information for various studies relying on GNSS observations.

Synthetic data protection: Towards a paradigm change in data regulation?

Synthetic data generated through machine learning algorithms from original real-world data is gaining prominence across sectors due to their potential to provide privacy-preserving alternatives to traditional data sources. However, recent studies have raised concerns about the re-identification risks of synthetic data. This article examines the legal challenges surrounding synthetic data protection, with a focus on the European Union's General Data Protection Regulation (GDPR). After briefly explaining the methods of synthetic data generation and discussing their potential for privacy preservation, the article analyses the shortcomings of the personal/non-personal dualist approach under the GDPR. It then assesses the possibility of a paradigm change in data protection legislation, moving beyond this binary categorisation. The article argues in favour of establishing clear guidelines for the generation and processing of synthetic data, prioritising the principles of transparency, accountability and fairness.

Augmenting business statistics information by combining traditional data with textual data: a composite indicator approach

Combining traditional and digital trace data is an emerging trend in statistics. In this respect, new data sources represent the basis for multi-purpose extraction of different statistical indicators, which contribute to augmenting the statistical information, for feeding smart statistics. The production of business statistics can benefit from the use of unstructured data, especially to study novel aspects which are not covered by traditional data sources. This paper proposes a methodological general framework for augmenting information by combining data, both structured and non structured. The statistical challenges of using unstructured data and their integration with traditional data are discussed. The methodological general framework is applied to the construction of smart composite indicators using social media data and their metadata. An empirical exercise illustrates how to apply the methodology in practice.

A snapshot of injecting drug consumption from the analysis of used syringes within the Medically Supervised Injecting Centre in Sydney, Australia.

The administration of illicit drugs by injection is associated with considerable harm, including an increased risk of overdose. The chemical analysis of used syringes can enhance knowledge on injecting drug consumption beyond traditional data sources (self-report surveys). This additional information may be useful during significant global events like the COVID-19 pandemic. This study aimed to examine a snapshot of the drugs injected at the Medically Supervised Injecting Centre (MSIC) in Sydney, Australia, in 2019-2020. Used syringes were collected from MSIC across three periods throughout 2019 and 2020 (February 2019, March-April 2020 and June-September 2020). Drug residues were extracted from used syringes using methanol before detection by gas chromatography-mass spectrometry and ultra-performance liquid chromatography-tandem mass spectrometry. The chemical analysis results were compared to self-report data obtained from MSIC clients. Heroin (46-53%), methamphetamine (24-34%) and pharmaceutical opioids (15-27%) were the most common drug residues detected. The chemically detected drugs had declining coherence with the drugs self-reported by MSIC clients across the time periods examined. There was no significant change in the drugs injected (heroin, methamphetamine and pharmaceutical opioids) across the three periods collected throughout varying COVID-19 lockdown restrictions. Changes in the frequency of other drugs injected and discrepancies between chemical analysis and self-report were potentially related to regulatory changes, degradation or misinformed sales. Routine chemical analysis of used syringes has provided an alternative information source to promote awareness of current drug trends and aid harm reduction.

Web scraping for price statistics in the Philippines

Official price statistics in the Philippines are mainly sourced from the conduct of regular surveys and censuses which entail high costs. As businesses move into digital platforms, alternatives to these traditional data sources have become more available; one of which is web scraping, a process of collecting information from the web. As digital and online platforms become increasingly utilized for commerce, web scraping offers a way to increase the frequency of data collection while reducing its cost compared to price surveys. This paper provides a survey of experiences of various government statistical agencies in their conduct of web scraping for the Consumer Price Index (CPI). Moreover, it details the Philippines’ experience using web scraped data to estimate the food and alcoholic beverages CPI of the National Capital Region in the Philippines, and that is compared to the official CPI estimate of the Philippine Statistics Authority. Finally, this paper discusses the challenges encountered and the recommendations for enhancing the approach.

Tracking Socio-Economic Development in Rural India over Two Decades Using Satellite Imagery

Longitudinal analysis of socio-economic development at sub-national scales can reveal valuable insights about which areas tend to develop faster than others and why. However, such analysis is difficult to conduct with traditional data sources such as censuses and surveys which are not repeated frequently and may require assumptions for imputation of values at non-surveyed locations. Indicators of socio-economic development based on satellite data have emerged as a proxy to track development at fine spatial and temporal scales. We build a model using daytime and nightlights satellite data to estimate an index of socio-economic development at the village level in India. We evaluate our model for temporal robustness and use it to produce estimates at three time points over a two-decade period. We then use these estimates to understand the effect on village-level development of factors such as the geographic distance of a village to hubs of economic activity and the inequality of development in the district. Our findings provide evidence of the possible impact that policy changes during this period have had on village development.

ODRA2D – TWO-DIMENSIONAL CASCADE FLOOD MODELLING FOR THE ODRA RIVER – DEVELOPMENT AND APPLICATIONS

Aim of the study: Development of new hydrodynamic numerical tools for simulations of the flow in the Odra River valley for the flood hazard assessment and management Material and methods: Use of the MIKE21 software for models building. Geospatial (ArcGis) topographical data processing and analysis for model inputs and outputs. Extensive use of remote sensing data (digital terrain model and orthophoto maps) combined with traditional data sources. Results and conclusions: A cascade of twenty five MIKE21 models was developed to cover a nearly 600 km long section of the Odra River. The models were used in different studies, including: the assessment of flood hazards with regards to various scenarios, the assessment of flooding due to ice jamming and breaks in embankments, the evaluation of the effectiveness of flood mitigating works within the complex Hydrosystem of the city of Wrocław, the verification of historic flood data and stage-discharge relations, and also the verification of design discharges.

UAV Atmosphere Sounding for Rocket Launch Support.

One of the crucial branches of activity at the Łukasiewicz Research Network-Institute of Aviation is developing a suborbital rocket vehicle capable of launching small payloads beyond the Earth's atmosphere, reaching over 100 km in altitude. Ensuring safety is a primary concern, particularly given the finite flight zone and impact area. Crucial to safety analysis is the wind profile, especially in the very first seconds of a flight, when rocket velocity is of the same order as the wind speed. Traditional near-ground wind data sources, ranging from wind towers to numerical models of the atmosphere, have limitations. Wind towers are costly and unfeasible at many test ranges used for launches, while numerical modeling may not reflect the specific ground profile near the launcher due to their large cell size (2 to +10 km). Meteorological balloons are not favorable for such measurements as they aim to provide the launch operator with a wind profile at high altitudes, and are launched only 1-2 times per flight attempt. Our study sought to prototype a wind measurement system designed to acquire near-ground wind profile data. It focuses on measuring wind direction and speed at near-ground altitudes with higher flight frequency, offering data on demand shortly before launch to help ensure safety. This atmosphere sounding system consists of an Unmanned Aerial Vehicle (UAV) equipped with an onboard ultrasonic wind sensor. Some reports in the literature have discussed the possibility of using UAV-borne anemometers, but the topic of measurement errors introduced by placing the anemometer onboard an UAV remains under studied. Limited research in this area underlines the need for experimental validation of design choices-for specific types of UAVs, anemometers, and mounting. This paper presents a literature review, a detailed overview of the prototyped system, and flight test results in both natural (outdoor) and controlled (indoor, no wind) conditions. Data from the UAV system's anemometer was benchmarked against a stationary reference weather station, in order to examine the influence of the UAV's rotor on the anemometer readings. Our findings show a wind speed Root Mean Square Error (RMSE) of 5 m/s and a directional RMSE of below 5.3° (both averaged for 1 min). The results were also compared with similar UAV-based wind measurements. The prototyped system was successfully used in a suborbital rocket launch campaign, thus demonstrating the feasibility of integrating UAVs with dedicated sensors for performing regular meteorological measurements in automatic mode.

Using natural language processing to characterize and predict homeopathic product-associated adverse events in consumer reviews: comparison to reports to FDA Adverse Event Reporting System (FAERS).

Apply natural language processing (NLP) to Amazon consumer reviews to identify adverse events (AEs) associated with unapproved over the counter (OTC) homeopathic drugs and compare findings with reports to the US Food and Drug Administration Adverse Event Reporting System (FAERS). Data were extracted from publicly available Amazon reviews and analyzed using JMP 16 Pro Text Explorer. Topic modeling identified themes. Sentiment analysis (SA) explored consumer perceptions. A machine learning model optimized prediction of AEs in reviews. Reports for the same time interval and product class were obtained from the FAERS public dashboard and analyzed. Homeopathic cough/cold products were the largest category common to both data sources (Amazon = 616, FAERS = 445) and were analyzed further. Oral symptoms and unpleasant taste were described in both datasets. Amazon reviews describing an AE had lower Amazon ratings (X2 = 224.28, P < .0001). The optimal model for predicting AEs was Neural Boosted 5-fold combining topic modeling and Amazon ratings as predictors (mean AUC = 0.927). Topic modeling and SA of Amazon reviews provided information about consumers' perceptions and opinions of homeopathic OTC cough and cold products. Amazon ratings appear to be a good indicator of the presence or absence of AEs, and identified events were similar to FAERS. Amazon reviews may complement traditional data sources to identify AEs associated with unapproved OTC homeopathic products. This study is the first to use NLP in this context and lays the groundwork for future larger scale efforts.

Big data in crime statistics: Using Google Trends to measure victimization in designated market areas across the United States

Google Trends (GT) data could potentially supplement traditional crime measurement strategies, allowing criminologists to better understand crime statistics on a macro level. This study assesses the validity of GT crime estimates. The findings indicate that GT data are reliable for estimating MVT, larceny, and rape. Additionally, we illustrate how to use GT to identify places with high rates of unreported offenses. The results of this study demonstrate the feasibility of leveraging open-source big data such as GT to supplement traditional sources of crime data, particularly for categories of crime with substantial underreporting rates. Results suggest the GT rape measure may be a more accurate estimate of the true incidence of rape than the measure drawn from the Uniform Crime Report (UCR). The limitations associated with the use of GT to generate estimates of crime are also discussed.

Building an intelligent system to identify trends in agricultural markets

In recent years, scholars and practitioners have discussed several challenges related to the fresh-food supply chains (SCs). One of these challenges is the lack of availability of curated, clean, and processed market information for the SC participants to use to aid in their decision-making process. For example, small growers often feel uncertain because they have little insight into which crops to plant so they will have a market to sell them to. Consequently, some growers are only able to sell parts of their harvest or decide not to harvest at all which causes food waste. This work aims to create a centralized market intelligence platform to acquire market information, curate it, analyze it, and connect this platform to the rest of an emerging intelligent supply chain. It is essential to take into consideration market intelligence to reduce losses related to market prices and demands, avoid scarcity events in which food availability and affordability decrease, and aid small growers by alerting them of future market opportunities A market intelligence layered system framework is developed in this work to collect relevant market indicator data including non-traditional data sources, monitor them, diagnose the market's state, and provide recommendations to the SC participants such as which crops and markets will be most promising in the future. A case study based on the 2019 market event in which celery prices increased across the United States terminal markets is provided. Through the case study, the proposed market intelligence framework is illustrated with focus on the first three layers: data collection, processing, and monitoring. Furthermore, the case study illustrates the usefulness of monitoring non-traditional data to identify market events with greater anticipation. Results indicate that the monitoring of non-traditional data was able to capture the anomalous event with greater time of anticipation than another traditional data source.