Tick Data Research Articles

Characterizing implied volatility functions from agricultural options markets

AbstractWe provide the first comprehensive characterization and comparison of implied volatility functions for five major agricultural options markets—corn, soybeans, soft red winter wheat, live cattle, and feeder cattle—using intraday tick data. Our results show that cattle markets exhibit a distinct leftward skew, which is puzzling and indicates that out‐of‐the‐money traded put options are theoretically overpriced. In contrast, we find that grain market implied volatility functions display a flatter, less pronounced smile pattern. We examine market sentiment induced short‐term hedging pressures using Commodity Futures Trading Commission reports, and market uncertainty around Cattle on Feed reports, as potential causes of the cattle markets skew. However, our results show that the explanatory power of our short‐term hedging pressure proxies are only helpful in isolated cases but overall cannot explain the large skews we observe in cattle markets.

HighFrequencyCovariance: A Julia Package for Estimating Covariance Matrices Using High Frequency Financial Data

High frequency data typically exhibit asynchronous trading and microstructure noise, which can bias the covariances estimated by standard estimators. While a number of specialized estimators have been proposed, they have had limited availability in open source software. HighFrequencyCovariance is the first Julia package which implements specialized estimators for volatility, correlation and covariance using high frequency financial data. It also implements complementary algorithms for matrix regularization. This paper presents the issues associated with exploiting high frequency financial data and describes the volatility, covariance and regularization algorithms that have been implemented. We then demonstrate the use of the package using foreign exchange market tick data to estimate the covariance of the exchange rates between different currencies. We also perform a Monte Carlo experiment, which shows the accuracy gains that are possible over simpler covariance estimation techniques.

Analysis and implementation of realtime stock prediction using reinforcement frameworks

Real time stock prediction is interesting research topic due to the risk involved with volatile scenarios. Modelling of the stocks by reducing the overestimation in ANN model, due to rapid fluctuations in the market guide fund managers risky decisions while building stock portfolio. This paper builds real time framework for stock prediction using deep reinforcement learning to buy, sell or hold the stocks. This paper models the transformed stock tick data and technical indicators using Transformed Deep-Q Learning. Our framework is cost reduced and transaction time optimized to get real time stock prediction using GPU and Memory containers. Stock predictor is architected using GRPC based clean architecture which has the benefits of easy updates, addition of new services with reduced integration costs. Data archive features of the cloud will give benefit of reduced cost of the new stock predictor framework.

Spatio-temporal modelling of tick life-stage count data with spatially varying coefficients.

There is a vast amount of geo-referenced data in many fields of study including ecological studies. Geo-referencing is usually by point referencing; that is, latitudes and longitudes or by areal referencing, which includes districts, counties, states, provinces and other administrative units. The availability of large geo-referenced datasets for modelling has necessitated the development and application of spatial statistical methods. However, spatial varying coefficients models exploring the abundance of tick counts remain limited. In this study we used data that was collected and prepared by researchers in the Department of Biological Sciences from the Old Dominion University, Virginia, USA. We modelled tick life-stage counts and abundance variability from 12 sampling locations, with 5 different habitats (numbered 1-5), three habitat types; namely: woods, edges and grass; collected monthly from May 2009 through December 2018. Spatio-temporal Poisson and spatio-temporal negative binomial (NB) count data models were fitted to the data and compared using the deviance information criteria (DIC). The NB model outperformed the Poisson models with all its DIC values being smaller than those of the Poisson model. Results showed that the covariates varied spatially across counties. There was a decreasing time (in years) effect over the study period. However, even though the time effect was decreasing over the study period, space-time interaction effects were seen to be increasing over time in York County.

Survey of ticks and severe fever with thrombocytopenia syndrome in Zhoushan, Zhejiang, 2018–2020

Objective To investigate the density and seasonal fluctuation of ticks and its association with the incidence of severe fever with thrombocytopenia syndrome (SFTS) in Zhoushan of Zhejiang province. Methods From 2018 to 2020, the surveillance data of free ticks were collected from Dinghai, Putuo, Daishan and Shengsi districts of Zhoushan and the information about laboratory confirmed cases of fever with thrombocytopenia syndrome were collected. The surveillance data of parasitic ticks and free ticks were collected from the areas where SFTS cases occurred. RT-PCR was used to detect SFTSV in ticks. Results A total of 612 free ticks were captured from 2018 to 2020. Haemaphysalis longhorni accounted for 83.82%, Rhipicephalus falciparum accounted for 14.05%. There were 20 cases of new fever with thrombocytopenia syndrome reported from 2018 to 2020, and the cases mainly occurred in Daishan county. A total of 51 free ticks and 86 parasitic ticks were captured near the area where the confirmed case occurred, all of which were Haemaphysalis longhorni. All the ticks tested by RT-PCR were negative for SFTSV. Conclusion The predominant species of ticks in Zhoushan was Haemaphysalis longicorni. The rural environment was the main habitat of ticks. The activity peak of ticks was during May - September. The incidence of fever with thrombocytopenia syndrome was related to the activity peak of ticks.

Gabor Features Extraction and Land-Cover Classification of Urban Hyperspectral Images for Remote Sensing Applications

The principles of the transform stage of the extract, transform and load (ETL) process can be applied to index the data in functional structures for the decision-making inherent in an urban remote sensing application. This work proposes a method that can be utilised as an organisation stage by reducing the data dimension with Gabor texture features extracted from grey-scale representations of the Hue, Saturation and Value (HSV) colour space and the Normalised Difference Vegetation Index (NDVI). Additionally, the texture features are reduced using the Linear Discriminant Analysis (LDA) method. Afterwards, an Artificial Neural Network (ANN) is employed to classify the data and build a tick data matrix indexed by the belonging class of the observations, which could be retrieved for further analysis according to the class selected to explore. The proposed method is compared in terms of classification rates, reduction efficiency and training time against the utilisation of other grey-scale representations and classifiers. This method compresses up to 87% of the original features and achieves similar classification results to non-reduced features but at a higher training time.

Profitability and liquidity provision of HFTs during large price shocks: Does relative tick size matter?

Using tick data identifying high-frequency traders (HFTs), this paper studies the trading profits and liquidity provision of HFTs during large price shocks. Previous studies report mixed evidence on whether HFTs provide or take liquidity in such events. Empirical findings in this paper suggest that relative tick size matters: HFTs provide liquidity when the shock is idiosyncratic and the relative tick size is large, but in this case, they do not earn profits from trading. On average, HFTs can earn profits against non-HFTs because they aggressively take liquidity and trade in the direction of the shocks for stocks of small relative tick sizes.

Monitoring the patterns of submission and presence of tick-borne pathogens in Ixodes scapularis collected from humans and companion animals in Ontario, Canada (2011\u20132017)

BackgroundThe universal nature of the human–companion animal relationship and their shared ticks and tick-borne pathogens offers an opportunity for improving public and veterinary health surveillance. With this in mind, we describe the spatiotemporal trends for blacklegged tick (Ixodes scapularis) submissions from humans and companion animals in Ontario, along with pathogen prevalence.MethodsWe tested tick samples submitted through passive surveillance (2011–2017) from humans and companion animals for Borrelia burgdorferi, Borrelia miyamotoi, Anaplasma phagocytophilum and Babesia microti. We describe pathogen prevalence in ticks from humans and from companion animals and constructed univariable Poisson and negative binomial regression models to explore the spatiotemporal relationship between the rates of tick submissions by host type.ResultsDuring the study, there were 17,230 blacklegged tick samples submitted from humans and 4375 from companion animals. Tick submission rates from companion animals were higher than expected in several public health units (PHUs) lacking established tick populations, potentially indicating newly emerging populations. Pathogen prevalence in ticks was higher in PHUs where established blacklegged tick populations exist. Borrelia burgdorferi prevalence was higher in ticks collected from humans (maximum likelihood estimate, MLE = 17.5%; 95% confidence interval, CI 16.97–18.09%) than from companion animals (9.9%, 95% CI 9.15–10.78%). There was no difference in pathogen prevalence in ticks by host type for the remaining pathogens, which were found in less than 1% of tested ticks. The most common co-infection B. burgdorferi + B. miyamotoi occurred in 0.11% of blacklegged ticks from humans and animals combined. Borrelia burgdorferi prevalence was higher in unengorged (21.9%, 95% CI 21.12–22.65%) than engorged ticks (10.0%, 95% CI 9.45–10.56%). There were no consistent and significant spatiotemporal relationships detected via regression models between the annual rates of submission of each host type.ConclusionsWhile B. burgdorferi has been present in blacklegged ticks in Ontario for several decades, other tick-borne pathogens are also present at low prevalence. Blacklegged tick and pathogen surveillance data can be used to monitor risk in human and companion animal populations, and efforts are under consideration to unite surveillance efforts for the different target populations.Graphical

Potential for online crowdsourced biological recording data to complement surveillance for arthropod vectors.

Voluntary contributions by citizen scientists can gather large datasets covering wide geographical areas, and are increasingly utilized by researchers for multiple applications, including arthropod vector surveillance. Online platforms such as iNaturalist accumulate crowdsourced biological observations from around the world and these data could also be useful for monitoring vectors. The aim of this study was to explore the availability of observations of important vector taxa on the iNaturalist platform and examine the utility of these data to complement existing vector surveillance activities. Of ten vector taxa investigated, records were most numerous for mosquitoes (Culicidae; 23,018 records, 222 species) and ticks (Ixodida; 16,214 records, 87 species), with most data from 2019-2020. Case studies were performed to assess whether images associated with records were of sufficient quality to identify species and compare iNaturalist observations of vector species to the known situation at the state, national and regional level based on existing published data. Firstly, tick data collected at the national (United Kingdom) or state (Minnesota, USA) level were sufficient to determine seasonal occurrence and distribution patterns of important tick species, and were able to corroborate and complement known trends in tick distribution. Importantly, tick species with expanding distributions (Haemaphysalis punctata in the UK, and Amblyomma americanum in Minnesota) were also detected. Secondly, using iNaturalist data to monitor expanding tick species in Europe (Hyalomma spp.) and the USA (Haemaphysalis longicornis), and invasive Aedes mosquitoes in Europe, showed potential for tracking these species within their known range as well as identifying possible areas of expansion. Despite known limitations associated with crowdsourced data, this study shows that iNaturalist can be a valuable source of information on vector distribution and seasonality that could be used to supplement existing vector surveillance data, especially at a time when many surveillance programs may have been interrupted by COVID-19 restrictions.

Efficacy of a water-based botanical acaricide formulation applied in portable spray box against the southern cattle tick, Rhipicephalus (Boophilus) microplus (Acari: Ixodidae), infesting cattle

Ectoparasitism result in annual losses to livestock producers estimated to reach billions of dollars in tropical and subtropical parts of the world where the southern cattle tick, Rhipicephalus (Boophilus) microplus, is established because this invasive pest is also a vector of pathogens causing bovine babesiosis and anaplasmosis. Reintroduction of R. microplus could be economically devastating for the cattle industry in the United States. Novel technologies are necessary to manage populations of R. microplus that are resistant to multiple classes of synthetic chemicals widely used as acaricides to control ticks infesting livestock, and to mitigate the environmental impact of these synthetic acaricides. Bioassays with a water-based formulation of a commercially available botanical acaricide (Essentria® IC3) at 6.25 % against a laboratory strain of R. microplus was 100 % lethal against unfed larvae, and 94 % mortality was recorded against engorged female ticks. These results prompted documentation of efficacy on infested cattle using an acaricide delivery system mimicking field conditions, which is required to consider the use of a product by integrated tick management programs. Twelve tick naïve cattle were artificially infested with unfed larvae on days -19, -12 and -5. On day -2, ticks from the left-side of the body of each animal were manually counted. Depending on pre-treatment tick counts the animals were assigned into three separate treatment groups: i) water (untreated control); ii) coumaphos 0.3 % in water (positive control); and iii) botanical acaricide 6.25 % in water. Five days after the last infestation, cattle were doused in a spray box and moved to individual stalls to collect tick data for calculation of treatment efficacy. The percentage control, based on the reproduction of surviving engorged female ticks after treatment with the 6.25 % Essentria in water-based spray was 70 % against R. microplus. Coumaphos treatment had a mean percent control of 100 %. Whereas this application of Essentria® IC3 cannot be recommended as a stand-alone method to eradicate R. microplus, the 6.25 % botanical acaricide water-based spray could be part of integrated tick management to control populations of R. microplus resistant to organophosphates like coumaphos and other classes of commercially available synthetic acaricides.

Dynamic Pricing of Restaurant Food Delivery: An Explorative Market Efficiency Study

We use restaurant food delivery sales tick data to investigate how prices reflect demand, allocate resources and impact profitability. We find that real-time demand can be used to accurately predict future demand over a short time frame, and that dynamic prices based on it reduce demand variation. Our evidence suggests that dynamic pricing of restaurant food delivery can decrease resource waste and improve producer profitability.

Separate Noise and Jumps From Tick Data: An Endogenous Thresholding Approach

We study the problem of jump detection for ultra-high-frequency tick-by-tick data. We propose a novel easy-to-implement procedure that can separate the contribution of microstructure noise and that of finite activity price jumps from the price process, which may have interesting implications on asset pricing and forecasting problems. We provide theoretical grounds of our approach, and suggests practical guidelines for determining the tuning parameter. Making a comparison with the “star performers” in a recent comprehensive review for jump detection methods by Maneesoonthorn et al. (2020) as well as a test based on Christensen et al. (2014) on tick data, we show that our method performs admirably well via extensive simulation and rich empirical illustration.

Machine Learning in the Corporate Bond Market and Beyond: A New Classifier

Trade signing algorithms that rely on quote data, tick data or both have been used extensively to assign a trade as either a buy or a sell. Leveraging the availability of a large panel of signed trade data in the corporate bond market, we explore machine learning methods to uncover a new trade signing model that improves upon standard trade classification methods in both the bond and equity markets. We show that both the trading and information environment at the time of the trade critically affect the accuracy of existing trade classification rules in general, and also illustrate the importance of optimizing the feature set in correctly classifying trade direction. Importantly, our approach and the Random Forest algorithm we propose can be used in markets both with and without pre-trade transparency.

An efficient storage-optimizing tick data clustering model

Tick data is a large volume of data, related to a phenomenon such as stock market or weather change, with data values changing rapidly over time. An important issue is to store tick data table in a way that it occupies minimum storage space while at the same time it can provide fast execution of queries. In this paper, a mathematical model is proposed to partition tick data tables into clusters with the aim of minimizing the required storage space. The genetic algorithm is then used to solve the mathematical model which is indeed a clustering model. The proposed method has been evaluated on a real-world weather tick dataset and compared to the storage-optimizing hierarchical agglomerative clustering (SOHAC) algorithm. The experiments show that our proposed method substantially outperforms SOHAC in achieving smaller values for compression ratio while reducing the execution time for small number of clusters.

Analysis of intra-day fluctuations in the Mexican financial market index

In this paper, a statistical analysis of high frequency fluctuations of the IPC, the Mexican Stock Market Index, is presented. A sample of tick--to--tick data covering the period from January 1999 to December 2002 was analyzed, as well as several other sets obtained using temporal aggregation. Our results indicates that the highest frequency is not useful to understand the Mexican market because almost two thirds of the information corresponds to inactivity. For the frequency where fluctuations start to be relevant, the IPC data does not follows any $\alpha$-stable distribution, including the Gaussian, perhaps because of the presence of autocorrelations. For a long range of lower-frequencies, but still in the intra-day regime, fluctuations can be described as a truncated L\'evy flight, while for frequencies above two-days, a Gaussian distribution yields the best fit. Thought these results are consistent with other previously reported for several markets, there are significant differences in the details of the corresponding descriptions.

Monitoring of ticks and tick-borne pathogens through a nationwide research station network in Finland

In 2015 a long-term, nationwide tick and tick-borne pathogen (TBP) monitoring project was started by the Finnish Tick Project and the Finnish Research Station network (RESTAT), with the goal of producing temporally and geographically extensive data regarding exophilic ticks in Finland. In the current study, we present results from the first four years of this collaboration.Ticks were collected by cloth dragging from 11 research stations across Finland in May–September 2015–2018 (2012–2018 in Seili). Collected ticks were screened for twelve different pathogens by qPCR: Borrelia afzelii, Borrelia garinii, Borrelia valaisiana, Borrelia burgdorferi sensu stricto, Borrelia miyamotoi, Babesia spp., Anaplasma phagocytophilum, Rickettsia spp., Candidatus Neoehrlichia mikurensis, Francisella tularensis, Bartonella spp. and tick-borne encephalitis virus (TBEV).Altogether 15 067 Ixodes ricinus and 46 Ixodes persulcatus were collected during 68 km of dragging. Field collections revealed different seasonal activity patterns for the two species. The activity of I. persulcatus adults (only one nymph detected) was unimodal, with activity only in May–July, whereas Ixodes ricinus was active from May to September, with activity peaks in September (nymphs) or July–August (adults). Overall, tick densities were higher during the latter years of the study. Borrelia burgdorferi sensu lato were the most common pathogens detected, with 48.9 ± 8.4% (95% Cl) of adults and 25.3 ± 4.4% of nymphs carrying the bacteria. No samples positive for F. tularensis, Bartonella or TBEV were detected.This collaboration project involving the extensive Finnish Research Station network has ensured enduring and spatially extensive, long-term tick data collection to the foreseeable future.

Developing Bid-Ask Probabilities for High-Frequency Trading

Methods of path integrals are used to develop multi-factor probabilities of bid-ask variables to be used in high-frequency trading (HFT). Adaptive Simulated Annealing (ASA) is used to fit the nonlinear forms, so developed to a day of BitMEX tick data. Maxima algebraic code is used to develop the path integral codes into C codes, and a sampling code is used for the fitting process. After these fits, the resultant C code is very fast and useful for forecasting upcoming ‘ask’, bid, midprice, etc., when narrow and wide windows of incoming data are used. A bonus is the availability of canonical momenta indicators (CMI) useful to forecast direction and strengths of these variables.

Tick based clustering methodologies establishing support and resistance levels in the currency exchange market

We establish support and resistance levels from data in intraday currency exchange market activity based on machine learning methods. Specifically we design two semi-supervised classification neural networks. The first one is based on a variant of the K-means method while the second is based on a Gaussian mixture model with expectation maximisation. Each performs classification from tick data on very short time windows and produces the corresponding support and resistance price levels. We test the methodology on actual market data for the EUR-USD currency exchange. As a sanity check we also perform mock trades based on actual market data. We evaluate the results for statistical significance using a number of performance metrics while also comparing against traditional methods.

QuantCloud: Enabling Big Data Complex Event Processing for Quantitative Finance Through a Data-Driven Execution

Quantitative Finance (QF) utilizes increasingly sophisticated mathematic models and advanced computer techniques to predict the movement of global markets, and price the derivatives and other assets. Being able to react quickly and intelligently to fast-changing markets is a decisive success factor for trading companies. To date, the rise of QF requires an integrated toolchain of enabling technologies to carry out complex event processing on the explosive growth and diversified forms of market metadata, in pursuit of a microsecond latency on an Exabyte-level dataset. Inspired by this, we present a data-driven execution paradigm that untangles the dependencies of complex processing events and integrate the paradigm with a big data infrastructure that streams time series data. This integrated platform is termed as the QuantCloud platform. Essentially, QuantCloud executes the complex event processing in a data-driven mode and manages large amounts of diversified market data in a data-parallel mode. To show its practicability and performance, we develop a prototype and benchmark by applying real-world QF research models on the New York Stock Exchange (NYSE) data. Using this prototype, we demonstrate this platform with an application to: (i) data cleaning and aggregating (including the computing of logarithmic returns from tick data and the finding the medians of grouped data) and (ii) data modeling: the autoregressive-moving average (ARMA) model. The performance results show that (a) this platform obtains a high throughput (usually in the order of millions of tick messages per second) and a sub-microsecond latency; (b) it fully executes data-dependent tasks through a data-driven execution; and (c) it implements a modular design approach for rapidly developing these data-crunching methods and QF research models. This platform resulting from an aggregated effort of the data-driven execution and big data infrastructure, offers the financial engineers with new insights and enhanced capabilities for effective and efficient incorporation of big data complex event processing technologies in their workflow.

Using imperfect data in predictive mapping of vectors: a regional example of Ixodes ricinus distribution

BackgroundKnowledge of Ixodes ricinus tick distribution is critical for surveillance and risk management of transmissible tick-borne diseases such as Lyme borreliosis. However, as the ecology of I. ricinus is complex, and robust long-term geographically extensive distribution tick data are limited, mapping often relies on datasets collected for other purposes. We compared the modelled distributions derived from three datasets with information on I. ricinus distribution (quantitative I. ricinus count data from scientific surveys; I. ricinus presence-only data from public submissions; and a combined I. ricinus dataset from multiple sources) to assess which could be reliably used to inform Public Health strategy. The outputs also illustrate the strengths and limitations of these three types of data, which are commonly used in mapping tick distributions.MethodsUsing the Integrated Nested Laplace algorithm we predicted I. ricinus abundance and presence–absence in Scotland and tested the robustness of the predictions, accounting for errors and uncertainty.ResultsAll models fitted the data well and the covariate predictors for I. ricinus distribution, i.e. deer presence, temperature, habitat, index of vegetation, were as expected. Differences in the spatial trend of I. ricinus distribution were evident between the three predictive maps. Uncertainties in the spatial models resulted from inherent characteristics of the datasets, particularly the number of data points, and coverage over the covariate range used in making the predictions.ConclusionsQuantitative I. ricinus data from scientific surveys are usually considered to be gold standard data and we recommend their use where high data coverage can be achieved. However in this study their value was limited by poor data coverage. Combined datasets with I. ricinus distribution data from multiple sources are valuable in addressing issues of low coverage and this dataset produced the most appropriate map for national scale decision-making in Scotland. When mapping vector distributions for public-health decision making, model uncertainties and limitations of extrapolation need to be considered; these are often not included in published vector distribution maps. Further development of tools to better assess uncertainties in the models and predictions are necessary to allow more informed interpretation of distribution maps.