Abstract
Concentration inequalities are widely used for analyzing machine learning algorithms. However, the current concentration inequalities cannot be applied to some non-causal processes which appear for instance in Natural Language Processing (NLP). This is mainly due to the non-causal nature of such involved data, in the sense that each data point depends on other neighboring data points. In this paper, we establish a framework for modeling non-causal random fields and prove a Hoeffding-type concentration inequality. The proof of this result is based on a local approximation of the non-causal random field by a function of a finite number of i.i.d. random variables.
Highlights
Concentration inequalities are widely used in statistical learning
The current concentration inequalities cannot be applied to some non-causal processes which appear for instance in Natural Language Processing (NLP)
The proof of this result is based on a local approximation of the non-causal random field by a function of a finite number of i.i.d. random variables
Summary
Concentration inequalities are widely used in statistical learning. For example, model selection techniques rely heavily on concentration inequalities [28]. The BERT model [21] has become a staple for a very large range of NLP tasks, such as translation, part-of-speech tagging, sentiment analysis Despite their success in practical applications, it lacks a theoretical framework to analyze such non-causal models. The natural extension of Markov chains to random fields leads to causal random fields: here the dependence is propagated along with preferential directions (see [17] for an example of application). It is natural to model the generation of pictures by a non-causal random field defined over a two-dimensional lattice. In this case, the completion problem consists in filling missing pixels using neighboring pixels [4]. Another application of importance is the case of completing geographical data sets which make sense for an ecology setting (see http://doukhan.u-cergy.fr/EcoDep.html), for which applications are of fundamental importance
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
