Quantitative Big Data: where chemometrics can contribute

Abstract

overwhelmed. We do not limit ourselves to over-simplified versions of “The Scientific Method”—the testing of hypotheses and searching for p-values—we also know how to listen and learn from real-world data, and leap forward from there. We have Moore’s Law on our side, but use the increasing computer power differently from many other fields: we tend to avoid the alienating “black box” modeling of machine learning and the scientific hubris of overly confident causal mechanistic modeling. Instead, we analyze big, real-world data sets with transparent data modeling methods that help us overview complex systems: Our main data modeling tools—“factor-analytic” decomposition methods like Principal Component Analysis (PCA) and Partial Least Squares Regression (PLSR) and the many extensions thereof—help us find, quantify and display the essential relationships—expected or unexpected—within and between data tables. These transparent, open-ended methods reveal the systematic relationships, not as magic, but for the eyes of scientist to see. Because meaningful data-driven modeling requires good data, we insist on representative sampling and pragmatic, understandable statistical assessments. Thereby we can get a good grip on the complexity of the real physical world. We can also use this approach to study the behavior of humans, and even of complex mathematical models. In the following, I present my view of Chemometrics as a science culture for the future, and outline a philosophical framework forthat.Ithendescribesometopicsforfutureworkinthe field.