SciQLop: an open source project for in situ data analysis

Nicolas Aunai,Benjamin Renard,Gautier Nguyen,Michotte De Welle Bayane,Vincent Génot,Alexis Jeandet,Alexandre Schulz

doi:10.5194/egusphere-egu22-5774

Abstract

&lt;p&gt;The SCIentific Qt application for Learning from Observations of Plasmas (SciQLop) project allows to easily discover, retrieve, plot and label in situ space physic measurements from remote servers such as Coordinated Data Analysis Web (CDAWeb) or Automated Multi-Dataset Analysis (AMDA). &amp;#160;Analyzing data from a single instrument on a given mission can rise some technical difficulties such as finding where to get them, how to get them and sometimes how to read them. &amp;#160;Thus building for example a machine-learning pipeline involving multiple instruments and even multiple spacecraft missions can be very challenging. Our goal here is to remove all these technical difficulties without sacrificing performances to allow scientist to focus on data analysis.&lt;br&gt;SciQLop development has started in 2015 as a C++ graphical application funded by the Paris-Saclay Center for Data Science (CDS) then by Paris-Saclay SPACEOBS and finally it joined the Plasma Physics Data Center (CDPP) in 2019. It has evolved from &amp;#160;a monolithic C++ graphical application to a collection of simple and reusable Python or C++ packages solving one problem at a time, increasing our chances to reach users and contributors.&lt;/p&gt;&lt;p&gt;The SciQLop project is composed of the following tools:&lt;/p&gt;&lt;ul&gt;&lt;li&gt;Speasy: An easy to use Python package to retrieve data from remote servers with multi-layer cache support.&lt;/li&gt; &lt;li&gt;Speasy_proxy: A self-hostable, chainable remote cache for Speasy written as a simple Python package.&lt;/li&gt; &lt;li&gt;Broni: A Python package which finds intersections between spacecraft trajectories and simple shapes or physical models such as magnetosheath.&lt;/li&gt; &lt;li&gt;Orbit-viewer: A Python graphical user interface (GUI) for Broni.&lt;/li&gt; &lt;li&gt;TSCat: A Python package used as backend for catalogs of events storage.&lt;/li&gt; &lt;li&gt;TSCat-GUI: A Python graphical user interface (GUI).&lt;/li&gt; &lt;li&gt;SciQLop-GUI: An extensible and efficient user interface to visualize and label time-series with an embedded IPYthon terminal.&lt;/li&gt; &lt;/ul&gt;&lt;p&gt;While some components are production ready and already used for science, SciQLop is still in development and the landscape is moving quite fast.&lt;br&gt;In this presentation we will give an overview of SciQLop, demonstrate its benefits using some specific cases studies and finally discuss the planned features development.&lt;/p&gt;

Full Text