Inference Capabilities Research Articles

Long-term validation of inner-urban mobility metrics derived from Twitter/X

Urban mobility analysis using Twitter as a proxy has gained significant attention in various application fields; however, long-term validation studies are scarce. This paper addresses this gap by assessing the reliability of Twitter data for modeling inner-urban mobility dynamics over a 27-month period in the. metropolitan area of Rio de Janeiro, Brazil. The evaluation involves the validation of Twitter-derived mobility estimates at both temporal and spatial scales, employing over 1.6 × 1011 mobile phone records of around three million users during the non-stationary mobility period from April 2020 to. June 2022, which coincided with the COVID-19 pandemic. The results highlight the need for caution when using Twitter for short-term modeling of urban mobility flows. Short-term inference can be influenced by Twitter policy changes and the availability of publicly accessible tweets. On the other hand, this long-term study demonstrates that employing multiple mobility metrics simultaneously, analyzing dynamic and static mobility changes concurrently, and employing robust preprocessing techniques such as rolling window downsampling can enhance the inference capabilities of Twitter data. These novel insights gained from a long-term perspective are vital, as Twitter - rebranded to X in 2023 - is extensively used by researchers worldwide to infer human movement patterns. Since conclusions drawn from studies using Twitter could be used to inform public policy, emergency response, and urban planning, evaluating the reliability of this data is of utmost importance.

On-device Training: A First Overview on Existing Systems

The recent breakthroughs in machine learning (ML) and deep learning (DL) have catalyzed the design and development of various intelligent systems over wide application domains. While most existing machine learning models require large memory and computing power, efforts have been made to deploy some models on resource-constrained devices as well. A majority of the early application systems focused on exploiting the inference capabilities of ML and DL models, where data captured from different mobile and embedded sensing components are processed through these models for application goals such as classification and segmentation. More recently, the concept of exploiting the mobile and embedded computing resources for ML/DL model training has gained attention, as such capabilities allow (i) the training of models via local data without the need to share data over wireless links, thus enabling privacy-preserving computation by design, (ii) model personalization and environment adaptation, and (iii) deployment of accurate models in remote and hardly accessible locations without stable internet connectivity. This work summarizes and analyzes state-of-the-art systems research that allows such on-device model training capabilities and provides a survey of on-device training from a systems perspective.

TabMoE: A General Framework for Diverse Table-Based Reasoning with Mixture-of-Experts

Tables serve as a widely adopted data format, attracting considerable academic interest concerning semantic understanding and logical inference of tables. In recent years, the prevailing paradigm of pre-training and fine-tuning on tabular data has become increasingly prominent in research on table understanding. However, existing table-based pre-training methods frequently exhibit constraints, supporting only single tasks while requiring substantial computational resources, which hinders their efficiency and applicability. In this paper, we introduce the TabMoE, a novel framework based on mixture-of-experts, designed to handle a wide range of tasks involving logical reasoning over tabular data. Each expert within the model specializes in a distinct logical function and is trained through the utilization of a hard Expectation–Maximization algorithm. Remarkably, this framework eliminates the necessity of dependency on tabular pre-training, instead exclusively employing limited task-specific data to significantly enhance models’ inferential capabilities. We conduct empirical experiments across three typical tasks related to tabular data: table-based question answering, table-based fact verification, and table-to-text generation. The experimental results underscore the innovation and feasibility of our framework.

Large language models as surrogate models in evolutionary algorithms: A preliminary study

Large Language Models (LLMs) have demonstrated remarkable advancements across diverse domains, manifesting considerable capabilities in evolutionary computation, notably in generating new solutions and automating algorithm design. Surrogate-assisted selection plays a pivotal role in evolutionary algorithms (EAs), especially in addressing expensive optimization problems by reducing the number of real function evaluations. However, whether LLMs can serve as surrogate models remains an unknown. In this study, we propose a novel surrogate model based purely on LLM inference capabilities, eliminating the need for training. Specifically, we formulate model-assisted selection as a classification problem or a regression problem, utilizing LLMs to directly evaluate the quality of new solutions based on historical data. This involves predicting whether a solution is good or bad, or approximating its value. This approach is then integrated into EAs, termed LLM-assisted EA (LAEA). Detailed experiments compared the visualization results of 2D data from 9 mainstream LLMs, as well as their performance on 5-10 dimensional problems. The experimental results demonstrate that LLMs have significant potential as surrogate models in evolutionary computation, achieving performance comparable to traditional surrogate models only using inference. This work offers new insights into the application of LLMs in evolutionary computation. Code is available at: https://github.com/hhyqhh/LAEA.git.

High-throughput systolic array-based accelerator for hybrid transformer-CNN networks

In this era of Transformers enjoying remarkable success, Convolutional Neural Networks (CNNs) remain highly relevant and useful. Indeed, hybrid Transformer-CNN network architectures, which combine the benefits of both approaches, have achieved impressive results. Vision Transformer (ViT) is a significant neural network architecture that features a convolutional layer as its first layer, primarily built on the transformer framework. However, owing to the distinct computation patterns inherent in attention and convolution, existing hardware accelerators for these two models are typically designed separately and lack a unified approach toward accelerating both models efficiently. In this paper, we present a dedicated accelerator on a field-programmable gate array (FPGA) platform. The accelerator, which integrates a configurable three-dimensional systolic array, is specifically designed to accelerate the inferential capabilities of hybrid Transformer-CNN networks. The Convolution and Transformer computations can be mapped to a systolic array by unifying these operations for matrix multiplication. Softmax and LayerNorm which are frequently used in hybrid Transformer-CNN networks were also implemented on FPGA boards. The accelerator achieved high performance with a peak throughput of 722 GOP/s at an average energy efficiency of 53 GOPS/W. Its respective computation latencies were 51.3 ms, 18.1 ms, and 6.8 ms for ViT-Base, ViT-Small, and ViT-Tiny. The accelerator provided a 12× improvement in energy efficiency compared to the CPU, a 2.3× improvement compared to the GPU, and a 1.5× to 2× improvement compared to existing accelerators regarding speed and energy efficiency.

Melon ripeness detection by an improved object detection algorithm for resource constrained environments

BackgroundRipeness is a phenotype that significantly impacts the quality of fruits, constituting a crucial factor in the cultivation and harvesting processes. Manual detection methods and experimental analysis, however, are inefficient and costly.ResultsIn this study, we propose a lightweight and efficient melon ripeness detection method, MRD-YOLO, based on an improved object detection algorithm. The method combines a lightweight backbone network, MobileNetV3, a design paradigm Slim-neck, and a Coordinate Attention mechanism. Additionally, we have created a large-scale melon dataset sourced from a greenhouse based on ripeness. This dataset contains common complexities encountered in the field environment, such as occlusions, overlapping, and varying light intensities. MRD-YOLO achieves a mean Average Precision of 97.4% on this dataset, achieving accurate and reliable melon ripeness detection. Moreover, the method demands only 4.8 G FLOPs and 2.06 M parameters, representing 58.5% and 68.4% of the baseline YOLOv8n model, respectively. It comprehensively outperforms existing methods in terms of balanced accuracy and computational efficiency. Furthermore, it maintains real-time inference capability in GPU environments and demonstrates exceptional inference speed in CPU environments. The lightweight design of MRD-YOLO is anticipated to be deployed in various resource constrained mobile and edge devices, such as picking robots. Particularly noteworthy is its performance when tested on two melon datasets obtained from the Roboflow platform, achieving a mean Average Precision of 85.9%. This underscores its excellent generalization ability on untrained data.ConclusionsThis study presents an efficient method for melon ripeness detection, and the dataset utilized in this study, alongside the detection method, will provide a valuable reference for ripeness detection across various types of fruits.

Insertable Glucose Sensor Using a Compact and Cost-Effective Phosphorescence Lifetime Imager and Machine Learning.

Optical continuous glucose monitoring (CGM) systems are emerging for personalized glucose management owing to their lower cost and prolonged durability compared to conventional electrochemical CGMs. Here, we report a computational CGM system, which integrates a biocompatible phosphorescence-based insertable biosensor and a custom-designed phosphorescence lifetime imager (PLI). This compact and cost-effective PLI is designed to capture phosphorescence lifetime images of an insertable sensor through the skin, where the lifetime of the emitted phosphorescence signal is modulated by the local concentration of glucose. Because this phosphorescence signal has a very long lifetime compared to tissue autofluorescence or excitation leakage processes, it completely bypasses these noise sources by measuring the sensor emission over several tens of microseconds after the excitation light is turned off. The lifetime images acquired through the skin are processed by neural network-based models for misalignment-tolerant inference of glucose levels, accurately revealing normal, low (hypoglycemia) and high (hyperglycemia) concentration ranges. Using a 1 mm thick skin phantom mimicking the optical properties of human skin, we performed in vitro testing of the PLI using glucose-spiked samples, yielding 88.8% inference accuracy, also showing resilience to random and unknown misalignments within a lateral distance of ∼4.7 mm with respect to the position of the insertable sensor underneath the skin phantom. Furthermore, the PLI accurately identified larger lateral misalignments beyond 5 mm, prompting user intervention for realignment. The misalignment-resilient glucose concentration inference capability of this compact and cost-effective PLI makes it an appealing wearable diagnostics tool for real-time tracking of glucose and other biomarkers.

Static analysis driven enhancements for comprehension in machine learning notebooks

Jupyter notebooks have emerged as the predominant tool for data scientists to develop and share machine learning solutions, primarily using Python as the programming language. Despite their widespread adoption, a significant fraction of these notebooks, when shared on public repositories, suffer from insufficient documentation and a lack of coherent narrative. Such shortcomings compromise the readability and understandability of the notebook. Addressing this shortcoming, this paper introduces HeaderGen, a tool-based approach that automatically augments code cells in these notebooks with descriptive markdown headers, derived from a predefined taxonomy of machine learning operations. Additionally, it systematically classifies and displays function calls in line with this taxonomy. The mechanism that powers HeaderGen is an enhanced call graph analysis technique, building upon the foundational analysis available in PyCG. To improve precision, HeaderGen extends PyCG’s analysis with return-type resolution of external function calls, type inference, and flow-sensitivity. Furthermore, leveraging type information, HeaderGen employs pattern matching techniques on the code syntax to annotate code cells. We conducted an empirical evaluation on 15 real-world Jupyter notebooks sourced from Kaggle. The results indicate a high accuracy in call graph analysis, with precision at 95.6% and recall at 95.3%. The header generation has a precision of 85.7% and a recall rate of 92.8% with regard to headers created manually by experts. A user study corroborated the practical utility of HeaderGen, revealing that users found HeaderGen useful in tasks related to comprehension and navigation. To further evaluate the type inference capability of static analysis tools, we introduce TypeEvalPy, a framework for evaluating type inference tools for Python with an in-built micro-benchmark containing 154 code snippets and 845 type annotations in the ground truth. Our comparative analysis on four tools revealed that HeaderGen outperforms other tools in exact matches with the ground truth.

Advances on MXene-Based Memristors for Neuromorphic Computing: A Review on Synthesis, Mechanisms, and Future Directions.

Neuromorphic computing seeks to replicate the capabilities of parallel processing, progressive learning, and inference while retaining low power consumption by drawing inspiration from the human brain. By further overcoming the constraints imposed by the traditional von Neumann architecture, this innovative approach has the potential to revolutionize modern computing systems. Memristors have emerged as a solution to implement neuromorphic computing in hardware, with research based on developing functional materials for resistive switching performance enhancement. Recently, two-dimensional MXenes, a family of transition metal carbides, nitrides, and carbonitrides, have begun to be integrated into these devices to achieve synaptic emulation. MXene-based memristors have already demonstrated diverse neuromorphic characteristics while enhancing the stability and reducing power consumption. The possibility of changing the physicochemical properties through modifications of the surface terminations, bandgap, interlayer spacing, and oxidation for each existing MXene makes them very promising. Here, recent advancements in MXene synthesis, device fabrication, and characterization of MXene-based neuromorphic artificial synapses are discussed. Then, we focus on understanding the resistive switching mechanisms and how they connect with theoretical and experimental data, along with the innovations made during the fabrication process. Additionally, we provide an in-depth review of the neuromorphic performance, making a connection with the resistive switching mechanism, along with a compendium of each relevant performance factor for nonvolatile and volatile applications. Finally, we state the remaining challenges in MXene-based devices for artificial synapses and the next steps that could be taken for future development.

Immersive analytics with augmented reality in meteorology: an exploratory study on ontology and linked data

Although Augmented Reality (AR) has been extensively studied in supporting Immersive Analytics (IA), there are still many challenges in visualising and interacting with big and complex datasets. To deal with these datasets, most AR applications utilise NoSQL databases for storing and querying data, especially for managing large volumes of unstructured or semi-structured data. However, NoSQL databases have limitations in their reasoning and inference capabilities, which can result in insufficient support for certain types of queries. To fill this gap, we aim to explore and evaluate whether an intelligent approach based on ontology and linked data can facilitate visual analytics tasks with big datasets on AR interface. We designed and implemented a prototype of this method for meteorological data analytics. An experiment was conducted to evaluate the use of a semantic database with linked data compared to a conventional approach in an AR-based immersive analytics system. The results significantly highlight the performance of semantic approach in helping the users analysing meteorological datasets and their subjective appreciation in working with the AR interface, which is enhanced with ontology and linked data.

Let’s Discover More API Relations: A Large Language Model-based AI Chain for Unsupervised API Relation Inference

APIs have intricate relations that can be described in text and represented as knowledge graphs to aid software engineering tasks. Existing relation extraction methods have limitations, such as limited API text corpus and affected by the characteristics of the input text. To address these limitations, we propose utilizing large language models (LLMs) (e.g., gpt-3.5) as a neural knowledge base for API relation inference. This approach leverages the entire Web used to pre-train LLMs as a knowledge base and is insensitive to the context and complexity of input texts. To ensure accurate inference, we design an AI chain consisting of three AI modules: API Fully Qualified Name (FQN) Parser, API Knowledge Extractor, and API Relation Decider. The accuracy of the API FQN Parser and API Relation Decider is 0.81 and 0.83, respectively. Using the generative capacity of the LLM and our approach’s inference capability, we achieve an average F1 value of 0.76 under the three datasets, significantly higher than the state-of-the-art method’s average F1 value of 0.40. Compared to the original CoT and modularized CoT methods, our AI chain design has improved the performance of API relation inference by 71% and 49%, respectively. Meanwhile, the prompt ensembling strategy enhances the performance of our approach by 32%. The API relations inferred by our method can be further organized into structured forms to provide support for other software engineering tasks.

Artificial intelligence model for automated surgical instrument detection and counting: an experimental proof-of-concept study

BackgroundRetained surgical items (RSI) are preventable events that pose a significant risk to patient safety. Current strategies for preventing RSIs rely heavily on manual instrument counting methods, which are prone to human error. This study evaluates the feasibility and performance of a deep learning-based computer vision model for automated surgical tool detection and counting.MethodsA novel dataset of 1,004 images containing 13,213 surgical tools across 11 categories was developed. The dataset was split into training, validation, and test sets at a 60:20:20 ratio. An artificial intelligence (AI) model was trained on the dataset, and the model’s performance was evaluated using standard object detection metrics, including precision and recall. To simulate a real-world surgical setting, model performance was also evaluated in a dynamic surgical video of instruments being moved in real-time.ResultsThe model demonstrated high precision (98.5%) and recall (99.9%) in distinguishing surgical tools from the background. It also exhibited excellent performance in differentiating between various surgical tools, with precision ranging from 94.0 to 100% and recall ranging from 97.1 to 100% across 11 tool categories. The model maintained strong performance on a subset of test images containing overlapping tools (precision range: 89.6–100%, and recall range 97.2–98.2%). In a real-time surgical video analysis, the model maintained a correct surgical tool count in all non-transition frames, with a median inference speed of 40.4 frames per second (interquartile range: 4.9).ConclusionThis study demonstrates that using a deep learning-based computer vision model for automated surgical tool detection and counting is feasible. The model’s high precision and real-time inference capabilities highlight its potential to serve as an AI safeguard to potentially improve patient safety and reduce manual burden on surgical staff. Further validation in clinical settings is warranted.

Deep causal inference for understanding the impact of meteorological variations on traffic

Understanding the causal impact of meteorological variations on traffic conditions (e.g., traffic flow and speed) is crucial for effective traffic prediction and management, as well as the mitigation of adverse weather effects on traffic. However, many existing studies focused on establishing associations between meteorological situations and traffic, rather than delving into causal relationships, especially with deep learning techniques. Consequently, the ability to identify specific meteorological conditions that significantly contribute to traffic congestion or delays is still limited. To address this issue, this study proposes the Meteorological-Traffic Causal Inference Variational Auto-Encoder Model (MT-CIVAE) to estimate the causal impact of fine-grained meteorological variations (e.g., rain and temperature) on traffic. Specifically, MT-CIVAE is based on the Variational Auto-Encoder and consists of an encoder to recover the distribution of latent confounders and a decoder to estimate the conditional probabilities of treatments. Transformer encoder layers are incorporated to analyze the spatial and temporal correlations of historical traffic data to further enhance the inference capability. To evaluate the effectiveness of the proposed approach for causal inference, real-world traffic flow and speed datasets collected from California, along with corresponding fine-grained meteorological datasets, are employed. The counterfactual analysis is conducted using artificially generated meteorological conditions as treatments, which allows for the simulation of hypothetical meteorological scenarios and the evaluation of their potential impact on traffic conditions. This study develops deep learning methods for assessing the causal impact of meteorological variations on traffic dynamics, offering explanations and insights that can assist transportation institutions in guiding post-meteorology traffic management strategies.

V-DAFT: visual technique for texture image defect recognition with denoising autoencoder and fourier transform

Texture is the surface qualities and visual attributes of an object, determined by the arrangement, size, shape, density, and proportion of its fundamental components. In the manufacturing industry, products typically have uniform textures, allowing for automated visual inspections of the product surface to recognize defects. During this process, texture defect recognition techniques can be employed. In this paper, we propose a method that combines a convolutional autoencoder architecture with Fourier transform analysis. We employ a normal reconstructed template as defined in this study. Despite its simple structure and rapid training and inference capabilities, it offers recognition performance comparable to state-of-the-art methods. Fourier transform is a powerful tool for analyzing the frequency domain of images and signals, which is essential for effective defect recognition as texture defects often exhibit characteristic changes in specific frequency ranges. The experiment evaluates the recognition performance using the AUC metric, with the proposed method showing a score of 93.7%. To compare with existing approaches, we present experimental results from previous research, an ablation study of the proposed method, and results based on the high-pass filter used in the Fourier mask.

Multi-species identification and number counting of fish passing through fishway at hydropower stations with LigTraNet

Fishway monitoring can verify the effectiveness of the fishway, optimise the operation mode, and achieve scientific management of fishway operations. Traditional fishway monitoring approaches, hindered by their inefficiency and substantial disruption of fish, are ill-suited for long-term surveillance; thus, employing video monitoring coupled with object detection technology presents an alternative or complementary solution. However, challenges such as the constrained computational capacity of onsite equipment in fishways, complexities involved in model deployment, and sluggish pace of detection are significant hurdles. In this study, by utilising the YOLOv8n model as a benchmark, we engineered a cross-stage partial module with a single convolution (C1) module to replace the existing C2f module with the aim of enhancing performance. We replaced the conventional 2D convolutions in the bottleneck configuration with depthwise separable convolutions and integrated the SimAM module to extract the detailed characteristics of the fish species. By amalgamating LigObNet detection with the DeepSORT algorithm, we established LigTraNet, which is designed to enable precise tracking, identification, and counting of individual fish. The results showed that LigObNet exhibited the lowest complexity and fastest detection speed for underwater fish among similar object recognition and detection models. Compared with the benchmark YOLOv8n model, there were reductions of 8.9% in the network layers, 40.5% in the parameter count, 39.3% in the memory footprint, and 35.8% in the giga floating-point operations and a 38.1% improvement in the inference speed. LigTraNet achieved a total count accuracy rate of 91.8%, demonstrating superior species quantification capabilities over other models with minimal resource usage and rapid inference capabilities, thus offering enhanced practicality for deployment on devices in real-world engineering contexts. This represents a departure from traditional manual monitoring methods for assessing fishway effectiveness, revolutionising aquatic ecological monitoring tools and methodologies and fostering the collaborative advancement of water resource project operations and ecological conservation.

Abstraction and decision fusion architecture for resource-aware image understanding with application on handwriting character classification

Resource-aware image understanding aims to achieve a soft equilibrium by effectively managing the constraints imposed by computational resources while improving the inferential capabilities of image processing systems. It serves a broad range of applications, spanning from embedded systems and IoT devices to budget-friendly smartphones and tablets. The proposed Abstraction and Decision Fusion Architecture (ADFA) addresses this problem through three tiers: abstraction, computation, and decision fusion. The first tier contains various views to generate different abstractions from the original data. These views are processed independently by an array of lightweight models forming the computation tier. They make independent decisions, where the final output is produced using a decision fusion tool in the last tier. To assess the capability of the proposed architecture, we have developed several ADFA-based models for the classification of handwriting data. In this regard, we first defined three data abstractions. Then, we trained support vector machines and fully connected neural networks according to the abstractions, which led to a set of independent basic models. Finally, an adaptive neuro-fuzzy inference system is employed as their hub to increase the accuracy by performing the decision fusion. Our experiments on the EMNIST dataset verify the efficiency and high accuracy of the proposed architecture in recognizing handwritten data, where the model size and the number of Multiply-Accumulate (MAC) operations are significantly smaller than state-of-the-art computational models.

On-tree fruit image segmentation comparing Mask R-CNN and Vision Transformer models. Application in a novel algorithm for pixel-based fruit size estimation

In situ automatic fruit monitoring is of great interest for more accurate and cost-efficient decision making in agriculture. For this purpose, the development of computer vision-based tools is essential. Deep Learning techniques have shown good performance in fruit detection and segmentation. Recently, new models based on Transformer architecture have emerged with promising potential and zero-shot inference capability. In this paper, a Deep Learning model, Mask R-CNN, was trained for on-tree pomegranate fruit segmentation and compared with foundational models based on Vision Transformer, Grounding DINO and Segment Anything Model. Results with Mask R-CNN proved a better performance, according to F1 score and AP metrics, and a lower computational cost, according to prediction time. One of the most interesting derived applications from fruit segmentation is fruit size estimation. However, segmented fruit masks are frequently incomplete due to occlusions. Therefore, image fruit size estimation is not a straightforward process. In this work, we also propose a novel algorithm to estimate and monitor the fruit size in pixel units from the automated masks. A median relative error of 1.39% was obtained, demonstrating the potential and feasibility of future fully-automatic fruit size estimators.

Foundations of reasoning with uncertainty via real-valued logics

Interest in logics with some notion of real-valued truths has existed since at least Boole and has been increasing in AI due to the emergence of neuro-symbolic approaches, though often their logical inference capabilities are characterized only qualitatively. We provide foundations for establishing the correctness and power of such systems. We introduce a rich class of multidimensional sentences, with a sound and complete axiomatization that can be parameterized to cover many real-valued logics, including all the common fuzzy logics, and extend these to weighted versions, and to the case where the truth values are probabilities. Our multidimensional sentences form a very rich class. Each of our multidimensional sentences describes a set of possible truth values for a collection of formulas of the real-valued logic, including which combinations of truth values are possible. Our completeness result is strong, in the sense that it allows us to derive exactly what information can be inferred about the combinations of truth values of a collection of formulas given information about the combinations of truth values of a finite number of other collections of formulas. We give a decision procedure based on linear programming for deciding, for certain real-valued logics and under certain natural assumptions, whether a set of our sentences logically implies another of our sentences. The generality of this work, compared to many previous works on special cases, may provide insights for both existing and new real-valued logics whose inference properties have never been characterized. This work may also provide insights into the reasoning capabilities of deep learning models.

Exploration of an Open Vocabulary Model on Semantic Segmentation for Street Scene Imagery

This study investigates the efficacy of an open vocabulary, multi-modal, foundation model for the semantic segmentation of images from complex urban street scenes. Unlike traditional models reliant on predefined category sets, Grounded SAM uses arbitrary textual inputs for category definition, offering enhanced flexibility and adaptability. The model’s performance was evaluated across single and multiple category tasks using the benchmark datasets Cityscapes, BDD100K, GTA5, and KITTI. The study focused on the impact of textual input refinement and the challenges of classifying visually similar categories. Results indicate strong performance in single-category segmentation but highlighted difficulties in multi-category scenarios, particularly with categories bearing close textual or visual resemblances. Adjustments in textual prompts significantly improved detection accuracy, though challenges persisted in distinguishing between visually similar objects such as buses and trains. Comparative analysis with state-of-the-art models revealed Grounded SAM’s competitive performance, particularly notable given its direct inference capability without extensive dataset-specific training. This feature is advantageous for resource-limited applications. The study concludes that while open vocabulary models such as Grounded SAM mark a significant advancement in semantic segmentation, further improvements in integrating image and text processing are essential for better performance in complex scenarios.

VIRA: An exoplanet atmospheric retrieval framework for JWST transmission spectroscopy

Abstract JWST observations are leading to important new insights into exoplanetary atmospheres through transmission spectroscopy. In order to harness the full potential of the broad spectral range and high sensitivity of JWST, atmospheric retrievals of exoplanets require a high level of robustness and accuracy in the underlying models. We present the VIRA retrieval framework which implements a range of modelling and inference capabilities motivated by early JWST observations of exoplanet transmission spectra. This includes three complementary approaches to modelling atmospheric composition, three atmospheric aerosol models, including a physically-motivated Mie scattering approach, and consideration of correlated noise. VIRA enables a cascading retrieval architecture involving a sequence of retrievals with increasing sophistication. We demonstrate VIRA using a JWST transmission spectrum of the hot Saturn WASP-39 b in the ∼1–5 μm range. In addition to confirming prior chemical inferences, we retrieve molecular abundances for H2O, CO, CO2, SO2 and H2S, resulting in super-solar elemental abundances of log(O/H) = −2.0 ± 0.2, log(C/H) = −2.1 ± 0.2 and log(S/H) = −3.6 ± 0.2, along with C/O and S/O ratios of $0.83^{+0.05}_{-0.07}$ and $0.029^{+0.012}_{-0.009}$, respectively, in the free chemistry case. The abundances correspond to $20.1^{+10.5}_{-8.1}\times$, $28.2^{+16.3}_{-12.1}\times$ and $20.8^{+10.3}_{-7.5}\times$ solar values for O/H, C/H and S/H, respectively, compared to C/H =8.67 ± 0.35 × solar for Saturn. Our results demonstrate how JWST transmission spectroscopy combined with retrieval frameworks like VIRA can measure multi-elemental abundances for giant exoplanets and enable comparative characterisation with solar system planets.