Real Benchmark Research Articles

Lightcone bounds for quantum circuit mapping via uncomplexity

Efficiently mapping quantum circuits onto hardware is integral for the quantum compilation process, wherein a circuit is modified in accordance with a quantum processor’s connectivity. Many techniques currently exist for solving this problem, wherein SWAP-gate overhead is usually prioritized as a cost metric. We reconstitute quantum circuit mapping using tools from quantum information theory, showing that a lower bound, which we dub the lightcone bound, emerges for a circuit executed on hardware. We also develop an initial placement algorithm based on graph similarity search, aiding us in optimally placing circuit qubits onto a device. 600 realistic benchmarks using the IBM Qiskit compiler and a brute-force method are then tested against the lightcone bound, with results unambiguously verifying the veracity of the bound, while permitting trustworthy estimations of minimal overhead in near-term realizations of quantum algorithms. This work constitutes the first use of quantum circuit uncomplexity to practically-relevant quantum computing.

Neutrosophic set and optimized deep learning for classification of chicken Eimeria species: a practical solution for poultry industry

AbstractTo optimize infection control and bolster productivity within the poultry industry, it is imperative to accurately classify Chicken Eimeria species. There are several methods for determining Eimeria disease in chickens. Traditional methods involve watching for clinical symptoms, and macroscopic lesions, and studying the parasite’s biology and oocyst morphology. These methods are frequently time-consuming and labor-intensive, necessitating the manual collection and analysis of samples, which can be especially difficult in large chicken farms. Deep learning algorithms, on the other hand, provide automated, accurate, and non-invasive methods for the detection of Eimeria. This paper proposed a classification model for the automatic classification of chicken Eimeria species. The proposed model is mainly based on integrating neutrosophic set theory and InceptionV3 deep-learning architecture. Three primary phases make up the proposed chicken Eimeria species classification model: the data preprocessing phase, the neutrosophic image conversion phase, and the image classification phase. To address the issue of class imbalance in the adopted dataset and enhance the model’s generalizability, the random oversampling method, and data augmentation techniques are employed during the data preprocessing phase. The preprocessed data is considered to feed the neutrosophic set-based segmentation algorithm, where true, false, and intermediate subsets are extracted. Finally, the true subset is utilized to feed the optimized InceptionV3. To determine the optimal hyperparameter values for InceptionV3, a modified version of the Brown Bear optimization algorithm is proposed in this paper. To evaluate the effectiveness of the proposed model, a real benchmark dataset comprising images of different Eimeria species is adopted. The experimental results revealed that the proposed model offers a more efficient and accurate alternative to traditional methods and state-of-the-art models, enabling faster and more effective diagnosis and treatment of Eimeria infections. It achieved an overall accuracy, specificity, sensitivity, and F1-score of nearly 100%. Additionally, the results showed that the high performance of the proposed model can reduce labor costs and boost throughput, thereby enhancing economic viability even more.

From lawns to meadows: spiders (Arachnida: Araneae) as indicators to measure urban grassland restoration success

In the present study, we investigate how spiders can be used to assess the effectiveness of restoring mesic grasslands on former urban lawns. We compile and analyze a comprehensive dataset, including both past and current data, focusing on the Aachen region. By systematically examining this data, we identify various indicators using different analytical methods. This approach allows us to distinguish distinct species communities, making them useful as diagnostic tools at various stages of habitat development. Additionally, we identify further parameters that are essential for evaluating meadow restoration in urban settings. We highlight the crucial importance of understanding the local species repertoire, as this knowledge is vital for setting realistic benchmarks for restoration projects.

Wasm-R3: Record-Reduce-Replay for Realistic and Standalone WebAssembly Benchmarks

WebAssembly (Wasm for short) brings a new, powerful capability to the web as well as Edge, IoT, and embedded systems. Wasm is a portable, compact binary code format with high performance and robust sandboxing properties. As Wasm applications grow in size and importance, the complex performance characteristics of diverse Wasm engines demand robust, representative benchmarks for proper tuning. Stopgap benchmark suites, such as PolyBenchC and libsodium, continue to be used in the literature, though they are known to be unrepresentative. Porting of more complex suites remains difficult because Wasm lacks many system APIs and extracting real-world Wasm benchmarks from the web is difficult due to complex host interactions. To address this challenge, we introduce Wasm-R3, the first record and replay technique for Wasm. Wasm-R3 transparently injects instrumentation into Wasm modules to record an execution trace from inside the module, then reduces the execution trace via several optimizations, and finally produces a replay module that is executable standalone without any host environment-on any engine. The benchmarks created by our approach are (i) realistic, because the approach records real-world web applications, (ii) faithful to the original execution, because the replay benchmark includes the unmodified original code, only adding emulation of host interactions, and (iii) standalone, because the replay benchmarks run on any engine. Applying Wasm-R3 to web-based Wasm applications in the wild demonstrates the correctness of our approach as well as the effectiveness of our optimizations, which reduce the recorded traces by 99.53% and the size of the replay benchmark by 9.98%. We release the resulting benchmark suite of 27 applications, called Wasm-R3-Bench, to the community, to inspire a new generation of realistic and standalone Wasm benchmarks.

Hyperspectral endmember extraction using convexity based purity index

The endmember extraction is a challenging problem in spectral unmixing (SU) of a mixed pixel in hyperspectral imagery. There are plenty of attempts to solve the endmember extraction problem. Still, the pure pixel assumption-based algorithms have probably been used most in solving the endmember extraction of SU due to the light computational burden. These pure pixel assumption-based algorithms usually follow one of the criteria: (1) Maximum simplex volume or (2) Extreme projection on a subspace. We propose a novel integrated framework that uses both the criteria mentioned above and the proposed one is referred to as the Convexity-based Pure Index (CPI) algorithm. The CPI generates a fixed number of convex sets based on the number of available bands in the hyperspectral image. The algorithm defines the purity score based on the availability of pixels in the convex sets for the two-band data. The CPI has been compared with contemporary algorithms such as Automatic Target Generation Process (ATGP), Vertex Component Analysis (VCA), Pixel Purity Index (PPI), Successive Volume MAXimization (SVMAX), Alternating Volume MAXimization (AVMAX), TRIple-P: P-norm based Pure Pixel identification (TRIP), Successive Decoupled Volume Max–Min (SDVMM), Negative ABundance-Oriented (NABO), and Entropy-based Convex Set Optimization (ECSO). The metrics, Spectral Angle Distance (SAD) and Spectral Information Divergence (SID) used in the comparison were improved up to 5.9% and 9%, respectively. The CPI outperforms prevailing algorithms on real benchmark data and new AVIRIS-NG data. The robustness of the CPI is also tested for various noisy synthetic data. The efficacy of the proposed algorithm is also tested by using qualitative analysis by visualizing the spectra comparison, and abundance maps for all real data.

A realistic benchmark for differential abundance testing and confounder adjustment in human microbiome studies

BackgroundIn microbiome disease association studies, it is a fundamental task to test which microbes differ in their abundance between groups. Yet, consensus on suitable or optimal statistical methods for differential abundance testing is lacking, and it remains unexplored how these cope with confounding. Previous differential abundance benchmarks relying on simulated datasets did not quantitatively evaluate the similarity to real data, which undermines their recommendations.ResultsOur simulation framework implants calibrated signals into real taxonomic profiles, including signals mimicking confounders. Using several whole meta-genome and 16S rRNA gene amplicon datasets, we validate that our simulated data resembles real data from disease association studies much more than in previous benchmarks. With extensively parametrized simulations, we benchmark the performance of nineteen differential abundance methods and further evaluate the best ones on confounded simulations. Only classic statistical methods (linear models, the Wilcoxon test, t-test), limma, and fastANCOM properly control false discoveries at relatively high sensitivity. When additionally considering confounders, these issues are exacerbated, but we find that adjusted differential abundance testing can effectively mitigate them. In a large cardiometabolic disease dataset, we showcase that failure to account for covariates such as medication causes spurious association in real-world applications.ConclusionsTight error control is critical for microbiome association studies. The unsatisfactory performance of many differential abundance methods and the persistent danger of unchecked confounding suggest these contribute to a lack of reproducibility among such studies. We have open-sourced our simulation and benchmarking software to foster a much-needed consolidation of statistical methodology for microbiome research.

Nighttime Pothole Detection: A Benchmark

In the field of computer vision, the detection of road potholes at night represents a critical challenge in enhancing the safety of intelligent transportation systems. Ensuring road safety is of paramount importance, particularly in promptly repairing pothole issues. These abrupt road depressions can easily lead to vehicle skidding, loss of control, and even traffic accidents, especially when water has pooled in or submerged the potholes. Therefore, the detection and recognition of road potholes can significantly reduce vehicle damage and the incidence of safety incidents. However, research on road pothole detection lacks high-quality annotated datasets, particularly under low-light conditions at night. To address this issue, this study introduces a novel Nighttime Pothole Dataset (NPD), independently collected and comprising 3831 images that capture diverse scene variations. The construction of this dataset aims to counteract the insufficiency of existing data resources and strives to provide a richer and more realistic benchmark. Additionally, we develop a baseline detector, termed WT-YOLOv8, for the proposed dataset, based on YOLOv8. We also evaluate the performance of the improved WT-YOLOv8 method and eight state-of-the-art object detection methods on the NPD and the COCO dataset. The experimental results on the NPD demonstrate that WT-YOLOv8 achieves a 2.3% improvement in mean Average Precision (mAP) over YOLOv8. In terms of the key metrics—AP@0.5 and AP@0.75—it shows enhancements of 1.5% and 2.8%, respectively, compared to YOLOv8. The experimental results provide valuable insights into each method’s strengths and weaknesses under low-light conditions. This analysis highlights the importance of a specialized dataset for nighttime pothole detection and shows variations in accuracy and robustness among methods, emphasizing the need for improved nighttime pothole detection techniques. The introduction of the NPD is expected to stimulate further research, encouraging the development of advanced algorithms for nighttime pothole detection, ultimately leading to more flexible and reliable road maintenance and road safety.

Rewriting the Infinite Chase for Guarded TGDs

Guarded tuple-generating dependencies (GTGDs) are a natural extension of description logics and referential constraints. It has long been known that queries over GTGDs can be answered by a variant of the chase —a quintessential technique for reasoning with dependencies. However, there has been little work on concrete algorithms and even less on implementation. To address this gap, we revisit Datalog rewriting approaches to query answering, where a set of GTGDs is transformed to a Datalog program that entails the same base facts on each base instance. We show that a rewriting consists of “shortcut” rules that circumvent certain chase steps, we present several algorithms that compute a rewriting by deriving such “shortcuts” efficiently, and we discuss important implementation issues. Finally, we show empirically that our techniques can process complex GTGDs derived from synthetic and real benchmarks and are thus suitable for practical use.

Joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering

Recent advancements in multiple kernel learning-based graph clustering methods have demonstrated significant promise in effectively learning a consensus kernel matrix from various candidate kernel matrices. This approach enables the creation of a low-dimensional representation in the kernel space of high-dimensional data through self-expressiveness. However, a key challenge remains in capturing the latent geometric properties embedded within different kernel matrices to enhance data representation. In this paper, we propose a novel method called joint consensus kernel learning and adaptive hypergraph regularization for graph-based clustering (JKHR). Our approach integrates an innovative adaptive hypergraph Laplacian regularizer, which is characterized by the fusion of multiple nearest neighbor kernel graphs, into the multiple kernel learning-based graph clustering framework. JKHR jointly and adaptively optimizes both the consensus kernel matrix and the hypergraph Laplacian regularizer to achieve a low-dimensional representation that effectively preserves the intrinsic geometry of the data. Experimental results on both synthetic and real benchmark datasets demonstrate that JKHR outperforms state-of-the-art self-expressiveness-based graph clustering methods as well as traditional clustering techniques.

SAT-based Formal Verification of Fault Injection Countermeasures for Cryptographic Circuits

Fault injection attacks represent a type of active, physical attack against cryptographic circuits. Various countermeasures have been proposed to thwart such attacks, however, the design and implementation of which are intricate, error-prone, and laborious. The current formal fault-resistance verification approaches are limited in efficiency and scalability. In this paper, we formalize the fault-resistance verification problem and show that it is coNP-complete. We then devise a novel approach for encoding the fault-resistance verification problem as the Boolean satisfiability (SAT) problem so that modern off-the-shelf SAT solvers can be utilized. The approach is implemented in an open-source tool FIRMER which is evaluated extensively on realistic cryptographic circuit benchmarks. The experimental results show that FIRMER is able to verify fault-resistance of almost all (72/76) benchmarks in 3 minutes (the other three are verified in 35 minutes and the hardest one is verified in 4 hours). In contrast, the prior approach fails on 31 fault-resistance verification tasks even after 24 hours (per task).

CICIoMT2024: A benchmark dataset for multi-protocol security assessment in IoMT

The Internet of Things (IoT) is increasingly integrated into daily life, particularly in healthcare, through the Internet of Medical Things (IoMT). IoMT devices support services like continuous health monitoring but raise significant cybersecurity concerns due to their vulnerability to various attacks. The complexity and data volume of IoMT network traffic requires advanced methods to enhance security and reliability. Machine Learning (ML) offers techniques to detect, prevent, and mitigate cyberattacks. However, existing benchmark datasets lack essential features for robust IoMT security solutions, such as a reduced number of real devices, a limited variety of attacks, and a lack of extensive profiling. We propose a realistic benchmark dataset for IoMT security solutions development and evaluation to address these gaps. We executed 18 attacks on an IoMT testbed with 40 devices (25 real and 15 simulated), using protocols like Wi-Fi, MQTT, and Bluetooth. Supporting technologies, including dedicated network traffic collectors and a Faraday Cage, ensured data quality. The attacks fall into five categories: DDoS, DoS, Recon, MQTT, and spoofing. We aim to establish a baseline that complements existing datasets, aiding researchers in creating secure healthcare systems using ML. Beyond simulating attacks, we capture the lifecycle of IoMT devices from network entry to exit through profiling, allowing classifiers to identify device anomalies. The resulting CICIoMT2024 dataset, published on the CIC dataset page, demonstrates that various methods can classify IoMT cyberattacks. This effort supports new IoMT security solutions and contributes to the broader field of cybersecurity in healthcare, ensuring more reliable IoMT device deployment.

BERT-ASC: Auxiliary-sentence construction for implicit aspect learning in sentiment analysis

Aspect-based sentiment analysis (ABSA) aims to associate a text with a set of aspects and infer their respective sentimental polarities. State-of-the-art approaches are built on fine-tuning pre-trained language models, focusing on learning aspect-specific representations from the corpus. However, aspects are often expressed implicitly, making implicit mapping challenging without sufficient labeled examples, which may be scarce in real-world scenarios. This paper proposes a unified framework to address aspect categorization and aspect-based sentiment subtasks. We introduce a mechanism to construct an auxiliary-sentence for the implicit aspect using the corpus’s semantic information. We then encourage BERT to learn aspect-specific representation in response to this auxiliary-sentence, not the aspect itself. We evaluate our approach on real benchmark datasets for both ABSA and Targeted-ABSA tasks. Our experiments show that it consistently achieves state-of-the-art performance in aspect categorization and aspect-based sentiment across all datasets, with considerable improvement margins. The BERT-ASC code is available at https://github.com/amurtadha/BERT-ASC.

Tier-specific energy benchmarking for disparate schools in developing South Africa

The increasing impacts of human-induced climate change in developing countries have spurred government policies, activism, and sustainability research aimed at reducing energy consumption. Understanding the electricity usage of buildings is crucial to cutting carbon emissions and achieving cost savings. This study addresses the challenge of establishing realistic and relevant energy benchmarks for educational institutions in developing countries, specifically focusing on the Western Cape, South Africa. Schools in this region exhibit significant differences in energy intensities but are currently assessed using the same reference standard. A top-down analysis was performed using descriptive statistics to develop energy performance benchmarks tailored to unique patterns of energy consumption in schools. Data from 31 less affluent schools were collected using smart meters to ensure accuracy. The proposed reference benchmarks, ranging from 12 to 37kWh/m2 per year, are significantly lower than the existing 60kWh/m2 per year benchmark, demonstrating substantial potential for energy savings. This nuanced benchmarking approach accounts for seasonal and term variations in energy usage, providing a more accurate comparison across schools. The research introduces a novel, context-sensitive benchmarking method that extends beyond existing standards by incorporating these variations. It underscores the importance of localized benchmarks for achieving school energy efficiency, contributing to environmental preservation and financial savings. The proposed benchmarks offer a robust framework for policymakers, standard bureaus, and education departments to craft energy efficiency policies that drive progress in the education sector. By addressing the unique energy usage patterns of schools, this approach facilitates targeted interventions, leading to improved energy management and sustainability.

Stochastic image spectroscopy: a discriminative generative approach to hyperspectral image modelling and classification

This paper introduces a new latent variable probabilistic framework for representing spectral data of high spatial and spectral dimensionality, such as hyperspectral images. We use a generative Bayesian model to represent the image formation process and provide interpretable and efficient inference and learning methods. Surprisingly, our approach can be implemented with simple tools and does not require extensive training data, detailed pixel-by-pixel labeling, or significant computational resources. Numerous experiments with simulated data and real benchmark scenarios show encouraging image classification performance. These results validate the unique ability of our framework to discriminate complex hyperspectral images, irrespective of the presence of highly discriminative spectral signatures.

Interpretable Multiscale Machine Learning‐Based Parameterizations of Convection for ICON

AbstractMachine learning (ML)‐based parameterizations have been developed for Earth System Models (ESMs) with the goal to better represent subgrid‐scale processes or to accelerate computations. ML‐based parameterizations within hybrid ESMs have successfully learned subgrid‐scale processes from short high‐resolution simulations. However, most studies used a particular ML method to parameterize the subgrid tendencies or fluxes originating from the compound effect of various small‐scale processes (e.g., radiation, convection, gravity waves) in mostly idealized settings or from superparameterizations. Here, we use a filtering technique to explicitly separate convection from these processes in simulations with the Icosahedral Non‐hydrostatic modeling framework (ICON) in a realistic setting and benchmark various ML algorithms against each other offline. We discover that an unablated U‐Net, while showing the best offline performance, learns reverse causal relations between convective precipitation and subgrid fluxes. While we were able to connect the learned relations of the U‐Net to physical processes this was not possible for the non‐deep learning‐based Gradient Boosted Trees. The ML algorithms are then coupled online to the host ICON model. Our best online performing model, an ablated U‐Net excluding precipitating tracer species, indicates higher agreement for simulated precipitation extremes and mean with the high‐resolution simulation compared to the traditional scheme. However, a smoothing bias is introduced both in water vapor path and mean precipitation. Online, the ablated U‐Net significantly improves stability compared to the non‐ablated U‐Net and runs stable for the full simulation period of 180 days. Our results hint to the potential to significantly reduce systematic errors with hybrid ESMs.

Multi-scale hierarchical model for long-term time series forecasting

Long-term time series forecasting (LTSF) has become an urgent requirement in many applications, such as wind power supply planning. This is a highly challenging task because it requires considering both the complex frequency-domain and time-domain information in long-term time series simultaneously. However, existing work only considers potential patterns in a single domain (e.g., time or frequency domain), whereas a large amount of time-frequency domain information exists in real-world LTSFs. In this paper, we propose a multi-scale hierarchical network (MHNet) based on time-frequency decomposition to solve the above problem. MHNet first introduces a multi-scale hierarchical representation, extracting and learning features of time series in the time domain, and gradually builds up a global understanding and representation of the time series at different time scales, enabling the model to process time series over lengthy periods of time with lower computational complexity. Then, the robustness to noise is enhanced by employing a transformer that leverages frequency-enhanced decomposition to model global dependencies and integrates attention mechanisms in the frequency domain. Meanwhile, forecasting accuracy is further improved by designing a periodic trend decomposition module for multiple decompositions to reduce input-output fluctuations. Experiments on five real benchmark datasets show that the forecasting accuracy and computational efficiency of MHNet outperform state-of-the-art methods.

Effective semi-supervised graph clustering with pairwise constraints

Semi-supervised graph clustering with constraints has received considerable attention in the last decade. They use pre-given constraints to guide the clustering process and improve the performance. Nonetheless, most of related research works still face two main issues: (1) cannot-link problem: how to ensure that instances under the cannot-link constraint are located in different clusters as much as possible. (2) The high computational cost caused by eigendecomposition. To tackle these problems, this paper proposes a novel method for directly solving the clustering indicator matrix without high complexity computational steps. Then, we propose an effective and simple algorithm guided by the pre-given constraints, which is able to simultaneously optimize the associated multiple pairs of constraints, allowing the associated constraints to achieve the related co-optimal solution. Extensive experiments are conducted to verify that our approach significantly reduces the violation rate of the prior constraints and the clustering performance of the proposed method outperforms seven other state-of-the-art semi-supervised clustering methods on 16 real benchmark datasets.

On Challenges of Monocular Pose Estimation for Endoluminal Navigation

Accurate endoscope navigation is vital for precise sampling and drug delivery during endoluminal procedures. Current systems rely on sensorized endoscopes for real-time bronchoscope tracking, but they are costly and require specialised settings. Camera pose estimation offers an alternative for endoscope motion tracking and navigation. In this study, we assess state-of-the-art monocular pose estimation methods for endoluminal navigation. We analyze multiple datasets and diverse feature tracking frameworks, including SIFT, ORB, BRISK, FREAK, AKAZE, SuperPoint, LoFTR, and EndoSfMLearner. Our findings reveal that, while modern deep learning-based feature detectors like SuperPoint and LoFTR yield denser features compared to traditional methods, their overall quality and reliability remain suboptimal. This insight is crucial for understanding the limitations of current monocular odometry in endoluminal environments. We also emphasize the challenges in generalizing learned frameworks developed for other odometry applications to endoscopic applications, stressing the need for more publicly available datasets. To fill this gap, we introduce a clinical bronchoscopy dataset with ground truth camera pose data. This dataset serves as a realistic benchmark for improving feature-based odometry in surgical navigation and encourages tailored framework development for learned pose estimation in endoluminal interventions.

Extract Implicit Semantic Friends and Their Influences from Bipartite Network for Social Recommendation

Social recommendation often incorporates trusted social links with user-item interactions to enhance rating prediction. Although methods that aggregate explicit social links have shown promising prospects, they are often constrained by the absence of explicit social data and the assumption of homogeneity, thus overlooking variations in social influence and consistency. These limitations hinder semantic expression and recommendation performance. Therefore, we propose a novel framework for social recommendation. First, we design a bipartite network embedding scheme, which learns vertex representations in the embedding space by modeling 1st-order explicit relations and higher-order implicit relations between vertices. Then, the similarity of the embedding vectors is used to extract top-k semantically consistent friends for each user. Next, we design an algorithm to assign a specific influence value to each user. Finally, we combine the top-k friends of the user and their influence values into an ensemble and add it as a regularization term to the rating prediction process of the user to correct the bias. Experiments on three real benchmark datasets show significant improvements in EISF over state-of-the-art methods.

TA-Detector: A GNN-based Anomaly Detector via Trust Relationship

With the rise of mobile Internet and AI, social media integrating short messages, images, and videos has developed rapidly. As a guarantee for the stable operation of social media, information security, especially graph anomaly detection (GAD), has become a hot issue inspired by the extensive attention of researchers. Most GAD methods are mainly limited to enhancing the homophily or considering homophily and heterophilic connections. Nevertheless, due to the deceptive nature of homophily connections among anomalies, the discriminative information of the anomalies can be eliminated. To alleviate the issue, we explore a novel method TA-Detector in GAD by introducing the concept of trust into the classification of connections. In particular, the proposed approach adopts a designed trust classier to distinguish trust and distrust connections with the supervision of labeled nodes. Then, we capture the latent factors related to GAD by graph neural networks (GNN), which integrate node interaction type information and node representation. Finally, to identify anomalies in the graph, we use the residual network mechanism to extract the deep semantic embedding information related to GAD. Experimental results on two real benchmark datasets verify that our proposed approach boosts the overall GAD performance in comparison to benchmark baselines.