Real Benchmark Research Articles

Convex Subspace Clustering by Adaptive Block Diagonal Representation.

Subspace clustering is a class of extensively studied clustering methods where the spectral-type approaches are its important subclass. Its key first step is to desire learning a representation coefficient matrix with block diagonal structure. To realize this step, many methods were successively proposed by imposing different structure priors on the coefficient matrix. These impositions can be roughly divided into two categories, i.e., indirect and direct. The former introduces the priors such as sparsity and low rankness to indirectly or implicitly learn the block diagonal structure. However, the desired block diagonalty cannot necessarily be guaranteed for noisy data. While the latter directly or explicitly imposes the block diagonal structure prior such as block diagonal representation (BDR) to ensure so-desired block diagonalty even if the data is noisy but at the expense of losing the convexity that the former's objective possesses. For compensating their respective shortcomings, in this article, we follow the direct line to propose adaptive BDR (ABDR) which explicitly pursues block diagonalty without sacrificing the convexity of the indirect one. Specifically, inspired by Convex BiClustering, ABDR coercively fuses both columns and rows of the coefficient matrix via a specially designed convex regularizer, thus naturally enjoying their merits and adaptively obtaining the number of blocks. Finally, experimental results on synthetic and real benchmarks demonstrate the superiority of ABDR to the state-of-the-arts (SOTAs).

On Exact Feature Screening in Ultrahigh-Dimensional Binary Classification

We propose a new model-free feature screening method based on energy distances for ultrahigh-dimensional binary classification problems. With a high probability, the proposed method retains only relevant features after discarding all the noise variables. The proposed screening method is also extended to identify pairs of variables that are marginally undetectable but have differences in their joint distributions. Finally, we build a classifier that maintains coherence between the proposed feature selection criteria and discrimination method, and also establish its risk consistency. An extensive numerical study with simulated and real benchmark datasets shows clear and convincing advantages of our proposed method over the state-of-the-art methods. Supplementary materials for this article are available online.

Developing realistic benchmarks for glaucoma care delivery

ABSTRACT Clinical Relevance Realistic benchmarks can serve as comparators for optometrists wishing to engage in clinical practice audits of their glaucoma care. Background The iCareTrack study established the appropriateness of glaucoma care delivery through clinical record audits of Australian optometry practices. Benchmarks required for monitoring and improving glaucoma care delivery do not exist. This study developed realistic benchmarks for glaucoma care and then benchmarked the performance of practices from the iCareTrack study to establish aspects of care that warrant attention from quality improvement initiatives. Methods Benchmarks were developed from the pre-existing iCareTrack dataset using the Achievable Benchmarks of Care (ABC) method. The iCareTrack study had audited the appropriateness of glaucoma care delivery against 37 clinical indicators for 420 randomly sampled glaucoma patient records from 42 Australian optometry practices. The four-step ABC method calculates benchmarks based on the top 10% of best-performing practices adjusted for low patient encounter numbers. iCareTrack results were compared to the benchmarks to explore the distribution of practices that were at, above or below benchmark. Results Benchmarks were developed for 34 of 37 iCareTrack indicators. For 26 (of 34) indicators, the benchmarks were at or above 90% appropriateness. The benchmarks for 14 (of 34) iCareTrack indicators were met by more than 80% of eligible practices, indicating excellent performance. Some aspects of glaucoma care such as peripheral anterior angle assessment, applanation tonometry, and visual field assessment appeared to be delivered sub-optimally by optometrists when compared to the benchmarks. Conclusion This study established benchmarks for glaucoma care delivery in optometry practices that reflect realistic and top achievable performance. The large number of indicators with benchmarks above 90% confirmed that glaucoma care can and should be delivered by optometrists at very high levels of appropriateness. Benchmarking identified pockets of sub-optimal performance that can now be targeted by quality improvement initiatives.

“I do not know! but why?” — Local model-agnostic example-based explanations of reject

Machine learning based decision making systems in safety critical areas place high demands on the accuracy and generalization ability of the underlying model. A common strategy to deal with uncertainties and possible mistakes is offered by learning with reject option, i.e. a model can refrain from prediction in ambiguous cases and leave the decision to a human expert. Yet, as for the models themselves, human decision-making is hampered by the fact that reject options are often implemented as black-box rules: Experts cannot readily understand the reasons for rejection.In this work, we propose a model-agnostic framework that enriches classification with reject option by explanation mechanisms. More specifically, we combine conformal prediction as a popular mathematically based technology of certainty estimation with local surrogates derived for the region of interest. This allows us to provide local explanations in terms of example-based explanation methods, including counterfactual, semi-factual, and factual methods. We demonstrate the performance of this technology through a series of benchmarks using 6 different data sets; the associated code is open source.11This article represents a substantially extended version of the conference paper Artelt et al. (2022). In addition to a more detailed explanation and derivation of the algorithms, we explore semi-factual and factual explanation methods and provide evaluations on realistic benchmark data sets.

A Machine Learning Approach to Simulate Gene Expression and Infer Gene Regulatory Networks.

The ability to simulate gene expression and infer gene regulatory networks has vast potential applications in various fields, including medicine, agriculture, and environmental science. In recent years, machine learning approaches to simulate gene expression and infer gene regulatory networks have gained significant attention as a promising area of research. By simulating gene expression, we can gain insights into the complex mechanisms that control gene expression and how they are affected by various environmental factors. This knowledge can be used to develop new treatments for genetic diseases, improve crop yields, and better understand the evolution of species. In this article, we address this issue by focusing on a novel method capable of simulating the gene expression regulation of a group of genes and their mutual interactions. Our framework enables us to simulate the regulation of gene expression in response to alterations or perturbations that can affect the expression of a gene. We use both artificial and real benchmarks to empirically evaluate the effectiveness of our methodology. Furthermore, we compare our method with existing ones to understand its advantages and disadvantages. We also present future ideas for improvement to enhance the effectiveness of our method. Overall, our approach has the potential to greatly improve the field of gene expression simulation and gene regulatory network inference, possibly leading to significant advancements in genetics.

Fourier-DeepONet: Fourier-enhanced deep operator networks for full waveform inversion with improved accuracy, generalizability, and robustness

Full waveform inversion (FWI) infers the subsurface structure information from seismic waveform data by solving a non-convex optimization problem. Data-driven FWI has been increasingly studied with various neural network architectures to improve accuracy and computational efficiency. Nevertheless, the applicability of pre-trained neural networks is severely restricted by potential discrepancies between the source function used in the field survey and the one utilized during training. Here, we develop a Fourier-enhanced deep operator network (Fourier-DeepONet) for FWI with the generalization of seismic sources, including the frequencies and locations of sources. Specifically, we employ the Fourier neural operator as the decoder of DeepONet, and we utilize source parameters as one input of Fourier-DeepONet, facilitating the resolution of FWI with variable sources. To test Fourier-DeepONet, we develop three new and realistic FWI benchmark datasets (FWI-F, FWI-L, and FWI-FL) with varying source frequencies, locations, or both. Our experiments demonstrate that compared with existing data-driven FWI methods, Fourier-DeepONet obtains more accurate predictions of subsurface structures in a wide range of source parameters. Moreover, the proposed Fourier-DeepONet exhibits superior robustness when handling data with Gaussian noise or missing traces and sources with Gaussian noise, paving the way for more reliable and accurate subsurface imaging across diverse real conditions.

Improving power and performance of on-chip network through virtual channel sharing and power gating

Networks-on-Chip (NoCs) are becoming the communication backbone in multicore chips. The router buffer holds a critical significance for the communication performance of NoC. However, the bursty traffic means that the buffers are idle for many periods and generate a significant amount of static power. Moreover, due to the uneven traffic distribution, some buffers within the router stand idle even at network saturation. Both the performance and power budget of NoC heavily depends on the router’s buffer budget. Therefore, we desperately demand a mechanism to maximize the usage of idle buffers to balance the load and improve the performance ceiling of NoC. Dynamically based on network traffic, early switching on or off of buffers to save power consumption. We propose a modified microarchitecture of the virtual channel sharing router, MVCR. The MVCR innovatively applies aggressive fine-grained power gating mechanisms to the idle buffers of each directional input port (east, west, south, and north) to achieve static power savings. Moreover, the MVCR further extends the sleeping VC period through an efficient packet-transmission adjustment mechanism, thus achieving maximum static power savings and improved network performance. Synthetic applications and real benchmarks have been used to validate MCVR. Compared to traditional router scheme, MVCR increases throughput by 30%, reduces power consumption by 47%, and has a latency overhead of about 6%. The key performance metrics are better than other solutions, and the added area cost is moderate.

Bugs4Q: A benchmark of existing bugs to enable controlled testing and debugging studies for quantum programs

Realistic benchmarks of reproducible bugs and fixes are vital to good experimental evaluation of debugging and testing approaches. However, there is no suitable bug benchmark suite that can systematically evaluate the debugging and testing methods of quantum programs until now. This paper proposes Bugs4Q, a benchmark of forty-two real, manually validated Qiskit bugs from three popular platforms (GitHub, StackOverflow, and Stack Exchange) in programming, supplemented with test cases to reproduce buggy behaviors. Bugs4Q also provides interfaces for accessing the buggy and fixed versions of the Qiskit programs and executing the corresponding source code and unit tests, facilitating the reproducible empirical studies and comparisons of Qiskit program debugging and testing tools. Bugs4Q is publicly available at https://github.com/Z-928/Bugs4Q-Framework.Editor’s note: Open Science material was validated by the Journal of Systems and Software Open Science Board.

Sparsity Fuzzy C-Means Clustering With Principal Component Analysis Embedding

The clustering method has been widely used in data mining, pattern recognition, and image identification. Fuzzy c-means (FCM) is a soft clustering method that introduces the concept of membership. In this method, the fuzzy membership matrix is obtained by calculating the distance between data points in the original space. However, these methods may yield suboptimal results owing to the influence of redundant features. Moreover, FCM is always sensitive to noise points and heavily subject to outliers. In this paper, we propose a method called sparsity FCM clustering with principal component analysis embedding (P_SFCM). We simultaneously conduct principal component analysis (PCA) and membership learning, and then add an additional weighting factor for each data point. The goal of this operation is to identify the noise or outliers. Overall, the benefit of our framework is that it retains most of the information in the subspace while improving the robustness of the noise. In this paper, we employ an iterative optimization algorithm to efficiently solve our model. To verify the reliability of the proposed method, we conduct a convergence analysis, noise robustness analysis, and multi-cluster experiments. Furthermore, comparative experiments are conducted on both synthetic and real benchmark datasets. The experimental results show that the P_SFCM is competitive with comparable methods.

Supervised Gradual Machine Learning for Aspect-Term Sentiment Analysis

Abstract Recent work has shown that Aspect-Term Sentiment Analysis (ATSA) can be effectively performed by Gradual Machine Learning (GML). However, the performance of the current unsupervised solution is limited by inaccurate and insufficient knowledge conveyance. In this paper, we propose a supervised GML approach for ATSA, which can effectively exploit labeled training data to improve knowledge conveyance. It leverages binary polarity relations between instances, which can be either similar or opposite, to enable supervised knowledge conveyance. Besides the explicit polarity relations indicated by discourse structures, it also separately supervises a polarity classification DNN and a binary Siamese network to extract implicit polarity relations. The proposed approach fulfills knowledge conveyance by modeling detected relations as binary features in a factor graph. Our extensive experiments on real benchmark data show that it achieves the state-of-the-art performance across all the test workloads. Our work demonstrates clearly that, in collaboration with DNN for feature extraction, GML outperforms pure DNN solutions.

Concurrent Multi-Label Prediction in Event Streams

Streams of irregularly occurring events are commonly modeled as a marked temporal point process. Many real-world datasets such as e-commerce transactions and electronic health records often involve events where multiple event types co-occur, e.g. multiple items purchased or multiple diseases diagnosed simultaneously. In this paper, we tackle multi-label prediction in such a problem setting, and propose a novel Transformer-based Conditional Mixture of Bernoulli Network (TCMBN) that leverages neural density estimation to capture complex temporal dependence as well as probabilistic dependence between concurrent event types. We also propose potentially incorporating domain knowledge in the objective by regularizing the predicted probability. To represent probabilistic dependence of concurrent event types graphically, we design a two-step approach that first learns the mixture of Bernoulli network and then solves a least-squares semi-definite constrained program to numerically approximate the sparse precision matrix from a learned covariance matrix. This approach proves to be effective for event prediction while also providing an interpretable and possibly non-stationary structure for insights into event co-occurrence. We demonstrate the superior performance of our approach compared to existing baselines on multiple synthetic and real benchmarks.

Only a Few Classes Confusing: Pixel-Wise Candidate Labels Disambiguation for Foggy Scene Understanding

Not all semantics become confusing when deploying a semantic segmentation model for real-world scene understanding of adverse weather. The true semantics of most pixels have a high likelihood of appearing in the few top classes according to confidence ranking. In this paper, we replace the one-hot pseudo label with a candidate label set (CLS) that consists of only a few ambiguous classes and exploit its effects on self-training-based unsupervised domain adaptation. Specifically, we formulate the problem as a coarse-to-fine process. In the coarse-level process, adaptive CLS selection is proposed to pick a minimal set of confusing candidate labels based on the reliability of label predictions. Then, representation learning and label rectification are iteratively performed to facilitate feature clustering in an embedding space and to disambiguate the confusing semantics. Experimentally, our method outperforms the state-of-the-art methods on three realistic foggy benchmarks.

Scalable and Globally Optimal Generalized L₁ K-center Clustering via Constraint Generation in Mixed Integer Linear Programming

The k-center clustering algorithm, introduced over 35 years ago, is known to be robust to class imbalance prevalent in many clustering problems and has various applications such as data summarization, document clustering, and facility location determination. Unfortunately, existing k-center algorithms provide highly suboptimal solutions that can limit their practical application, reproducibility, and clustering quality. In this paper, we provide a novel scalable and globally optimal solution to a popular variant of the k-center problem known as generalized L_1 k-center clustering that uses L_1 distance and allows the selection of arbitrary vectors as cluster centers. We show that this clustering objective can be reduced to a mixed-integer linear program (MILP) that facilitates globally optimal clustering solutions. However, solving such a MILP may be intractable for large datasets; to remedy this, we present a scalable algorithm that leverages constraint generation to efficiently and provably converge to its global optimum. We further enhance outlier handling through a simple but elegant extension to our MILP objective. We first evaluate our algorithm on a variety of synthetic datasets to better understand its properties and then validate on 20 real benchmark datasets where we compare its performance to both traditional L_1 distance k-center and k-medians baselines. Our results demonstrate significant suboptimality of existing algorithms in comparison to our approach and further demonstrate that we can find optimal generalized L_1 k-center clustering solutions up to an unprecedented 1,000,000 data points.

New Distributed Optimization Method for TSO–DSO Coordinated Grid Operation Preserving Power System Operator Sovereignty

Electrical power system operators (SOs) are free to realize grid operations according to their own strategies. However, because resulting power flows also depend on the actions of neighboring SOs, appropriate coordination is needed to improve the resulting system states from an overall perspective and from an individual SO perspective. In this paper, a new method is presented that preserves the data integrity of the SOs and their independent operation of their grids. This method is compared with a non-coordinated local control and another sequential method that has been identified as the most promising distributed optimization method in previous research. The time series simulations use transformer tap positioning as well as generation unit voltage setpoints and reactive power injections as flexibilities. The methods are tested on a multi-voltage, multi-SO, realistic benchmark grid with different objective combinations of the SOs. In conclusion, the results of the new method are much closer to the theoretical optimum represented by central optimization than those of the other two methods. Furthermore, the introduced method integrates a sophisticated procedure to provide fairness between SOs that is missing in other methods.

Generative Models Should at Least Be Able to Design Molecules That Dock Well: A New Benchmark.

Designing compounds with desired properties is a key element of the drug discovery process. However, measuring progress in the field has been challenging due to the lack of realistic retrospective benchmarks, and the large cost of prospective validation. To close this gap, we propose a benchmark based on docking, a widely used computational method for assessing molecule binding to a protein. Concretely, the goal is to generate drug-like molecules that are scored highly by SMINA, a popular docking software. We observe that various graph-based generative models fail to propose molecules with a high docking score when trained using a realistically sized training set. This suggests a limitation of the current incarnation of models for de novo drug design. Finally, we also include simpler tasks in the benchmark based on a simpler scoring function. We release the benchmark as an easy to use package available at https://github.com/cieplinski-tobiasz/smina-docking-benchmark. We hope that our benchmark will serve as a stepping stone toward the goal of automatically generating promising drug candidates.

Mutation-Based Minimal Test Suite Generation for Boolean Expressions

Boolean expressions are highly involved in control flows of programs and software specifications. Coverage criteria for Boolean expressions aim at producing minimal test suites to detect software faults. There exist various testing criteria, efficiency of which is usually evaluated through mutation analysis. This paper proposes an integer programming-based minimal test suite generation technique relying on mutation analysis. The proposed technique also takes into account the cost of fault detection. The technique is optimal such that the resulting test suite guarantees to detect all the mutants under given fault assumptions, while maximizing the average percentage of fault detection of a test suite. Therefore, the approach presented can also be considered as a reference method to check the efficiency of any common technique. The method is evaluated using four well-known real benchmark sets of Boolean expressions and is also exemplary compared with MCDC criterion. The results show that the test suites generated by the proposed method provide better fault coverage values and faster fault detection.

Examining Antenatal Corticosteroid Administration at a Single Center via a New Proposed Quality Metric [ID: 1375947

INTRODUCTION: The Society for Maternal-Fetal Medicine (SMFM) proposed a new quality metric in 2022 to increase the rate of antenatal corticosteroid (ACS) provision for early preterm deliveries and minimize ACS for pregnancies ending in term deliveries. This study evaluates compliance with this quality metric and examines which delivery indications are associated with suboptimal ACS timing. METHODS: This is a single-center retrospective cohort of patients receiving ACS from January 2017 to September 2022. The primary outcome, measuring compliance with the SMFM ACS quality metric, is the proportion of all early preterm deliveries that received optimally timed ACS (one dose of ACS within 6 hours–7 days before birth). The secondary outcome is the proportion of term deliveries that had previously received ACS compared to all patients who received ACS. A sample of convenience of 500 patients was investigated to determine the indication for ACS. RESULTS: Of the 36,080 deliveries between 2017 and 2022, 2,022 patients received ACS (initial or rescue) and delivered at this center. There were 3,720 early preterm deliveries. Of early preterm deliveries, 14.67% received ACS in the optimal window. Of all the patients that received ACS, 20.9% delivered at term. ACS were more optimally dosed for hypertension relative risk (RR) 1.71 (95% CI 1.36–2.14) and preterm prelabor rupture of membranes RR 2.08 (95% CI 1.67–2.59). CONCLUSION: Based on the SMFM quality metric, rates of optimal ACS administration remain lower than minimum realistic benchmarks. Continuing efforts are needed to optimize ACS administration for preterm deliveries. In order to meet these goals, institutions may examine which clinical situations lead to suboptimal ACS.

Text Indexing for Long Patterns: Anchors are All you Need

In many real-world database systems, a large fraction of the data is represented by strings: sequences of letters over some alphabet. This is because strings can easily encode data arising from different sources. It is often crucial to represent such string datasets in a compact form but also to simultaneously enable fast pattern matching queries. This is the classic text indexing problem. The four absolute measures anyone should pay attention to when designing or implementing a text index are: (i) index space; (ii) query time; (iii) construction space; and (iv) construction time. Unfortunately, however, most (if not all) widely-used indexes (e.g., suffix tree, suffix array, or their compressed counterparts) are not optimized for all four measures simultaneously, as it is difficult to have the best of all four worlds. Here, we take an important step in this direction by showing that text indexing with locally consistent anchors (lc-anchors) offers remarkably good performance in all four measures, when we have at hand a lower bound l on the length of the queried patterns --- which is arguably a quite reasonable assumption in practical applications. Specifically, we improve on the construction of the index proposed by Loukides and Pissis, which is based on bidirectional string anchors (bd-anchors), a new type of lc-anchors, by: (i) designing an average-case linear-time algorithm to compute bd-anchors; and (ii) developing a semi-external-memory implementation to construct the index in small space using near-optimal work. We then present an extensive experimental evaluation, based on the four measures, using real benchmark datasets. The results show that, for long patterns, the index constructed using our improved algorithms compares favorably to all classic indexes: (compressed) suffix tree; (compressed) suffix array; and the FM-index.

Optimization of design variables and control rules in field development under uncertainty: A case of intelligent wells and CO2 water alternating gas injection

This paper focuses on life-cycle optimization of oil field development plan under uncertainty. The optimization problem included a wide range of design variables and control rules related to wells and platform in a realistic benchmark case (UNISIM–II–D) with a known ground truth reservoir model, UNISIM–II–R, that resembles Brazilian pre-salt fractured carbonate reservoirs in their early development stage. The design variables were the number and location of wells, the fluid processing capacity of the platform, and the location of internal control valves (ICVs), whereas the control rules were the setting of ICVs, production and injection rates, and the duration of the water-alternating-gas (WAG-CO2) cycle. An iterative sequential optimization framework was developed to deal with the massive search space and complex parameterization. The optimization further took into consideration the subsurface, operational and economic uncertainties, and used iterative discrete Latin hypercube method as the search algorithm. The robust optimization was carried out on a subset of representative models derived from the reduction of a large ensemble of data-assimilated models. As a low-fidelity representation of the compositional fluid model, a black-oil model was used to reduce simulation runtime. To validate our optimization framework, we applied the optimal development strategy to the ensemble of compositional simulation models, as well as the ground truth model. The true model's responses were within the ranges predicted by the compositional ensemble, confirming the optimization framework's reliability. The general methodology developed in this study, as well as our findings, can be used to optimize other similar real-world complex and high-risk field development projects, and are especially useful in closed-loop field development and management practices.

Solving Rehabilitation Scheduling Problems via a Two-Phase ASP Approach

Abstract A core part of the rehabilitation scheduling process consists of planning rehabilitation physiotherapy sessions for patients, by assigning proper operators to them in a certain time slot of a given day, taking into account several legal, medical, and ethical requirements and optimizations, for example, patient’s preferences and operator’s work balancing. Being able to efficiently solve such problem is of upmost importance, in particular after the COVID-19 pandemic that significantly increased rehabilitation’s needs. In this paper, we present a two-phase solution to rehabilitation scheduling based on Answer Set Programming, which proved to be an effective tool for solving practical scheduling problems. We first present a general encoding and then add domain-specific optimizations. Results of experiments performed on both synthetic and real benchmarks, the latter provided by ICS Maugeri, show the effectiveness of our solution as well as the impact of our domain-specific optimizations.