Unseen Instances Research Articles

Granular correlation-based label-specific feature augmentation for multi-label classification

Multi-label classification is an extension of single-label classification with generations of multi-output for unseen instances. Label correlation is an essential component in constructing multi-label classifiers. How to optimize the representation of label correlation while preserving the semantics of label-specific remains an uncertain issue. Instead of estimating label correlation by a holistic feature representation, we present an augmented label correlation model by generating multi-granularity label-specific features. Firstly, we devise a mixture distance measure to characterize the closeness of an instance by weighing the Pearson correlation coefficient with cosine similarity. Secondly, we explore the local label-specific relative discrimination by leveraging from both the instance-level and class-level correlation distribution within k nearest neighborhood. Finally, we conduct an information fusion strategy to comprehensively integrate the positive and the negative tendencies at the neighborhood level. Instances with salient positive tendency and compact neighborhood structure receive larger values while receiving smaller values with salient negative tendency and sparse neighborhood structure. With the concatenation of original features and augmented features, we examine the classification performance of the proposed granule correlation-based feature augmentation (GOFA) on well-established second-order multi-label classification methods. Extensive comparisons on thirteen benchmarks demonstrate the statistical superiority of GOFA over state-of-the-art multi-label classifications.

A cooperative coevolutionary genetic programming hyper-heuristic for multi-objective makespan and cost optimization in cloud workflow scheduling

This study presents a novel multi-objective approach for NP-hard workflow scheduling in cloud computing environments. Traditional rule-based heuristics offer flexibility but lack consistent superiority across all criteria and scenarios. The development of specific rules usually requires tedious manual refinement by experts. To overcome this limitation, our method leverages evolutionary computation and simulation to automatically derive new rules. Moreover, workflow scheduling involves two crucial and related aspects: task scheduling and the allocation of virtual machines. Our contribution includes three distinct algorithms: two that address each decision separately and a third approach that coevolves priority rules for both decisions simultaneously. Computational tests demonstrate superior performance, with an exploration of rules yielding a 72.91% larger hypervolume for optimizing makespan and costs compared to benchmark heuristics from the literature. The validation on unseen instances shows a 90.26% improvement in hypervolume performance, highlighting the robustness of our approach.

Accuracy tradeoffs between individual bone and joint-level statistical shape models of knee morphology

Statistical shape models (SSMs) are useful tools in evaluating variation in bony anatomy to assess pathology, plan surgical interventions, and inform the design of orthopaedic implants and instrumentation. Recently, by considering multiple bones spanning a joint or the whole lower extremity, SSMs can support studies investigating articular conformity and joint mechanics. The objective of this study was to assess tradeoffs in accuracy between SSMs of the femur or tibia individually versus a combined joint-level model. Three statistical shape models were developed (femur-only, tibia-only, and joint-level) for a training set of 179 total knee arthroplasty (TKA) patients with osteoarthritis representing both genders and several ethnicities. Bone geometries were segmented from preoperative CT scans, meshed with triangular elements, and registered to a template for each SSM. Principal component analysis was performed to determine modes of variation. The statistical shape models were compared using measures of compactness, accuracy, generalization, and specificity. The generalization evaluation, assessing the ability to describe an unseen instance in a leave-one-out analysis, showed that errors were consistently smaller for the individual femur and tibia SSMs than for the joint-level model. However, when additional modes were included in the joint-level model, the errors were comparable to the individual bone results, with minimal additional computational expense. When developing more complex SSMs at the joint, lower limb, or whole-body level, the use of an error threshold to inform the number of included modes, instead of 95 % of the variation explained, can help to ensure accurate representations of anatomy.

Constructing selection hyper-heuristics for open vehicle routing with time delay neural networks using multiple experts

Hyper-heuristics are general purpose search methods for solving computationally difficult problems. A selection hyper-heuristic is composed of two key components: a heuristic selection method and move acceptance criterion. Under an iterative single-point search framework, a solution is modified by selecting and applying a predefined low-level heuristic, with a decision then taken to accept or reject the resulting solution. Designing a selection hyper-heuristic is an extremely challenging task. In this study, we investigate computer-aided design of a selection hyper-heuristic for the open vehicle routing problem. A time delay neural network is used as an offline apprenticeship learning method. Our approach first observes the search behaviour of multiple expert human-designed selection hyper-heuristics on a selected sample of training instances, before automatically generating a selection hyper-heuristic capable of solving unseen instances effectively. The proposed approach is tested on open vehicle routing problem instances of different sizes to examine the performance and generality of the selection hyper-heuristics generated. Improved performance is demonstrated over a set of well-known benchmarks from the literature when compared to using the existing expert systems directly.

Regular paper Comparative Analysis of Deep Learning Models for Sentiment Analysis on IMDB Reviews

Within the domain of natural language processing, sentiment analysis assumes a fundamental position, facilitating the comprehension of societal perspectives and opinions, thereby playing a pivotal role in understanding public sentiment. In this study, an examination and comparison of deep learning architectures was conducted on IMDB movie reviews. We evaluated the performance of Basic Recurrent Neural Networks, Long Short-Term Memory (LSTM), Gated Recurrent Neural Unit (GRU), Bidirectional LSTM, Bidirectional GRU and 1D Convolutional Neural Networks (Conv1D) based on their training, validation, and testing accuracies. Our results indicate that while LSTM achieved the highest accuracy of 99.91% on the training data, GRU demonstrated superior performance (88.28%) on the validation dataset. Interestingly, Bidirectional GRU emerged as the top-performing model (87.54%) on the testing data, showcasing its robustness in generalizing to unseen instances. These findings highlight the importance of evaluating model performance across multiple datasets to assess their real-world effectiveness. Furthermore, our comparative analysis provides valuable understanding of the advantages and limitations of each model, offering practical guidance for selecting the optimal framework for sentiment analysis endeavors. Overall, this research contributes to the progress of such methodologies and deep learning approaches in natural language processing.

Scheduling of twin automated stacking cranes based on Deep Reinforcement Learning

Effective scheduling of twin automated stacking cranes (ASCs) in automated storage yard is critical to maximize operational efficiency. While Deep Reinforcement Learning (DRL) is promising in solving NP-hard scheduling problems, twin ASCs scheduling is challenging due to its unique properties including sequence-dependent setup and potential ASC interferences. In this paper, we propose a novel DRL method to learn high-quality policy for scheduling twin ASCs. We propose a Markov Decision Process model that enables the DRL agent to learn to minimize makespan and possible interferences. Based on the problem characteristics, we design a self-attention based neural architecture to effectively capture the relationships between containers under certain block state. Experiments show that the agent with the proposed feature extraction network can learn high-quality policies from training instances. These learned policies can be employed to produce effective scheduling solutions within seconds. Compared to traditional scheduling methods, the learned policy performs best in most problem sizes, and the performance improvement amplifies as the scales increase. Moreover, the policies show remarkable generalization ability on unseen instances with different distributions or scales.

Implicit Modeling of Non-rigid Objects with Cross-Category Signals

Deep implicit functions (DIFs) have emerged as a potent and articulate means of representing 3D shapes. However, methods modeling object categories or non-rigid entities have mainly focused on single-object scenarios. In this work, we propose MODIF, a multi-object deep implicit function that jointly learns the deformation fields and instance-specific latent codes for multiple objects at once. Our emphasis is on non-rigid, non-interpenetrating entities such as organs. To effectively capture the interrelation between these entities and ensure precise, collision-free representations, our approach facilitates signaling between category-specific fields to adequately rectify shapes. We also introduce novel inter-object supervision: an attraction-repulsion loss is formulated to refine contact regions between objects. Our approach is demonstrated on various medical benchmarks, involving modeling different groups of intricate anatomical entities. Experimental results illustrate that our model can proficiently learn the shape representation of each organ and their relations to others, to the point that shapes missing from unseen instances can be consistently recovered by our method. Finally, MODIF can also propagate semantic information throughout the population via accurate point correspondences.

Combinatorial Optimization-Enriched Machine Learning to Solve the Dynamic Vehicle Routing Problem with Time Windows

With the rise of e-commerce and increasing customer requirements, logistics service providers face a new complexity in their daily planning, mainly due to efficiently handling same-day deliveries. Existing multistage stochastic optimization approaches that allow solving the underlying dynamic vehicle routing problem either are computationally too expensive for an application in online settings or—in the case of reinforcement learning—struggle to perform well on high-dimensional combinatorial problems. To mitigate these drawbacks, we propose a novel machine learning pipeline that incorporates a combinatorial optimization layer. We apply this general pipeline to a dynamic vehicle routing problem with dispatching waves, which was recently promoted in the EURO Meets NeurIPS Vehicle Routing Competition at NeurIPS 2022. Our methodology ranked first in this competition, outperforming all other approaches in solving the proposed dynamic vehicle routing problem. With this work, we provide a comprehensive numerical study that further highlights the efficacy and benefits of the proposed pipeline beyond the results achieved in the competition, for example, by showcasing the robustness of the encoded policy against unseen instances and scenarios. History: This paper has been accepted for the Transportation Science special issue on DIMACS Implementation Challenge: Vehicle Routing Problems. Funding: This work was supported by Deutsche Forschungsgemeinschaft [Grant 449261765].

Learning Unified Anchor Graph for Joint Clustering of Hyperspectral and LiDAR Data.

The joint clustering of multimodal remote sensing (RS) data poses a critical and challenging task in Earth observation. Although recent advances in multiview subspace clustering have shown remarkable success, existing methods become computationally prohibitive when dealing with large-scale RS datasets. Moreover, they neglect intrinsic nonlinear and spatial interdependencies among heterogeneous RS data and lack generalization ability for out-of-sample data, thereby restricting their applicability. This article introduces a novel unified framework called anchor-based multiview kernel subspace clustering with spatial regularization (AMKSC). It learns a scalable anchor graph in the kernel space, leveraging contributions from each modality instead of seeking a consensus full graph in the feature space. To ensure spatial consistency, we incorporate a spatial smoothing operation into the formulation. The method is efficiently solved using an alternating optimization strategy, and we provide theoretical evidence of its scalability with linear computational complexity. Furthermore, an out-of-sample extension of AMKSC based on multiview collaborative representation-based classification is introduced, enabling the handling of larger datasets and unseen instances. Extensive experiments on three real heterogeneous RS datasets confirm the superiority of our proposed approach over state-of-the-art methods in terms of clustering performance and time efficiency. The source code is available at https://github.com/AngryCai/AMKSC.

Machine learning-based ransomware classification of Bitcoin transactions

Ransomware presents a significant threat to the security and integrity of cryptocurrency transactions. This research paper explores the intricacies of ransomware detection in cryptocurrency transactions using the Bitcoinheist dataset. The dataset encompasses 28 distinct families classified into three ransomware categories: Princeton, Montreal, and Padua, along with a white category representing legitimate transactions. We propose a novel hybrid supervised and semi-supervised multistage machine learning framework to tackle this challenge. Our framework effectively classifies known ransomware families by leveraging ensemble learning techniques such as Decision Tree, Random Forest, XGBoost, and Stacking. Additionally, we introduce a novel semi-supervised approach to accurately identify previously unseen ransomware instances within the dataset. Through rigorous evaluation employing comprehensive classification metrics, including accuracy, precision, recall, F1 score, RoC score, and prediction time, our proposed approach demonstrates promising results in ransomware detection within cryptocurrency transactions.

Human addictive behavior prediction by using lime with ensemble model

<span>The data-driven techniques have utilized data mining and machine learning (ML) techniques in the biomedical and healthcare fields. The process of decision-making in uncertain contextual related to human addictions and emotions play an important role in the present research. The main aim of the research is to perform classification and generate a support system for uncertain addiction circumstances by proposing a technique for drug addiction treatment. The human behavior has majority shown challenges for the prediction of human behaviors that includes body poses estimation, movements and interaction with objects. This pose estimation has showed complexity with more pose aspects and the proposed research attempts to understand the human behaviors. The present research uses the local interpretable model-agnostic explanations (LIME) for finding the input features which are most important to generate a particular output based on decision service. LIME understands the model to perturb the data samples as an input and understands shows predictions change. Also, the ensemble classifier contains classifiers group that combines for performing the prediction of all unseen instances based on voting. The proposed LIME Feature-Ensemble classifier obtained 97.54% of accuracy when compared to the existing convolutional neural network (CNN) of 59.33% and Ensemble model of 93.33% accuracy.</span>

Exploratory machine learning with unknown unknowns

In conventional supervised learning, a training dataset is given with ground-truth labels from a known label set, and the learned model will classify unseen instances to known labels. This paper studies a new problem setting in which there are unknown classes in the training data misperceived as other labels, and thus their existence appears unknown from the given supervision. We attribute the unknown unknowns to the fact that the training dataset is badly advised by the incompletely perceived label space due to the insufficient feature information. To this end, we propose the exploratory machine learning, which examines and investigates training data by actively augmenting the feature space to discover potentially hidden classes. Our method consists of three ingredients including rejection model, feature exploration, and model cascade. We provide theoretical analysis to justify its superiority, and validate the effectiveness on both synthetic and real datasets.

Learning geometric consistency and discrepancy for category-level 6D object pose estimation from point clouds

Category-level 6D object pose estimation aims to predict the position and orientation of unseen object instances, which is a fundamental problem in robotic applications. Previous works mainly focused on exploiting visual cues from RGB images, while depth images received less attention. However, depth images contain rich geometric attributes about the object’s shape, which are crucial for inferring the object’s pose. This work achieves category-level 6D object pose estimation by performing sufficient geometric learning from depth images represented by point clouds. Specifically, we present a novel geometric consistency and geometric discrepancy learning framework called CD-Pose to resolve the intra-category variation, inter-category similarity, and objects with complex structures. Our network consists of a Pose-Consistent Module and a Pose-Discrepant Module. First, a simple MLP-based Pose-Consistent Module is utilized to extract geometrically consistent pose features of objects from the pre-computed object shape priors for each category. Then, the Pose-Discrepant Module, designed as a multi-scale region-guided transformer network, is dedicated to exploring each instance’s geometrically discrepant features. Next, the NOCS model of the object is reconstructed according to the integration of consistent and discrepant geometric representations. Finally, 6D object poses are obtained by solving the similarity transformation between the reconstruction and the observed point cloud. Experiments on the benchmark datasets show that our CD-Pose produces superior results to state-of-the-art competitors.

An actor-critic algorithm with policy gradients to solve the job shop scheduling problem using deep double recurrent agents

In this work, we present a method that applies Deep Reinforcement Learning, an approximate dynamic programming procedure using deep neural networks, to the job shop scheduling problem (JSSP). The aim is to show that a greedy-like heuristic trained on a subset of problems, can effectively generalize to some extent to unseen instances, and be competitive compared to other methods. We model the JSSP as a Markov Decision Process and we exploit the efficacy of reinforcement learning to solve the problem. We adopt an actor-critic scheme based on policy gradients, specifically the Proximal Policy Gradient method, where the action taken by the agent is influenced by policy considerations on the state-value function. The procedures take into account the challenging nature of JSSP, where the state and the action space change for every instance and after each decision. To tackle this variability, we introduced a novel model based on two incident Long-Short Term Memory networks, followed by an encoding model, different in structure for both the actor and the critic. Experiments show the algorithm reaches good solutions in a short time, proving that is possible to generate new greedy heuristics just from learning-based methodologies. We compared our algorithms against several established heuristics, an adaptive method, a commercial solver based on branch and cut, and another approach based on Deep Reinforcement Learning, proving the validity of the proposed method in terms of time and makespan. The model can generalize, to some extent, to larger problems originating from a different distribution.

Deep learning optimal quantum annealing schedules for random Ising models

A crucial step in the race towards quantum advantage is optimizing quantum annealing using ad-hoc annealing schedules. Motivated by recent progress in the field, we propose to employ long-short term memory neural networks to automate the search for optimal annealing schedules for random Ising models on regular graphs. By training our network using locally-adiabatic annealing paths, we are able to predict optimal annealing schedules for unseen instances and even larger graphs than those used for training.

Large-State Reinforcement Learning for Hyper-Heuristics

Hyper-heuristics are a domain-independent problem solving approach where the main task is to select effective chains of problem-specific low-level heuristics on the fly for an unseen instance. This task can be seen as a reinforcement learning problem, however, the information available to the hyper-heuristic is very limited, usually leading to very limited state representations. In this work, for the first time we use the trajectory of solution changes for a larger set of features for reinforcement learning in the novel hyper-heuristic LAST-RL (Large-State Reinforcement Learning). Further, we introduce a probability distribution for the exploration case in our epsilon-greedy policy that is based on the idea of Iterated Local Search to increase the chance to sample good chains of low-level heuristics. The benefit of the collaboration of our novel components is shown on the academic benchmark of the Cross Domain Heuristic Challenge 2011 consisting of six different problem domains. Our approach can provide state-of-the-art results on this benchmark where it outperforms recent hyper-heuristics based on reinforcement learning, and also demonstrates high performance on a benchmark of complex real-life personnel scheduling domains.

Tag Recommendation System for Marathi News Articles by using Multi-label Classification

Abstract: Multi-label classification is the variant of a classification problem where multiple labels are assigned to each instance. In multi-label classification, the training set is composed of instances each associated with a set of labels, and the task is to predict the label sets of unseen instances through analyzing training instances with known label sets. This paper demonstrates the use of multi-label classification to determine tags for news articles written in the Marathi Language of India. The proposed study uses Binary Relevance (One vs Rest) technique of multi-label classification to establish the tags for the given input of a Marathi news article. Tag recommendation systems for Marathi news articles can greatly enhance the user experience for readers and help them find the articles that are most relevant to their interests.

Learning to schedule (L2S): adaptive job shop scheduling using double deep Q network

ABSTRACT The stochasticity and randomly changing nature of the production environment posed a significant challenge in developing real-time responsive scheduling solutions. Many previous scheduling solutions assumed static environments, user-anticipated, and hand-crafted dynamic scenarios. However, real-world production environment events are random and unpredictable. This study considers Job Shop Scheduling Problem (JSSP) as an iterative decision-making problem, and Deep Reinforcement Learning (DRL)-based solution is designed to address these challenges. A deep neural network is utilized for function approximation, and the input feature vectors are extracted iteratively to be used in the sequential decision-making process. The production states are expressed with randomly changing feature vectors of each job’s operations and the corresponding machines. This work proposes Double Deep Q Network (DDQN) methods to train the model. Results are evaluated on the renowned OR-Library benchmark problems. The evaluation result indicates that the proposed approach is comparative in benchmark problems, and the scheduling agent can get good results in unseen instances with an average of 94.86% of the scheduling score.

Improving the Performance of Open-Set Recognition with Generated Fake Data

Open-set recognition models, in addition to generalizing to unseen instances of known categories, have to identify samples of unknown classes during the training phase. The main reason the latter is much more complicated is that there is very little or no information about the properties of these unknown classes. There are methodologies available to handle the unknowns. One possible method is to construct models for them by using generated inputs labeled as unknown. Generative adversarial networks are frequently deployed to generate synthetic samples representing unknown classes to create better models for known classes. In this paper, we introduce a novel approach to improve the accuracy of recognition methods while reducing the time complexity. Instead of generating synthetic input data to train neural networks, feature vectors are generated using the output of a hidden layer. This approach results in a less complex structure for the neural network representation of the classes. A distance-based classifier implemented by a convolutional neural network is used in our implementation. Our solution’s open-set detection performance reaches an AUC value of 0.839 on the CIFAR-10 dataset, while the closed-set accuracy is 91.4%, the highest among the open-set recognition methods. The generator and discriminator networks are much smaller when generating synthetic inner features. There is no need to run these samples through the first part of the classifier with the convolutional layers. Hence, this solution not only gives better performance than generating samples in the input space but also makes it less expensive in terms of computational complexity.

Online Routing Over Parallel Networks: Deterministic Limits and Data-driven Enhancements

Over the past decade, GPS-enabled traffic applications such as Google Maps and Waze have become ubiquitous and have had a significant influence on billions of daily commuters’ travel patterns. A consequence of the online route suggestions of such applications, for example, via greedy routing, has often been an increase in traffic congestion since the induced travel patterns may be far from the system optimum. Spurred by the widespread impact of traffic applications on travel patterns, this work studies online traffic routing in the context of capacity-constrained parallel road networks and analyzes this problem from two perspectives. First, we perform a worst-case analysis to identify the limits of deterministic online routing. Although we find that deterministic online algorithms achieve finite, problem/instance-dependent competitive ratios in special cases, we show that for a general setting the competitive ratio is unbounded. This result motivates us to move beyond worst-case analysis. Here, we consider algorithms that exploit knowledge of past problem instances and show how to design data-driven algorithms whose performance can be quantified and formally generalized to unseen future instances. We then present numerical experiments based on an application case for the San Francisco Bay Area to evaluate the performance of the proposed data-driven algorithms compared with the greedy algorithm and two look-ahead heuristics with access to additional information on the values of time and arrival time parameters of users. Our results show that the developed data-driven algorithms outperform commonly used greedy online-routing algorithms. Furthermore, our work sheds light on the interplay between data availability and achievable solution quality. History: Accepted by Andrea Lodi, Area Editor for Design and Analysis of Algorithms–Discrete. Funding: This work was supported by National Science Foundation (NSF) Award 1830554 and by the German Research Foundation (DFG) under [Grant 449261765]. Supplemental Material: The e-companion is available at https://doi.org/10.1287/ijoc.2023.1275 .