Multiple Problems Research Articles

Reinforcement Learning for Solving Multiple Vehicle Routing Problem with Time Window

Vehicle routing problem with time window (VRPTW) is of great importance for a wide spectrum of services and real-life applications, such as online take-out and car-hailing platforms. A promising method should generate high-qualified solutions within limited inference time, and there are three major challenges: (a) directly optimizing the goal with several practical constraints; (b) efficiently handling individual time-window limits; and (c) modeling the cooperation among the vehicle fleet. In this article, we present an end-to-end reinforcement learning framework to solve VRPTW. First, we propose an agent model that encodes constraints into features as the input and conducts harsh policy on the output when generating deterministic results. Second, we design a time penalty augmented reward to model the time-window limits during gradient propagation. Third, we design a task handler to enable the cooperation among different vehicles. We perform extensive experiments on two real-world datasets and one public benchmark dataset. Results demonstrate that our solution improves the performance by up to 11.7% compared to other RL baselines and could generate solutions for instances within seconds, while existing heuristic baselines take for minutes as well as maintain the quality of solutions. Moreover, our solution is thoroughly analyzed with meaningful implications due to the real-time response ability.

АДАПТАЦІЙНЕ УПРАВЛІННЯ ПОКАЗОМ РЕКЛАМНИХ РОЛИКІВ В СОЦІАЛЬНИХ МЕРЕЖАХ ЗА СХЕМОЮ БАГАТОРУКОГО БАНДИТА

In a rapidly changing marketing environment, to ensure the success of an advertising campaign, it is necessary to minimize the time from the idea of advertising to the implementation of communication. There is a need for simultaneous advertising, study and adjustment of communication processes. Accordingly, it is necessary to make a certain sequence of decisions, the sum of which ensures the right balance between learning and earning. One of the most convenient adaptation mechanisms for practical implementation is the two-armed bandit scheme. A two-armed bandit is a model of multiple choices of one of two alternatives with an a priori unknown distribution of payoffs between them. Each alternative is associated with one arm of the bandit. As a result of the choices made, the probabilistic distribution of payoffs is identified by step to step and used to exploit the best alternative. In practice, there may be more alternatives, in which case the multiple decision problem is maped to the multi-armed bandit scheme. A new multi-armed bandit-based model for advertising control in social networks is proposed. The idea is that multiple advertisements are displayed simultaneously, and the frequency of display of each depends on its performance - from the level of viewer retention to the contact frame. Based on the retention rates, a certain share of resources is redistributed among the commercials on a daily basis. Simple and effective formulas for the reassignment of resources are proposed, taking into account both the current unevenness of retention rates and the statistics of video demonstrations. A simulation of the control model for a 35-day campaign for three commercials is performed. The performance of the control model is demonstrated both under steady-state conditions and in the case of a viral message insertion. The phase trajectories show how the response of the system to the insertion of a viral message changes with different control parameters.

Iteration with Bisection to Approximate the Solution of a Boundary Value Problem

Due to the restrictive growth and/or monotonicity requirements inherent in their employment, classical iterative fixed-point theorems are rarely used to approximate solutions to an integral operator with Green’s function kernel whose fixed points are solutions of a boundary value problem. In this paper, we show how one can decompose a fixed-point problem into multiple fixed-point problems that one can easily iterate to approximate a solution of a differential equation satisfying one boundary condition, then apply a bisection method in an intermediate value theorem argument to meet a second boundary condition. Error estimates on the iterates are also established. The technique will be illustrated on a second-order right focal boundary value problem, with an example provided showing how to apply the results.

Secure Data Aggregation Using Authentication and Authorization for Privacy Preservation in Wireless Sensor Networks.

Existing secure data aggregation protocols are weaker to eliminate data redundancy and protect wireless sensor networks (WSNs). Only some existing approaches have solved this singular issue when aggregating data. However, there is a need for a multi-featured protocol to handle the multiple problems of data aggregation, such as energy efficiency, authentication, authorization, and maintaining the security of the network. Looking at the significant demand for multi-featured data aggregation protocol, we propose secure data aggregation using authentication and authorization (SDAAA) protocol to detect malicious attacks, particularly cyberattacks such as sybil and sinkhole, to extend network performance. These attacks are more complex to address through existing cryptographic protocols. The proposed SDAAA protocol comprises a node authorization algorithm that permits legitimate nodes to communicate within the network. This SDAAA protocol's methods help improve the quality of service (QoS) parameters. Furthermore, we introduce a mathematical model to improve accuracy, energy efficiency, data freshness, authorization, and authentication. Finally, our protocol is tested in an intelligent healthcare WSN patient-monitoring application scenario and verified using an OMNET++ simulator. Based upon the results, we confirm that our proposed SDAAA protocol attains a throughput of 444 kbs, representing a 98% of data/network channel capacity rate; an energy consumption of 2.6 joules, representing 99% network energy efficiency; an effected network of 2.45, representing 99.5% achieved overall performance of the network; and time complexity of 0.08 s, representing 98.5% efficiency of the proposed SDAAA approach. By contrast, contending protocols such as SD, EEHA, HAS, IIF, and RHC have throughput ranges between 415-443, representing 85-90% of the data rate/channel capacity of the network; energy consumption in the range of 3.0-3.6 joules, representing 88-95% energy efficiency of the network; effected network range of 2.98, representing 72-89% improved overall performance of the network; and time complexity in the range of 0.20 s, representing 72-89% efficiency of the proposed SDAAA approach. Therefore, our proposed SDAAA protocol outperforms other known approaches, such as SD, EEHA, HAS, IIF, and RHC, designed for secure data aggregation in a similar environment.

Reward Penalties on Augmented States for Solving Richly Constrained RL Effectively

Constrained Reinforcement Learning employs trajectory-based cost constraints (such as expected cost, Value at Risk, or Conditional VaR cost) to compute safe policies. The challenge lies in handling these constraints effectively while optimizing expected reward. Existing methods convert such trajectory-based constraints into local cost constraints, but they rely on cost estimates, leading to either aggressive or conservative solutions with regards to cost. We propose an unconstrained formulation that employs reward penalties over states augmented with costs to compute safe policies. Unlike standard primal-dual methods, our approach penalizes only infeasible trajectories through state augmentation. This ensures that increasing the penalty parameter always guarantees a feasible policy, a feature lacking in primal-dual methods. Our approach exhibits strong empirical performance and theoretical properties, offering a fresh paradigm for solving complex Constrained RL problems, including rich constraints like expected cost, Value at Risk, and Conditional Value at Risk. Our experimental results demonstrate superior performance compared to leading approaches across various constraint types on multiple benchmark problems.

CREAD: A Classification-Restoration Framework with Error Adaptive Discretization for Watch Time Prediction in Video Recommender Systems

The watch time is a significant indicator of user satisfaction in video recommender systems. However, the prediction of watch time as a target variable is often hindered by its highly imbalanced distribution with a scarcity of observations for larger target values and over-populated samples for small values. State-of-the-art watch time prediction models discretize the continuous watch time into a set of buckets in order to consider the distribution of watch time. However, it is highly uninvestigated how these discrete buckets should be created from the continuous watch time distribution, and existing discretization approaches suffer from either a large learning error or a large restoration error. To address this challenge, we propose a Classification-Restoration framework with Error-Adaptive-Discretization (CREAD) to accurately predict the watch time. The proposed framework contains a discretization module, a classification module, and a restoration module. It predicts the watch time through multiple classification problems. The discretization process is a key contribution of the CREAD framework. We theoretically analyze the impacts of the discretization on the learning error and the restoration error, and then propose the error-adaptive discretization (EAD) technique to better balance the two errors, which achieves better performance over traditional discretization approaches. We conduct detailed offline evaluations on a public dataset and an industrial dataset, both showing performance gains through the proposed approach. Moreover, We have fully launched our framework to an online video platform, which resulted in a significant increase in users' video watch time by 0.29% through A/B testing. These results highlight the effectiveness of the CREAD framework in watch time prediction in video recommender systems.

Parallel Beam Search Algorithms for Domain-Independent Dynamic Programming

Domain-independent dynamic programming (DIDP), a model-based paradigm based on dynamic programming, has shown promising performance on multiple combinatorial optimization problems compared with mixed integer programming (MIP) and constraint programming (CP). The current DIDP solvers are based on heuristic search, and the state-of-the-art solver, complete anytime beam search (CABS), uses beam search. However, the current DIDP solvers cannot utilize multiple threads, unlike state-of-the-art MIP and CP solvers. In this paper, we propose three parallel beam search algorithms and develop multi-thread implementations of CABS. With 32 threads, our multi-thread DIDP solvers achieve 9 to 39 times speedup on average and significant performance improvement over the sequential solver, finding the new best solutions for two instances of the traveling salesperson problem with time windows. In addition, our solvers outperform multi-thread MIP and CP solvers in four of the six combinatorial optimization problems evaluated.

Development of a Telehealth Simulation Using Intraprofessional Collaboration.

Telehealth allows access to high-quality, holistic patient care, including diagnosis, interventions, treatments, monitoring, and patient education. As the use of telehealth continues to increase, faculty considered the need for entry-level nursing students to be introduced to telehealth and its services. Faculty from the medical-surgical II and mental health courses developed a learning experience for students that blends concepts from both courses, as patients often present with multiple problems. The telehealth experience helped students utilize assessment skills, learn delegation, and connect concepts from two courses to provide care for a patient remotely.

ScDrugPrio: a framework for the analysis of single-cell transcriptomics to address multiple problems in precision medicine in immune-mediated inflammatory diseases.

Ineffective drug treatment is a major problem for many patients with immune-mediated inflammatory diseases (IMIDs). Important reasons are the lack of systematic solutions for drug prioritisation and repurposing based on characterisation of the complex and heterogeneous cellular and molecular changes in IMIDs. Here, we propose a computational framework, scDrugPrio, which constructs network models of inflammatory disease based on single-cell RNA sequencing (scRNA-seq) data. scDrugPrio constructs detailed network models of inflammatory diseases that integrate information on cell type-specific expression changes, altered cellular crosstalk and pharmacological properties for the selection and ranking of thousands of drugs. scDrugPrio was developed using a mouse model of antigen-induced arthritis and validated by improved precision/recall for approved drugs, as well as extensive in vitro, in vivo, and in silico studies of drugs that were predicted, but not approved, for the studied diseases. Next, scDrugPrio was applied to multiple sclerosis, Crohn's disease, and psoriatic arthritis, further supporting scDrugPrio through prioritisation of relevant and approved drugs. However, in contrast to the mouse model of arthritis, great interindividual cellular and gene expression differences were found in patients with the same diagnosis. Such differences could explain why some patients did or did not respond to treatment. This explanation was supported by the application of scDrugPrio to scRNA-seq data from eleven individual Crohn's disease patients. The analysis showed great variations in drug predictions between patients, for example, assigning a high rank to anti-TNF treatment in a responder and a low rank in a nonresponder to that treatment. We propose a computational framework, scDrugPrio, for drug prioritisation based on scRNA-seq of IMID disease. Application to individual patients indicates scDrugPrio's potential for personalised network-based drug screening on cellulome-, genome-, and drugome-wide scales. For this purpose, we made scDrugPrio into an easy-to-use R package ( https://github.com/SDTC-CPMed/scDrugPrio ).

Dynamic-MLCS: Fast searching for dynamic multiple longest common subsequences in sequence stream data

In recent years, data stream mining has become a popular research topic, wherein the dynamic multiple longest common subsequence problem for sequence streams (i.e., D-MLCS) is an important issue with a vast range of applications. However, the existing methods are only for static multiple longest common subsequence problem (i.e., S-MLCS) and there are no algorithms for the D-MLCS problem. Using the existing algorithms for the S-MLCS problem to solve the D-MLCS problem often leads to an overload of the main memory and is extremely time consuming as all the sequence segments should be stored into the main memory when the new segments flow in, and all updated problems should be resolved. To overcome this shortcoming, we designed a scheme for constructing dynamic Directed Acyclic Graphs (DAGs). It only needs to update the current DAG post each sequence segment flows in, without constructing the DAG from the beginning. We propose a novel algorithm for the D-MLCS problem called Dynamic-MLCS, which only needs to store a part of the key information of each streaming sequence segments, instead of storing all streaming sequence segments and scanning them repeatedly. Theoretical analyses demonstrate that the time and space complexities of the proposed algorithm are only relevant and linear to the number of the incomplete points and newly generated nodes, and the experiments conducted on the datasets of sequence streams indicate that the proposed Dynamic-MLCS is efficient and performs better than the compared state-of-the-art algorithms.

Multiple Drug Resistant Streptococcus Strains-An Actual Problem in Pig Farms in Western Romania.

Streptococci are a type of bacteria that can cause severe illnesses in humans and animals. Some typical species like S. suis, or atypical species like S. porcinus and, S. dysgalactiae subsp. dysgalactiae, can cause infections like septicemia, meningitis, endocarditis, arthritis, and septic shock. S. suis is considered a newly emerging zoonotic pathogen. Although human streptococcal infection outbreaks are rare, it is appropriate to review the main streptococcal species isolated in pig farms in western Romania, due to the high degree of antibiotic resistance among most isolates commonly used in human treatment. This study examines the resistance patterns of these isolates over 5 years (2018-2023). The research investigated the antimicrobial susceptibility of 267 strains of Streptococcus spp. isolated from pigs, primarily from lung and brain tissues. This report is the first to describe the distribution of atypical Streptococcus species (SDSE, S. porcinus, S. hyovaginalis, S. pluranimalium, S. canis) in Romania, as well as the antibiotic resistance profile of these potentially zoonotic species. It is important to re-evaluate and consider the high rates of resistance of S. suis to tetracyclines, lincosamides, macrolides, and aminoglycosides, as well as the high recovery rates of S. suis from the lungs and brain when treating swine diseases.

In-vitro queen rearing protocol for stingless bee, Tetragonula iridipennis Smith

South-East Asia falls behind in comparison to South America and Australia in beekeeping with stingless bees. This is partly contributed by the lack of proper management techniques and slow rate of natural colony multiplication in different bee species found in this part of the world. Production of natural queen bees in stingless bee colonies depends upon several factors, such as adequate nutrition, temperature regulation, seasonality, colony size, natural enemies among many others. The capacity to produce a large number of queen bees without depending upon the production of natural queen inside colonies makes in vitro rearing of stingless bee as a good help for the problem of slow colony multiplication. The present study describes in vitro queen rearing protocol for the Asian stingless bee, Tetragonula iridipennis Smith. The larval provision required for producing a queen bee was three times more than a natural worker bee. The in vitro queen bees were morphologically similar to the natural queens. The reproductive success of these in vitro queen bees also supported the proposed protocol. The technique would be useful in colony multiplication of this species at a faster rate and in developing management tactics for large-scale adoption in Indo-Malayan and Australasia regions.

Statistical inference for diagnostic test accuracy studies with multiple comparisons.

Diagnostic accuracy studies assess the sensitivity and specificity of a new index test in relation to an established comparator or the reference standard. The development and selection of the index test are usually assumed to be conducted prior to the accuracy study. In practice, this is often violated, for instance, if the choice of the (apparently) best biomarker, model or cutpoint is based on the same data that is used later for validation purposes. In this work, we investigate several multiple comparison procedures which provide family-wise error rate control for the emerging multiple testing problem. Due to the nature of the co-primary hypothesis problem, conventional approaches for multiplicity adjustment are too conservative for the specific problem and thus need to be adapted. In an extensive simulation study, five multiple comparison procedures are compared with regard to statistical error rates in least-favourable and realistic scenarios. This covers parametric and non-parametric methods and one Bayesian approach. All methods have been implemented in the new open-source R package cases which allows us to reproduce all simulation results. Based on our numerical results, we conclude that the parametric approaches (maxT and Bonferroni) are easy to apply but can have inflated type I error rates for small sample sizes. The two investigated Bootstrap procedures, in particular the so-called pairs Bootstrap, allow for a family-wise error rate control in finite samples and in addition have a competitive statistical power.

Application of Indigenous Knowledge in Agriculture in Pandemic Situation

This article highlights the social, economic and daily life conditions of the global public during the terrible epidemic of Corona virus that started in Wuhan province of China in December 2019. What kind of problems common people had to face in their daily life during this terrible life-destroying and social-disconnecting epidemic situation that has come down abnormally in the lives of common people all over the world has been highlighted here. Food, drink and shelter are one of the three pillars of people’s lives, one of the problems of people during the corona virus was food and what kind of problems arose in the field of farmers and agriculture producing these crops are highlighted here. In this article, the farmers were temporarily deprived of various uses of science and technology in the field of agriculture during the corona period, so the contribution of the indigenous technology and knowledge that the farmers maintained equally in the agricultural field and crop production by using indigenous technology and knowledge as an alternative How it was used in agriculture is presented here. During the Corona period, the public life was disturbed and the common people did not have the supply of various types of chemical substances used daily to maintain the food supply and the prosperity of the crops. However, the advantages and disadvantages faced by farmers in crop production using traditional indigenous methods and knowledge are presented here. The way farmers and their crops did not deprive the common people of food in the quarantine situation during the corona period despite multiple problems is considered as one of the feats to disrupt the public life then and now.

Attaining robust performance targets in data envelopment analysis with application to efficiency evaluation of airports

We consider a ratio-based efficiency analysis in view of achieving performance targets by Decision Making Units. The targets refer to the efficiency scores or ranks and the truth of pairwise preference relations. These outcomes can be instantiated for at least one or all feasible weights associated with inputs and outputs. We discuss Multiple Objective Optimization problems that minimize the required reductions of consumed resources or increases of produced results needed to reach the pre-defined aims. The proposed models offer great flexibility to the Decision Maker, who can indicate which factors should be modified and to which extent. The evolutionary optimization algorithm computes a set of solutions that exhibit the trade-offs between expected modifications of various factors. They can serve as the basis for interactively selecting the most preferred solution to be implemented in practice. The use of the proposed framework is illustrated in the problem of analyzing the efficiency of Polish airports.

Multi-Objective Operating Parameters Optimization for the Start Process of Proton Exchange Membrane Fuel Cell Stack with Non-Dominated Sorting Genetic Algorithm II

Proton exchange membrane fuel cell (PEMFC) is one of the most promising energy conversion devices in the world. Performance, durability, and cost are the key issues currently limiting its large-scale commercial application. This paper proposes a non-dominated sorting genetic algorithm II (NSGA-II) for optimizing operating parameters of the start process to improve the output performance of a PEMFC stack. First, a Simulink model of the PEMFC stack including the anode module, cathode module, water transfer module, output voltage module, and output net power module is established, and the accuracy of the stack model is verified through experiments. The three performances are then optimized simultaneously based on NSGA-II. The results show that the optimized operating parameters for the start process results in a PEMFC stack that outperforms the base case in steady-state voltage, percentage of undershoot, and net power these three indicators with the same response time, demonstrating the success of the method in solving multiple optimization problems. This study presents an effective approach for the multi-objective optimization of the PEMFC stack, which is of guidance for engineering practice.

Finite mixture models for multiscale dynamics in Chinese financial markets

A novel statistical machine learning technique for decomposing, analyzing, and modeling complex time series is presented in this paper. This methodology is based on unsupervised classification methods using finite mixtures, which have been widely used in factor analysis to extract essential information from multivariate static data in a parsimonious manner. Here, they are applied in a purely dynamic framework to create a new family of autoregressive models. The selection of the number of components and appropriate geometric template in these models is based on a set of information criteria that require solving a multiple objective optimization problem. To ensure the most adequate models are selected, the problem is augmented with a simulation instruction, providing a robust criterion for assignment. Experiments on real financial data sampled over the COVID-19 pandemic demonstrate the potential of this approach as a promising alternative to the well-known Box-Jenkins methodology. The proposed method offers flexibility, accuracy, and stability, making it suitable for a variety of applications in financial modeling and other fields where complex time series data analysis is required.

FLC-Based Ultra-Low-Frequency Oscillation Suppression Scheme for Interconnected Power Grids

The appearance of ultra-low-frequency oscillations in the grid at the sending end, after asynchronous grid interconnection, poses a significant threat to the stable operation of the system. For post-asynchronous interconnection in a multi-DC transmission system, an investigation is conducted to analyze the causes of ultra-low-frequency oscillations and the utilization of a Frequency Limit Controller (FLC) which aims to suppress these oscillations. Furthermore, a method is developed to rank DC sensitivity, considering the hydroelectric distribution in the sending-end grid, by combining the DC FLC impact factor and DC control sensitivity. Subsequently, a novel approach for ultra-low-frequency oscillation suppression is proposed. This approach employs the stochastic subspace method for parameter estimation and the NSGA-II optimization algorithm to convert the multi-DC optimization challenge into multiple sequential cyclic optimization problems, each focusing on a single DC, ensuring a more effective suppression of ultra-low-frequency oscillations. The proposed scheme’s effectiveness is validated through simulations using a specific locations’ interconnected power grid.

SHAP value-based ERP analysis (SHERPA): Increasing the sensitivity of EEG signals with explainable AI methods.

Conventionally, event-related potential (ERP) analysis relies on the researcher to identify the sensors and time points where an effect is expected. However, this approach is prone to bias and may limit the ability to detect unexpected effects or to investigate the full range of the electroencephalography (EEG) signal. Data-driven approaches circumvent this limitation, however, the multiple comparison problem and the statistical correction thereof affect both the sensitivity and specificity of the analysis. In this study, we present SHERPA - a novel approach based on explainable artificial intelligence (XAI) designed to provide the researcher with a straightforward and objective method to find relevant latency ranges and electrodes. SHERPA is comprised of a convolutional neural network (CNN) for classifying the conditions of the experiment and SHapley Additive exPlanations (SHAP) as a post hoc explainer to identify the important temporal and spatial features. A classical EEG face perception experiment is employed to validate the approach by comparing it to the established researcher- and data-driven approaches. Likewise, SHERPA identified an occipital cluster close to the temporal coordinates for the N170 effect expected. Most importantly, SHERPA allows quantifying the relevance of an ERP for a psychological mechanism by calculating an "importance score". Hence, SHERPA suggests the presence of a negative selection process at the early and later stages of processing. In conclusion, our new method not only offers an analysis approach suitable in situations with limited prior knowledge of the effect in question but also an increased sensitivity capable of distinguishing neural processes with high precision.

On upper bounds for the multiple knapsack assignment problem

The Multiple Knapsack Assignment Problem is a strongly NP-hard combinatorial optimisation problem, with several applications. We show that an upper bound for the problem, due to Kataoka and Yamada, can be computed in linear time. We then show that some bounds due to Martello and Monaci dominate the Kataoka-Yamada bound. Finally, we define an even stronger bound, which turns out to be particularly effective when the number of knapsacks is not a multiple of the number of item classes.