Vanilla Version Research Articles

Advancing Session-Based Recommendations with Atten-Mixer+: Dynamic and Adaptive Multi-Level Intent Mining

Session-based recommendation (SBR) systems, traditionally reliant on complex graph neural networks (GNNs), often face challenges with marginal performance improvements despite increased model complexity. In this paper, we dissect the classical GNN-based SBR models and empirically find that the sophisticated GNN propagations might be redundant, given the readout module plays a significant role in GNN-based models. Based on this observation, we introduce Atten-Mixer+, an advanced iteration of our previously developed Multi-Level Attention Mixture Network (Atten-Mixer). Atten-Mixer+ forgoes GNN propagation in favor of a dynamic and adaptive readout process, tailored to the unique characteristics of each session. Different from the vanilla version, Atten-Mixer+ features the Adaptive Intent Scaler (AIS) layer, which dynamically determines the depth of multi-level user intent analysis, and a soft allocation approach for generating user intent queries across entire user interaction sequences. This innovative design allows Atten-Mixer+ to capture a nuanced and comprehensive understanding of user behaviors, overcoming the limitations of fixed-length analysis. Empirical evaluations on benchmark datasets highlight Atten-Mixer+'s superior efficiency and effectiveness, marking a significant step forward in the predictive accuracy of session-based recommendation systems.

Data-Driven Abstractions via Binary-Tree Gaussian Processes for Formal Verification

To advance formal verification of stochastic systems against temporal logic requirements for handling unknown dynamics, researchers have been designing data-driven approaches inspired by breakthroughs in the underlying machine learning techniques. As one promising research direction, abstraction-based solutions based on Gaussian process (GP) regression have become popular for their ability to learn a representation of the latent system from data with a quantified error. Results obtained based on this model are then translated to the true system via various methods. In a recent publication, GPs using a so-called binary-tree kernel have demonstrated a polynomial speedup w.r.t. the size of the data compared to their vanilla version, outcompeting all existing sparse GP approximations. Incidentally, the resulting binary-tree Gaussian process (BTGP) is characteristic for its piecewise-constant posterior mean and covariance functions, naturally abstracting the input space into discrete partitions. In this paper, we leverage this natural abstraction of the BTGP for formal verification, eliminating the need for cumbersome abstraction and error quantification procedures. We show that the BTGP allows us to construct an interval Markov chain model of the unknown system with a speedup that is polynomial w.r.t. the size of the abstraction compared to alternative approaches. We provide a delocalized error quantification via a unified formula even when the true dynamics do not live in the function space of the BTGP. This allows us to compute upper and lower bounds on the probability of satisfying reachability specifications that are robust to both aleatoric and epistemic uncertainties.

The magnifying role of the banking sector during depressions

Is there a magnifying role of the banking sector during depressions? How can a financial perturbation, possibly small-sized, create large impacts on output and enduring recessions? What is the role played by the massive stock of due claims brought about by a financial crisis? We address these questions by putting forth a general equilibrium model endowed with a banking system in which due claims – henceforth named due loans – and occasionally binding credit restrictions coexist and their effects reinforce each other. Under “bad” financially-driven perturbations due loans hike and the concomitant opportunity, holding, regulatory, and impairment costs trigger sizable increases in external finance premia and promote credit restrictiveness. Firms’ net worth collapses as they are called in to finance banks’ problems, and their ability to invest and accumulate capital becomes compromised. The amplification effect can be very large: the full-fledged banking model may only require a perturbation of one-tenth the size of a plain vanilla version to deliver the same output drop at the trough. Effects are non-linear, since credit restrictions remain slack on the boundary of the steady state, and asymmetric, since these restrictions play no role whatsoever under “good shocks.”

CrysGNN: Distilling Pre-trained Knowledge to Enhance Property Prediction for Crystalline Materials

In recent years, graph neural network (GNN) based approaches have emerged as a powerful technique to encode complex topological structure of crystal materials in an enriched repre- sentation space. These models are often supervised in nature and using the property-specific training data, learn relation- ship between crystal structure and different properties like formation energy, bandgap, bulk modulus, etc. Most of these methods require a huge amount of property-tagged data to train the system which may not be available for different prop- erties. However, there is an availability of a huge amount of crystal data with its chemical composition and structural bonds. To leverage these untapped data, this paper presents CrysGNN, a new pre-trained GNN framework for crystalline materials, which captures both node and graph level structural information of crystal graphs using a huge amount of unla- belled material data. Further, we extract distilled knowledge from CrysGNN and inject into different state of the art prop- erty predictors to enhance their property prediction accuracy. We conduct extensive experiments to show that with distilled knowledge from the pre-trained model, all the SOTA algo- rithms are able to outperform their own vanilla version with good margins. We also observe that the distillation process provides significant improvement over the conventional ap- proach of finetuning the pre-trained model. We will release the pre-trained model along with the large dataset of 800K crys- tal graph which we carefully curated; so that the pre-trained model can be plugged into any existing and upcoming models to enhance their prediction accuracy.

Beyond ADMM: A Unified Client-Variance-Reduced Adaptive Federated Learning Framework

As a novel distributed learning paradigm, federated learning (FL) faces serious challenges in dealing with massive clients with heterogeneous data distribution and computation and communication resources. Various client-variance-reduction schemes and client sampling strategies have been respectively introduced to improve the robustness of FL. Among others, primal-dual algorithms such as the alternating direction of method multipliers (ADMM) have been found being resilient to data distribution and outperform most of the primal-only FL algorithms. However, the reason behind remains a mystery still. In this paper, we firstly reveal the fact that the federated ADMM is essentially a client-variance-reduced algorithm. While this explains the inherent robustness of federated ADMM, the vanilla version of it lacks the ability to be adaptive to the degree of client heterogeneity. Besides, the global model at the server under client sampling is biased which slows down the practical convergence. To go beyond ADMM, we propose a novel primal-dual FL algorithm, termed FedVRA, that allows one to adaptively control the variance-reduction level and biasness of the global model. In addition, FedVRA unifies several representative FL algorithms in the sense that they are either special instances of FedVRA or are close to it. Extensions of FedVRA to semi/un-supervised learning are also presented. Experiments based on (semi-)supervised image classification tasks demonstrate superiority of FedVRA over the existing schemes in learning scenarios with massive heterogeneous clients and client sampling.

Exploring lottery ticket hypothesis in few-shot learning

Lottery Ticket Hypothesis (LTH) [14] has gathered great focus since being proposed. Researchers then succeed in figuring out alternative ways to find the ”winning ticket” and extending the vanilla version to various kinds of situations ranging from image segmentation to language pretraining. However, these works all emphasize fully supervised learning with plenty of training instances, whilst they ignore the important scenario of learning from few examples, i.e., Few-Shot Learning (FSL). Different from classical many-shot learning tasks, the common FSL setting assumes the disjoint of source and target categories. To the best of our knowledge, the lottery ticket hypothesis has for the first time, systematically studied in few-shot learning scenarios. To validate the hypothesis, we conduct extensive experiments on several few-shot learning methods with three widely-used datasets, miniImageNet, CUB, CIFARFS. Results reveal that we can even find “winning tickets” for some high-performance methods. In addition, our experiments on Cross-Domain FSL further validates the transferability of the found “winning tickets”. Furthermore, the process of finding LTH can be costly. So we study the early-stage LTH for FSL via exploring the Inverse Scale Space(ISS). Empirical results validate the efficacy of early-stage LTH.

An Efficient Algorithm for Distance-based Structural Graph Clustering

Structural graph clustering (SCAN) is a classic graph clustering algorithm. In SCAN, a key step is to compute the structural similarity between vertices according to the overlap ratio of one-hop neighborhoods. Given two vertices u and v, existing studies only consider the case when u and v are neighbors. However, the structural similarity between non-neighboring vertices in SCAN is always zero, and using only one-hop neighbors on weighted graphs discards the weights on each edge. Both may not reflect the true closeness of two vertices and may fail to return high-quality clustering results. To tackle this issue, we define and study the distance-based structural graph clustering problem. Given a distance threshold d and two vertices u and v, the structural similarity between u and v is defined as the ratio of their respective neighbors within a distance of no more than d. We show that the newly defined distance-based SCAN achieves better clustering results compared to the vanilla version of SCAN. However, the new definition brings challenges in the computation of final clustering results. To tackle this efficiency issue, we propose DistanceSCAN, an efficient approximate algorithm for solving the distance-based SCAN problem. The main idea of DistanceSCAN is to use all-distances bottom-k sketches (ADS) to speed up the computation of similarities. Given the ADS, we can derive the similarity between two vertices with a bounded cost of O(k). However, to ensure that the estimated similarity has an approximation guarantee, the value of k still needs to be set to as large as thousands. This brings high computational costs when computing the similarities between neighboring vertices. To tackle this issue, we further construct histograms to prune the structural similarity computations of vertices pairs. Extensive experiments on real datasets validate the effectiveness and efficiency of DistanceSCAN.

GöwFed: A novel federated network intrusion detection system

Network intrusion detection systems are evolving into intelligent systems that perform data analysis while searching for anomalies in their environment. Indeed, the development of deep learning techniques paved the way to build more complex and effective threat detection models. However, training those models may be computationally infeasible in most Edge or IoT devices. Current approaches rely on powerful centralized servers that receive data from all their parties — violating basic privacy constraints and substantially affecting response times and operational costs due to the huge communication overheads. To mitigate these issues, Federated Learning emerged as a promising approach, where different agents collaboratively train a shared model, without exposing training data to others or requiring a compute-intensive centralized infrastructure. This work presents GöwFed, a novel network threat detection system that combines the usage of Gower Dissimilarity matrices and Federated averaging. Different approaches of GöwFed have been developed based on state-of the-art knowledge: (1) a vanilla version — achieving a median point of [0.888, 0.960] in the PR space and a median accuracy of 0.930; and (2) a version instrumented with an attention mechanism — achieving comparable results when 0.8 of the best performing nodes contribute to the model. Furthermore, each variant has been tested using simulation oriented tools provided by TensorFlow Federated framework. In the same way, a centralized analogous development of the Federated systems is carried out to explore their differences in terms of scalability and performance — the median point of the experiments is [0.987, 0.987]) and the median accuracy is 0.989. Overall, GöwFed intends to be the first stepping stone towards the combined usage of Federated Learning and Gower Dissimilarity matrices to detect network threats in industrial-level networks.

Facial expression recognition in videos using hybrid CNN & ConvLSTM.

The three-dimensional convolutional neural network (3D-CNN) and long short-term memory (LSTM) have consistently outperformed many approaches in video-based facial expression recognition (VFER). The image is unrolled to a one-dimensional vector by the vanilla version of the fully-connected LSTM (FC-LSTM), which leads to the loss of crucial spatial information. Convolutional LSTM (ConvLSTM) overcomes this limitation by performing LSTM operations in convolutions without unrolling, thus retaining useful spatial information. Motivated by this, in this paper, we propose a neural network architecture that consists of a blend of 3D-CNN and ConvLSTM for VFER. The proposed hybrid architecture captures spatiotemporal information from the video sequences of emotions and attains competitive accuracy on three FER datasets open to the public, namely the SAVEE, CK + , and AFEW. The experimental results demonstrate excellent performance without external emotional data with the added advantage of having a simple model with fewer parameters. Moreover, unlike the state-of-the-art deep learning models, our designed FER pipeline improves execution speed by many factors while achieving competitive recognition accuracy. Hence, the proposed FER pipeline is an appropriate candidate for recognizing facial expressions on resource-limited embedded platforms for real-time applications.

Optimal Subsampling Bootstrap for Massive Data

The bootstrap is a widely used procedure for statistical inference because of its simplicity and attractive statistical properties. However, the vanilla version of bootstrap is no longer feasible computationally for many modern massive datasets due to the need to repeatedly resample the entire data. Therefore, several improvements to the bootstrap method have been made in recent years, which assess the quality of estimators by subsampling the full dataset before resampling the subsamples. Naturally, the performance of these modern subsampling methods is influenced by tuning parameters such as the size of subsamples, the number of subsamples, and the number of resamples per subsample. In this article, we develop a novel hyperparameter selection methodology for selecting these tuning parameters. Formulated as an optimization problem to find the optimal value of some measure of accuracy of an estimator subject to computational cost, our framework provides closed-form solutions for the optimal hyperparameter values for subsampled bootstrap, subsampled double bootstrap and bag of little bootstraps, at no or little extra time cost. Using the mean square errors as a proxy of the accuracy measure, we apply our methodology to study, compare and improve the performance of these modern versions of bootstrap developed for massive data through numerical study. The results are promising.

Deep learning-based schemes for singularly perturbed convection-diffusion problems

Deep learning-based numerical schemes such as Physically Informed Neural Networks (PINNs) have recently emerged as an alternative to classical numerical schemes for solving Partial Differential Equations (PDEs). They are very appealing at first sight because implementing vanilla versions of PINNs based on strong residual forms is easy, and neural networks offer very high approximation capabilities. However, when the PDE solutions are low regular, an expert insight is required to build deep learning formulations that do not incur in variational crimes. Optimization solvers are also significantly challenged, and can potentially spoil the final quality of the approximated solution due to the convergence to bad local minima, and bad generalization capabilities. In this paper, we present an exhaustive numerical study of the merits and limitations of these schemes when solutions exhibit low-regularity, and compare performance with respect to more benign cases when solutions are very smooth. As a support for our study, we consider singularly perturbed convection-diffusion problems where the regularity of solutions typically degrades as certain multiscale parameters go to zero.

Vision-based bicycle and motorcycle detection using a YOLO-based Network

This paper describes a system that can distinguish between visually-similar objects, specifically bicycles and motorcycles, successfully from the vantage point of traffic surveillance cameras. The You Only Look Once (YOLO) is used as the main framework in this research due to its speed performance among various machine learning models and methods evaluated. We built a dataset consisting of motorcycle and bicycle images from different CCTV footage for this project. CCTV footage may vary on the angle of view from the object, image resolution, and ambient environment settings. Using this dataset, we trained YOLOv3-based models, and their performances were compared to the vanilla version of YOLOv3 and other pre-trained models. Four (4) models were trained and compared; the best-performing model is shown to be associated with a dataset with properly labeled data (i.e., marking every instance of the object of interest) and having the most number of instances in the training and testing set.Introduction.

WORD SEARCH USING BOYER-MOORE ALGORITHM

Boyer-Moore is one of the algorithms used for the search words that is said to be one of the fastest to complete the search process. This project aims to implement hash table and see the effect it has in the searching process. A hash table is a table that contains key that maps to a value using a hash function. By doing so, the hash table should improve the speed of searching process. The vanilla version of Boyer-Moore will be used as the baseline to test the speed and accuracy of this method and check whether further improvement are needed.

Effect of Term Weighting on Keyword Extraction in Hierarchical Category Structure

While there have been several studies related to the effect of term weighting on classification accuracy, relatively few works have been conducted on how term weighting affects the quality of keywords extracted for characterizing a document or a category (i.e., document collection). Moreover, many tasks require more complicated category structure, such as hierarchical and network category structure, rather than a flat category structure. This paper presents a qualitative and quantitative study on how term weighting affects keyword extraction in the hierarchical category structure, in comparison to the flat category structure. A hierarchical structure triggers special characteristic in assigning a set of keywords or tags to represent a document or a document collection, with support of statistics in a hierarchy, including category itself, its parent category, its child categories, and sibling categories. An enhancement of term weighting is proposed particularly in the form of a series of modified TFIDF's, for improving keyword extraction. A text collection of public-hearing opinions is used to evaluate variant TFs and IDFs to identify which types of information in hierarchical category structure are useful. By experiments, we found that the most effective IDF family, namely TF-IDFr, is identity>sibling>child>parent in order. The TF-IDFr outperforms the vanilla version of TFIDF with a centroid-based classifier.

An edge streaming data processing framework for autonomous driving

In recent years, with the rapid development of sensing technology and the Internet of Things (IoT), sensors play increasingly important roles in traffic control, medical monitoring, industrial production and etc. They generated high volume of data in a streaming way that often need to be processed in real time. Therefore, streaming data computing technology plays an indispensable role in the real-time processing of sensor data in high throughput but low latency. However, there are two problems in deploying streaming data process ability in cloud computing data centre. Firstly, massive sensor nodes simultaneously upload data to the remote cloud computing data centre, which requires a large number of bandwidth resources supports. The existing network infrastructure cannot provide enough bandwidth at a reasonable price. Secondly, due to the geographical distribution characteristics of the cloud computing data centre, there will inevitably be large transmission delay during the process of data transmission. Such end-to-end delay is intolerable to mobile applications especially for those latency sensitive tasks. In view of the above problems, this paper proposes an autonomous driving oriented edge streaming data processing framework, which migrates the computing and storage capability from the remote cloud data centre to the edge data centre. It focuses on the change of vehicle flow in a specific geographical area, and uses the computing power sunk to edge node to process the massive streaming data generated by autonomous vehicles nearby. The proposed framework is implemented on top of Spark Streaming, which builds up a gray model based traffic flow monitor, a traffic prediction orientated prediction layer and a fuzzy control based Batch Interval dynamic adjustment layer for Spark Streaming. It could forecast the variation of sensors data arrive rate, make streaming Batch Interval adjustment in advance and implement real-time streaming process by edge. Therefore, it can realise the monitor and prediction of the data flow changes of the autonomous driving vehicle sensor data in geographical coverage of edge computing node area, meanwhile minimise the end-to-end latency but satisfy the application throughput requirements. The experiments show that it can predict short-term traffic with no more than 4% relative error in a whole day. By making batch consuming rate close to data generating rate, it can maintain system stability well even when arrival data rate changes rapidly. The Batch Interval can be converged to a suitable value in two minutes when data arrival rate is doubled. Compared with vanilla version Spark Streaming, where there has serious task accumulation and introduces large delay, it can reduce 35% latency by squeezing Batch Interval when data arrival rate is low; it also can significantly improve system throughput by only at most 25% Batch Interval increase when data arrival rate is high.

Fittest Survival: An Enhancement Mechanism for Monte Carlo Tree Search

Monte Carlo tree search (MCTS), which constructs a search tree of game states and evaluates expected rewards by thousands of Monte Carlo simulations, has become the pre-eminent approach for many challenging games. One severe challenge of MCTS is the contradiction between the accuracy of states' evaluation and practical time consumption for simulations, both of which are critical for a competitive game program. This paper proposes and evaluates fittest survival Monte Carlo tree search (FS-MCTS) which provides a novel mechanism to enhance MCTS towards correct direction with fewer simulations. The key idea of FS-MCTS is to keep states with significant advantages to survive while eliminate the others. This is the meaning of 'fittest survival'. In this sense, FS-MCTS no longer completely depends on the evaluation accuracy of game states which severely relies on adequate simulation times. We evaluate FS-MCTS in the problems of poker. Experimental results show that FS-MCTS, combining with several variants of popular UCB policies, performs better than their vanilla versions when a certain number of simulations are guaranteed for its theoretical prerequisites.

Efficient Biomedical Image Segmentation on EdgeTPUs at Point of Care

The U-Net architecture is a state-of-the-art neural network for semantic image segmentation that is widely used in biomedical research. It is based on an encoder-decoder framework and its vanilla version shows already high performance in terms of segmentation quality. Due to its large parameter space, however, it has high computational costs on both, CPUs and GPUs. In a research setting, inference time is relevant, but not crucial for the results. However, especially in mobile, clinical applications a light and fast variant would allow deep-learning assisted, objective diagnosis at the point of care. In this work, we suggest an optimized, tiny-weight U-Net for an inexpensive hardware accelerator. We first mined the U-Net architecture to reduce computational complexity to increase runtime performance by simultaneously keeping the accuracy on a high level. Using an open, biomedical dataset for high-speed videoendoscopy (BAGLS), we show that we can dramatically reduce the parameter space and computations by over 99.8% while keeping the segmentation performance at 95% of our baseline. Using a custom upscaling routine, we further successfully deployed our optimized U-Net to an EdgeTPU hardware accelerator to gain cost-effective speed improvements on conventional computers and to showcase the applicability of EdgeTPUs for biomedical imaging processing of large images on portable devices. Combining the optimized architecture and the EdgeTPU, we gain a speedup of >79-times compared to our initial baseline while keeping high accuracy. This combination allows to provide immediate results to the clinician, especially in constrained computational environments, and an objective diagnosis at the point of care.

Canonical statistical model analysis of p−p , p -Pb, and Pb-Pb collisions at energies available at the CERN Large Hadron Collider

The system-size dependence of hadrochemistry at vanishing baryon density is considered within the canonical statistical model (CSM) with local exact conservation of three conserved charges, allowing for a possibility of strangeness undersaturation, i.e. $\gamma_S \leq 1$. Exact baryon number conservation is found to be even more important than that of strangeness in the canonical suppression picture at the LHC, in contrast to intermediate and low collision energies. The model is applied to p-p, p-Pb, and Pb-Pb data of the ALICE collaboration. A chemical equilibrium CSM with a fixed $T_{\rm ch} = 155$ MeV describes the trends seen in most yield ratios. However, this vanilla version of CSM predicts an enhancement of the $\phi/\pi$ ratio at smaller multiplicities, in stark contrast to the suppression seen in the data. The data are described with a 15% relative accuracy level whence a multiplicity dependence of both the temperature and the strangeness saturation parameter $\gamma_S \leq 1$ is accepted. Both the canonical suppression and the strangeness undersaturation effects are small at $d N_{\rm ch} / d\eta \gtrsim 100$, but they do improve substantially the description of hadron yields in p-p collisions, in particular the $\Omega$ yields. A possibility to constrain the rapidity correlation volume using net-proton fluctuation measurements is pointed out.

Runtime performance evaluation and optimization of type-2 hypervisor for MIPS64 architecture

Over the last decade, virtualization technologies have seen unprecedented growth in the system-level domain and promptly have moved to the embedded system domain. While there are numerous applications of virtualization for off-the-shelf hardware such as security, sandboxing, testing tools, etc. However, there are few open-source virtualization solutions available for embedded systems. For real-time virtualization, good runtime performance is a deterministic factor for its practical use in the embedded domain. In this paper, the internal architecture of HTTM (an open-source type-2 hypervisor for MIPS64 architecture) was thoroughly analyzed. ISA virtualization unit and execution cycle control were two primary units that were profiled and explored for optimization. Dynamic Binary Translation (DBT) unit was modified to generate highly efficient code. While the reduction in switching between the translated code and management layer improved the overall execution cycle. The implementation of these strategies resulted in a 72–91% improvement in bandwidth benchmarks. Similarly, latency benchmarks show a 2–92% improvement from its vanilla version. Collectively producing an overall 1–5 times the improvement in execution time. The performance of optimized HTTM is also compared with Quick Emulator (QEMU). HTTM performs 44–80% better than QEMU in bandwidth benchmarks while QEMU performs better in latency operations.

Diversification-based learning simulated annealing algorithm for hub location problems

PurposeThe purpose of this paper is to examine the efficacy of diversification-based learning (DBL) in expediting the performance of simulated annealing (SA) in hub location problems.Design/methodology/approachThis study proposes a novel diversification-based learning simulated annealing (DBLSA) algorithm for solving p-hub median problems. It is executed on MATLAB 11.0. Experiments are conducted on CAB and AP data sets.FindingsThis study finds that in hub location models, DBLSA algorithm equipped with social learning operator outperforms the vanilla version of SA algorithm in terms of accuracy and convergence rates.Practical implicationsHub location problems are relevant in aviation and telecommunication industry. This study proposes a novel application of a DBLSA algorithm to solve larger instances of hub location problems effectively in reasonable computational time.Originality/valueTo the best of the author’s knowledge, this is the first application of DBL in optimisation. By demonstrating its efficacy, this study steers research in the direction of learning mechanisms-based metaheuristic applications.