Tuning Overhead Research Articles

Faster MIL-based Subgoal Identification for Reinforcement Learning by Tuning Fewer Hyperparameters

Various methods have been proposed in the literature for identifying subgoals in discrete reinforcement learning (RL) tasks. Once subgoals are discovered, task decomposition methods can be employed to improve the learning performance of agents. In this study, we classify prominent subgoal identification methods for discrete RL tasks in the literature into the following three categories: graph-based, statistics-based, and multi-instance learning (MIL)-based. As contributions, firstly, we introduce a new MIL-based subgoal identification algorithm called EMDD-RL and experimentally compare it with a previous MIL-based method. The previous approach adapts MIL’s Diverse Density (DD) algorithm, whereas our method considers Expected-Maximization Diverse Density (EMDD). The advantage of EMDD over DD is that it can yield more accurate results with less computation demand thanks to the expectation-maximization algorithm. EMDD-RL modifies some of the algorithmic steps of EMDD to identify subgoals in discrete RL problems. Secondly, we evaluate the methods in several RL tasks for the hyperparameter tuning overhead they incur. Thirdly, we propose a new RL problem called key-room and compare the methods for their subgoal identification performances in this new task. Experiment results show that MIL-based subgoal identification methods could be preferred to the algorithms of the other two categories in practice.

Pattern unlocking guided multi‐modal continuous authentication for smartphone with multi‐branch context‐aware representation learning and auto encoder

AbstractWidely accepted explicit authentication protocols are vulnerable to a series of attacks, for example, shoulder surfing and smudge attacks, leaving users with the constant burden of periodic password changes. As such, we propose a novel framework for continuous authentication on smartphones. This approach is guided by pattern unlocking, which is widely used and will not cause learning cost. After collecting multi‐modal data that describe both behavioral and contextual information, we employ a multi‐branch context‐aware attention network as the representation learner to perform feature extraction, then an auto encoder is then used for authentication. To overcome challenges, including cold‐start and few‐shot training, which is less discussed in other works, we incorporate transfer learning with a coarse‐to‐fine pre‐training workflow. Additionally, we deploy a hierarchical approach to offload model tuning overhead from smartphones. Extensive experiments on more than 68 000 real‐world recordings validate the effectiveness of the proposed method, with an EER (equal error rate) of 2.472% under mixed contexts, which consistently outperforms state‐of‐the‐art approaches under both static and mixed contexts.

A Fast Precision Tuning Solution for Always-On DNN Accelerators

Due to the non-volatility nature of Resistive RAM (ReRAM), dynamic operations in the arrays contribute to a much larger portion of power in ReRAM based neural networks than static power. To reduce the dynamic power of in-situ operations with neural parameters, precisiontuning is considered a viable approach of approximate computing to trade-off excessive computation exactness for power and efficiency gains. However, the switching overhead of precision tuning in hardware severely impacts its effectiveness when the systems need to quickly react to the change of environment, user constraint or input quality. This work for the first time investigates the feasibility of agile precision tuning for neural network accelerators to benefit from approximate computing. The proposed Computing in Memory (CiM) CNN accelerators fully utilize the normally-off characteristics of memristor crossbars to achieve instant network precision tuning without worrying about the model reloading penalty. The ReRAM-based accelerator, with the proposed neural parameter mapping policy and the novel mixed-model training method, induces negligible precision-switching latency and power consumption when compared with traditional variable precision accelerators. In evaluation with state-of-the-art workloads, the proposed ReRAM DNN architecture saves 58.3%-62.47% area overhead over the baseline design. We also leverage the proposed ReRAM accelerator architecture to build a novel always-on Key-Word Spotting (KWS) system. The KWS design can switch between different precision modes to capture the relevant sound with high accuracy. The experimental results show the precisionadjustable KWS architecture saves considerable operating energy when fed with realistic test-sets of audio data.

A Systematic Comparison of search-Based approaches for LDA hyperparameter tuning

Context: Latent Dirichlet Allocation (LDA) has been successfully used in the literature to extract topics from software documents and support developers in various software engineering tasks. While LDA has been mostly used with default settings, previous studies showed that default hyperparameter values generate sub-optimal topics from software documents. Objective: Recent studies applied meta-heuristic search (mostly evolutionary algorithms) to configure LDA in an unsupervised and automated fashion. However, previous work advocated for different meta-heuristics and surrogate metrics to optimize. The objective of this paper is to shed light on the influence of these two factors when tuning LDA for SE tasks. Method: We empirically evaluated and compared seven state-of-the-art meta-heuristics and three alternative surrogate metrics (i.e., fitness functions) to solve the problem of identifying duplicate bug reports with LDA. The benchmark consists of ten real-world and open-source projects from the Bench4BL dataset. Results: Our results indicate that (1) meta-heuristics are mostly comparable to one another (except for random search and CMA-ES), and (2) the choice of the surrogate metric impacts the quality of the generated topics and the tuning overhead. Furthermore, calibrating LDA helps identify twice as many duplicates than untuned LDA when inspecting the top five past similar reports. Conclusion: No meta-heuristic and/or fitness function outperforms all the others, as advocated in prior studies. However, we can make recommendations for some combinations of meta-heuristics and fitness functions over others for practical use. Future work should focus on improving the surrogate metrics used to calibrate/tune LDA in an unsupervised fashion.

QoS-Aware Virtualization Resource Management Mechanism in 5G Backhaul Heterogeneous Networks

An efficient resource management strategy can save network resources and improve network quality of service (QoS). In order to solve the problem of resource management difficulties of multiple SPs sharing the same backhaul network and QoS mismatch in heterogeneous networks, a downstream resource management in 5G virtualized backhaul network is proposed in this paper. Firstly, the wavelength assignment problem is described as a mathematics model with the consideration of minimizing turning overhead and load balancing. The quantum-behaved particle swarm optimization algorithm is used to solve the mathematics model. Secondly, the 5G traffic is mapping to different priority queues in TWDM-PON by the proposed QoS mapping algorithm. Finally, the downstream traffic is scheduled by the proposed resource scheduling algorithm. Numerical results show that the proposed mechanism can maintain high resource utilization, reduce wavelength tuning overhead and improve the delay of traffic.

Techniques for dynamic hardware management of streaming media applications using a framework for system scenarios

Many modern applications exhibit dynamic behavior, which can be exploited for reduced energy consumption. We employ a two-phase combined design-time/run-time methodology that identifies different run-time situations and clusters similar behaviors into system scenarios. This methodology is integrated with our framework for system scenario based designs, which dynamically tune the hardware to match the application behavior. We focus on streaming media applications, and achieve significant energy reductions for an extracted control structure of a video codec widely used in hand-held devices today. We encode a video stream consisting of different frame sizes based on measured available wireless bandwidth. Energy consumption is measured with a modified microcontroller board from Atmel with two alternative voltage and frequency settings. While maintaining the perceptual video quality and frame rate, our method results in up to 49% energy reduction for our encoded streams. The average tuning overhead of our mechanism is negligible after applying simple loop transformations to the encoder control structure. We furthermore show how to obtain up to 34.3% energy reductions for a system with limited slack available, by dynamically exploiting the available sleep modes.

Automatic Scale Parameters in Affinity Matrix Construction for Improved Spectral Clustering

Spectral clustering partitions data into similar groups in the eigenspace of the affinity matrix. The accuracy of the spectral clustering algorithm is affected by the affine equivariance realized in the translation of distance to similarity relationship. The similarity value computed as a Gaussian of the distance between data objects is sensitive to the scale factor [Formula: see text]. The value of [Formula: see text], a control parameter of drop in affinity value, is generally a fixed constant or determined by manual tuning. In this research work, [Formula: see text] is determined automatically from the distance values i.e. the similarity relationship that exists in the real data space. The affinity value of a data pair is determined as a location estimate of the spread of distance values of the data points with the other points. The scale factor [Formula: see text] corresponding to a data point [Formula: see text] is computed as the trimean of its distance vector and used in fixing the scale to compute the affinity matrix. Our proposed automatic scale parameter for spectral clustering resulted in a robust similarity matrix which is affine equivariant with the distance distribution and also eliminates the overhead of manual tuning to find the best [Formula: see text] value. The performance of spectral clustering using such affinity matrices was analyzed using UCI data sets and image databases. The obtained scores for NMI, ARI, Purity and F-score were observed to be equivalent to those of existing works and better for most of the data sets. The proposed scale factor was used in various state-of-the-art spectral clustering algorithms and it proves to perform well irrespective of the normalization operations applied in the algorithms. A comparison of clustering error rates obtained for various data sets across the algorithms shows that the proposed automatic scale factor is successful in clustering the data sets equivalent to that obtained using manually tuned best [Formula: see text] value. Thus the automatic scale factor proposed in this research work eliminates the need for exhaustive grid search for the best scale parameter that results in best clustering performance.

Phase distance mapping: a phase-based cache tuning methodology for embedded systems

Networked embedded systems typically leverage a collection of low-power embedded systems (nodes) to collaboratively execute applications spanning diverse application domains (e.g., video, image processing, communication, etc.) with diverse application requirements. The individual networked nodes must operate under stringent constraints (e.g., energy, memory, etc.) and should be specialized to meet varying applications' requirements in order to adhere to these constraints. Phase-based tuning specializes a system's tunable parameters to the varying runtime requirements of an application's different phases of execution to meet optimization goals. Since the design space for tunable systems can be very large, one of the major challenges in phase-based tuning is determining the best configuration for each phase without incurring significant tuning overhead (e.g., energy and/or performance) during design space exploration. In this paper, we propose phase distance mapping, which directly determines the best configuration for a phase, thereby eliminating design space exploration. Phase distance mapping applies the correlation between a known phase's characteristics and best configuration to determine a new phase's best configuration based on the new phase's characteristics. Experimental results verify that our phase distance mapping approach, when applied to cache tuning, determines cache configurations within 1 % of the optimal configurations on average and yields an energy delay product savings of 27 % on average.

Optimal Scheduling for Minimum Delay in Passive Star Coupled WDM Optical Networks

In this paper, a local optimization framework is proposed, which is able to include system constraints including channel availability, receiver availability and tuning overhead by linear mathematical formulations so that it is sufficient to obtain the optimal performance in terms of message delay. A mixed integer linear programming (MILP) based scheme for passive star coupled WDM optical networks is presented. Based on the new solution, the wavelength assignment and message sequence that guarantee the delivery of the given traffic request, while minimizing the average delay can be achieved. Moreover, the negative effect of the tuning overhead has been incorporated into the new algorithm, which has been ignored in most of the previous work. Numerical results obtained suggest that the proposed scheme is a promising approach for optimizing the network performance in terms of average message delay.

Message scheduling in WDM optical networks with reduced transmitter tuning overhead

Relatively large transceiver tuning overhead is one major difficulty when designing a medium access control scheme for a single-hop passive-star coupled WDM optical network. To overcome this difficulty, we propose two algorithms, namely continuous channel scheduling (CCS) and continuous channel-minimum scheduling latency (CC-MSL), to reduce the negative impact of tuning overhead as much as possible. Extensive simulations have been conducted. And the simulation results show that significant improvement in average delay can be achieved by our new algorithms.

Efficient scheduling with reduced message delay for passive star coupled WDM optical networks

A novel reservation-based multiple access scheme for single-hop passive star coupled wavelength division multiplexing optical networks has been proposed. Unlike other existing reservation-based scheduling algorithms for the same type of networks, the major feature of the proposed scheme is that it has considered carefully the physical features of the network and traffic, which may affect the performance of the networks, for traffic scheduling. Those features include message length, channel availability, destination availability and transceivers tuning overhead. The algorithm has been deliberately designed with considerations on the timing relationships among those factors in order to reduce the idle time on the data channels resulting in its superior ability to reduce the end-to-end delay and to enhance the utilisation of the transmission channels. The performance of the proposed algorithm has been studied extensively by numerical simulations.

Persistent reservation protocol for variable-length messages in WDM-based local networks

The author proposes a persistent reservation protocol in a wavelength division multiplexing (WDM)-based local network using a passive star topology. With this protocol, once a node reserves a data channel, the node persistently uses the channel until its message is completely transmitted. Therefore, the protocol needs only one tuning time for a node to begin to transmit its message, which reduces deterioration of throughput due to tuning overhead. The proposed protocol can efficiently support variable-length messages. The network can accommodate a variable number of nodes and can operate independently of any change of the number of nodes. Hence, any ‘new’ node can join the network at anytime without network re-initialisation. Moreover, with the protocol, data channel and destination conflicts can be avoided. Throughput–delay characteristics are analysed with a finite population model, and are verified by simulation results.

Scheduling variable-length messages in a single-hop multichannel local lightwave network

The design of a medium access control scheme for a single-hop, wavelength-division-multiplexing-(WDM) multichannel local lightwave network poses two major difficulties: relatively large transmitter/receiver tuning overhead and large ratio of propagation delay to packet transmission time. Most schemes proposed so far have ignored the tuning overhead, and they can only schedule fixed-length packet transmissions. To overcome these two difficulties, the authors propose several scheduling algorithms which can reduce the negative impact of tuning overhead and schedule variable-length messages. A separate channel (control channel) is employed for transmission of control packets, and a distributed scheduling algorithm is invoked at each node every time it receives a control packet. By allowing the length of messages to be variable, a long message can be scheduled with a single control packet transmission, instead of fragmenting it into many fixed-length packets, thereby significantly reducing the overhead of control packet transmissions and improving the overall system performance. Three novel scheduling algorithms are proposed, varying in the amount of global information and processing time they need. Two approximate analytical models are formulated to study the effect of tuning time and the effect of having a limited number of data channels. Extensive simulations are conducted. Average message delays are compared for all of the algorithms. >

Scalable High-Speed Protocols for WDM Optical Star Networks

This paper proposes scalable conflict-free media-access protocols for WDM optical passive star networks using shared parallel queueing and wormhole scheduling. Shared parallel queueing is to associate several parallel queues with a data channel, rather than one deep queue (as in DT-WDMA) or N static queues (as in [5] and [6], N is the number of stations). This significantly reduces both the hardware resource requirement and the control signalling demand. Wormhole scheduling is to schedule multiple successive packets which are destined to the same receiver at one time, decreasing the overhead of tuning caused by relatively slow filter and relieving the electronic processing bottleneck. An analytical model is presented and verified through simulation. Numerical results from analysis and simulation indicate that with only 5-6 parallel queues, the protocols are capable of achieving high throughput. Thus the protocols can be scaled up for networks with a large number of stations and could lead to a potential implementation.