Set Of Available Actions Research Articles

Embracing the Game-Like Character of Science and Engineering Activities: A Perspective to Guide Teachers’ Instructional Decisions

ABSTRACT Recent science education reforms emphasize the goal of developing students’ science and engineering practices. To foreground those practices during science instruction, teachers need conceptual tools to guide their instructional decision-making. In this conceptual paper, I propose that thinking about science and engineering activities as educative games provides a useful model for designing productive educational experiences. Central to this argument is a recognition of the ways that games allow learners to experience and cultivate new forms of agency—including those aligned with the practice of science and engineering. When immersed in a game, a student/player pursues clearly defined goals using a limited set of available actions, and in doing so they act out and experience a form of agency that they might not otherwise have been able to access. Viewed in this way, science and engineering activities can be analyzed and evaluated with respect to the forms of agency that they cultivate in students, including the extent to which those forms of agency are consistent with those of science and engineering. In addition to presenting this way of thinking about science and engineering activities, this paper proposes a practical model to assist teachers in analyzing and designing science and engineering learning activities.

Single-trial stimuli classification from detected P300 for augmented Brain–Computer Interface: A deep learning approach

The purpose of advanced Brain–Computer Interfaces (BCIs) is to connect the human brain with an external device without using the muscular system. To do this, they must effectively process mental activity and infer information on the users’ intentions and directives. This work proposes a novel and explainable BCI system capable of recognizing P300 deflection in single-trial EEGs with higher accuracy compared to the literature gold standard. Moreover, the proposed deep-learning approach allows us to go beyond the mere P300 detection, which is, to our best knowledge, the current state of the art. Indeed, we first identify the P300-related signal in the single-trial EEG signal, and then, we further discriminate the ERPs associated to the detected P300 between visual and auditory stimuli-related. To do this, we employ a CNN–LSTM neural network, which manages a 3D data representation of the acquired EEG signals. The performance of the approach is tested on experiments carried out on 22 subjects, revealing a 82.4% F1-score in P300 identification and 82.4% discriminating between visual and auditory stimuli. The employed algorithmic procedure also reports the most relevant each EEG channels in determining the predictions, adding interpretability to the proposed AI-based tools. These results pave the way for more sophisticated BCIs, capable of extending the set of available actions for the patients. The project was pre-approved by the Research Assessment Committee of the Department of Psychology (CRIP) for minimal risk projects, under the aegis of the Ethical Committee of University of Milano-Bicocca, on May 27th, 2019, protocol number RM-2019-193.

The Efficient Resource Scheduling Strategy in Cloud: A Metaheuristic Approach

The cloud computing is evolving as a high-performance computing platform due to broad-scale, flexible computational architecture and heterogeneous collection of autonomous systems. Cloud technology uses concept of virtualization for managing resources, which develops resource scheduling as a key issue. The scheduling of cloud tasks is an NP-complete problem and therefore irreconcilable with particular solution. Also, with the huge collection of a database system, the management of resources and tasks becomes complex with specific to the completion time requirements and cost constraints. To resolve this problem, a number of meta-heuristic algorithms have been developed. Due to redundant wastage of resources and time, the under and over-provisioning is one kind of issues leads to either degradation in performance or wastage of cloud resources. To overcome these kinds of problems, we introduce a task scheduling approach by incorporating reinforcement learning along with the nature-inspired meta-heuristic optimization to maximizing cloud throughput, minimizing completion time & production cost in IaaS cloud. By reinforcement learning, the agent will choose appropriate action among a set of available actions and the scheduler succeeds towards task allocation resulted to decrease makespan and increasing system utilization rate.

Rate and Channel Adaptation in Cognitive Radio Networks Under Time-Varying Constraints

We consider dynamic rate and channel adaptation in a cognitive radio network serving heterogeneous applications under dynamically varying channel availability and rate constraint. We formalize it as a Bayesian learning problem, and propose a novel learning algorithm, called Volatile Constrained Thompson Sampling (V-CoTS), which considers each rate-channel pair as a two-dimensional action. The set of available actions varies dynamically over time due to variations in primary user activity and rate requirements of the applications served by the users. Our algorithm learns to adapt its rate and opportunistically exploit spectrum holes when the channel conditions are unknown and channel state information is absent, by using acknowledgment only feedback. It uses the monotonicity of the transmission success probability in the transmission rate to optimally tradeoff exploration and exploitation of the actions. Numerical results demonstrate that V-CoTS achieves significant gains in throughput compared to the state-of-the-art methods.

Maximising goals achievement through abstract argumentation frameworks: An optimal approach

Argumentation is a prominent AI research area, focused on approaches and techniques for performing common-sense reasoning, that is of paramount importance in a wide range of real-world applications, such as decision support and recommender systems. In this work we introduce an approach for updating an abstract Argumentation Framework (AF) so that achievement with respect to a given set of goals is maximised. The set of goals identifies arguments for which a specific acceptability status (a labelling) will be pursued, distinguishing between “in” and “out” goals. Given an AF, a set of goals and a set of available actions allowing to add or remove arguments and attacks from the AF, our approach will select the strategy (set of actions) that should be applied in order to obtain a new AF where the goals achievement is maximised. Moreover, the selected strategy will be optimal with respect to the number of actions to be applied. In the context of argumentation-based expert and intelligent systems, our approach will provide tools allowing the user to interact with the argumentative reasoning process carried out by the system, learning how the strategy she undertakes will affect the recommendations she receives. For that, we propose an encoding of the AF, the available actions and goals as weighted Boolean formulas, and rely on MaxSAT techniques for selecting the optimal strategy. We provide an experimental analysis of our approach, and formally show that the results we obtain correspond to the optimal strategy.

Logic Multiplicity and Organizational Competitiveness

Researchers predict that organizations that integrate highly conflicting logics into their core activities are less likely to succeed than organizations that prioritize institutional templates. This paper studies two cases that challenge these predictions, investigating how, and under what conditions, the instantiation of conflicting logics affect organizations’ competitiveness. The results show that organizations that present high logic centrality access a broader set of alternative market strategies, which may translate into a higher competitive agility and a privileged access to primary resources, which are essential for the organizations’ commercial success. However, while at the strategy level such organizations access a multiplicity of options, at the practice level the compromise amongst conflicting logics translates into a reduced set of available actions, pushing organizations towards a “compromise zone lock-in effect”. In volatile and highly competitive markets, where non-commercial constituents provide a primary resource, the benefits of increased competitive agility and privileged access to primary resources might overcome the costs of being “locked into the compromise zone”. Under these conditions, organizations that present high logic centrality may be more competitive than organizations that have a clear logic hierarchy.

Deep Learning for Signal Authentication and Security in Massive Internet-of-Things Systems

Secure signal authentication is arguably one of the most challenging problems in the Internet of Things (IoT), due to the large-scale nature of the system and its susceptibility to man-in-the-middle and data-injection attacks. In this paper, a novel watermarking algorithm is proposed for dynamic authentication of IoT signals to detect cyber-attacks. The proposed watermarking algorithm, based on a deep learning long short-term memory structure, enables the IoT devices (IoTDs) to extract a set of stochastic features from their generated signal and dynamically watermark these features into the signal. This method enables the IoT gateway, which collects signals from the IoTDs, to effectively authenticate the reliability of the signals. Moreover, in massive IoT scenarios, since the gateway cannot authenticate all of the IoTDs simultaneously due to computational limitations, a game-theoretic framework is proposed to improve the gateway's decision making process by predicting vulnerable IoTDs. The mixed-strategy Nash equilibrium (MSNE) for this game is derived, and the uniqueness of the expected utility at the equilibrium is proven. In the massive IoT system, due to the large set of available actions for the gateway, the MSNE is shown to be analytically challenging to derive, and thus, a learning algorithm that converges to the MSNE is proposed. Moreover, in order to handle incomplete information scenarios, in which the gateway cannot access the state of the unauthenticated IoTDs, a deep reinforcement learning algorithm is proposed to dynamically predict the state of unauthenticated IoTDs and allow the gateway to decide on which IoTDs to authenticate. Simulation results show that with an attack detection delay of under 1 s, the messages can be transmitted from IoTDs with an almost 100% reliability. The results also show that by optimally predicting the set of vulnerable IoTDs, the proposed deep reinforcement learning algorithm reduces the number of compromised IoTDs by up to 30%, compared to an equal probability baseline.

Online adaptive teleoperation via motion primitives for mobile robots

Assistive teleoperation aims to help operators control robotic systems with ease. In this work, we present a novel adaptive teleoperation approach that is amenable to mobile systems using motion primitives for long-duration teleoperation, such as exploration using mobile vehicles or walking for humanoid systems. We first describe teleoperation using motion primitives, which are dynamically feasible and safe local trajectories based on a kinematic or dynamic model. We take a predict-and-adapt approach to assistive teleoperation, whereby adaptation is based on the predicted user intent. By representing the operator as an optimizing controller, a probabilistic distribution can be constructed for the available future actions based on some reward function. Adaptation is provided in the form of subsampling, which tailors the set of available actions based on the likelihood of action selection. We describe the framework for general systems and delineate the extrapolation to ground, air, and legged mobile robots, and demonstrate generalizability of this framework on two systems via simulation and experimentation; namely, a quadrotor micro air vehicle, and a simulated 3D humanoid system. Both systems show provably better performance in teleoperation by measures of behavioral entropy.

Dynamic choice in a complex world

We consider an environment in which a Decision Maker (DM) finds it sufficiently complex to even describe the state space, let alone guess the parameters of the underlying data generating process. He is therefore unable to use the standard Bayesian methods. Instead, at each moment in time, the DM constructs a preference relation on the set of available actions based on their past performance. We postulate a set of axioms on this family of preference relations indexed by histories (of rewards). Two key conditions, a typical Exchangeability axiom and another labeled Consistency, that constraints the DM not to update her preferences after certain events (loosely based the “the principle of insufficient reason”), characterize thumb rules for ranking actions that are akin to the familiar fictitious play in game theory. Moreover, if in fact the stochastic process that generates the rewards is exchangeable, a DM that obeys our axioms (almost surely) cannot be distinguished in the long run from one that is fully cognizant of the environment and satisfies the expected utility hypothesis. The main result requires proving a representation result on multisets which may be of independent interest.

Learning Hurdles for Sleeping Experts

We study the online decision problem in which the set of available actions varies over time, also called the sleeping experts problem. We consider the setting in which the performance comparison is made with respect to the best ordering of actions in hindsight. In this article, both the payoff function and the availability of actions are adversarial. Kleinberg et al. [2010] gave a computationally efficient no-regret algorithm in the setting in which payoffs are stochastic. Kanade et al. [2009] gave an efficient no-regret algorithm in the setting in which action availability is stochastic. However, the question of whether there exists a computationally efficient no-regret algorithm in the adversarial setting was posed as an open problem by Kleinberg et al. [2010]. We show that such an algorithm would imply an algorithm for PAC learning DNF , a long-standing important open problem. We also consider the setting in which the number of available actions is restricted and study its relation to agnostic-learning monotone disjunctions over examples with bounded Hamming weight.

DYNAMICS OF CHOICE RESTRICTION IN LARGE GAMES

We study games in which the number of players are large, and hence outcomes are independent of the identities of the players. Game models typically study how choices made by individual rational players determine game outcomes. We extend this model to include an implicit player — the society, who makes actions available to players and incurs certain costs in doing so. In the course of play, an option a may be chosen only by a small number of players and hence may become too expensive to maintain, so the society may remove it from the set of available actions. This results in a change in the game and the players strategize afresh taking this change into account. We highlight the mutual recursiveness of individual rationality and societal rationality in this context. Specifically, we study two questions: When players play according to given strategy specifications, which actions of players should the society restrict and when, so that the social cost is minimized eventually? Conversely, assuming a set of rules by which society restricts choices, can players strategize in such a way as to ensure certain outcomes? We discuss solutions in finite memory strategies.

Testing timed systems modeled by Stream X-machines

Stream X-machines have been used to specify real systems where complex data structures. They are a variety of extended finite state machine where a shared memory is used to represent communications between the components of systems. In this paper we introduce an extension of the Stream X-machines formalism in order to specify systems that present temporal requirements. We add time in two different ways. First, we consider that (output) actions take time to be performed. Second, our formalism allows to specify timeouts. Timeouts represent the time a system can wait for the environment to react without changing its internal state. Since timeous affect the set of available actions of the system, a relation focusing on the functional behavior of systems, that is, the actions that they can perform, must explicitly take into account the possible timeouts. In this paper we also propose a formal testing methodology allowing to systematically test a system with respect to a specification. Finally, we introduce a test derivation algorithm. Given a specification, the derived test suite is sound and complete, that is, a system under test successfully passes the test suite if and only if this system conforms to the specification.

Regret and Convergence Bounds for a Class of Continuum-Armed Bandit Problems

We consider a class of multi-armed bandit problems where the set of available actions can be mapped to a convex, compact region of Ropfd, sometimes denoted as the ldquocontinuum-armed banditrdquo problem. The paper establishes bounds on the efficiency of any arm-selection procedure under certain conditions on the class of possible underlying reward functions. Both finite-time lower bounds on the growth rate of the regret, as well as asymptotic upper bounds on the rates of convergence of the selected control values to the optimum are derived. We explicitly characterize the dependence of these convergence rates on the minimal rate of variation of the mean reward function in a neighborhood of the optimal control. The bounds can be used to demonstrate the asymptotic optimality of the Kiefer-Wolfowitz method of stochastic approximation with regard to a large class of possible mean reward functions.

Joint optimization of customer segmentation and marketing policy to maximize long-term profitability

Abstract With the advent of one-to-one marketing media, e.g. targeted direct mail or internet marketing, the opportunities to develop targeted marketing (customer relationship management) campaigns are enhanced in such a way that it is now both organizationally and economically feasible to profitably support a substantially larger number of marketing segments. However, the problem of what segments to distinguish, and what actions to take towards the different segments increases substantially in such an environment. A systematic analytic procedure optimizing both steps would be very welcome. In this study, we present a joint optimization approach addressing two issues: (1) the segmentation of customers into homogeneous groups of customers, (2) determining the optimal policy (i.e. what action to take from a set of available actions) towards each segment. We implement this joint optimization framework in a direct-mail setting for a charitable organization. Many previous studies in this area highlighted the importance of the following variables: R(ecency), F(requency), and M(onetary value). We use these variables to segment customers. In a second step, we determine which marketing policy is optimal using markov decision processes, following similar previous applications. The attractiveness of this stochastic dynamic programming procedure is based on the long-run maximization of expected average profit. Our contribution lies in the combination of both steps into one optimization framework to obtain an optimal allocation of marketing expenditures. Moreover, we control segment stability and policy performance by a bootstrap procedure. Our framework is illustrated by a real-life application. The results show that the proposed model outperforms a CHAID segmentation.

Controlled one dimensional diffusions with switching costs—average cost criterion

This paper deals with a one-dimensional controlled diffusion process on a compact interval with reflecting boundaries. The set of available actions is finite and the action can be changed only at countably many stopping times. The cost structure includes both a continuous movement cost rate depending on the state and the action, and a switching cost when the action is changed. The policies are evaluated with respect to the average cost criterion. The problem is solved by looking at, for each stationary policy, an embedded stochastic process corresponding to the state intervals visited in the sequence of switching times. The communicating classes of this process are classified into closed and transient groups and a method of calculating the average cost for the closed and transient classes is given. Also given are conditions to guarantee the optimality of a stationary policy. A Brownian motion control problem with quadratic cost is worked out in detail and the form of an optimal policy is established.