Trading Agent Research Articles

Intelligent Trading Agents for Massively Multi-Player Game Economies

As massively multi-player gaming environments become more detailed, developing agents to populate these virtual worlds as capable non-player characters poses an increasingly complex problem. Human players in many games must achieve their objectives through financial skills such as trading and supply chain management as well as through combat and diplomacy. In this paper, we examine the problem of creating intelligent trading agents for virtual markets. Using historical data from EVE Online, a science-fiction based MMORPG, we evaluate several strategies for buying, selling, and supply chain management. We demonstrate that using reinforcement learning to determine policies based on the market microstructure gives trading agents a competitive advantage in amassing wealth. Imbuing agents with the ability to adapt their trading policies can make them more resistant to exploitation by other traders and capable of participating in virtual economies on an equal footing with humans.

The “Golden City” under Embargo: Prague's International Trade during the Hussite Wars, 1420–1436

AbstractAfter the outbreak of the First Hussite Wars (spring 1420), the Hussite capital Prague faced—at least in theory—a total embargo on all trade and commerce. However, trade evidently continued in spite of this embargo. The present article systematically assesses our knowledge on this trade and highlights articles, geographical structures and agents of long-distance trade to and from the Czech metropolis during the war, thus furthering our knowledge about the economic history of early fifteenth-century Central Europe in general. Furthermore, the author uses the example of the anti-Hussite embargo to address important and hitherto largely-neglected methodological questions concerning the analysis of medieval trade prohibitions in general.

Russia Trade with Africa: The Current State of Affairs

Russia is strengthening its geopolitical influence. An example of this is the holding of the first Russia-Africa summit and economic forum in Sochi in October 2019. This event showed how important Africa is for Russia and how important Russia is for Africa. It was an epoch-making event that demonstrated to the world the degree of Russian-African mutual interest.With what has Russia approached this event and how will economic cooperation develop? This work is devoted to this. The article deals with the issues of mutual trade between Russia and African countries. The main African trade agents of the Russian Federation in recent years have been identified. The state of trade between Russia and African countries has been analyzed, according to which the indicators have grown significantly due to the increase in the volume of Russian exports to African countries. This, especially in the context of solving the import substitution program and developing new markets, is extremely promising, since the African market is large and Russian goods are competitive on it, which opens up new opportunities for increasing trade and cooperation.The main export and import positions have been determined. According to the study, it is necessary to revise export positions on the part of Russia, in particular, the transition from the fuel and energy direction to industrial goods, the provision of services and new technologies. Diversification of African exports is also necessary, by increasing the added value of goods, a transition to the industrialization of production and the use of high technologies.In this regard, the most promising seems to be the strengthening of cooperation between Russia and Africa in the direction of increasing the volume of trade, promoting Russian education, high technologies, services, medicine, increasing investment in the fuel and energy complex of African countries, cooperation in agriculture, fish farming, construction, water purification and other mutually beneficial directions. This will strengthen the positions of Russia and Africa as leading players in the world arena and will serve to strengthen the polycentric world order.

Deep Q-Learning for Trading Cryptocurrency

This article sets forth a framework for deep reinforcement learning as applied to trading cryptocurrencies. Specifically, the authors adopt Q-Learning, which is a model-free reinforcement learning algorithm, to implement a deep neural network to approximate the best possible states and actions to take in the cryptocurrency market. Bitcoin, Ethereum, and Litecoin were selected as representatives to test the model. The Deep Q trading agent generated an average portfolio return of 65.98%, although it showed extreme volatility over the 2,000 runs. Despite the high volatility of deep reinforcement learning, the experiment demonstrates that it has exceptionally high potential to be employed and provides a solid foundation on which to build further research. <b>TOPICS:</b>Currency, big data/machine learning, performance measurement <b>Key Findings</b> ▪ The authors use deep neural networks to create a Deep Q-Learning trading agent that approximates the best actions to take based on rewards to maximize returns from trading the three cryptocurrencies with the largest market capitalization. ▪ The Deep Q-Learning agent generates a return of 65.98% on average over the course of 2,000 episodes; however, the returns do exhibit a large standard deviation given the highly volatile nature of the cryptocurrencies. ▪ The authors introduce a framework on which future deep reinforcement learning and rewards-based trading agents can be built and improved.

Spoofing the Limit Order Book: A Strategic Agent-Based Analysis

We present an agent-based model of manipulating prices in financial markets through spoofing: submitting spurious orders to mislead traders who learn from the order book. Our model captures a complex market environment for a single security, whose common value is given by a dynamic fundamental time series. Agents trade through a limit-order book, based on their private values and noisy observations of the fundamental. We consider background agents following two types of trading strategies: the non-spoofable zero intelligence (ZI) that ignores the order book and the manipulable heuristic belief learning (HBL) that exploits the order book to predict price outcomes. We conduct empirical game-theoretic analysis upon simulated agent payoffs across parametrically different environments and measure the effect of spoofing on market performance in approximate strategic equilibria. We demonstrate that HBL traders can benefit price discovery and social welfare, but their existence in equilibrium renders a market vulnerable to manipulation: simple spoofing strategies can effectively mislead traders, distort prices and reduce total surplus. Based on this model, we propose to mitigate spoofing from two aspects: (1) mechanism design to disincentivize manipulation; and (2) trading strategy variations to improve the robustness of learning from market information. We evaluate the proposed approaches, taking into account potential strategic responses of agents, and characterize the conditions under which these approaches may deter manipulation and benefit market welfare. Our model provides a way to quantify the effect of spoofing on trading behavior and market efficiency, and thus it can help to evaluate the effectiveness of various market designs and trading strategies in mitigating an important form of market manipulation.

Diversity-driven knowledge distillation for financial trading using Deep Reinforcement Learning

Deep Reinforcement Learning (RL) is increasingly used for developing financial trading agents for a wide range of tasks. However, optimizing deep RL agents is notoriously difficult and unstable, especially in noisy financial environments, significantly hindering the performance of trading agents. In this work, we present a novel method that improves the training reliability of DRL trading agents building upon the well-known approach of neural network distillation. In the proposed approach, teacher agents are trained in different subsets of RL environment, thus diversifying the policies they learn. Then student agents are trained using distillation from the trained teachers to guide the training process, allowing for better exploring the solution space, while “mimicking” an existing policy/trading strategy provided by the teacher model. The boost in effectiveness of the proposed method comes from the use of diversified ensembles of teachers trained to perform trading for different currencies. This enables us to transfer the common view regarding the most profitable policy to the student, further improving the training stability in noisy financial environments. In the conducted experiments we find that when applying distillation, constraining the teacher models to be diversified can significantly improve their performance of the final student agents. We demonstrate this by providing an extensive evaluation on various financial trading tasks. Furthermore, we also provide additional experiments in the separate domain of control in games using the Procgen environments in order to demonstrate the generality of the proposed method.

Equilibrium asset pricing with transaction costs

We study risk-sharing economies where heterogeneous agents trade subject to quadratic transaction costs. The corresponding equilibrium asset prices and trading strategies are characterised by a system of nonlinear, fully coupled forward–backward stochastic differential equations. We show that a unique solution exists provided that the agents’ preferences are sufficiently similar. In a benchmark specification with linear state dynamics, the empirically observed illiquidity discounts and liquidity premia correspond to a positive relationship between transaction costs and volatility.

Trading Algorithm Model Based on Technical Indicators

Today the rapid proliferation of the internet provides an environment where efficient e-commerce solutions can be developed. The electronic market is gaining more attention in the global economy, it gives buyers and sellers more liberty to trade cost-effectively and allows access to an adequate amount of data for analysis. New trading agents have been developed for the best utilization of such data. These agents design strategies using financial analysis techniques such as technical indicators. Two very well-known technical indicators used to develop strategies are Convergence-Divergence (MACD) and Stochastic Oscillator (SO). This paper aims to devise a trading algorithm that combines MACD and SO in a single strategy and check the reliability of the combined signals it generates. JTAP simulation system has been used to test the proposed strategy. In this paper, we evaluated the performance of our proposed strategy when implemented on shares of Karachi Stock Exchange, Pakistan which proves improvement of strategy.

Rival principals and shrewd agents: Military assistance and the diffusion of warfare

AbstractMilitary assistance is a perennial feature of international relations. Such programmes typically aim to improve the effectiveness of local partners, exporting the donor's way of war through the provision of training and equipment. By remaking indigenous armies in their own image, donors likewise hope to mitigate the profound agency costs associated with the transfer of military capability. But, while technical and organisational transformations can provide notable battlefield advantages, the philosophies underlying such innovations are not so easily propagated. Instead, new tactics, structures, and technologies typically intersect with pre-existing local schemata of war, producing novel if sometimes dysfunctional hybrid praxes. According to principal-agent theory, the application of greater conditionality in the provision of military assistance should improve the fidelity of military diffusion, aligning agents’ divergent interests with their principals’ goals. In practice, however, principal-agent exchanges rarely exist in isolation. Examining the modernisation of nineteenth-century Japan as a case study in military diffusion, this article argues that competition between rival patrons allows recipient states to play would-be principals off against each other, bypassing conditionality by replicating a marketplace for military assistance. In so doing, however, agents trade functionality for sovereignty in their military diffusion.

Connecting Disconnected Financial Markets?

In most financial markets, securities are traded in isolation. Such a disconnected market design can be inefficient if agents trade more than one security. I assess welfare effects of connecting markets by allowing orders for one security to depend on prices of other securities. I show that everyone trades identical amounts under both market structures if and only if the clearing prices are perfectly correlated or all are price-takers. Prices in disconnected markets might allow strategic traders to extract higher rents from nonstrategic traders. In expectation, connected markets generate higher welfare, but all markets become efficient as they grow large. (JEL D44, D47, G10, H82)

A multi-block ADMM based approach for distribution market clearing with distribution locational marginal price

This work investigates a distribution locational marginal price (DLMP) scheme for promoting the market penetration of small-scale prosumers/consumers connected at distribution level and boosting their potential of demand response. In the distribution market, the distribution market operator (DMO) negotiates with each participant a node-specific DLMP and the respective demand profile of customer, so as to maximize the social welfare while respecting the operational constraints of the distribution system and load devices. First, different customers at demand side are modeled, including prosumers with photovoltaic (PV) generation, distributed storages and electric vehicles. All of them communicate with DMO and realize automated demand response by intelligent trading agent. Then, the distribution market clearing model for a massive number of customers in an ex-ante trade scheme is formulated as a market-based multi-period alternating current optimal power flow (ACOPF). As this clearing model is separable, the augmented Lagrangian relaxation (ALR) method is adopted to solve the market clearing model in a parallel manner. In the ALR, a second-order penalty term at each node is introduced to ensure the convergence, whereas it is not separable for customers. Hence, there are a multi-block demand variable coupled within the second-order penalty term at each node. A standard alternating direction method of multipliers (ADMM) algorithm cannot converge in this situation. To tackle this multi-block problem, we exploit the multi-block ADMM and distributed Dauglas-Rarchford Splitting method (d-DRSM), both of which proximally regularize the customer-specific subproblem to ensure global convergence. Furthermore, the prediction-correction mechanism in d-DRSM algorithm can achieve better rate of convergence in comparison with the multi-block ADMM. Finally, case studies are conducted and numerical results have shown the effectiveness of the proposed methodology, and the performance of different algorithms are analyzed for improving the clearing method’s efficiency.

Liability Limits of Transport Commission Agent

Considering the importance of transport commission agent's work and through the examination of transport commission agent's liability, the intensity and burden of transport commission agent's liability is shown. The legal nature of a transport commission agent's obligation is an obligation to achieve a result, thus he usually has to contract out of liability, either through the legal limits of liability where the law established the legal exemptions that the transport commission agent would benefit from, as well as the agreement limits of liability through the cases in which the law allowed the transport commission agent to establish a condition under the agreement exempts him from liability or specify his damage liability.&#x0D; Since the law prohibited the transport commission agent from establishing any condition that exempted him from liability in the cases of damage and injury, thus he is bound by compensating his agent fully for the occurring damages, except the cases in which the transport commission agent establishes a condition to limit his liability within the limits permitted by the law. &#x0D; Lastly, the development and importance of the transport commission agent's trade and profession and its resulting legal conflicts and problems are the reason behind conducting this study in order to identify the liability limits of the transport commission agent.

Adversarial Attacks Against Reinforcement Learning-Based Portfolio Management Strategy

Many researchers have incorporated deep neural networks (DNNs) with reinforcement learning (RL) in automatic trading systems. However, such methods result in complicated algorithmic trading models with several defects, especially when a DNN model is vulnerable to malicious adversarial samples. Researches have rarely focused on planning for long-term attacks against RL-based trading systems. To neutralize these attacks, researchers must consider generating imperceptible perturbations while simultaneously reducing the number of modified steps. In this research, an adversary is used to attack an RL-based trading agent. First, we propose an extension of the ensemble of the identical independent evaluators (EIIE) method, called enhanced EIIE, in which information on the best bids and asks is incorporated. Enhanced EIIE was demonstrated to produce an authoritative trading agent that yields better portfolio performance relative to that of an EIIE agent. Enhanced EIIE was then applied to the adversarial agent for the agent to learn when and how much to attack (in the form of introducing perturbations).In our experiments, our proposed adversarial attack mechanisms were > 30% more effective at reducing accumulated portfolio value relative to the conventional attack mechanisms of the fast gradient sign method (FSGM) and iterative FSGM, which are currently more commonly researched and adapted to compare and improve.

Designing Strategies for Autonomous Stock Trading Agents using a Random Forest Approach

Machine learning-based autonomous agents are valuable for back-testing stock trading strategies, including algorithmic trading. Several studies in the financial literature have proposed artificial intelligence-based algorithms that support decision making for financial investment, but few studies have provided systematic processes for designing intelligent trading agents. This paper overviews the steps involved in designing agents that forecast stock prices in a trading strategy. These steps include data preprocessing, time-series segmentation, dimensionality reduction, clustering, and others. Our main contributions are: (i) a systematic process that guides the design and development of trading agents, and (ii) a random forest forecasting model.

Machine Learning, Market Manipulation and Collusion on Capital Markets: Why the 'Black Box' matters

This paper offers a novel perspective on the implications of increasingly autonomous and “black box” algorithms, within the ramification of algorithmic trading, for the integrity of capital markets. Artificial intelligence (AI) and particularly its subfield of machine learning (ML) methods have gained immense popularity among the great public and achieved tremendous success in many real-life applications by leading to vast efficiency gains. In the financial trading domain, ML can augment human capabilities in both price prediction, dynamic portfolio optimization, and other financial decision-making tasks. However, thanks to constant progress in the ML technology, the prospect of increasingly capable and autonomous agents to delegate operational tasks and even decision-making is now beyond mere imagination, thus opening up the possibility for approximating (truly) autonomous trading agents anytime soon. Given these spectacular developments, this paper argues that such autonomous algorithmic traders may involve significant risks to market integrity, independent from their human experts, thanks to self-learning capabilities offered by state-of-the-art and innovative ML methods. Using the proprietary trading industry as a case study, we explore emerging threats to the application of established market abuse laws in the event of algorithmic market abuse, by taking an interdisciplinary stance between financial regulation, law & economics, and computational finance. Specifically, our analysis focuses on two emerging market abuse risks by autonomous algorithms: market manipulation and “tacit” collusion. We explore their likelihood to arise on global capital markets and evaluate related social harm as forms of market failures. With these new risks in mind, this paper questions the adequacy of existing regulatory frameworks and enforcement mechanisms, as well as current legal rules on the governance of algorithmic trading, to cope with increasingly autonomous and ubiquitous algorithmic trading systems. It shows how the “black box” nature of specific ML-powered algorithmic trading strategies can subvert existing market abuse laws, which are based upon traditional liability concepts and tests (such as “intent” and “causation”). In concluding, by addressing the shortcomings of the present legal framework, we develop a number of guiding principles to assist legal and policy reform in the spirit of promoting and safeguarding market integrity and safety.

Transactive Energy Management of PV-Based EV Integrated Parking Lots

Electric vehicles (EVs) are a viable alternative for a sustainable mode of transportation. A huge penetration of EVs in future will lead to increasing demand in commercial parking lots equipped with charging systems. Hence, this research area is being pursued meaningfully in recent years. This article proposes a transactive energy management system (EMS) for commercial parking lots equipped with EV charging system and rooftop PV system. Initially, the EMS is optimized with the objective of balancing charging demand with supply. To make the EMS more realistic, factors such as battery degradation cost and photovoltaic levelized cost of energy are considered. At the later stage, the EMS of each parking lot communicates the energy requirement and excess energy to the local trading agent to initiate the proposed transactive energy transaction mechanism. The double-sided auction bidding mechanism is price flexible and based on the valuation of the energy requirement of parking lots. For a system with six parking lots and 25 EVs in each parking lot, the proposed scheme results in cost savings in the range of 2% to 7% for different cases of fixed and variable feed-in tariff. The uncertainty analysis shows that the cost savings vary from a minimum of 2.41% to maximum 12.09% with an average of 6.11%. The case studies demonstrate the potential economic benefit of the proposed scheme.

Deep Reinforcement Learning Agent for S&amp;P 500 Stock Selection

This study investigated the performance of a trading agent based on a convolutional neural network model in portfolio management. The results showed that with real-world data the agent could produce relevant trading results, while the agent’s behavior corresponded to that of a high-risk taker. The data used were wide in comparison with earlier reported research and was based on the full set of the S&amp;P 500 stock data for twenty-one years supplemented with selected financial ratios. The results presented are new in terms of the size of the data set used and with regards to the model used. The results provide direction and offer insight into how deep learning methods may be used in constructing automatic trading systems.

A Simple Mechanism Causing Wealth Concentration.

We study mechanisms leading to wealth condensation. As a natural starting point, our model adopts a neoclassical point of view, i.e., we completely ignore work, production, and productive relations, and focus only on bilateral link between two randomly selected agents. We propose a simple matching process with deterministic trading rules and random selection of trading agents. Furthermore, we also neglect the internal characteristic of traded goods and analyse only the relative wealth changes of each agent. This is often the case in financial markets, where a traded good is money itself in various forms and various maturities. We assume that agents trade according to the rules of utility and decision theories. Agents possess incomplete knowledge about market conditions, but the market is in equilibrium. We show that these relatively frugal assumptions naturally lead to a wealth condensation. Moreover, we discuss the role of wealth redistribution in such a model.

A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning

The adoption of computer-aided stock trading methods is gaining popularity in recent years, mainly because of their ability to process efficiently past information through machine learning to predict future market behavior. Several approaches have been proposed to this task, with the most effective ones using fusion of a pile of classifiers decisions to predict future stock values. However, using prices information in single supervised classifiers has proven to lead to poor results, mainly because market history is not enough to be an indicative of future market behavior. In this paper, we propose to tackle this issue by proposing a multi-layer and multi-ensemble stock trader. Our method starts by pre-processing data with hundreds of deep neural networks. Then, a reward-based classifier acts as a meta-learner to maximize profit and generate stock signals through different iterations. Finally, several metalearner trading decisions are fused in order to get a more robust trading strategy, using several trading agents to take a final decision. We validate the effectiveness of the approach in a real-world trading scenario, by extensively testing it on the Standard & Poor’s 500 future market and the J.P. Morgan and Microsoft stocks. Experimental results show that the proposed method clearly outperforms all the considered baselines (which still performs very well in the analysed period), and even the conventional Buy-and-Hold strategy, which replicates the market behaviour.

A Q-learning agent for automated trading in equity stock markets

Trading strategies play a vital role in Algorithmic trading, a computer program that takes and executes automated trading decisions in the stock market. The conventional wisdom is that the same trading strategy is not profitable for all stocks all the time. The selection of a trading strategy for the stock at a particular time instant is the major research problem in the stock market trading. An optimal dynamic trading strategy generated from the current pattern of the stock price trend can attempt to solve this problem. Reinforcement Learning can find this optimal dynamic trading strategy by interacting with the actual stock market as its environment. The representation of the state of the environment is crucial for performance. We have proposed two different ways to represent the discrete states of the environment. In this work, we trained the trading agent using the Q-learning algorithm of Reinforcement Learning to find optimal dynamic trading strategies. We experimented with the two proposed models on real stock market data from the Indian and American stock markets. The proposed models outperformed the Buy-and-Hold and Decision-Tree based trading strategy in terms of profitability.