Cloning strategies from trading records using agent-based reinforcement learning algorithm

Chiao Ting Chen, An-Pin Chen, Szu-Hao Huang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

5 Scopus citations

Abstract

Investment decision making is considered as a series of complicated processes, which are difficult to be analyzed and imitated. Given large amounts of trading records with rich expert knowledge in financial domain, extracting its original decision logics and cloning the trading strategies are also quite challenging. In this paper, an agent-based reinforcement learning (RL) system is proposed to mimic professional trading strategies. The concept of continuous Markov decision process (MDP) in RL is similar to the trading decision making in financial time series data. With the specific-designed RL components, including states, actions, and rewards for financial applications, policy gradient method can successfully imitate the expert's strategies. In order to improve the convergence of RL agent in such highly dynamic environment, a pre-Trained model based on supervised learning is transferred to the deep policy networks. The experimental results show that the proposed system can reproduce around eighty percent trading decisions both in training and testing stages. With the discussion of the tradeoff between explorations and model updating, this paper tried to fine-Tuning the system parameters to get reasonable results. Finally, an advanced strategy is proposed to dynamically adjust the number of explorations in each episode to achieve better results.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE International Conference on Agents, ICA 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages34-37
Number of pages4
ISBN (Print)9781538681800
DOIs
StatePublished - 10 Sep 2018
Event2018 IEEE International Conference on Agents, ICA 2018 - Singapore, Singapore
Duration: 28 Jul 201831 Jul 2018

Publication series

NameProceedings - 2018 IEEE International Conference on Agents, ICA 2018

Conference

Conference2018 IEEE International Conference on Agents, ICA 2018
CountrySingapore
CitySingapore
Period28/07/1831/07/18

Keywords

  • Policy gradient financial trading
  • Reinforcement learning
  • Strategy cloning
  • Transfer learning

Fingerprint Dive into the research topics of 'Cloning strategies from trading records using agent-based reinforcement learning algorithm'. Together they form a unique fingerprint.

  • Cite this

    Chen, C. T., Chen, A-P., & Huang, S-H. (2018). Cloning strategies from trading records using agent-based reinforcement learning algorithm. In Proceedings - 2018 IEEE International Conference on Agents, ICA 2018 (pp. 34-37). [8460078] (Proceedings - 2018 IEEE International Conference on Agents, ICA 2018). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/AGENTS.2018.8460078