Reinforcement Learning for Financial Trading
Reinforcement Learning For Financial Trading ?
How to use Reinforcement learning for financial trading using Simulated Stock Data using MATLAB.
Setup
To run:
Open RL_trading_demo.prj
Open workflow.mlx
Run workflow.mlx
Environment and Reward can be found in: myStepFunction.m
Overview:
The goal of the Reinforcement Learning agent is simple. Learn how to trade the financial markets without ever losing money.
Note, this is different from learn how to trade the market and make the most money possible.
The aim of this example was to show:
1. What reinforcement learning is
2. How it can be applied to trading the financial markets
3. Leave a starting point for financial professionals to use and enhance using their own domain expertise.
The example use an environment consisting of 3 stocks, $20000 cash & 15 years of historical data.
Stocks are:
Simulated via Geometric Brownian Motion or
Historical Market data (source: AlphaVantage: www.alphavantage.co)
Copyright 2020 The MathWorks, Inc.
Cite As
David Willingham (2021). Reinforcement Learning for Financial Trading (https://github.com/matlab-deep-learning/reinforcement_learning_financial_trading), GitHub. Retrieved .
Comments and Ratings (15)
MATLAB Release Compatibility
Platform Compatibility
Windows macOS LinuxCategories
Tags
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!Discover Live Editor
Create scripts with code, output, and formatted text in a single executable document.
data
docs
Select a Web Site
Choose a web site to get translated content where available and see local events and offers. Based on your location, we recommend that you select: .
Select web siteYou can also select a web site from the following list:
Americas
- América Latina (Español)
- Canada (English)
- United States (English)
Europe
- Belgium (English)
- Denmark (English)
- Deutschland (Deutsch)
- España (Español)
- Finland (English)
- France (Français)
- Ireland (English)
- Italia (Italiano)
- Luxembourg (English)
- Netherlands (English)
- Norway (English)
- Österreich (Deutsch)
- Portugal (English)
- Sweden (English)
- Switzerland
- United Kingdom (English)
Is it possible to use 'Reinforcement Learning for Financial Trading' on a cpu? I am a home user and do not have access to a gpu. It showed the training time as 80 hrs on a gpu. Can you suggest some alternative ways to use the above with a cpu?
Hi Hector, I'm not sure Simulink makes sense for defining a Financial Trading environment. What questions to do you have about the multi-agent examples? Feel free to email me them to me and I'll pass it onto the RL developers. My email is dwilling@mathworks.com
Hello David, Matlab2020b has included multiple-agents capabilities but for those like me that are not familiar with simulink becomes a bit difficult to understand the logic behind the examples provided in the new version. Is it any possibility that you kindly implement your example in simulink? single agent would be enough to follow your template. Thank you.
Thanks Hector! It was a lot of fun developing the example, great to hear you have found it useful.
Hello David, good job! This demo is what separate matlab from other softwares. It is helpful, documented and easy to start a project from a novice in RL for finance. I am looking forward to see the new version when you include -Transaction costs , Refined reward system and Compare different agents. Good job! and thank you
Thanks for the great tutorial! As the tutorial covers 15 years of historical data, one nuance for trading strategy development is the difference between RAW price data and ADJUSTED price data based on corporate actions such as splits and dividends [1]. Both paradigms are used in the quant investing arena, albeit under different use cases. There are also other common attributes a tech-savvy investor may look out for when choosing a stock market data source [2]. Assuming that the readers of this tutorial may use the reinforcement learning recipe as a "launchpad" for larger-scale trading strategy development, we just wanted to mention a couple interesting nuances on the data aspects.
Thanks again for the great content, David!
References:
1. https://www.investopedia.com/terms/a/adjusted_closing_price.asp
2. https://medium.com/@patrick.collins_58673/stock-api-landscape-5c6e054ee631
Hi, i found a bug in myStepFunction where Ind4 is computed, i.e.
Ind4 = (stock_price{1,:} ./ mean(lastweek(1,:))) - 1; % change from average of 7 days
You are not computing the mean price over the previous 7 days.
mean(lastweek(1,:)) is equal to lastweek(1,:) so Ind4 == Ind1.
It should be mean(lastweek) since you are looking for a Mean of Matrix Columns.
Could you chek it, please?
Thanks.
Great Work!
Code updated to work with R2020a
Hi Giovanni, I just added a PDF overview of the example, this may help in understanding the logic behind the code.
In the GitHub repo, the readme explains a lot about the example:
https://github.com/matlab-deep-learning/reinforcement_learning_financial_trading
Thank you for the great submission! More and deeper comments throughout the code would have been very helpful to better understand not only the workflow, but the various functions too, particularly the critical Step and Reset functions. BTW, it seems that when using fetchStockData, no matter the stocks chosen, the results are always those from the simulated data. The current value in the "Profit on Test Data" figure does change, but the stock prices do not change upon changing the tickers. But it may be just me not following the right flow.