Walk Forward Optimization in Python with Backtesting.py
HTML-код
- Опубликовано: 29 июн 2024
- We use backtesting.py to do walk-forward Analysis / Optimization in Python. This allows us to test our trading strategies more rigorously to ensure that we're not over-fitting.
⭐ Code:
github.com/ChadThackray/WFO-b...
⏰ Timestamps:
00:00 - Introduction
03:47 - Data Sourcing / importing
05:50 - Writing our Momentum strategy
20:10 - Walk forward Function
36:00 - Plotting our out of sample backtests
44:55 - Plotting combined equity curve
56:57 - Plotting sample and validation data ranges
Amazing content and with this high quality content, wish you the very best and soon million subscribers!
Loved the video, exactly what I needed. Would love to get access to your notebooks too.
I use an ML model that splits the data into training and validation sets within the strategy itself. So I just got rid of the validation backtest within the walk_forward function, and it worked fine.
Very nice code
Thanks
I've been binge watching your stuff for a few days now. This is a great video with a solid idea. My only problem is that my strategy allows for multiple trades to be opened, but as soon as the week ends, it closes all my open trades, and this causes my performance to tank a bit. If I run it over full data set I get very good results, but slicing it like this really kills my return %. Not sure if something can be done about it though...
Generally if the strategy does significantly worse in walk forward, you've probably over-fit.
Would recommend doing some forward testing / paper trading to get a better idea of what's going on. Especially if you're trading on shorter timeframes. Importantly you should not make any changes to the strategy whilst forward testing, or you'll risk more over-fitting.
I've been fixing my code and strategy. Currently optimising for two month, walking forward for a week. Also updated your code to get current equity from the walk forward week for the next week. In your experience, if walk forward does well, how does it compare with real life?
Hello Chad. Thanks for your videos, they have served very well for my formation as an algorithmic trader. I have a question, if I am not calculating indicators for other timeframes, do I still need the "warmup_bars"?
Depends on the indicators you use. Most require some kind of warmup time
Thank you so much for the information, it's very useful. Could you use the split data from sklearn?
I mean with cross-validation
Thanks for your work. Is any way get the code you show in the video?
cool things
I'm curious of your view regarding what would be a very good backtesting methodology. E.g. kfold validation, walk-forward optimization (anchored or not)... there are multiple ways to look at it and you've mentioned on your website that you're into the hard stuff... I'm curios of your view on what a default / go-to method would do.
For context: I started out splitting 60-20-20 and quickly realized I couldn't do that because I would, for example, train on a sideways market, validate on an upside and test on an upside. Then used kfold and split the timerange into 5 folds, each with its own training and testing set. That worked well enough but on higher timeframes I couldn't get enough trades to validate the results. This is my default framework but I sometimes use a walk-forward that is anchored... but I wanted to know how more experienced quants do it. If the 5 fold approach is better or worse than an unanchored walk forward or than an anchored one or if there is another way to look at this that is escaping me.
Hey I'm also trying to understand this topic. Can I please connect with you and possibly recieve some guidance?
Are going to going to make it available somewhere?
Hi, is there a way to download the .py you're explaining? Thanks
Nice content, Chad. I am new for back testing , I have no idea how to rewrite the walk forward if using different time frame with 2 moving average crossover strategy with optimisation. Thanks.
Perhaps try some of my other videos or general python tutorials until you're ready
@@ChadThackray Now I am using the max time frame of long ma for warmup_bars
I have a question, in the walk forward function, why bt.optimize not bt_training.optimize!
Any chance we can add a Monte Carlo and MAE/MFE analysis please?
+1 on the montecarlo
I don't understand how to choose parameters after such optimizations. For example we have bunch of results which is obtained with different strategy parameters. What would we choose for further trading in real market? Which parameters are best?
You walk forward to the current time so that the training data ends at the current date and then use the strategy parameters from that last test in your live strategy for as long as your validation data window was
this is the way
I think the most repetitive parameters through all the validation processes with good results should be selected for the live trading
Your view platform not yet clear for the viewers to see clear.....adjusted d zoom up.
how do we download your notebook?
Hey Chad. I can't install backtesting on terminal. What do I do? I'm using Jupyter Notebook VSC (Visual Studio Code).
Just install it however you normally install python packages
It seems that backtesting file contains a virus. It was blocked by antivirus. @@ChadThackray
source code please
Hi. My Jupiter says "No module named backtesting"
Run "pip install backtesting" inside or outside the jupyter notebook
Hey Chad. Can you share your code?
How can used metatrader data here,l?😁
I've never used metatrader but as long as you've got it in a pandas dataframe it'll work fine
@@ChadThackray Ok thx!
I had a problem with the optimized parameters for the backtests on the validation sets of data all being the same when printing them from the _strategy in the stats results.
When you tested it in your video you only checked a single _strategy result to see if it was different from the defaults, but not if they were different from each other so I don't know if you had the same problem, I think I copied your code exactly in a new jupiter notebook and still had the same problem, anyone else have this problem?
edit:
I fixed it like this but I don't know what the original problem was.
def walk_forward(strategy, data_full, warmup_bars, lookback_bars, validation_bars, cash=10_000_000, commission=0.002):
stats_master = []
params_master = []
for i in range(lookback_bars + warmup_bars, len(data_full) - validation_bars, validation_bars):
training_data = data_full.iloc[i - lookback_bars - warmup_bars:i]
validation_data = data_full.iloc[i - warmup_bars:i + validation_bars]
bt_training = Backtest(training_data, strategy, cash=cash, commission=commission)
stats_training = bt_training.optimize(
small_threshold=list(np.arange(0, 1, 0.1)),
large_threshold=list(np.arange(1, 3, 0.2)),
maximize="Equity Final [$]"
)
optimized_strategy = stats_training._strategy
strategy.small_threshold = optimized_strategy.small_threshold
strategy.large_threshold = optimized_strategy.large_threshold
bt_validation = Backtest(validation_data, strategy, cash=cash, commission=commission)
stats_validation = bt_validation.run()
stats_master.append(stats_validation)
params_master.append({
"small_threshold": optimized_strategy.small_threshold,
"large_threshold": optimized_strategy.large_threshold
})
return stats_master, params_master
lookback_bars = 28*400
validation_bars = 7*400
warmup_bars = 16*24
stats, params = walk_forward(
MomentumStrategy,
data,
lookback_bars=lookback_bars,
validation_bars=validation_bars,
warmup_bars=warmup_bars
)
stats
params[0], params[1], params[2], params[3], params[4], params[5], params[6], params[7], params[8], params[9], params[10]
i've same problem :( i couldnt find any solution
yeah I had the same problem and tried different indicators, sometimes it would change just 1 value but I never figured it out