I might be missing something basic here, but he is doing no such thing. He is not using or splicing together the out-of-sample numbers. They seem to serve only as confidence builders.
Thanks for explaining this. But I'm still a little confused. If we're re-optimizing our inputs for every year (based on the rolling previous 4 years), how is that not also curve-fitting? I'm first to admit I'm not a math genius or know much about statistics or probabilities or anything. But to me, it seems like as long as you have a large enough out-of-sample period that your strategy continues performing as expected, isn't that just as good?
The idea is to simply take advanatage of the recent occurring statistical edge before it reverts to the mean, and we do it repeatedly until the statistical edge degardes in the out of sample data.
Hey! Thank you for the detailed explanation of this type of analysis! I have a question though. When you have the results of the first 10 4-year optimizations, then I kinda lost you as to what you are supposed to do with the results. You take the average of the result-parameters and apply them to the live/demo account? You do again a walk forward analysis but now for every 2 years beginning from 2012 and optimizing the new data-result from each of the previous optimizations? I didn't quite understand what the next step is.
I always wondered 'what next?' myself ... it seems this isn't so much an optimization as a robustness test. Personally I religiously use strict InSample and OutSample testing to build strats ... however I could see myself using the equity curve of a walkforward to generate a worst-case scenario ... maybe do a monte carlo on that and then adjust my positon size based on 2 SD max drawdown (my particular preference)
Great content. What does it mean if the walk forward (out-sample) performance is better than optimized (in-sample) performance? Similarly, what if each phase seems to improve? I would expect performance to degrade over time. Thank you for this!
@@TheTransparentTrader I don't get it still, I guess the software you're using automatically takes in those inputs for multiple years of different params and finds an optimal set of params to use going forward?
If you run a backtest on all the out-of-sample windows and you vary the strategy from year to year, do you then also liquidate the portfolio at the end of each year? If not, then the strategy from the previous year would affect the following year since you might have stocks in your portfolio that you might not have bought with the new strategy. Either way, thanks for the clear video!
What type of "optimization" are you utilizing? It's helpful to know what you mean exactly. Thanks for your video. It's interesting to see someone do this in excel.
Great subject... one vital point: if out of sample data doest not look good at first DO NOT GO BACK AND MODIFY THE RULES.... from that point on you are on dream land and fooling your self
When running optimization one parameter at a time, doesnt that tend to make your choice of default input for the other parameters quite important? For instance, just as an example, the ideal "fast" lookback period for a MA crossover system would vary quite a bit depending on your choice of "slow" period. So how do you deal with this situation?
Yes this is important. The simplest way to do it is to optimise all parameters at the same time. I like to approach it from a logical point of view by doing them individually. You may well get different results from each method. Although I have found that a strategy which is generally robust should show up positive results for either method.
Been waiting for this one and super excited to see how your wfa proces looks like. I also use a 4:1 opt/wfa but where mines differ is the amount of stages. I start at 2010 and have 6 stages where the last opt 2017-2021. I just started my incubation today actually. I use MT5 and use quant analyzer to stich together my wfa results to see what my systems expectations are when going live. I am going to try your opt/wfa process on my current system and hopefully add more confidence in it. Thanks PS How about a video on portfolio diversification? 🤔
Currently I haven't really explored it very much. I'm sure it is very useful and probably saves time. I wanted to show doing it manually in the video though.
![060: Strategy Optimization with Robert Pardo [AUDIO ONLY]](/img/1.gif)
Actually one of the best vids I have seen on the subject as you show the splicing together of the out of sample outputs!
I might be missing something basic here, but he is doing no such thing. He is not using or splicing together the out-of-sample numbers. They seem to serve only as confidence builders.
Wow this is great! Informative as always!
Thanks for explaining this. But I'm still a little confused.
If we're re-optimizing our inputs for every year (based on the rolling previous 4 years), how is that not also curve-fitting? I'm first to admit I'm not a math genius or know much about statistics or probabilities or anything. But to me, it seems like as long as you have a large enough out-of-sample period that your strategy continues performing as expected, isn't that just as good?
The idea is to simply take advanatage of the recent occurring statistical edge before it reverts to the mean, and we do it repeatedly until the statistical edge degardes in the out of sample data.
Thanks for the video. If i understand it correctly; every end of year you will re optimize and use the new inputs for the next year?
Hey! Thank you for the detailed explanation of this type of analysis! I have a question though. When you have the results of the first 10 4-year optimizations, then I kinda lost you as to what you are supposed to do with the results. You take the average of the result-parameters and apply them to the live/demo account? You do again a walk forward analysis but now for every 2 years beginning from 2012 and optimizing the new data-result from each of the previous optimizations? I didn't quite understand what the next step is.
I always wondered 'what next?' myself ... it seems this isn't so much an optimization as a robustness test. Personally I religiously use strict InSample and OutSample testing to build strats ... however I could see myself using the equity curve of a walkforward to generate a worst-case scenario ... maybe do a monte carlo on that and then adjust my positon size based on 2 SD max drawdown (my particular preference)
Great content. What does it mean if the walk forward (out-sample) performance is better than optimized (in-sample) performance? Similarly, what if each phase seems to improve? I would expect performance to degrade over time. Thank you for this!
What settings do you end up using going forward on out of sample data? the very last leg of the optimization on the walk forward?
great video.
my questions too, thanks Jarrod!
You can see in the spreadsheet the settings for 2021. In January I will re optimise using 2018, 2019, 2020 and 2021 to get the inputs for 2022.
@@TheTransparentTrader thanks!
@@TheTransparentTrader I don't get it still, I guess the software you're using automatically takes in those inputs for multiple years of different params and finds an optimal set of params to use going forward?
what amount of time is good to be able to analyze a strategy ?, 1, 3, 5 or 10 years or do it progressively thanks Jarod
If you run a backtest on all the out-of-sample windows and you vary the strategy from year to year, do you then also liquidate the portfolio at the end of each year? If not, then the strategy from the previous year would affect the following year since you might have stocks in your portfolio that you might not have bought with the new strategy. Either way, thanks for the clear video!
Muilticharts has this built in. I don't get why you would do it this way?
Just a more manual way of doing it. It helps understand the process.
What type of "optimization" are you utilizing? It's helpful to know what you mean exactly. Thanks for your video. It's interesting to see someone do this in excel.
Great keep going
Great subject... one vital point: if out of sample data doest not look good at first DO NOT GO BACK AND MODIFY THE RULES.... from that point on you are on dream land and fooling your self
That's right!
When running optimization one parameter at a time, doesnt that tend to make your choice of default input for the other parameters quite important? For instance, just as an example, the ideal "fast" lookback period for a MA crossover system would vary quite a bit depending on your choice of "slow" period. So how do you deal with this situation?
Yes this is important. The simplest way to do it is to optimise all parameters at the same time. I like to approach it from a logical point of view by doing them individually. You may well get different results from each method. Although I have found that a strategy which is generally robust should show up positive results for either method.
Hi Jared, great video.
What's the minimum number of trades you try to accomodate within each in-sample period?
I tend to mostly just use 4 years for in sample. At minimum this would be about 50 trades. Most strategies trade more though.
Been waiting for this one and super excited to see how your wfa proces looks like. I also use a 4:1 opt/wfa but where mines differ is the amount of stages. I start at 2010 and have 6 stages where the last opt 2017-2021. I just started my incubation today actually. I use MT5 and use quant analyzer to stich together my wfa results to see what my systems expectations are when going live.
I am going to try your opt/wfa process on my current system and hopefully add more confidence in it.
Thanks
PS
How about a video on portfolio diversification? 🤔
Thanks for sharing, and the video idea.
Anchored won't keep up with market behavior changes as well.
I see that you got a WFT on you MultiCharts, is that no good?
Currently I haven't really explored it very much. I'm sure it is very useful and probably saves time. I wanted to show doing it manually in the video though.
Dear sir what about fxcm broker I want open an account with em for day trading or scalping indices and forex?
Thank you
too briton gayish
Has anyone explored plutohq? They support eft’s bot trades also. Not difficult for anyone getting started. Also offer paper algotrading games.