The graph looks only so impressive because there are multiple models which only forecast one single value. The real deal is doing a relatively long forecast based on an ARIMA model. Twenty (or so) ARIMA Models each fitted on a subset of real data and forecasting only one step are exactly as useful as taking the last value and adding a random number to it.
Really helpful tutorial. One question, when you try to predict future values that are not exist yet, what do you use at the part of actual_test_value = testing_set[i] training_set.append(actual_test_value)" because for the future we do not have "actual values". I used the model_predictions[i] as appending to the training_set (which actually the training_set is the actual whole data) and I get a straight line as a result at my graph which is not that good. any solutions? thank you in advance.
You might be getting a straight line because the model is predicting the price based on the same input features. Maybe for e.g. "you added input feature /actual value '200' to the data, the model will predict the price e.g. '250', now for next future day the input appended is same i.e. actual value '200' and hence model predicts again '250'. With this logic the model keeps on predicting '250' and you get a straight line as output. I believe you need to check the input features you're appending to the data set.
When running for loop, you add test data to training data and you run ARIMA model then the result absolutely great. However, it just like tell the computer learn by heart the data and tell it show the test data again
Hey Ritvik, you haven't checked the stationarity and seasonality of the data or it is not necessary for price prediction? You have done prediction for test data. How do predictions for unseen data or future data? Do we have to run for loop for combined trained and test data for making predictions of unseen data?
Yes, Very good point. I didn't mention it as it might make the video more lengthy and confusing for beginners. But Yes, we will do all the relevant tests before performing the model in a real world analysis.
Thanks once again for yet another captivating financial programming video. This is a video every crypto trader should watch to equip himself for financial gains.
Ritvik, I wish you all the best for you and your family in the coming year and express my gratitude for your hard work and highly educational videos. Thank you from Rainy California, and best wishes!
Sir thanks for the explanation, wanna ask one thing, can you plot the prediction of the stock price of the next 30 days? I mean not prediction in the test set, i want the prediction of the stock of the new days (next 30 days), is that possible to do? Thank you please answer it sir....
with this arima model can we implement it into other asset classes? Of course, changing the Arima model or would this code automatically choose the best-fit Arima model for stock X?
Yes, it can be used for any tradable securities, however, it is just a basic model and might have some limitations. Please do thorough research before putting real money on it.
@@FinancialProgrammingwithRitvik thanks for the reply much appreciated. I am a new follower of your RUclips account and I have seen many channels and I can say that you have some high quality content. I am a trader and currently doing my masters in finance from RMIT university Australia. But just learnt basic coding at uni with R studios. I can see you have a python course so if I take that course would I be able to make a model myself like the one you made like how advance is it what I mean to ask ? Thanks Mohammad khan
@@mohammadkhan5387 Thanks for your interest in my work. Yes, in that course you will learn to make the models on your own. In fact, sessions 4 and 5 are just about exploring some projects and how to optimize/automate them using Python. I guarantee you won't regret your decision to enrol for the course: fpritvik.com/python
Great video, I was happy to subscribe! I did find that when running the statsmodel package suggests to change the import to: "from statsmodels.tsa.arima.model import ARIMA" (so arima.model instead of arima_model) but this causes some errors with the line 'yhat = list(output[0])[0]' when training. Do you mind having a look?
Yes, pricing data is reliable as per my past experience, however, if you want to explore more financial data providers API, then I have made an entire playlist on it. Just check it out and you will see how much data is available in the market for free.
Amazing thank you so much! I notice that stats model has recently changed ARIMA, does it mean any code changes are required? I also couldn't work out how to amend the code to handle an MA value of > 0. I get error messages about convergence ...
@@FinancialProgrammingwithRitvik I got it working ! But then began to wonder how useful my “creation” was in the real world … I.e I couldn’t forecast out multiple time intervals into the unknown as I wouldn’t be able to re-informing the training dataset each time :(
Yes, Very good point. I didn't mention it as it might make the video more lengthy and confusing for beginners. But Yes, we will do all the relevant tests before performing the model in a real world analysis.
Is there any way to use this forecasting to analyze huge data and predict 5 or 15 min movements in a certain day? and if so, how can I do it, just to learn and compare with real market
Thanks a lot for really helpful tutorial! I have a doubt as why are you training ARIMA model in a for loop for every iteration of test data. So if you are building a rolling model, it should be rather done every time new data point arrives, we can do train test split and train ARIMA model once instead of doing it for every iteration of test data.
Hi,Nice video.I would like to know more theoretical aspects of ARIMA.Please do a video if possible. Also can you do video on how to do CNN based prediction?
Yes, we need to work on all biases before finalizing the code for real world applications such as Multicollinearity, Stationarity, heteroscedasticity, etc.
I have some doubts 1) Dont we need to check if our data is stationary or not? 2) And after if its not stationary we have to take differencing right 3) And if at all it comes stationary we need to feed that value in plot acf and plot pacf to see our MA and AR values i.e p and q values 4) Then we can come to conclusion of our p and q numbers right and then feed them to ARIMA model?? So how did you land those 4,0,1 numbers, they cant be random Please correct/guide me if Im missing something I mean im confused is this video centered for bitcoin users or those guys who wants to know how that code is working?
No, you are correct. We would need to follow all the steps exactly as you mentioned in the real world analysis. I made the video just to show how we can use ARIMA model. Yes 4,0,1 should not be random. I appreciate your detailed comment :)
@@FinancialProgrammingwithRitvik thank you for clearing... I mean you can also make a detailed video regarding the points I mentioned, people will appreciate it, and also its nice to see different perspectives over a problem.. Have a great day
it is good to check if data is stationary but with time series of financial instruments, the data is almost always non-stationary, especially as we expand the time period.
Thanks so much sir, this is straight forward Just a question please, what if I’m trying to pass multiple crypto asset instead of just bitcoin. Say 10 different assets how do i call it using yfinance?
thank you very much ritvik for your content that you bring to the channel and above all for the niche based on the markets and also for the clear and detailed explanations in the videos
It's difficult to understand the query here. I recommend you to get the google drive access and copy and paste my code and check line by line where is the mistake.
Yes, Very good point. I didn't mention it as it might make the video more lengthy and confusing for beginners. But Yes, we will do all the relevant tests before performing the model in a real world analysis.
I can't understand the part where you have append the training data in the function with actual_test_value and how I can predict future price i.e. price after 1 week with this approach?
Hello sir. Thank you for your video. Excellent job. Could you please provide me the code to forecast future one year values. Please help me for my thesis
hlo Sir goodevening I have been working on Nifty bank prediction using ARIMA in python since jan '23 . being from Commerce background everything seems a little bit difficult but i really want to do this . Now I am stuck at a point where I am getting straight line predictions . I don't know what to do . I have tried everything but all in vain. Please suggest me what to do next
The prices till 2022 are already known, How can i use 100% of the data as training data and predict the future values??? Here we are prediciting already known values. How to predicit future values?
The graph looks only so impressive because there are multiple models which only forecast one single value. The real deal is doing a relatively long forecast based on an ARIMA model. Twenty (or so) ARIMA Models each fitted on a subset of real data and forecasting only one step are exactly as useful as taking the last value and adding a random number to it.
I am happy there are people sharing passion with such preparation, congratulations on your great community support!
Thanks :)
Really helpful tutorial. One question, when you try to predict future values that are not exist yet, what do you use at the part of
actual_test_value = testing_set[i]
training_set.append(actual_test_value)"
because for the future we do not have "actual values". I used the model_predictions[i] as appending to the training_set (which actually the training_set is the actual whole data) and I get a straight line as a result at my graph which is not that good. any solutions? thank you in advance.
Same here … 😢
same here...
You might be getting a straight line because the model is predicting the price based on the same input features. Maybe for e.g. "you added input feature /actual value '200' to the data, the model will predict the price e.g. '250', now for next future day the input appended is same i.e. actual value '200' and hence model predicts again '250'. With this logic the model keeps on predicting '250' and you get a straight line as output. I believe you need to check the input features you're appending to the data set.
bro he is misguiding that's why he didn't replied to this comment.
When running for loop, you add test data to training data and you run ARIMA model then the result absolutely great. However, it just like tell the computer learn by heart the data and tell it show the test data again
Thanks for the valuable feedback
Very simple explanation to understand the application of ARIMA model. You da best.
Thanks :)
Hey Ritvik, you haven't checked the stationarity and seasonality of the data or it is not necessary for price prediction? You have done prediction for test data. How do predictions for unseen data or future data? Do we have to run for loop for combined trained and test data for making predictions of unseen data?
Yes, Very good point. I didn't mention it as it might make the video more lengthy and confusing for beginners. But Yes, we will do all the relevant tests before performing the model in a real world analysis.
Very helpful video! Thank you for posting.
You're very welcome!
Pretty good video! Thanks for sharing :)
Thanks for watching!
Thanks once again for yet another captivating financial programming video. This is a video every crypto trader should watch to equip himself for financial gains.
Much appreciated!
Thank you Ritvik, easy to understand and very useful!!!
Glad it was helpful!
Subscribed! Thanks for the video :)
Thanks for the sub!
Ritvik, I wish you all the best for you and your family in the coming year and express my gratitude for your hard work and highly educational videos. Thank you from Rainy California, and best wishes!
Thanks a ton
Great vid! very helpful
Thanks :)
thank you for this great content. i am studying for CMT and this is very helpful, thanks!
My pleasure :)
Nice work my friend!
Thank you! Cheers!
Great, Thank you!
My pleasure :)
Great Channel. This is the channel I was looking for.
I am glad to know that :)
Great video, thanks Ritvik
It's my pleasure :)
Ritvik, Great Work! :))) I had ARIMA during my BSc but obviously we never saw a computer so now i cna catch up.
I am glad :)
Great Video, keep up the good work!
Thanks, will do!
Excellent topic learned alot.
I am glad.
Great work buddy
Thank you so much for your appreciation :)
Great work!
Thank you! Cheers!
Loved the way you explained. It was really Helpful
Thanks :)
thanks! learn so much
I am glad to know that :)
Sir thanks for the explanation, wanna ask one thing, can you plot the prediction of the stock price of the next 30 days? I mean not prediction in the test set, i want the prediction of the stock of the new days (next 30 days), is that possible to do? Thank you please answer it sir....
Will try
This is great! Thanks for your knowledge share! ;)
My pleasure.
Thank you Ritvik, i liked the video a lot and found it usefull!
Thanks for the appreciation :)
Nicely explained
Thank you so much for your appreciation :)
Excellent topic I wish there was a similar video with Oil prices prediction using stock RSI or CCI indicators! Best of luck for this year
Noted! Thanks for the suggestion.
great content ! found your yt channel very useful
Glad you think so!
Which model is the best for time series predication? Is it LSTM, ARIMA or Prophet?
It depends on the instrument we are predicting. Cryptos like Bitcoin will have a complete different answer than Bonds.
Great video , thanks for sharing knowledge
My pleasure
good explanation, thanks! waiting from one month... and video is delivered today keep it up ritvit
Glad you liked it! Pardon for your wait. I will be more consistent from now on :)
with this arima model can we implement it into other asset classes? Of course, changing the Arima model or would this code automatically choose the best-fit Arima model for stock X?
Yes, it can be used for any tradable securities, however, it is just a basic model and might have some limitations. Please do thorough research before putting real money on it.
@@FinancialProgrammingwithRitvik thanks for the reply much appreciated. I am a new follower of your RUclips account and I have seen many channels and I can say that you have some high quality content. I am a trader and currently doing my masters in finance from RMIT university Australia. But just learnt basic coding at uni with R studios. I can see you have a python course so if I take that course would I be able to make a model myself like the one you made like how advance is it what I mean to ask ? Thanks Mohammad khan
@@mohammadkhan5387 Thanks for your interest in my work. Yes, in that course you will learn to make the models on your own. In fact, sessions 4 and 5 are just about exploring some projects and how to optimize/automate them using Python. I guarantee you won't regret your decision to enrol for the course: fpritvik.com/python
You are doing such an amazing job. Waiting for your next videos eagerly! Thanks a lot.
Thank you so much 😀
Merry Christmas ❤️buddy
Same to you :)
Great video, I was happy to subscribe! I did find that when running the statsmodel package suggests to change the import to: "from statsmodels.tsa.arima.model import ARIMA" (so arima.model instead of arima_model) but this causes some errors with the line 'yhat = list(output[0])[0]' when training. Do you mind having a look?
I will check it out and thanks for the sub
@@FinancialProgrammingwithRitvik Hello sir I have the same problem is there any solution for that ? Thank you for answear and excellent video :)
is the data from yfinance reliable ? and if there are any other libraries like yfinance please do suggest.
Yes, pricing data is reliable as per my past experience, however, if you want to explore more financial data providers API, then I have made an entire playlist on it. Just check it out and you will see how much data is available in the market for free.
All the codes are soo easy to understand and execute
Thanks for the appreciation :)
Yes we want to see vedio on the ARIMA 👍
Noted!
Cant wait to see the effect after params become optimized!
Yes, I am excited too :D
Good work Ritvik. your videos and codes are really helpful
Thanks :)
If I iunderstand correctly the code makes a prediction for time already passed?? How can it make prediction for the future? Thanks, sir
We need to incorporate more things in the model to make it for the real world
@@FinancialProgrammingwithRitvik I don't remember, does it predict one day into the future or not yet?
GOOd!
haha, thanks :)
What is the use of lagged future prediction?
I understand. It's just for the educational purpose.
Fantastic job, i hope in some new contents soon!!!!
Very soon!
Amazing thank you so much! I notice that stats model has recently changed ARIMA, does it mean any code changes are required? I also couldn't work out how to amend the code to handle an MA value of > 0. I get error messages about convergence ...
I would need to check it.
@@FinancialProgrammingwithRitvik I got it working ! But then began to wonder how useful my “creation” was in the real world … I.e I couldn’t forecast out multiple time intervals into the unknown as I wouldn’t be able to re-informing the training dataset each time :(
Your video is amazing
Thanks :)
how do you increase the length of the model prediction so you are able to predict the price for the next day?
We would need daily data and then run the model.
@@FinancialProgrammingwithRitvik Can you tell me how I can extend this for next 10 days
Thanks for sharing !
My pleasure!
I would like to ask you sir how did your model account for the seasonality ?
Yes, Very good point. I didn't mention it as it might make the video more lengthy and confusing for beginners. But Yes, we will do all the relevant tests before performing the model in a real world analysis.
seasonality in a time series = non-stationary data .... i think that is why he used differencing
Very interesting and complete explanation
Thanks for the appreciation :)
Thanks for all your videos. They are very helpful. Keep up the great work, Ritvik!
Sure... My pleasure
Thanks for your video. Is there any way to get your code from your videos?
You can get a free access of my google drive. Over there, this code is save along with many other codes + study material.
Thanks!
Welcome!
love your coding style
Thank you so much for your appreciation :)
Keep up the great work. Thanks for the video.
Thank you so much for your appreciation :)
Is there any way to use this forecasting to analyze huge data and predict 5 or 15 min movements in a certain day? and if so, how can I do it, just to learn and compare with real market
Prediction is possible, yet we would need to work on reliability on it. We can use the same model or any AI based models for the prediction.
how can i increase the lenght o the model prediction? like a prediction of a week instead of a day
Okay I am confused with one thing. We split data into train and test.
Shouldn't we test the trained 90% on the final 10% not from the start?
Thank you for your content!
My pleasure.
Genius
Thanks :)
This is a good approach and I really enjoyed following it, however I am curious how would you account for non-stationary data in the forecast?
Good question! I will make more videos covering nonstationarities.
@@FinancialProgrammingwithRitvik i think you would convert that data into stationary then?
Good prediction analysis!
Thank you so much for your appreciation :)
Thanks a lot for really helpful tutorial! I have a doubt as why are you training ARIMA model in a for loop for every iteration of test data. So if you are building a rolling model, it should be rather done every time new data point arrives, we can do train test split and train ARIMA model once instead of doing it for every iteration of test data.
I agree
Hi,Nice video.I would like to know more theoretical aspects of ARIMA.Please do a video if possible. Also can you do video on how to do CNN based prediction?
Noted.
Great video
Thanks :)
I am getting the predictions as a straight line no matter what values I take for p, d, q. Is it because of the stationarity of the data?
Yes, we need to work on all biases before finalizing the code for real world applications such as Multicollinearity, Stationarity, heteroscedasticity, etc.
Thank you for providing all this useful content !
My pleasure.
I have some doubts
1) Dont we need to check if our data is stationary or not?
2) And after if its not stationary we have to take differencing right
3) And if at all it comes stationary we need to feed that value in plot acf and plot pacf to see our MA and AR values i.e p and q values
4) Then we can come to conclusion of our p and q numbers right and then feed them to ARIMA model??
So how did you land those 4,0,1 numbers, they cant be random
Please correct/guide me if Im missing something
I mean im confused is this video centered for bitcoin users or those guys who wants to know how that code is working?
No, you are correct. We would need to follow all the steps exactly as you mentioned in the real world analysis. I made the video just to show how we can use ARIMA model. Yes 4,0,1 should not be random. I appreciate your detailed comment :)
@@FinancialProgrammingwithRitvik thank you for clearing...
I mean you can also make a detailed video regarding the points I mentioned, people will appreciate it, and also its nice to see different perspectives over a problem..
Have a great day
The same thought came to my mind why ain't he making the series stationary 😅
it is good to check if data is stationary but with time series of financial instruments, the data is almost always non-stationary, especially as we expand the time period.
hello can u make a option trading simulator for Indian stock.. (Nifty50)
Will try soon.
Thanks so much sir, this is straight forward
Just a question please, what if I’m trying to pass multiple crypto asset instead of just bitcoin. Say 10 different assets how do i call it using yfinance?
We need to write a for loop for that and store the data when it comes.
@@FinancialProgrammingwithRitvik example?
Very well explained Sir. Thank you
Thank you so much for your appreciation :)
Good content thank you
Thanks for the appreciation :)
Ritvik You made my day - thank you
I am glad to know that :)
Thank u !
You're welcome!
Thank so much sir
thank you very much ritvik for your content that you bring to the channel and above all for the niche based on the markets and also for the clear and detailed explanations in the videos
Thank you so much for your appreciation :)
I did everything as in the video but I have an error in yhat = list(output[0])[0] --> 'numpy.float64' object is not iterable
It's difficult to understand the query here. I recommend you to get the google drive access and copy and paste my code and check line by line where is the mistake.
@@FinancialProgrammingwithRitvik After many hours I was able to solve the error in this way yhat = output[0]
How to add the forcatested valeus at the same fugure with the actual values?
You mean the same dataframe?
@@FinancialProgrammingwithRitvik yeah
Is it a 1 day ahead prediction or what? Means the prediction is of what time spam?
what about Stationarity ? AIC tooo big, i think better to do auto_arima before, for choosing params
Yes, Very good point. I didn't mention it as it might make the video more lengthy and confusing for beginners. But Yes, we will do all the relevant tests before performing the model in a real world analysis.
hello ritvik,
can we able to take multiple columns to feed it
Yes, we can train the model with multiple features as well.
Great video man, really appreciate it. One question though, how do I predict the value for the next trading day?
I can't understand the part where you have append the training data in the function with actual_test_value and how I can predict future price i.e. price after 1 week with this approach?
great video, can you try to predict forward testing
We can mae prediction model for it.
Thanks Ritvik for sharing. Great tutorials
Thanks for watching!
sir please upload videos for data how to used from different platform or others
Hello sir. Thank you for your video. Excellent job. Could you please provide me the code to forecast future one year values. Please help me for my thesis
Ritvik sir. It's fantastic video. I am learning so much from you. I have developed my first intra day algo using python.
My pleasure.
It is predicted future price or historical price ?
Yes, it's predicting future prices. I have shown it on historical data
thank you : )
You are welcome!
buen vídeo!!
Gracias :)
great video !! can you please share details of instruments used while recording this great video
I don't have any setup. I just have a webcam and a shoulder mic
Ritvik, your videos are very easy to follow and extremely informational.
Glad you like them!
hlo Sir goodevening
I have been working on Nifty bank prediction using ARIMA in python since jan '23 . being from Commerce background everything seems a little bit difficult but i really want to do this . Now I am stuck at a point where I am getting straight line predictions . I don't know what to do . I have tried everything but all in vain. Please suggest me what to do next
It happens with everyone. Try some other model apart from ARIMA. You may like my fpProphet. I have made a video it that as well.
I did a prediction of BTC and I got of mape of 0.18 did I made any mistake ?
Difficult to say. MAPE may varying significantly depending on the time period.
The prices till 2022 are already known, How can i use 100% of the data as training data and predict the future values??? Here we are prediciting already known values. How to predicit future values?
Is the code from your videos avaliable for downloading somewhere?
You can get a free access of my google drive. Over there, this code is save along with many other codes + study material.