Code & Dataset for practice: github.com/pik1989/MLProject-ChurnPrediction Also watch, Churn Analysis dashboard creation using Power BI: Part 1: ruclips.net/video/Dryfm9VaEw4/видео.html Part 2: ruclips.net/video/mAPcW_YDUKQ/видео.html
This channel is one of the best I've come across for ML, Data Analysis and Python. The level of content uploaded here is so remarkable. Thank you so much Sir. ❤
I am at half the lecture and can not contain my gratitude: Totally awesome lecture and explanations! Please do other lectures related to ML- you have rare gift to impart understanding. Million Thanks! God Bless!
One of the best End to End ML project explanation and implementation I have seen till now.... Thank you @Satyajit for taking out your time and preparing such a wonderful video for us...
Thank you for providing such a detailed and clear explanation for this data science project. I am learning data science using Gemini and your videos, and I am so happy to have found you as a resource.
Hello, I wanted to thank you. I got the job as data analyst. In the technical interview, I was asked to do any dummy projects and walk through the process .I used this data and topic of churn analysis and did similar project like yours with the help of your code. I used Power for data visualization and used python for data modeling for the same data and was able to impress the interviewers . I also had the theoretical knowledge on what I was doing but I am not good at technical stuff because all I learnt in the school was to calculate eigen-value, PCA, manually, definition of p-value and regression analysis😂😂. I can understand what the code is doing but if I have to code,I will spent all day on youtube. Its my second day on job but I don't have proper understanding of relational database. They are using Powerbi and SSMS together for the daily reports and all. I need to sustain on the job😂😁 Please advise me with the link of videos if you have made on SQL,powerbi, and python on how they are working together.
Hi Satyajit, I believe performing resampling on entire data might be responsible for data leakage since some of the resampled examples will be the part of both training and testing datasets now contributing to a high F1 score.
@@mohitpandey5190 But my performance metrics dropped significantly after this, Came down to around 80% in almost all metrics. Do you have suggestions over other techniques that could work much better.
Finding telecom domain projects on youtube are very difficult. Thank you for sharing..Please share more end to end projects focused on telecom domain..
Please bring projects which can be showcased on CV. Like..customer segmentation, customer retention, customer call anlysis.It would be very helpful. Thanks for ur work!
Amazing work..clear explanation of EDA and explaining insights by analysing...Keep doing more end to end ML projects for us in different domains.. thank you
Thanks for the amazing End To End Explanation...... Sir can you please make a video playlist on Speech To Text Machine learning Model building from Scratch?
Yeah me too...could u please help..In Webpage i'm getting the error as internal server error and in anaconda prompt Value error as: X has 51 features,but RandomForestClassifier is expecting 50 features as input...
@@gauravmore8578 No..I'm not getting the output everytime...for most of the cases I'm getting Value error: X has 51 features, but randomforestclassifier expecting 50 has input...
Hey there love you teaching skills... I have a question that the EDA steps used over here can we use it as the base for different DS projects or there is something different
having problem while running it, could you please help me out. while running app.py error occur in browser.... Internal Server Error The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
Hi actually I am having doubt how to handle the imbalanced data set like having only two col x and y and y is having both Pos and negative values in it how to approach with it I tried transforming x to log x and many more but didn't worked
You have label encoded the data before splitting the data into test and train that is why you are getting such high precision and recall. There is data leakage here.
Hi, Every industry have predicting wheather a person churn or not churn how about making this project more realistic with prescriptive analytics by converting churn customer into non churn and if person is already a churn by providing him to good premium to convert churn customer into non churn customer..it will be helpfull to entire community..please make video on this
Good suggestion, but the part you explained is really difficult to implement as it's need cost analysis and ROI calculation, and it needs sales team involvement too, which is really difficult to show.
@@SatyajitPattnaik thanks for ur reply please can u make video on reinforcement learning on banking or insurance data it will helpfull for aspiring data scientist .hope that video soon 🎉
@@SatyajitPattnaik I have idea in customer retention in financial industry but i don't know how to implement reinforcement learning although it requires huge data ...my idea is to implement prescriptive analytics in churn prediction ..it needs normal any ensemble algorithm to predict churn or non churn customers then taking churn customer data we implement some prescriptive analytics by before customer going to churn we will recommend some premiums to them then using LSTM models and reinforcement learning and time series ..LSTM needs because we need to focus on customer past behaviour with the company and time series needs because it's customers travel how loyal they are with company and finally reinforcement learning needed because we need to train our model cantinously with the data then model learn it self for future data
I’m working on a project titled "Predictive Analytics for Customer Churn in E-commerce Using Machine Learning and Sentiment Analysis on Customer Reviews" for my master’s. I’ve explored many datasets, but I’m struggling to find one that has both transactional data and customer reviews from the same source. Most datasets either have one or the other. Could you please help me find a suitable dataset or guide me on how to approach this? Since my project integrates both traditional analysis and sentiment analysis, having data from a single source is crucial. Thanks in advance for your help!
thank you for the tutorial, i have a question, when im running all your code, ValueError: X has 64 features, but DecisionTreeClassifier is expecting 45 features as input. can you explain it? thank you
Shouldn't be, can u check again, many people are using this, And it works fine. If u can't figure out, upload your code on GitHub, share me the link and tell me which line is it failing..
The error message is quite clear, you must be passing some value that is not present in original data, hence once you are doing encoding a new feature is getting generated hence you got 2 extra features
I am getting error while implementing SMOTENN....when I am running it is giving me error 'nonetype' object has no attribute 'split'. Kindly help me to solve this error I am stucked
Project explanation is great, just one problem, when u applied smoteenn before splitting train and test data, u r getting 92% accuracy through wrong approach, on correcting this , accuracy comes down below 80%, if u could fix this issue.
Very detailed explanation. Thanks a lot for the efforts in creating this. I have a query regarding selection of training dataset for a subscription based model. If I select historical data with churners in the last 3months, for example, would I need to restrict the subscription start date in the training dataset for creating the base data?
@@SatyajitPattnaik Thanks for replying, but my question was should I filter out the data by subscription start date too like only taking subscriptions for a particular time period? I am taking churned customers for last 3months.
@@sandeepthukral3018 yes, i have deployed similar application in 2 of my previous companies, however the complexity in real production level projects is immense
Hi Satyajit. .In addition to predicting whether a customer churned out or not, we can also predict when the customer is going to churn out for e.g. 6 months /9 months etc. How can we accomplish that?
There's a column called as tenure, so if you are testing the model on customer X and he's in the system for 24 months, just pass tenure as 30 for that customer and u will get the results for that customer if he's going to be churned in next 6 months or not.
Very good explanantion but I have a doubt as in if we choose decision tree in this case is there a way to find out the root node? any code available to do so?
@@SatyajitPattnaik Thank you very much but I got it very small nodes I used fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (5,5), dpi=300) tree.plot_tree(model_dt_smote,filled=True) and I am getting nodes that are not clearly visible any way to increase size of nodes and make it look clear? That would be great help!!
A lot of things can be done in this model, this is not the most optimized model, like you said CV can be performed, other feature engineering steps plus hyperparameters Optimization can also be done.
@Satyajit Pattnaik Thanks a lot for helping out and providing the vedio for college project..but I'm getting an error at the last...after submitting the user values in webpage..I'm getting the error as internal server error and the same in Anaconda prompt..I'm getting the Value error : X has 51 features,but RandomForestClassifier is expecting 50 features as input...But I'm not getting the error always but mostly...for 1st input given in CSV file I'm getting the output but for 2nd input I'm getting the output...I followed the same code given in github...could u please help..
Hi Satyajit, Thank you for your detailed steps to implement End-to-End ML with deployment. I have few questions, can you please clarify? Please explain the usage of the following lines df_1=pd.read_csv("first_telc.csv") # how did you get this file "first_telc.csv? df_2 = pd.concat([df_1, new_df], ignore_index = True) single = model.predict(new_df__dummies.tail(1)) # as per this line, you are considering only the input which we enter from html so, df_1, df_2 steps may not be required. If I just process the input from html using pd.get_dummies and then do model.predict , this will be enough for prediction. Am I correct? Any how, Thank you once again for your contribution
Hi Satyajit, I found the reason for using 'first_telc.csv' and the above clarifications which I raised. I have figured it out, you have to use some sample data and then add your input data because, then only all combination of categorical values are covered and it will be converted into equal number of columns as compared with Analysis file using pd.get_dummies. To match 51 columns, you are loading that csv file. However to predict, you are using only the last row (tail) which is entered by user in UI. Thanks a lot.
For deployment can you please explain what is in df_1 ? and why you concat with new_df to create df_2. I am working on a similar project with this data set.
Sir As i am from Pharmaceutical sales background,is it possible make a video on end to end Business Analytics project on pharmaceutical company used cases like ur very nice Telecom project,it would be very helpful for us.
Thanks a lot for this session Satyajit .... I have a doubt that how data scientists typically finalise a machine learning model for a given problem since there are many machine learning models available?
The model that predicts better on test data is fine tuned and pushed into production to get tested on live data, further the results are evaluated by sales and various teams, they give feedbacks whether the model is actually doing good or not, and then further remodelling is done, it's a long process
I noticed one thing. You applied SMOTEENN function on the train and test data. Applying SMOTEENN function on the test set and later calculating f1 score, recall etc from predictions made from the test set does not give the correct results. When I didnt apply SMOTEENN to the test set my results were actually very similar to the decision tree classifier score calculated on data without SMOTEENN function. What makes sense is applying SMOTEENN only to the train data and not test data.
hi sir, i am anu, recently i joined in data analyst course, your video's are very help full for me i am facing issue on after installing masked rcnn then i fail to importing "from mrcnn import model as modellib " kindly resolve the issue, thank you
@@SatyajitPattnaik thanks for reply AttributeError Traceback (most recent call last) in () 1 from mrcnn.config import Config 2 from mrcnn import utils ----> 3 import mrcnn.model as modellib 4 from mrcnn import visualize 5 from mrcnn.model import log /usr/local/lib/python3.10/dist-packages/mrcnn/model.py in 253 254 --> 255 class ProposalLayer(KE.Layer): 256 """Receives anchor scores and selects a subset to pass as proposals 257 to the second stage. Filtering is done based on anchor scores and AttributeError: module 'keras.engine' has no attribute 'Layer' i search on stack overflow it tells about all instances keras application KE is changes to KL but i don't understand how to fix it
@@SatyajitPattnaik But I like your EDA approach. Kudos. I am a very new player in ML. Doing data science masters from Great learning. There are so much to learn from you. Will keep watching more case studies shared by you. Thank you for your amazing work.
hey man create an recommendations engine to the people not get churn recommended something so they got customers back N already have they get more interested
Awesome! i enjoyed practicing the project along with listening your insights thanks a lot .I am statistician by virtue of my profession wish to connect with you how can i reach you!
You can read more about eager and lazy learning, the algo used here in this use case is probably DecisionTree if I recall correctly, and DT is basically a eager learner. So how the learning happens is something that is internal to the algorithm, to know more about it, you have to dig in to the model's base codes, here we are just implementing the algorithm and performing our predictions. I hope you got my point, for example, if u want to know why Decision Trees are called eager Learners, you will have to check how a decision tree is implemented and what's the code within the algos. A link which can help you: towardsdatascience.com/machine-learning-classifiers-a5cc4e1b0623 In case you still have the doubts, we can probably connect over LinkedIn.
Yes, I'm experience some difficulties with understanding the pronunciation. You would be so kind if allow subtitles in this video. Thank you, again for this great content!
After training the model , you have shown the accuracy on test data i.e 93% but how we can get churn/Not churn against that in test dataset, not through API but in the python only
Yes, we can get it, just run the entire test data in a loop, and append the prediction score in a new column, for ex: df["Prediction"] = model.predict_proba(X_test) Just giving you ideas..
@@SatyajitPattnaik Thanks for the clarification. I will try that . One more ques . - We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful.
Thanks for the clarification. I will try that . Two more ques . - 1.We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful. 2.max_depth=6, min_samples_leaf=8 , how we have decided this in this e.g. ?
Can use the model to predict data in PBI right? Then, create 1 table to have churn values customers in PBI. I don't use file Excel but use data from a database to test. Can you help me ?
Predicting on PBI isn’t possible, but predictions can be integrated with PBI to showcase the results. In PBI you can do time series forecasting but no ML predictions
@@SatyajitPattnaik I have finished building the customer churn prediction model and saved it in the pickle library. Now how can I use that model to predict data on PBI (can use integration or some other way) and show convenient results for calculating customer churn? I don't know what to do :( Please help me. Please. Thank you very much.
@@SatyajitPattnaik I can't get data from PBI to predict outside because data in database db2 :(. How to connect VS code with database db2 to create churn customer in data on the database? Then, I can use the data it use in PBI, right ?
@@VyĐặngThịTường-i3m use chatGPT on how to connect db2 from your python notebook, once u r able to create connections, start working on the predictive model , and then u can connect with power bi and show predictions on pbi
@@akshajagarwal3519 well if u r using exact similar values then theres no way it Will fail, its tested and used by many, pls debug it, or else modify the input fields to be drop down values
Hello, thanks for making an interactive video. However, I advise reducing the size of the title of the video section, such as the Model Deployment title. It's very distracting and I can't see what you're doing because it's obscured by the title of the video section🙂
Hi Satyajit, thank you well-done project. I had a data science interview this week can I take this end-to-end project to present in front of the interviewer?
In the telecom churn dataset, "Monthly Charges" refers to the total amount charged to the customer for their telecom services each month , while "Total Charges" refers to the total amount charged to the customer over the entire duration of their service with the telecom company .
Your time related parameters will play a vital role here, let say, you want to test your existing customers, and latest tenure period of that customer is 27 months, if u want to test whether he stays after 6 months or not, u need to pass the tenure as 33 for that customer with other details..
sir i have a problem in model deployment in spyder. can you help me("UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte") this is the error anyways thanks for great and amazing video
Customer segmentation is an unsupervised technique, where you need to create customer clusters based on their behaviour, when you have a variable where you know who is churning, and who's not, you can have a classification model to predict whether a new customer will churn or not.
That's a second problem, churn or not churn is one classification problem, you can also train a multiple class classification by having classes like: not churned, churned due to service, churned due to cost etc, so it's a multi class classification 👍
Code & Dataset for practice: github.com/pik1989/MLProject-ChurnPrediction
Also watch, Churn Analysis dashboard creation using Power BI:
Part 1: ruclips.net/video/Dryfm9VaEw4/видео.html
Part 2: ruclips.net/video/mAPcW_YDUKQ/видео.html
can i get ppt of this ?? or can you tell how w can make such a wonderfull ppt on telecom customer churn like this ??
Thanks a lot 🎉🎉
@@WOW-vn5jd Canva 😁
Didn't get the insights that you find from describe() method like 75% clients have 55 months tenure and 25% clients pay some USD
@@jahangagan5396 i thnk 25 is wrongly typed it should be 75
This channel is one of the best I've come across for ML, Data Analysis and Python. The level of content uploaded here is so remarkable. Thank you so much Sir. ❤
😊
The best end-to-end machine learning project. I have learnt a lot from this video. Thank you, sir.
Thanks Rishab
I am at half the lecture and can not contain my gratitude: Totally awesome lecture and explanations! Please do other lectures related to ML- you have rare gift to impart understanding. Million Thanks! God Bless!
One of the best End to End ML project explanation and implementation I have seen till now....
Thank you @Satyajit for taking out your time and preparing such a wonderful video for us...
Thanks, that keeps me motivated.
Wow, you really have put in great effort to make such a detailed video. Thanks a ton! Keep sharing your knowledge 🙂
Thank you for providing such a detailed and clear explanation for this data science project. I am learning data science using Gemini and your videos, and I am so happy to have found you as a resource.
Highest way of teaching.. U covered each and every minute details in just one project.... 🙏thanks
Thanks a lot..
it felt so real life...like i am a da in an company and my manager is giving us walkthrough of the project...kudos
Hi, is this industry level project for data scientist?
Hello, I wanted to thank you. I got the job as data analyst. In the technical interview, I was asked to do any dummy projects and walk through the process .I used this data and topic of churn analysis and did similar project like yours with the help of your code.
I used Power for data visualization and used python for data modeling for the same data and was able to impress the interviewers . I also had the theoretical knowledge on what I was doing but I am not good at technical stuff because all I learnt in the school was to calculate eigen-value, PCA, manually, definition of p-value and regression analysis😂😂. I can understand what the code is doing but if I have to code,I will spent all day on youtube.
Its my second day on job but I don't have proper understanding of relational database. They are using Powerbi and SSMS together for the daily reports and all. I need to sustain on the job😂😁 Please advise me with the link of videos if you have made on SQL,powerbi, and python on how they are working together.
Check my playlists, there are videos on power bi and sql too (end to end)
@srijanakhatri7854 can you please guide me how to go on as i am someone who is starting from scratch
Thank you so much Satyajit for this wonderful end-to-end full tutorial. That was really helpful.
Thanks Anirban
Really appreciate your effort best project explanation so far for me ! Thank you !
Wonderful explanation. Love how you get into detail wrt business understanding and coding
Very great project and explanation! Thank you! ❤
thank you for the tutorial you are really amazing and really have a gift to teach. Keep the good work up
Thanks for your kind words 👍
Thank you Satyajit. The sessions was very informative and helpful!!
Thanks Sainath
Big fan for ur efforts and explanation. I feel motivated from u
What a Great project, thanks Sir
Hi Satyajit,
I believe performing resampling on entire data might be responsible for data leakage since some of the resampled examples will be the part of both training and testing datasets now contributing to a high F1 score.
Yes, that's a mistake I did here, you can do resampling on training data instead of the full data, I already acknowledged it in some other comment 👍
Noted @@SatyajitPattnaik, Make sense. Amazing work on churn modeling 🙌🏼
@@mohitpandey5190 But my performance metrics dropped significantly after this, Came down to around 80% in almost all metrics. Do you have suggestions over other techniques that could work much better.
Thank you for this, I really learnt a lot
Great to hear this 😁
X_resampled, y_resampled = sm.fit_sample(x,y) become X_resampled, y_resampled = sm.fit_resample(x,y) thanks for the video
Finding telecom domain projects on youtube are very difficult. Thank you for sharing..Please share more end to end projects focused on telecom domain..
Definitely
Please bring projects which can be showcased on CV. Like..customer segmentation, customer retention, customer call anlysis.It would be very helpful. Thanks for ur work!
@@riyatiwari4767 noted, pls wait for 2 weeks, you would see new Telecom projects 🔥🔥
Amazing work..clear explanation of EDA and explaining insights by analysing...Keep doing more end to end ML projects for us in different domains.. thank you
Thanks Srikanth.
@@SatyajitPattnaik thanks sir and please do one more end to end ml project like this in a different domain.
@@depeshkumarmohanty9522 Already done, pls search "end to end satyajit" you will get more projects
what an amazing video..thanks for it
Very very helpful!!
Awesome video!
thank you so much for this video
Thank you brother, You saved me from failing my assignment. You are really a Savior sent by god.
Thanks Jaideep ☺️😍
Lovely project..
Thanks for the amazing End To End Explanation...... Sir can you please make a video playlist on Speech To Text Machine learning Model building from Scratch?
Sure, will do 👍
Hey, I'm getting this error
What am I doing wrong?
ValueError: X has 65 features, but RandomForestClassifier is expecting 50 features as input.
Yeah me too...could u please help..In Webpage i'm getting the error as internal server error and in anaconda prompt Value error as: X has 51 features,but RandomForestClassifier is expecting 50 features as input...
I'm not getting the error always but 3 in 5 times..for the 2nd input i'm getting the output but for 1st i'm getting the error
@@ittaboyinaharshithayadav7759 Did you manage to get output ?
@@gauravmore8578 were you able ro solve it..?
@@gauravmore8578 No..I'm not getting the output everytime...for most of the cases I'm getting Value error: X has 51 features, but randomforestclassifier expecting 50 has input...
'SMOTEENN' object has no attribute 'fit_sample'
and
name 'X_resampled' is not defined
can you explain the error and correct answer sir please
Sir please reply
its changed to "fit_resample", please use chatGPT for faster resolution :)
Hey there love you teaching skills... I have a question that the EDA steps used over here can we use it as the base for different DS projects or there is something different
Same 😀
having problem while running it, could you please help me out. while running app.py error occur in browser.... Internal Server Error
The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.
Pls go through comments, many had this issue because of the input that they pass
This is a great project and I got to learn a lot from it. I wanted to ask will this be a good project for my final year dissertation?
Yes surely
Hi, i would want to know the procedure of what tweak in code for next 3 month churn prediction you were talking about
thank you sir.
welcome sir
Awesome content
Many thanks
Hi actually I am having doubt how to handle the imbalanced data set like having only two col x and y and y is having both Pos and negative values in it how to approach with it I tried transforming x to log x and many more but didn't worked
You have label encoded the data before splitting the data into test and train that is why you are getting such high precision and recall. There is data leakage here.
But isn't that normal to do
Hi,
Every industry have predicting wheather a person churn or not churn how about making this project more realistic with prescriptive analytics by converting churn customer into non churn and if person is already a churn by providing him to good premium to convert churn customer into non churn customer..it will be helpfull to entire community..please make video on this
Good suggestion, but the part you explained is really difficult to implement as it's need cost analysis and ROI calculation, and it needs sales team involvement too, which is really difficult to show.
@@SatyajitPattnaik thanks for ur reply please can u make video on reinforcement learning on banking or insurance data it will helpfull for aspiring data scientist .hope that video soon 🎉
@@VenuGopal-dr8ln Reinforcement learning is a huge topic, let me know a specific topic that you want, and I can make a video on it.
@@SatyajitPattnaik I have idea in customer retention in financial industry but i don't know how to implement reinforcement learning although it requires huge data ...my idea is to implement prescriptive analytics in churn prediction ..it needs normal any ensemble algorithm to predict churn or non churn customers then taking churn customer data we implement some prescriptive analytics by before customer going to churn we will recommend some premiums to them then using LSTM models and reinforcement learning and time series ..LSTM needs because we need to focus on customer past behaviour with the company and time series needs because it's customers travel how loyal they are with company and finally reinforcement learning needed because we need to train our model cantinously with the data then model learn it self for future data
I’m working on a project titled "Predictive Analytics for Customer Churn in E-commerce Using Machine Learning and Sentiment Analysis on Customer Reviews" for my master’s. I’ve explored many datasets, but I’m struggling to find one that has both transactional data and customer reviews from the same source. Most datasets either have one or the other.
Could you please help me find a suitable dataset or guide me on how to approach this? Since my project integrates both traditional analysis and sentiment analysis, having data from a single source is crucial.
Thanks in advance for your help!
thank you for the tutorial, i have a question, when im running all your code, ValueError: X has 64 features, but DecisionTreeClassifier is expecting 45 features as input. can you explain it? thank you
Shouldn't be, can u check again, many people are using this, And it works fine.
If u can't figure out, upload your code on GitHub, share me the link and tell me which line is it failing..
Ok I will check it again, I hope running well like others, thank you very much
@@SatyajitPattnaik Your code is awsome, it's really work finally, thank you very much
@@iqballatifable thanks Iqbal, do share my channel with your friends & colleagues 🔥🔥
@@iqballatifable I too got same error how did you resolve it
thank you so so much sir............
Welcome 🔥🔥
ValueError: X has 52 features, but DecisionTreeClassifier is expecting 50 features as input. How to Fix this problem , please?
The error message is quite clear, you must be passing some value that is not present in original data, hence once you are doing encoding a new feature is getting generated hence you got 2 extra features
Hi how tenure_group hot coded into dummies? and how it generated 0's and 1's
I am getting error while implementing SMOTENN....when I am running it is giving me error 'nonetype' object has no attribute 'split'. Kindly help me to solve this error I am stucked
Project explanation is great, just one problem, when u applied smoteenn before splitting train and test data, u r getting 92% accuracy through wrong approach, on correcting this , accuracy comes down below 80%, if u could fix this issue.
Very detailed explanation. Thanks a lot for the efforts in creating this. I have a query regarding selection of training dataset for a subscription based model. If I select historical data with churners in the last 3months, for example, would I need to restrict the subscription start date in the training dataset for creating the base data?
You can create tenure as a derived metrics based on expiration date - subscription date
@@SatyajitPattnaik Thanks for replying, but my question was should I filter out the data by subscription start date too like only taking subscriptions for a particular time period? I am taking churned customers for last 3months.
Hi, is this industry level project for data scientist?
@@sandeepthukral3018 yes, i have deployed similar application in 2 of my previous companies, however the complexity in real production level projects is immense
Hi Satyajit. .In addition to predicting whether a customer churned out or not, we can also predict when the customer is going to churn out for e.g. 6 months /9 months etc. How can we accomplish that?
There's a column called as tenure, so if you are testing the model on customer X and he's in the system for 24 months, just pass tenure as 30 for that customer and u will get the results for that customer if he's going to be churned in next 6 months or not.
Hi Satyajit. Nice End-to-End project. Could you let us know when we need to do Hyper-parameter tuning, and why is it required.
Once you build your prototype model, and want to take a further step in enhancing the model, you might have to do hyperparameters Optimization....
Hyper parameter tuning is required to reduce the variance.
With introducing small amount of bias we are able to reduce variance to a large extent.
@@sarfrazjaved330 Spot on 👍👍
Very good explanantion but I have a doubt as in if we choose decision tree in this case is there a way to find out the root node? any code available to do so?
You can plot a tree, sklearn.tree.plot_tree
@@SatyajitPattnaik Thank you very much but I got it very small nodes I used
fig, axes = plt.subplots(nrows = 1,ncols = 1,figsize = (5,5), dpi=300)
tree.plot_tree(model_dt_smote,filled=True) and I am getting nodes that are not clearly visible any way to increase size of nodes and make it look clear? That would be great help!!
why is cross validation not performed while building the model. Is there any criteria that you have considered to not use CV.
A lot of things can be done in this model, this is not the most optimized model, like you said CV can be performed, other feature engineering steps plus hyperparameters Optimization can also be done.
@Satyajit Pattnaik Thanks a lot for helping out and providing the vedio for college project..but I'm getting an error at the last...after submitting the user values in webpage..I'm getting the error as internal server error and the same in Anaconda prompt..I'm getting the Value error : X has 51 features,but RandomForestClassifier is expecting 50 features as input...But I'm not getting the error always but mostly...for 1st input given in CSV file I'm getting the output but for 2nd input I'm getting the output...I followed the same code given in github...could u please help..
Sorry,for 2nd input in CSV file I'm getting output,but for 1st input I'm getting Value Error and internal server error
@@harshithayadavittaboyina7390 Hey, did you find the solution as i am facing same kind of issue..
Hi Satyajit, Thank you for your detailed steps to implement End-to-End ML with deployment. I have few questions, can you please clarify? Please explain the usage of the following lines
df_1=pd.read_csv("first_telc.csv") # how did you get this file "first_telc.csv?
df_2 = pd.concat([df_1, new_df], ignore_index = True)
single = model.predict(new_df__dummies.tail(1)) # as per this line, you are considering only the input which we enter from html so, df_1, df_2 steps may not be required. If I just process the input from html using pd.get_dummies and then do model.predict , this will be enough for prediction. Am I correct?
Any how, Thank you once again for your contribution
Hi Satyajit, I found the reason for using 'first_telc.csv' and the above clarifications which I raised. I have figured it out, you have to use some sample data and then add your input data because, then only all combination of categorical values are covered and it will be converted into equal number of columns as compared with Analysis file using pd.get_dummies. To match 51 columns, you are loading that csv file. However to predict, you are using only the last row (tail) which is entered by user in UI. Thanks a lot.
Okay a
@@DilipKumar-ww5si very helpfull, Thanks
dear sir, what is first_telc.csv file which was in app.py , please explain
It's not used anywhere, pls ignore.
yes i also have the same doubt
For deployment can you please explain what is in df_1 ? and why you concat with new_df to create df_2. I am working on a similar project with this data set.
When I check that df_1 'first_telc.csv'. It has 19 columns and 75 rows ? can you please explain where this comes from
and why you concat this with the data coming from the user ?
Sir As i am from Pharmaceutical sales background,is it possible make a video on end to end Business Analytics project on pharmaceutical company used cases like ur very nice Telecom project,it would be very helpful for us.
there are some healthcare case studies on my channel
Thanks a lot for this session Satyajit .... I have a doubt that how data scientists typically finalise a machine learning model for a given problem since there are many machine learning models available?
The model that predicts better on test data is fine tuned and pushed into production to get tested on live data, further the results are evaluated by sales and various teams, they give feedbacks whether the model is actually doing good or not, and then further remodelling is done, it's a long process
Sir, can you please tell what is the use of concatenating df1 with the input queries ?
To test the model on a new data, I am just capturing the input values in a new dataframe and then calling model predict on that dataframe
I noticed one thing. You applied SMOTEENN function on the train and test data. Applying SMOTEENN function on the test set and later calculating f1 score, recall etc from predictions made from the test set does not give the correct results. When I didnt apply SMOTEENN to the test set my results were actually very similar to the decision tree classifier score calculated on data without SMOTEENN function. What makes sense is applying SMOTEENN only to the train data and not test data.
@@traeht yes thats a good observation, i did that mistake, balancing out should be done for training data only
Sir, Can we get the PPT you have used to explain the model.
I don't share ppts for free, but you can create the ppts by yourself by taking ideas from mine.
hi sir, i am anu,
recently i joined in data analyst course, your video's are very help full for me
i am facing issue on
after installing masked rcnn then i fail to importing "from mrcnn import model as modellib "
kindly resolve the issue,
thank you
Whats the error?
@@SatyajitPattnaik thanks for reply
AttributeError Traceback (most recent call last)
in ()
1 from mrcnn.config import Config
2 from mrcnn import utils
----> 3 import mrcnn.model as modellib
4 from mrcnn import visualize
5 from mrcnn.model import log
/usr/local/lib/python3.10/dist-packages/mrcnn/model.py in
253
254
--> 255 class ProposalLayer(KE.Layer):
256 """Receives anchor scores and selects a subset to pass as proposals
257 to the second stage. Filtering is done based on anchor scores and
AttributeError: module 'keras.engine' has no attribute 'Layer'
i search on stack overflow it tells about all instances keras application KE is changes to KL
but i don't understand how to fix it
Very thank u
Welcome Chaman
may I know what ML algorithm is used for this churn prediction sir?
Classification algorithms
does this come under predictive analytics sir?
@@tamizhmalar9843 yes
can you give exact pre reqrusite for this projects ? so i learn and directly do this project
Python, Stats, Machine Learning is what you need to learn.
I run project and i gave input to the fields,it does not showing any output
did you solve the issue?
In my understanding, SMOTE should be applied on training data set...not on entire dataset.
@@saurabhagrawal3691 yes I already mentioned that in multiple separate comments, it was a mistake that i did
@@SatyajitPattnaik But I like your EDA approach. Kudos. I am a very new player in ML. Doing data science masters from Great learning. There are so much to learn from you. Will keep watching more case studies shared by you. Thank you for your amazing work.
@@SatyajitPattnaik Could you please share any detailed work of yours which include saving model and deployment using pickling and Flask. Many thanks.
hey man create an recommendations engine to the people not get churn recommended something so they got customers back N already have they get more interested
Sure, will build one
Hi how can we compare two float numbers in python with greater than operation I am getting error saying >= is not a function in numpy?
Awesome! i enjoyed practicing the project along with listening your insights thanks a lot .I am statistician by virtue of my profession wish to connect with you how can i reach you!
All Social media links are provided in my video descriptions, you can reach me via those links :)
oooo okay
Hello sir,
Can I know the ML model of which you are using like linear regression, logistic regression, random forest
How can we apply Artifical Intelligence , eager and lazy learning in it, finding common patterns in it
You can read more about eager and lazy learning, the algo used here in this use case is probably DecisionTree if I recall correctly, and DT is basically a eager learner.
So how the learning happens is something that is internal to the algorithm, to know more about it, you have to dig in to the model's base codes, here we are just implementing the algorithm and performing our predictions.
I hope you got my point, for example, if u want to know why Decision Trees are called eager Learners, you will have to check how a decision tree is implemented and what's the code within the algos.
A link which can help you: towardsdatascience.com/machine-learning-classifiers-a5cc4e1b0623
In case you still have the doubts, we can probably connect over LinkedIn.
Hi..Satyajit ,can you make a video how anyone get fresher datascientist job in industry....and another question is,Are you from Odisha ?????
Sure i can make a video, and yes i am from Odisha
Hello sir can you tell me the source from where did you get the Dataset from?
It's a public dataset easily available on Kaggle and various other websites.
@@SatyajitPattnaik thankyou sir
Bro how to download your dataset bro
GitHub repository link is provided, pls fork the project on GitHub and use the data..
At first, thank you a lot for this video! At second )))) - can you provide a subtitles for it?
Well, it's in English already, let me know if you're facing any difficulties
Yes, I'm experience some difficulties with understanding the pronunciation. You would be so kind if allow subtitles in this video. Thank you, again for this great content!
After training the model , you have shown the accuracy on test data i.e 93% but how we can get churn/Not churn against that in test dataset, not through API but in the python only
Yes, we can get it, just run the entire test data in a loop, and append the prediction score in a new column, for ex: df["Prediction"] = model.predict_proba(X_test)
Just giving you ideas..
@@SatyajitPattnaik Thanks for the clarification. I will try that .
One more ques . - We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful.
Thanks for the clarification. I will try that .
Two more ques . -
1.We have done the correlation part, so where we used that? those variables which were not affecting the target variable can be deleted before model building.. Please let me know if i m thinking in right way ? Or correct me if i m wrong. would be really helpful.
2.max_depth=6, min_samples_leaf=8 , how we have decided this in this e.g. ?
Can use the model to predict data in PBI right? Then, create 1 table to have churn values customers in PBI. I don't use file Excel but use data from a database to test. Can you help me ?
Predicting on PBI isn’t possible, but predictions can be integrated with PBI to showcase the results. In PBI you can do time series forecasting but no ML predictions
@@SatyajitPattnaik I have finished building the customer churn prediction model and saved it in the pickle library. Now how can I use that model to predict data on PBI (can use integration or some other way) and show convenient results for calculating customer churn? I don't know what to do :( Please help me. Please. Thank you very much.
@@VyĐặngThịTường-i3m Predictions can be done outside Power BI, Power BI is just a visualisation tool
@@SatyajitPattnaik I can't get data from PBI to predict outside because data in database db2 :(. How to connect VS code with database db2 to create churn customer in data on the database? Then, I can use the data it use in PBI, right ?
@@VyĐặngThịTường-i3m use chatGPT on how to connect db2 from your python notebook, once u r able to create connections, start working on the predictive model , and then u can connect with power bi and show predictions on pbi
Asking if u. Have any playlist for the very first things of excel, Powerpoint or ms word. If u have, than pls provide me with the link.
how to get the customer chrun dataset
Checj video description
I am getting Internal server error when i click on submit on the webpage , kindly help ASAP
So your values arent matching with the expected values
@@SatyajitPattnaik I am inputting the values directly from the dataset itself , can you provide with the values to be entered.
@@akshajagarwal3519 well if u r using exact similar values then theres no way it Will fail, its tested and used by many, pls debug it, or else modify the input fields to be drop down values
@@SatyajitPattnaik Thank You
How did you fix this error??
Hi Satyajit, May I ask you to share the link for python code used in today’s session!!
I will upload it tomorrow first half for sure, it's a busy day for me today, thanks for understanding!!
github.com/pik1989/MLProject-ChurnPrediction
Is it mandatory to be good enough to write code on our own, If we are looking for data engineer jobs? .........from a final year student
Not mandatory
Hello, thanks for making an interactive video. However, I advise reducing the size of the title of the video section, such as the Model Deployment title. It's very distracting and I can't see what you're doing because it's obscured by the title of the video section🙂
Its a very old video of mine, checkout my channel for some latest end to end projects 😀
Hi Satyajit, thank you well-done project. I had a data science interview this week can I take this end-to-end project to present in front of the interviewer?
Ofcourse you can..
@@SatyajitPattnaik are you from odisha??
@@susantakumarsahoo1982 yes
Even am planning for the same. Thank you for the indepth explanation.
Hi ,How to create the churn data in retail industry.. ?
You have to read some papers about it.
Can any of you please tell me what is the difference between monthly charges and total charges? Thanks in advance.
In the telecom churn dataset, "Monthly Charges" refers to the total amount charged to the customer for their telecom services each month , while "Total Charges" refers to the total amount charged to the customer over the entire duration of their service with the telecom company .
@@SatyajitPattnaik Thanks a lot.
Can I know what are pre-requisites of this project?
Python and basics of ML should be enough
Could you please explain --telco_data.TotalCharges--...AS we dont know which columns contains the null data , then how should we direct write it ?
Can you point the time where you have this doubt
how do we incorporate the 6 month or 3 month prediction into this?
Your time related parameters will play a vital role here, let say, you want to test your existing customers, and latest tenure period of that customer is 27 months, if u want to test whether he stays after 6 months or not, u need to pass the tenure as 33 for that customer with other details..
there are other columns that would change as well, like total charges, age etc etc..
sir i have a problem in model deployment in spyder.
can you help me("UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte") this is the error
anyways thanks for great and amazing video
Its an encoding error, spot the error in the exact line, it must be failing while you are reading the file
smoothen function is showing error
I am getting Internal server error When we run the project
You need to see the error and fix it, its a very basic project and lot of error handling is required
Hi sir..can i use this in data analyst role resume?
Can, as Predictive Analytics is a part of Analytics :)
Someone please help me. I have a confusion between customer segmentation and customer churn
Customer segmentation is an unsupervised technique, where you need to create customer clusters based on their behaviour, when you have a variable where you know who is churning, and who's not, you can have a classification model to predict whether a new customer will churn or not.
@@SatyajitPattnaik thanks :)
@@anketsonawane6651 I can make a video on customer segmentation if you promise me 100 subs 😂
Send the code which you gave in video?
Check description
Sir how we can show the reason that he is churning
That's a second problem, churn or not churn is one classification problem, you can also train a multiple class classification by having classes like: not churned, churned due to service, churned due to cost etc, so it's a multi class classification 👍