Support Vector Machine (SVM) with R - Classification and Prediction Example
HTML-код
- Опубликовано: 11 мар 2017
- Includes an example with,
- brief definition of what is svm?
- svm classification model
- svm classification plot
- interpretation
- tuning or hyperparameter optimization
- best model selection
- confusion matrix
- misclassification rate
Machine Learning videos: goo.gl/WHHqWP
Becoming Data Scientist: goo.gl/JWyyQc
Introductory R Videos: goo.gl/NZ55SJ
Deep Learning with TensorFlow: goo.gl/5VtSuC
Image Analysis & Classification: goo.gl/Md3fMi
Text mining: goo.gl/7FJGmd
Data Visualization: goo.gl/Q7Q2A8
Playlist: goo.gl/iwbhnE
svm is an important machine learning tool related to analyzing big data or working in data science field.
R is a free software environment for statistical computing and graphics, and is widely used by both academia and industry. R software works on both Windows and Mac-OS. It was ranked no. 1 in a KDnuggets poll on top languages for analytics, data mining, and data science. RStudio is a user friendly environment for R that has become popular.
Thumbs up !!
most of your tutorials are pretty useful.
you have a good knack of explaining complicated techniques in a simplified way.
Thanks for the feedback!
Your tutorials are priceless. Thank you for sharing your knowledge. This was easy to understand and to the point.
Thanks for comments!
Thank you for your made simple and easy to follow video tutorials. You are awesome!
Thanks for your feedback!
you're the best teacher ever
Thanks for your comments!
Thank you so much. A great explanation of the SVM model.
You are welcome!
Very clear and helpful. Thank you sir!
Welcome!
#Learning From "Support Vector Machine (SVM) with R-Classification and Prediction Example
#准备工作，加载数据，并看一下数据的分布
data("iris")
str(iris)
library(ggplot2)
qplot(Petal.Length, Petal.Width, data=iris, color=Species)
#第一步：运行SVM，选择合适的Kernel方法
library(e1071)
mymodel=svm(Species~., data=iris, kernel = "polynomial")
#-------将mymodel的Kernel方法改为radial，linear,也可以改为polynomial
summary(mymodel)
#第二步：Tuning，即超平面优化，选择最佳模型
set.seed(123)
tmodel=tune(svm, Species~., data = iris, ranges = list(epsilon =
seq(0,1,0.1), cost = 2^(2:9)))
#-------seq生成一个序列，0开始，1结束，中间相隔0.1，一共11位数；
#-------cost取值为2到9，一共8位数，11x8=88个参数模型，如果数据很大，则需要很久
plot(tmodel)
summary(tmodel)
#第三步：选择最佳的模型，并作图
mymodel=tmodel$best.model
summary(mymodel)
plot(mymodel, data = iris, Petal.Width~Petal.Length,
slice = list(Sepal.Width = 3, Sepal.Length = 4))
##Petal.Width~Petal.Length，定义谁是X，谁是Y
#第四步：计算预测能力
##Confusion Matrix and MisClassification Error
pred=predict(mymodel, iris)
tab = table(Predicted = pred, Actual = iris$Species)
tab #tab用来查看预测的结果
1-sum(diag(tab))/sum(tab) #计算预测失败的概率
Not sure about your question.
@@bkrai Thanks. It's the R code for this video.
Excellent Session sir on SVM...Very Useful
Thanks!
Thank you again for these complete episodes. You have been of a great help to me "Rai". Please, I'd appreciate a complete episode on the ensembles, essentially, heterogeneous ensemble using DT, SVM etc. inclusive as the base classifiers.
Comprehensive videos on ensembles are not common, in fact, I haven't come across any. It will go a long way If you could put something together on this. Thank you for your help!
Thanks for the suggestion, I'll do it in near future!
Very well explained and very useful!
Thanks!
I am an avid subscriber of yours. Your videos are simply outstanding and very helpful for self study. Thank you very much for your videos and all the hard work.
Thanks for feedback and comments!
god blesses you sir. You are the best and much appreciate!!!
Thanks for comments!
One Word --- Awesome , Thanks Sir..
Welcome!
Thank you Dr. Rai. This video was really helpful and entertaining.
You are welcome!
Excellent video!! Thanks for sharing.
Thanks for comments!
Very nice video. Easy to understand. Appreciated your effort.
Thanks for comments!
very informative and well explained.
Thanks for your comments!
Thank you Mr. Rai for this excellent demonstration and explanation of SVM.
Regards.
thanks for feedback!
Dear Sir, Thank u very much for the video and code. I can say I learned ML and r coding using your tutorial much more than udemy, lynda, and other works. Good Job. Your channel is the best indeed!
You are most welcome!
Very good explanation! Instantly subscribed to your channel.
Thanks for comments!
Thank you very much from the bottom of my heart.
You are very welcome!
Thank you sir....
Welcome!
Sir will you please explain me what does Cost, gamma and radial means and what they do? Also explain me Radial and Sigmoid. I'm sorry too many questions I have asked but since you always help me to understand the concept clearly it's my request. Thank you Sir.
excellent really worth
Thanks!
You are really great Sir!!!!
Thanks for comments!
Sir your teaching is excellent please post some videos on how handle semi supervised machine learning algorithm in R especially in case of SVM
Thanks for the suggestion!
Thank you so much Sir!
Most welcome!
Thank you sir for this video
Most welcome!
Thanks for your video!! How to calculate AIC and BIC in SVM?
I see that many videos say let us predict and use the predict command. What are you trying to predict? What is the output is being expected?
Many thanks again for your amazing video.
Can you let me know how we evaluate the variables?
Such as we have 10 variables but only 5 of them are significant (for ex; in logistic regression, we evaluate them by P-value and OR (95%CI)).
Some said that we use weight to evaluate them, every variable has its weight, the higher the weight, the more signficant.
And can you give me the code for that?
Thankyou Sir, This tutorial was quite useful but I am trying to create a user-defined function for SVM analysis in which I can define the data set kernel, and other parameter for the data set in function calling. How can I do that ?
Incredible explain sir....plz made a video list of parametric and non parametric test..as early as possible
Thanks for the suggestion!
Hello Dr. Rai, Thanks for your great tutorials. I shoud say I learnt ML and r coding using your tutorial much more than udemy, lynda, and other works. Good Job. Your channel is the best indeed! I suggested to all my frineds!
I was wondering that would you teach us some machine learning in python?
Thanks for your comments! I'll plan to do python in few months.
brilliant, brilliant, brilliant sir.....request= can you do one please for regression
Thanks, I've added it to my list.
thankyou sir, can you please share the link
Here is the link:
drive.google.com/open?id=0B5W8CO0Gb2GGc1ZZQWhmMmpuWWc
good explanation
Thanks for comments!
Thank you so much for your wonderful videos!
There is one question about this video, that is , when using the function "tune", it always says that "Error in if (tunecontrol$cross > n) stop(sQuote("cross"), " must not exceed sampling size!") :
argument is of length zero"
Have searched for solutions and tried to convert the data used to a list but still did not work.
Would you please suggest how to fix it?
Thank you!
I saw this today, probably by now you must have addressed this.
Why when you used the slice function you set Sepal.Width = 3 and Sepal.Length = 4 ? Is this just for convenience since they are the last two variables that need to be accounted for? Are these the boundaries that are created when you created the graph?
Hi, Rai thanks for this clear lecture. But I have a question: I follow the exactly same steps as yours, but when use tune function, I get a different result from you. I get the best parameter: cost 4 (instead of 8 as yours), the best performance 0.04 (instead of yours 0.033). But all the steps i just exactly the same with you. Do you have any idea why it happened?
Thanks a lot Dr. Rai for uploading this tutorial. I would like to apply this SVM method to calculate a susceptibility index able to be plotted in ArcGIS, so I need to know the predicted values of the dependence variable:
1. How can be calculated?
2. Can I use for that the same coding as in the case of neural network?
Thank you very much
Hii, I am also facing a similar issue. I have developed the model using the training dataset and tested it. But I am not sure how to import the developed model in ArcGIS to apply it to the actual raster layers!!
Can you help me out?
very good job
Thanks for comments!
is there anything i can do to get the size of every specie? i get the number of support vectors alright but it doesn't show the distribution... and also, i have 38 variables... how do i plot the graph for all of them?
Thank you very much, please can you give me how to downsampling And oversampling the positive data samples to avoid data imbalance
Here is the link:
ruclips.net/video/Ho2Klvzjegg/видео.html
Sir, how to identify the important variables in SVM when we have a set of variables?
Sir , can u explain the inutution for three classes what is going on, as u explained for the two classes..on e hyperplane is drawn between two classes ..if the third class is there how does it separate
Hello. Thanks for your videos. I was wondering that could you teach us about genetic programming in R if there is any? Thanks
Thanks for the suggestion, I;ve added this to my list.
super sir, here there is clear separation but "cleveland heart" from UCI is complex and have lot of overlapping...
That's right. And for data that have lot of overlapping, it is always a good idea to try more methods.
Sir for large sample value what could be the value of epsilon and cost..
Okay , if i got the model ... how can i do to get an equation to for example use it in an application ? i mean to reproduce the classification results without R ? Thank you
Very nice video to watch during my exam preparations! The music would be nicer if it was maybe 50% of the volume at any point where you are talking. Otherwise well explained and great to watch :)
Thanks for the tip!
@@bkrai epsilon doesn't seem to have any effect of the results when I use tune like you do. But I found that another example used "gamma" instead of "epsilon" for another model and that had an effect on SVM for me (surprisingly). Do you know why it's like that?
Sir your videos are excellent and very easy to understand...!! Can you please post a video on regression models using SVM and ANN? That would be a great help in understanding the differences in results and validation parameters observed by using same algorithms. Thank you.
For ANN, you can use:
ruclips.net/video/SrQw_fWo4lw/видео.html
@@bkrai Yes sir... I had already went through that video but I wasn't able to perform that with my data. That's why I'm requesting you for the same.
I cannot understand why do we use slice ?Could you please explain more about it.
Sir, may i know why sepal length and sepal width assigned with constant values. that means we can't plot model with more than 2 variables. if I have assign constant values, how to decide the constant values like you have assigned 3 and 4. suppose I have used boruta algorithm for variables selection before running SVM model. i got 5 variables out of 10 variables as important. then how to plot SVM model. please help me by replying to my comment
Does SVM capture the nonlinear interaction effects across variables when using RBF?
That's correct.
I did this
tuned_model
Thanks for the wonderful session on SVM. I have a question regarding how did you choose value for epsilon , cost for the tuned model. If it is a trial and error method, I would like to know how did you end up getting that.
The best values are chosen by the model itself from the range that we provide.
yes I agree that sir. But how did you come up with this range. it looks like the optimal value is entirely depends on the range which we provide. is that right?.
Yes I agree sir. But how did you come up with that range. It looks like that the optimum value for cost & epsilon is entirely depends on range we provide. Is that right sir?.
For epsilon the range has to be between 0 and 1. So you can try 0.1 increments. If the plot suggests further fine-tuning, you can even try 0.05 or 0.01 increments. For cost default value is 1. And as mentioned in the video, you need to try very wide range and that's why we have used 2^2 etc. For most situation this approach will help you to get best values for these parameters. The idea is to have very wide range for both so that you don't miss the best values.
oh fine sir.
Sir, for discrete independent variables, can we use them as factors model?
Yes, should work fine.
Sir,
I am unable to understand this line:
slice = list(Sepal.Width = 3, Sepal.Length = 4))
What is the use and why 3 and 4?
hi sir we need svm treat binary database on java would help us with this?
Hello sir, in the 14 line from script (4.56 mins in vedio) we have slice, how to select the values in it and if many variables are the in the data, should we take SVM seperately between two variables each time?
This is what slice represents - "a list of named values for the dimensions held constant (only needed if more than two variables are used). The defaults for unspecified dimensions are 0 (for numeric variables) and the first level (for factors). Factor levels can either be specified as factors or character vectors of length 1."
In the video we used values that are more reasonable than default zero.
on tuning im getting this error..please help sir...Error in do.call(method, c(list(train.x, data = data, subset = train.ind[[sample]]), :
'what' must be a function or character string
>
Hi sir,
Can you please explain the significance of the parameters epsilon!
Regards
It affects the number of support vectors.
graet video sir.. sir can u make a video on Taylors diagram.
Thanks for comments and suggestion!
Can you show us in other video how to do the support vector regreesion with a dataset with many variables? It will be great
thanks for the suggestion, I've added it to my list.
Very good content Sirji!
Sir how to used the best model for testing data set ?
Instead of iris data with the model, you can use test data.
@@bkrai Thanks Sirji
welcome!
Hi Bharatendra, I am trying to run SVM model on dataset with 15 features and the label is binary, it looks something like this
y_test$SurveyYes
I would suggest try and use the same format as shown in the video.
Dr, do you have any numeric svm (regression) tutorial?
Not yet.
Hi! Excellent tutorial! all very clear.. I have a data set with four columns only, these are location, duration, date and time. I implemented the svm model for prediction, but all predicted values are incorrect. How can I approach date and time? I did normalize the data but still prediction rate is bad.
If one of the variables is date/time related, I would say use time series. Facebook recently open sourced its time series forecasting package. Here is the link:
ruclips.net/user/edit?o=U&video_id=7xDAYa6Ouo8
Hi! thank you, but the link is pointing to an empty page of youtube.
Here is the correct link:
ruclips.net/video/7xDAYa6Ouo8/видео.html
Sir what does slice =list (sepal. Width=3,sepal.length=4 ) indicates?
This is what slice represents - "a list of named values for the dimensions held constant (only needed if more than two variables are used). The defaults for unspecified dimensions are 0 (for numeric variables) and the first level (for factors). Factor levels can either be specified as factors or character vectors of length 1."
In the video we used values that are more reasonable than default zero.
1. While ploting the model at 4:06, why did u choose "Petal.Width~Petal.Lenght"? Is it because these variables have low correlation?
2. Also what is the reason to select Sepal.Width = 3 and Sepal.Length = 4? Is it because while using these values we see a better classifier while plotting the model?
I found this
From ?plot.svm
slice a list of named numeric values for the dimensions held constant (only needed if more than two variables are used). Dimensions not specified are fixed at 0.
In other words, when visualising the effect of predictor variables on the response you can specify which other predictor variables are to be hold constant (i.e. at a fixed value).
So in your example, you're visualising the effect of the predictor variables Petal.Length and Petal.Width on the response while keeping Sepal.Width and Sepal.Length constant at the specified values
Sir,as kernel changes number of support vector change.Can this number be measure of accuracy of the model?
For accuracy you should use info in the confusion matrix.
So you only used the Petal length and width to do the svm test and ignored the Spetal characteristics ? Or did they affect the algorithm ?
The others can be tried in the same way.
Its very pretty, sir please share the link of R script
Hi why are you doing the typical training and test data in this case?
That can be easily done here too.
SVM separate those factor levels like a cluster? If it is so why are having those many vectors?
It's outcome of the algorithm and depends on type of data.
sorry typo in the previous question, for discrete independent variables, can we use them as factors in our model
Factor variables are usually of "nominal" type. For definitions you can use this link:
ruclips.net/video/1hF0x7WsVOI/видео.html
thank so much for this video sir....can i apply this to a Raster image (i.e., Array) and could you please share the R script as well sir
it depends on what type of data you have, no harm in trying. Here is the link to R code:
drive.google.com/open?id=0B5W8CO0Gb2GGc1ZZQWhmMmpuWWc
Ok sir, thanks sir..... do u also have videos on KNN, Naive bayes and R codes for ROC, PCA and Multiple linear regression
@@bkrai thank you so much guru ji
sir, cost function = should it always start from 2 or we can have 3 to the power of ?
with 2 square, we start at cost value of 4 and then go to 8, 16, etc.. With 3 square, it will start at 9 and then jump to 27, 81, etc. But you can try it and see if it helps or not.
Many thanks sir，thank you！I have a question for you. In the following statement: "mymodel
It's because of 2D plot only 2 variables can be accommodated.
@@bkrai Thanks for the answer
@@bkrai Sir, you have assigned constant values for other variables. how you have decided those constant values sir?
Sir, I am getting the following error. could you say what can be done
> plot(mymodel, data = iris,
+ Petal.Width~Petal.Length,
+ slice = list(Sepal.Width = 3, Sepal.length = 4))
Error in `[.data.frame`(expand.grid(lis), , labels(terms(x))) :
undefined columns selected
I see a typo in Sepal.length = 4
use "L" in length.
very wonderful and useful
i have a problem in install package in R can you help me
the problem is [ unable to install packages (default library 'c:/program files/r/r-3.4.3/library' is not writeable)]
probably you can restart RStudio and retry installing the package.
thanks
Hello sir! This was very helpful thank you so much.. Can you please tell me how to split the data into train and test because I didn't understand quite well how you split the data here.. Or if there is a link to w pervious tutorial.. Thank you so much
Here is a link that has more details:
ruclips.net/p/PL34t5iLfZddspfUiv-9EaOVNUG64_fwFq
Thank you 😁
welcome!
how qplot done
if we more number of variable then what can I use qplot
In a scatter plot, we can only have two numeric variables at a time. If you have more variables, select two most important and see if they are helping to classify response or not.
Is there any way to extract varibale importance in SVM ?. If so could you please suggest how to do that. Thanks
You can try feature extraction using the link below before doing svm:
ruclips.net/video/VEBax2WMbEA/видео.html
Dr. Bharatendra Rai Thanks.
Good
thx
Thanks again, sir! please upload the R file sir.
hi how to work with high frequency data with SVM, thanks
From high frequency data you can extract features and then use svm.
ROC Curve & AUC value Demo should be here
You can find them here: ruclips.net/video/ypO1DPEKYFo/видео.html
Sir I have one question. Why didn't you divide the data into train and test.
Since it was already a part of many videos, I try to focus just on SVM. But you are right, it's always better to partition the dataset.
Error in svm.default(x, y, scale = scale, ..., na.action = na.action) :
Need numeric dependent variable for regression.
why do I always get this error whenever I'm using this formula?
mymodel
What is dependent variable in your data?
Thank you for your response. I also tried the iris data and follow the tutorial, but still got the same error.
Thanks. Why the iris data is not partitioned to train and test in this tutorial?
I did it to keep length of the video small. But data partitioning should be done for all machine learning methods.
@@bkrai Thanks Sir.
welcome!
Hello sir! This was very helpful thank you so much.. Can you please tell me how to split the data into train and test because I didn't understand quite well how you split the data here.. Or if there is a link to w pervious tutorial.. Thank you so much
@@asmam-k7150 You can see ruclips.net/video/RLjSQdcg8AM/видео.html
Every time I try to plot after running the SVM model
> plot(SVM Model name, data = data file name, Y axis variable~X axis variable)
I get this error:
> Error in Summary.factor(c(26L, 20L, 50L, 29L, 33L, 43L, 29L, 9L, 3L, 10L, :
‘min’ not meaningful for factors
How do I correct this error?
Instead of factor, use a numeric variable.
@@bkrai But Dependent variable is binary , so I have to say factor, isnt it? Even in your video, species is factor.
My dataset is multi variable how can i apply svm on it, can u help me??
What do you mean by multi variable? Does it mean more than one variable? If yes, then you should have no problem applying svm.
@@bkrai R is telling me "all arguments must have the same length" how can I solve this problem ?
My data is qualitative it contains all variables are categorical...is svm applicable to my data??
Try random forest.
what is set.seed ?how do we decide set .seed value?
you can choose any number you like. And then you can use that same number when you try to repeat analysis with same results.
That music....kept me awake
😊
tab
Make sure pred and actual have same number of data points.
where can i find your r code ???
Here is the link:
drive.google.com/open?id=0B5W8CO0Gb2GGc1ZZQWhmMmpuWWc
Why this was not divided into test/train?
Here just illustrated how to do SVM in R. But you are 100% correct, if you are applying it to any problem, make sure to split data in test/train.
@@bkrai thank you sir for your response. Also if you could answere, i tried this on pima indian diabetes dataset (very famous); except for sigmoid I coudn't see colored boundaries (+ve and -ve catagory) for any other function and the misclassification error is least for linear, yet the algorithm (your method to find out best function) says that radial is the best one, can you guess what could be happening under the hood?
Sir, please attach R command file.