Видео

Thank you. To your question: If you are using the dual formulation, you are optimising the lambdas. In the cost function are only lambdas. Then, you can directly calculate the weights with w=sum lambda*x*y

The subsricpts i and j represent summations over the N data points to train the model.

Hi. This is pretty diffecult to answer, especially without more detailed knowledge about your data. But typical values are 0.1, 1.0, 10.0. Best regards Christian

Thank you for your nice comment. I hope to find time to make new videos, but the lectutes are beside my daily job just for fun...and fun time is at the momente rare ;-) But to your question: After my phd in physics, i started a job as a Data Scientist in the industry. I wanted to learn all the ML basics and read some books. To reflect my knowledge, as a side project, i wrote a lecture on ML for my former university. I hold the lecture and realized that i missed a lot of explanations and proofs. And, as you also mentioned, for some of these it is diffecult to find something in the internet. But often this does not matter (if you have some knowledge about mathematical proofs - i took a lot of math and physics courses at university). Start with looking up how a statement like "convex shape" is defined. Then write down all the math formulation of your "assumtions" like cost function and so on. They have to be substituted into the "left side" of the statement. Now, of course, the tricky part is to calculate that the "right side" of the statement shows up. This needs some experience you might become by studying math, physics or something similar. But I think most important is, to start by writing the first steps down on paper and not geting afraid that this problem is to diffecult or so. Hope this gives you a quick insight... Best regards

Hi, sorry, right kow, there is no website where you can download the slides. If they are available, i give a note here on RUclips.

Thank you for your comment and question. I can stongly recommend among other the books "Hands-On Machine Learning with Scikit-Learn and Tensorflow" by Aurelien Geron, "Deep Learning" by Ian Goodfellow, Yoshua Bengio, Aaron Courville, "The Elements of Statistical Learning" by Travor Hastie, Robert Tibshirani, Jerome Friedman. The first book gives a nice lightweigt introduction with Python implementations whereas the latter show a more mathematical-based approach to machine learning. Besides these books I recommend the videos by Andrew Ng ruclips.net/user/Deeplearningai and Patrick Winston ruclips.net/user/mitocw . Best regards Christian

Thanks for your question/comment. Yes, in general, you can either modify the loss function or the classification-function to obtain a convex optimization task for classification. Logistic regession make use of the sigmoid function for solving classification tasks. It is a very common used function for classification. But indeed, you can use different functions for classification as well. For example, a trival choice is the linear function which gives you a convex optimization problem. However, this function has problems when is comes to outliers (ruclips.net/video/9fTHqy51N_o/видео.html). Because the logistic function has nice properties when it comes to classification, the easy way is to modify the loss function.

Thanks for your question. Yes, Support Vector Regression (SVR) does not separate groups in your data. SVR fits a curve to the data points. And this curve is used for a regression problem to predict a concrete numeric value for new data. To find the best curve, margin boundaries are specified which include all data points (at least for hard margins). The curve for the predictions is then in the middle of the margin boundaries. The curve is linear for the linear SVR and can be non-linear if the kernel trick is applied (see: ruclips.net/video/6fqONx4mMI8/видео.html). In contrast, a Support Vector Machine (SVM) used for classification (also SVC) separates two groups of different classes by a curve. Then, new data is classified with respect to this curve. I hope this explanation helps. If you still have questions, please dont hesitate to ask.

Thanks for your comment. I agree, a two sided parametric ReLU is a good choice. You have to find a tradeoff between incorporating non-linearity and at the same time gradients of approximately 1. Most often I prefer LeakyReLU, but more different slopes can be advantagous depending on the use case.

Thanks for your question and sorry for the late response. Here is a concrete example for one vs. one classification: Let's assume we have 4 classes A, B, C, and D. 1.) First, one-hot encoding is applied to the data to obtain a vector representation, i.e., we have 4 variables in a vector which can be 0 or 1. Then, the test vector has 4 entries and each entry can be either 0 or 1. For example, a test vector for class A is (1,0,0,0). 2.) Let's consider a logistic regression model. For the 4 different classes, we have the following logistic regression models: (1) a model for A or B; (2) a model for B or C; (3) a model for C or D; (4) a model for A or C; (5) a model for A or D; (6) a model for B or C. Each model outputs typtically a probability value between 0 and 1 to decide between the two classes. For example, model (1) outputs the probability for class B and the inverse is the probability for class A. So, if the model outputs 0.8, this model would prefer class B over class A. 3.) Application of the models: assume an input which corresponds to a test vector output with class A. If all models are well trained, we excpect model (1), (4), and (5) to vote for class A. So class A has a score of +3. Model (2), (3), and (6) vote randomly for the other classes. However, non of these classes can have a score geather than 2 (because non of the 3 classes B, C, D is in all 3 models). So, in total, the one vs. one approach selects class A. I hope this explanations helps. If there are still questions, please do not hesitate to ask.

@@amile-machinelearningwithc4547 Thanks a lot