Very helpful. One thing to note. The real reason we square the residuals is to ensure that that we are not minimizing the sum with positives and negatives. Being on either side of the fit is an error, but if left unchecked a residual of 0.1 and -0.1 would equate to zero when in fact they should be agnostic to which side the error occurred and sum to 0.2 . To fix that we square the values and everything is positive, thus finding the "least squares" gives us the best fit. I don't know if that is what you were implying in the video, but I did not hear that called out.
I have watched so many videos on how to do this and this is the first one that actually made sense and where I actually understood what components went where. Amazing! Thank you so much!
I'm here from July 2023 and I'd like to say that this is EXACTLY how neural networks are trained. I watched Andrej Karpathy's full coding of ChatGPT and this is exactly what he did there. We have a certain expected output from the network. But the real output is different. We measure the difference i.e. residual i.e. error and work on minimizing that. That's literally all there is to it. The code is just there to make this process easier and find each parameter for billions of neurons.
Yeah, a lot of people think machine learning and AI are more sophisticated than they really are. If you are interested in materials informatics I have a whole playlist from a course that I teach here at the University of Utah. Check it out
@@TaylorSparks Wow, didn't expect you to reply so quickly. Or at all 😂 Thanks for the video. It really helped me out. I'll check out your other stuff too when I get the chance 😃
bruh this explanation goes hard it was so clear and concise, I've tried to read how to use solver from other sources but they were always convoluted and not too useful. this video in contrast was good and efficient thank you!
Thank you so much! Do me a favor and check out our podcast Materialism. You can find it on any platform and we would love to get an iTunes review if you've got a second. ;)
You made it very easy. In order to fit a model, bring any equation and very the parameters then go to Solver to minimise the sum of the squared Residuals (SSR) and this gives the best values for parameters.
Great job leading us through this so smoothly. I suspect that most people coming here are like me and know what they want Excel to do but had no idea it could do it. It took me about 60 seconds to learn how to use solver from your video, as opposed to never on my own. Thanks!
Hi Taylor, thank you for such precise explanation. I want to know how can we find uncertainty in fitting parameters and eventually the goodness of the fit?
Thank you for the clear introduction! Like another commenter I am trying to fit dose response curves which all have starts of 0, and it goes very odd, but you've got me further in a few minutes than I expected and opened my eyes to what's possible in excel (I don't get coding, despite trying to learn).
Thank you for mentioning that the Solver needs to have a initial best fit that is somewhat close to the existing data! I tried this method after watching another video, the author failed to mention this and I was very frustrated after spending over an hour going thorough my excel sheet looking for problems that were not there.
Great explanation! Clear and straight to the point. However, the main problem with fitting curve is to find the potential fitting equation to work on :)
Thank you for you comprehensive video. I have a simple question, How can we define the error or standard deviation from the fitting constant A, k, n ?? e.g. A+/- something, k +/- something and so on.
@@TaylorSparks I am looking for parameter estimation for a system of ODE direclty without finding its solution. I would be grateful if you suggest anything to me.
Thank you for the clear explanation. May I know, how to solve the linearization if the question Y=a exp (-kt) + b? Will it be ln (Y-b) = ln a - kt? and how to solve in excel when we only have data for Y and t and need to find the constant value, a, b and k?
Thank you for the explanation! Which model is better to use if I need to fix one curve of catalytic descomposition? It seems like the curve of your video. Can I use the same Avrami equation modified?
Thank you so much for this video ! It helped solved a problem I've been working on since Friday. Just one quick question, do you have a source to find the type of equation to use or is it emprical knowledge ? For your case I would've tried a sigmoid function but obviously I would have never gotten the fit you have.
Unfortunately that's where I rely on domain knowledge. There are other routes, such as symbolic regression that can try to find generic shapes but I think domain knowledge is best.
Thank you for this clear and succinct explanation!! So, to be clear, is y = A x 1-exp(-kt^n)) the same as writing y = A x 1-(e^-kt^n))? I hope that makes sense...
Thanks Taylor for the video. The solver in excel i understood well from your explanation. Can you please elaborate more that how i can start with the Y_fit equation just by looking at my data set ??? i mean you choose the exponential equation for your data set ???
Ooh, that's the hardest part! Unfortunately there's no easy shortcut for this. Technically, you can try symbolic regression to try many functions until it finds one that fits well, but mostly you have to do some math text searching.
Thanks for the video. Would you please explain how to fit the non-linear data when one do not know the equation at all? For this video, you initially had the idea that which equation may fit such curve. My data have strain-hardening behavior, how may I determine which equation may be suitable for my case? Thanks
Absolute legend! I manage to model the behaviour of nitinol using a similar equation, but is there a way of taking the SSR value and converting it to something like R-squared?
Hi, thank you so much for your video, the explanation is excellent. If I may ask, In R software there is no solver function (If I am not wrong), you would know a function that would do this work just good as solver in R?
Thank you sir! I now have an answer for my drill T^T. I would like to ask if there is anyway I could check the consistency of the computed constants on the given data? Will I just substitute it on the equation and check if they have the same y? Or do I need correlation?
You mention in the beginning of the video that Python would be better software to use to do this kind of fitting. I liked this video and it was helpful but I would like to see you do tutorial on how to do the same thing in Python.
Thank you very much for a very nice explanation. I understand the whole thing but the constant value. I mean, how do we can decide the best constant value? If anyone can explain it to me, I would appreciate it!!
Very helpful. One thing to note. The real reason we square the residuals is to ensure that that we are not minimizing the sum with positives and negatives. Being on either side of the fit is an error, but if left unchecked a residual of 0.1 and -0.1 would equate to zero when in fact they should be agnostic to which side the error occurred and sum to 0.2 . To fix that we square the values and everything is positive, thus finding the "least squares" gives us the best fit. I don't know if that is what you were implying in the video, but I did not hear that called out.
You are absolutely correct. Thanks for pointing that out.
As a researcher myself investigating heterogeneous behavior, this short tutorial was extremely helpful. Brilliant explanation, Subscribed!
Thank you!! Let me know what else would be helpful!
@@TaylorSparks Is there any other way of further optimising the solvers fit within excel?
@@michealcairns4971 I'm sure there is. I'm not an expert in using Excel's solver though.
I have watched so many videos on how to do this and this is the first one that actually made sense and where I actually understood what components went where. Amazing! Thank you so much!
Feedback like this makes me so happy.
This is amazingly helpful! I watched numerous videos trying to do logistic regression fitting and this is the only one that helped me!
So glad to help!
Same happened to me. Thank you.
Same for me
Great explanation. Straight to the point and without missing any necessary details. I'll look at your videos for future questions.
I'm here from July 2023 and I'd like to say that this is EXACTLY how neural networks are trained. I watched Andrej Karpathy's full coding of ChatGPT and this is exactly what he did there. We have a certain expected output from the network. But the real output is different. We measure the difference i.e. residual i.e. error and work on minimizing that. That's literally all there is to it. The code is just there to make this process easier and find each parameter for billions of neurons.
Yeah, a lot of people think machine learning and AI are more sophisticated than they really are. If you are interested in materials informatics I have a whole playlist from a course that I teach here at the University of Utah. Check it out
@@TaylorSparks Wow, didn't expect you to reply so quickly. Or at all 😂 Thanks for the video. It really helped me out. I'll check out your other stuff too when I get the chance 😃
all these years I've always used Matlab or Python... Often find myself in Excel... this will work well. Thank you!
bruh this explanation goes hard it was so clear and concise, I've tried to read how to use solver from other sources but they were always convoluted and not too useful. this video in contrast was good and efficient thank you!
I gotchooo boo
Hey, Taylor! Just wanted to thank for your all work, from material science to statistics!!!
Thank you so much! Do me a favor and check out our podcast Materialism. You can find it on any platform and we would love to get an iTunes review if you've got a second. ;)
You made it very easy. In order to fit a model, bring any equation and very the parameters then go to Solver to minimise the sum of the squared Residuals (SSR) and this gives the best values for parameters.
You explained it in 6 minutes better than my docent in 2 90min sessions, thank you so much!!
Glad to help dude!
Great job leading us through this so smoothly. I suspect that most people coming here are like me and know what they want Excel to do but had no idea it could do it. It took me about 60 seconds to learn how to use solver from your video, as opposed to never on my own. Thanks!
I'm so glad it was helpful!
Thanks, you literally saved me trying to interpolate a sine wave to data that looked like a sine wave. Again thanks!!!!!
My pleasure!!
Dude, helped me out so much, thought I had to redo all of my measurements for a uni project. Very well done!
My pleasure! Give the video a like, sub and share and it will help me keep making content like this
ruclips.net/video/-rW0q11JMag/видео.html
Explain everything in very easy manner... Nice job
Heck ya
i LOVE THE VIDEO, I ASKED MANY PEOPLE ABOUT IT, BUT NONE COULD EXPLAIN ME BETTER. Thank you
Glad to help!!
Great video. There is a lot of people out there doing super complex explanations for this, but this one is perfect. Thanks a lot.
Thanks dude!!
Short, direct and extremely helpful
Glad it was useful!
Thank You, Taylor. I was trying to fit using MATLAB. However, your video is awesome
Super glad to help. Thanks for tuning in.
a good solution to find the equations for solving "the area of hysteresis" problem. Thank you so much
Great point. Very happy to help.
I never knew about plot digitizer before watching your video and going through the description
It's so great
Hi Taylor, thank you for such precise explanation. I want to know how can we find uncertainty in fitting parameters and eventually the goodness of the fit?
This is one of the most beautiful video on youtube. God! It's so good.. Thank you!
Lol. Glad to help friend!
You’re amazing boyyyy!!! You just solved one of my chemical engineering project .Thanks
Glad to help!
hi fellow chemical engineer
My english listening is very very bad, but it was no a problem for I reproduce your procedure, excellent, thank you so much bro
Super glad to help!
Thank you! I just started a statistics class and I did not understand the residual until I saw it happen, visually, on your graph.
This is the best video that explained the regression fitting. Thank you!
🙌🙌🙌
Absolutely excellent, enjoyable, clear and precise. Well done sir!
Thank you so much. I'm super glad to help
Thank you. Saving a students life rn.
ruclips.net/video/-rW0q11JMag/видео.html
Thank you for the clear introduction! Like another commenter I am trying to fit dose response curves which all have starts of 0, and it goes very odd, but you've got me further in a few minutes than I expected and opened my eyes to what's possible in excel (I don't get coding, despite trying to learn).
Thank you for mentioning that the Solver needs to have a initial best fit that is somewhat close to the existing data! I tried this method after watching another video, the author failed to mention this and I was very frustrated after spending over an hour going thorough my excel sheet looking for problems that were not there.
Right? Makes a big difference
Amazing video, love your work, saved my chemistry degree xxxx
Woohoo!! This feedback makes me so happy. You might like our materials science podcast "Materialism." Drop us a review if you can.
Thank you so much. It helped me a lot. Your explanations are simple and very efficient.
Thank you for help.. I tried to obtain wavefunction (800-data). Result: Great!! thanks 🎉🎉
I have one word. Beautiful video. Thanks
Heck ya!
Perfect video! Thanks for your great instruction
@@roykeyer2174 Happy to help!
Awesome video! Helping me with my studies. Thanks a lot!
Absolutely awesome video. Clear and to the point. Great work mate.
Great explanation! Clear and straight to the point. However, the main problem with fitting curve is to find the potential fitting equation to work on :)
Ya. There's no substitute to some basic mathematics. They really are helpful when getting an equation of the right form.
Excellent video - I now use the solver option a great deal in my work. Thank you.
Hi Taylor, this video is very helpful, thank you. Just have a quick question, for inhibitory non-linear response curve, what equation should be used?
this make me more understand about sigmoid value. Thank you!!!
Heck ya homie!
Thank you for you comprehensive video. I have a simple question, How can we define the error or standard deviation from the fitting constant A, k, n ?? e.g. A+/- something, k +/- something and so on.
thank you so much, Sparks. now I can defend my thesis!
wow! I'm so glad to help! Good luck on your defense.
Wow.... you teach me what i really want. Thanks a lot!!!
That was a simple and straightforward explanation. Can you say how did you assume the equation?
From domain knowledge. I've seen that shape before so I figured it would work.
Thank you 🙏 solve my question within 6 min lol. I’m trying to do my homework for mass transfer and needed this
Pleasure to serve!
Its great video. Very short and only informative. Thanks
Glad to help!!
@@TaylorSparks I am looking for parameter estimation for a system of ODE direclty without finding its solution. I would be grateful if you suggest anything to me.
Thank you. Great Explanation. Very helpful
This is really helpful.Thank you so much.😃😃
Really helpful video I used this in conjunction with Gompertz forecasting equation and found the optimal constants for forecasting shipment nos
Glad it helped!!
THANKS! Your explanation was super clear!
Glad to help!
very helpful,keep uploading this type of content
Got a specific need?
This helped me A LOT! Thank you!
Glad to help!
Helped me a lot, thanks
Mr. Sparks thank you very much!! You did me a big favor!!!!!!!!
oh oh awesome, I'm always glad to help. Do me a favor and subscribe like and share. I'm trying to grow this channel so that more people can find it.
Wonderfull video!!! Very precise and usefull information. I want to try doing this using Origin or R, but I need a video like this.
ruclips.net/video/-rW0q11JMag/видео.html
good information. Goes pretty fast. I can't quite get my S shape curves to fit close enough. Any suggestions?
Thank you for the clear explanation.
May I know, how to solve the linearization if the question Y=a exp (-kt) + b? Will it be ln (Y-b) = ln a - kt? and how to solve in excel when we only have data for Y and t and need to find the constant value, a, b and k?
Thanks, very useful and quick video.
Now this is magic! Thanks alot, very helpful!
Glad to help! Please like subscribe and share!
Thank you for the explanation! Which model is better to use if I need to fix one curve of catalytic descomposition? It seems like the curve of your video. Can I use the same Avrami equation modified?
Sure, give it a try and see how it fits. You can calculate the r squared if you like.
This was really useful. If i've calculated the fit how doi calculate the upper and lower limits at 95 % confidence level in excel?
Great explanation but I wounder if I got different trend which equation should be used or same yours?
It totally depends on your trend. What does it look like?
Thank you so much for this video ! It helped solved a problem I've been working on since Friday. Just one quick question, do you have a source to find the type of equation to use or is it emprical knowledge ? For your case I would've tried a sigmoid function but obviously I would have never gotten the fit you have.
Unfortunately that's where I rely on domain knowledge. There are other routes, such as symbolic regression that can try to find generic shapes but I think domain knowledge is best.
Thank you Taylor for sharing this very important video
Glad to help! Sub to see more content.
Greatest curve fitting vedio.. Thank you. 🌹 🌹
Why thank you!
@@TaylorSparks you did great jobe. You deserve a big thanks. 🌹
Wow thank you so much...i now have a clear understanding of this work
Well explained. Thank you.
💪🏻💪🏻💪🏻
It was very helpful, thank you
Thanks. I am using this for my thesis regression analysis!!!
Thank you for this clear and succinct explanation!! So, to be clear, is y = A x 1-exp(-kt^n)) the same as writing y = A x 1-(e^-kt^n))? I hope that makes sense...
That was a fantastic video. Great presentation. Thank you
So glad that it was helpful! Let me know if I can help with another topic. I also appreciate if you can like and subscribe to help me grow my channel.
You are a legend.
Fantastic video and explanation. Please let me know if you are familiar with other functions similar to Avrami without an upper bound. Thanks!
Just exp(x)?
Thanks Taylor for the video. The solver in excel i understood well from your explanation. Can you please elaborate more that how i can start with the Y_fit equation just by looking at my data set ??? i mean you choose the exponential equation for your data set ???
Ooh, that's the hardest part! Unfortunately there's no easy shortcut for this. Technically, you can try symbolic regression to try many functions until it finds one that fits well, but mostly you have to do some math text searching.
Thanks for the video. Would you please explain how to fit the non-linear data when one do not know the equation at all? For this video, you initially had the idea that which equation may fit such curve. My data have strain-hardening behavior, how may I determine which equation may be suitable for my case? Thanks
Thank you, that was really helpful.
This video helped me so much thank you man
wow very helpful demo, thank you!
very helpful, thank you sir!
THANKS! Your explanation is awesome! It is helpful!
Very glad to help
Thanks a lot, very helpful. I got it
Very helpful. How would this apply to a stepped y plot? For example X would be 1-10, Y 15%,15%,30%,30%,60%,60%,75%,75%,90%,100%
Yes! Thank you! I will be using this for my research as well as for my own educational purposes of modeling covid-19 confirmed cases
That's wonderful!
Got here for the same reason. Thank you, Taylor!
Reported covid19 cases are forged...
Jannie Schlüter Wear an aluminum foil hat. Problem solved.
@@tecnocato you already wearing it. Nothing gets through your thick melon. 🙉
Hi Taylor, great video. How would the equation change if the curve fitting is not an S-curve (as in your video) but something like a reverse S-curve?
Do negative of expression and then plus 1?
well explained , Thank you
Love the way you explained. Very clear. Thank you so much,
Glad it helped!
Excellent!
Thank you so much, great explanation.
Thanks Taylor .
That's very helpful:)
Absolute legend! I manage to model the behaviour of nitinol using a similar equation, but is there a way of taking the SSR value and converting it to something like R-squared?
Sadly, no. R squared is not valid for nonlinear models. statisticsbyjim.com/regression/r-squared-invalid-nonlinear-regression/
Great video
Hi, thank you so much for your video, the explanation is excellent. If I may ask, In R software there is no solver function (If I am not wrong), you would know a function that would do this work just good as solver in R?
Unfortunately, I'm not as familiar with the solver function in R, I mostly use python
Thank you sir! I now have an answer for my drill T^T. I would like to ask if there is anyway I could check the consistency of the computed constants on the given data? Will I just substitute it on the equation and check if they have the same y? Or do I need correlation?
Nope, plug them in. The sum squared residual can generate the r^2 correlation too
You mention in the beginning of the video that Python would be better software to use to do this kind of fitting. I liked this video and it was helpful but I would like to see you do tutorial on how to do the same thing in Python.
ruclips.net/video/HWwLZP14DG4/видео.html
@@TaylorSparks Oh perfect, thank you very much!
This was so helpful! Thank you!
Thank you sir
Nice explanation
Very helpfull tutorial, thanks!
ruclips.net/video/-rW0q11JMag/видео.html
Thank you very much for a very nice explanation. I understand the whole thing but the constant value. I mean, how do we can decide the best constant value? If anyone can explain it to me, I would appreciate it!!