Great video, and thank you for sharing your work. I am wondering if you can provide a definition of a Left suspension. Is it the same as a truncation? I am trying to determine weibull parameters for a population of assets where failure records do not exist prior to a certain year. Thank you
In reliability, where "operating age at failure" is modeled by a random variable, a Left Suspension would be: examining the asset and discovering that it has ALREADY FAILED, but you don't know when the failure happened. You only know that it happened sometime between age = zero, and age = present-age. This scenario happens frequently in complex machines. The machine is disassembled, or examined, and some component within the machine is found to have "failed", but it may not be clear when the failure actually occurred -- unless of course, the failed component CAUSED the machine to fail, and you know the age when this happened. A "truncation" is normally a Right Suspension. Again, in reliability, if an asset is pulled out of service BEFORE it fails, but for some reason it is never returned to the population (i.e. to operation) to experience failure, then that asset counts as a truncated data point, and serves as a Right Suspension in the parameter estimation process. To complete the picture, an Interval Suspension is a failure that occurs at some unknown time BETWEEN two ages. Suppose a complex machine is disassembled at 100 operating-hours, and component X is examined and seen to be fine. Then at 200 operating-hours, the machine is disassembled again, only this time component X is examined and found to have failed. Since you don't know when the component failed, only that it failed between 100 and 200 hours, its failure is an interval suspension. Turns out, left and right suspensions are actually only specific cases of the more general Interval suspension. In a left suspension, the left-end of the interval is time = zero. In a right suspension, the right-end of the interval is time = infinity. In your case, where no failure data exists before a certain point in time, I'm not sure there is any data there to harvest. But that's the question to ask yourself: "Is there ANYTHING I know about the assets operating before failure data was kept?" If you know they succeeded (and you don't end up double counting them because they operated into the time where failure data is kept), you MIGHT have some good suspension data.
I just want to say thanks for posting this and sharing your spreadsheet. I had never heard of MLE until a couple of weeks ago, and your explanations as well as my digging into the details of your spreadsheet really helped me put it all together, and it's likely I will expand what you've done for reliability engineering work in the future. Good stuff. I also appreciate you keeping up with the comments as I also didn't understand the purpose of epsilon until you explained it. While I thought that's what you were doing, I wasn't 100% sure.
Thanks for your kind words. I'm interested in your expanded work, if you don't mind sharing it. Where I worked before, "rank regression" was the more common method of parameter estimation when using a Weibull distribution. While the rank regression method isn't necessarily difficult, I just found the MLE method to be much more elegant and intuitive, especially because it can be used with ANY distribution (as long as you can form the PDF and CDF via a formula). This feature allowed me to experiment with alternative distributions, and I ultimately settled on a 5-parameter distribution for modeling most jet-engine reliability. A 2 or 3-parameter Weibull just could not adequately model the infant mortality we frequently saw in reality. Check out this link: www.weibull.com/hotwire/issue16/relbasics16.htm for some validation of my leanings. I actually had not done the comparisons they have, so I'm glad to see someone has put in the work to compare the methods.
Hi David, thank you very much for posting this. I wonder where can I add interval censor for failures? It seems that I could only add interval censor for suspensions in interval suspension right and left column. Thank you.
Sorry, I don't know what "interval censor" means. I suspect you *may* be using this term the same way I use "Interval Suspension". If so, then please see my response to Kevin Taylor where I discuss this. The spreadsheet is actually setup to input interval suspensions. Take a look at my response to Kevin Taylor and see if I'm addressing your situation. If not, please explain you situation more precisely, with examples, and maybe I, or someone else can figure out how to modify the spreadsheet for your situation. OK, so I looked up "interval censored data", and it appears to be exactly what I described as "interval suspensions". I'm not sure why the spreadsheet can't be used with your data. Again, please explain (after reading the response to Kevin Taylor) why it can't be used with your data.
Nice template. I do have a question though. Why didn't you use the solver to just maximize log likelihood by changing alpha and beta? In that case, you didn't have to compute the derivative of loglikelihood and hence could have done away with the epsilon parameter in the likelihood column. It would have made the template less complicated. But hey, that's just me .
It's not just you. I spoke about it briefly when replying to Long Nguyen. But, in full disclosure, I only learned of the free availability of solver after I had published the video. But when I applied solver to finding best parameters, it seemed like it wasn't quite as robust as the more brutish technique I used. Nevertheless, as you point out, it would have made the template MUCH simpler. Also, as a teaching device, I believe there is utility in NOT hiding too many of the details. Solver is like magic. I am confident that most undergrads will have absolutely no clue how solver is doing its magic in the background. The approach I used will (or should) help students understand how to solve this type of problem, and be able to apply the concepts to other problems. Or, maybe they should just use solver on all other problems! ;-) Thanks for your input.
I commend the academic purpose behind the method that you have employed in the video. You say that solver is not as robust as goal seek. Aren't these using the same kind of optimization technique? I tried solver with your dataset and I got the same set of parameters as you did. Is there any way that you could share the dataset that you used when you got different results? But it was really informative. Whenever i wanted to estimate parameters for non-linear regression, I always went for R or python, but not once thought of the ubiquitous MS Excel. I will definitely try this method next time to verify my results. Great work, Sir!!! (y)
Thanks for the compliment. No, Solver gives the same results, as I recall: it's been a while since I did the comparison. The issue I was having was that the starting points had to be closer to the correct answer when using Solver. Otherwise, Solver was more inclined wander off to infinity than my approach was. Again, I refer to my previous responses to Emiel van Maurik, and Long Nguyen. Yes, I imagine Solver uses a modified Newton's method, just as I imagine Goal-Seek does. I mean, other than guessing at a solution (like the evolution method), what other way is there to hone in on the zeros of a function. The references I've studied for solving non-linear vector functions have been consistent in this regard. You (numerically) form the Jacobian at some starting point in the solution space, invert it, and solve for the vector of differences between the starting point, and the zeros. If the function is perfectly linear within the region, that vector of differences nails the solution. If not linear, then it over- or under-shoots the zeros. Just like with Newton's method. So, while Solver is inherently vector (and must use the Jacobian), Goal-Seek is inherently scalar, where the Jacobian reduces to just the derivative of the function. But I'm getting way far off from the MLE topic... Perhaps some other video addresses the pros/cons of using Solver as a vector solver, versus using Goal-Seek in a round-robin, scalar-solver approach, like I did it. But definitely, thanks for asking. Your question caused me to revisit and reconsider some things. Thanks again.
Hi David, Thanks a lot for your tutorial and further explanation here. Still I have questions of understanding your Excel spreadsheet, I hope you can help me with that. First I will explain what I am working on for better understanding. I'm also doing a project on failure data and what would be the optimal interval for replacing a component. This could be from any component, so any distribution could be possible. Therefore I was asked to create an Excel spreadsheet just like you did. In this sheet, users could then insert their own data and determine easily step by step which distributions fits the data best (Normal/Exponential/Weibull). Then they should get a MTBF with a margin of error at a chosen confidence interval. This was not the biggest problem. I was also asked to take censored data into account, again just like you did. Unfortunately, this was the point where I got stuck. I analyzed your Excel spreadsheet first and I have various questions. In the first place I am a bit confused on your epsilon value. What it exactly means and how you use it in your eventual calculation. I only want to use the MLE since I mostly consider having random data. Then I also do not understand the 'Param #' cell, what happens when making this choice and why do you inserted 5 parameters. Could you easily add one if you want? Since it is also possible to use the exponential distribution for for example Electronic parts, did you also made such an Excel spreadsheet for that. Finally I was trying to insert a macro just like you did but It failed, unfortunately I have never worked with macro's. I still have till June 17 to finish my Excel spreadsheet. Hope you can help me. If you want I can send you my spreadsheet so you can see what I have done. Emiel
The epsilon value is internal to the solver mechanism. You don't really have to mess with it. I spent some time in the video explaining how it worked, but understanding it is not really necessary to performing an MLE analysis. You just have to have SOME mechanism for finding the parameters that maximize the "Likelihood" scalar value. I initially kind-of "maximized" it (in the video) by GUESSING at the parameters and trying to do a best-fit by visually comparing the data-CDF curve with the computed CDF curve. You can do that too. It's just not very elegant to do it that way, and it's very difficult to hone in on the BEST value(s) when doing it with guess-work. I set up the spreadsheet to use the "Goal Seek" function, and as such, had to have a value that indicated "maximum". This is the reason for forming the derivative value: "d(Log_Likelihood)/d(parameter)" Recall that maximums or minimums are found by setting the derivative to zero. The spreadsheet numerically computes the derivative via the "epsilon" terms, and the GoalSeek attempts to find values of the parameters that make that derivative value zero. It would have been simpler to just use the "Solver" add-in routine since you can just ask Solver to look for value-sets (parameters) that maximize a given scalar value (the likelihood value). Using Solver would allow us to skip all that epsilon stuff, and forming the derivative manually. You just tell solver which value you want maximized, which values (the distribution parameters) to change to do it, and hit the go button. It takes care of everything else. I spoke of this briefly when answering a question by Long Nguyen. As I noted to him, in my experience, Solver tends to wander off into never-never land more often than the approach I used. I setup the spreadsheet to use 5 parameters because the distribution I was experimenting with had 5 parameters in it: 2 parameters for the "wear-out" failure mode, and 3 parameters for the "infant mortality" failure mode. I wrote an article on the distribution but never published it. If I get around to publishing it, I'll come back here and put a link to it.
Hi, I used LSM to find shape parameter then used gamma function and Maximum likelihood method to find scale parameter. I want to know whether my approach is correct and reliable. Seems like I'm getting the closet values to what I supposed to get. Thank you.. Will be waiting for your reply.
+Lizy Ae: I can't really understand your question. Probably best to just ask your professor. He or she can see exactly what you're doing, and can help guide you.
Hello David. So my average value is in negative value. I can't execute Weibull's Distribution. Can you please tell me what's the alternative to Weibull's?
No, not without seeing the data, or understanding the physical meaning of the data. Sorry, I don't know how to recommend any distribution without those things. But you can perhaps do it yourself by first plotting a histogram of the data using the technique I used beginning at about minute (3:20) of the video running through minute (4:15). Once you have the histogram in front of you, compare the shape of it to any of the standard shapes like normal, log-normal, gamma, Weibull, beta, etc. If the shape looks similar to one of the standard shapes, it might fit if you can transform the independent variable (let's call it "time" so I don't have to keep typing "independent variable") in some way. For example, you say that you have time-values that are less than zero, which immediately rules out a gamma or Weibull curve. But what if, instead of computing the PDF or CDF of time, you computed PDF(-1*time). Does that make all values positive? If not, would a shift on the x-axis do it. That is, if you just added a single constant to every value of time, would that make them all positive and begin the curve at time=zero. You'll just have to be creative and try different things. Hope this helps.
+Marcello Cusma The data can NOTIONALLY represent "time to failure". Since I work in reliability, that makes the most sense to me, but the data could be from any process modeled by a random variable. For example, it could be the length of molecules grown in a polymerization process. Or it could be the weight of items received at a shipping node. The data I used in the video was not actual data taken from any real thing. I generated the data sets using random-number generation techniques, and Excel. Not all data sets will have suspensions. In reliability, we get all the data-suspension variants: left-suspension, interval-suspension, and right-suspension. That is why it made sense to me to demonstrate the MLE method using reliability-type data. In the video, I only demonstrated analyzing a data-set that included right-suspensions, but the technique works for all suspensions. Finally, let me repeat: the data in the video notionally represents TIME TO FAILURE of each item, NOT MTBF. MTBF is one single parameter that you would use to characterize a set of time-to-failure data. It only gives the mean (or average) value of the data set, and says nothing about the "spread" of the data.Hope this helps.
Thank you a lot, David. I was going to use your spreadsheet, but I couldn't. I just replace my data in column A (Failures). I used Normal data to see how your macro works. I already knew that mean and Standard Deviation are 38.8 and 11.4 respectively. But after click on (MLE Estimate Parameters) I got following results for mean and SD: Mean: 1327.8 SD: 1086.82 My data are: 26, 33, 65, 28, 34, 55, 25, 44, 50, 36, 26, 37, 43, 62, 35, 38, 45, 32, 28, 34 Why does this happen?
Like virtually all solving routines (and the built-in Excel solver is no different in this regard), it needs a reasonable starting point. My suspicion is that you only replaced the failure data (and possibly only the first screen-full) then hit the ".. Estimate" button. You need to make sure all the suspension data is cleared out of column E, and all the left-over data is cleared from column A. Then you need to put REASONABLE starting point values into the estimates of mean and SD in cells T4 and U4. If you don't, it'll begin with the mean and SD from the original example data, which is way off, and probably won't converge to the right answer. The reason this can happen is that the solver routine "falls into a dimple" on the solution landscape. That is, in well-behaved problems, the true answer lies at the bottom of a giant canyon, or a crater (or in our case, since we're seeking a maximum, at the peak of a large mountain). If you begin looking for that maximum or minimum far enough away from the true peak or outside of the crater, the solver gets tricked into falling into a small dimple, or resting on top of a mole-hill instead of the top of the mountain. These are called "local" maxima or minima, and they're NOT what you want. But, start off close enough to the mountain (or inside the crater), and the solver always recognizes which way is up, and always climbs to the "global" maxima. Just to test it, I tried just clearing out the original values from columns A & E, inputting your data, then pressing the button. Sure-enough, I got garbage answers. Then I tried putting in reasonable starting-point-estimates into cells T4 and U4, (I used 50 and 5) and it quite quickly gave the expected results of 38.8 and 11.4. How far off your starting points can be from the correct values is related to how well-behaved your function is, and to some degree, on your data. For the set you provided, when I put in a value of 20 for SD, the process diverged and ran off to some point way in the distance (414 million). So, apparently, a good starting point for the SD is pretty important! On the flip-side, I tried it with starting points as low as 0.7, and it still worked. Lower than 0.7, however, caused the macro to crash. Hope this helps.
Thanks. It definitely helped. Yes, the starting point for SD is very important, but it seems the starting point for mean has less sensitivity. I have another question. I have 300 data and I know the normal distribution will be fit on them. Is it reasonable to find the distribution parameters (mean and SD) from MLE, or It would be better if I calculate them based on data values (I mean calculate mean and SD using data based on their formula)? In fact I was wondering if there is any advantageous in obtaining distribution parameters using MLE when you have the values of the distribution? For instance, about previous example, My data are: 26, 33, 65, 28, 34, 55, 25, 44, 50, 36, 26, 37, 43, 62, 35, 38, 45, 32, 28, 34. Is it reasonable to obtain the parameters of normal distribution by using MLE?
No, I think using MLE is overkill in cases like yours, where you know the population is Normal, and you don't have any suspensions. In these cases, just calculating the mean and standard deviation by text-book formulas is DEFINITELY the way to go. But, how do you estimate the parameters if there are suspensions? Maybe there are ways, I just don't know what they are. Or, what about if you are working with distributions that don't have closed formulas for the parameters? These are the cases where the MLE can become very helpful.
Dear David, I was wondering if MLE is efficient for a small dataset (6-7 data points). If it's not, which method is more suitable for fitting distribution to such a small dataset. I can give you these data points as follows: 6.8, 13.9, 14.6, 16, 17.5, 18.6, 17 Your advice would be much appreciated. Thank you. Regards,
"Efficiency" is really not the right question to ask. The questions you need to ask yourself are: 1) Does the MLE method produce a good and "unbiased" estimate for the parameter in question? 2) How confident do I need to be in my parameter estimates? I didn't address either of these issues in the video, nor in any prior question responses. If you are unfamiliar with "bias", think about the way we compute standard deviation. We know that the definition of standard deviation ends with a division by 'N': the number of data-points in the set (population). See en.wikipedia.org/wiki/Standard_deviation#Basic_examples. BUT, if we are using the conventional formula to estimate the *POPULATION* standard deviation, we don't divide by N, but rather by N-1. This is because the standard formula applied to a SAMPLE does in fact give us the standard deviation of the sample, but it gives us a *biased estimate* of the standard deviation of the POPULATION. We need to make that distinction -- the standard deviation of a SAMPLE is NOT the *best* estimate for the standard deviation of the POPULATION (assuming a normally distributed population). Turns out, we get the same behavior with MLE for standard deviation (sigma) for a normal distribution. The MLE for sigma produces the conventional formula, where the final division is by N, not N-1. Thus, the MLE parameter estimate for sigma for the population is slightly biased. In your case (7 data points), assuming your population is normal and that you are trying to estimate the standard deviation, the MLE parameter will be off about 14% from the unbiased estimate. In much larger data sets the difference between the MLE value and an unbiased estimate may become negligible. This doesn't mean MLE is bad. The article en.wikipedia.org/wiki/Bias_of_an_estimator gives an example of when an MLE estimate is actually better (for Poisson distribution) than the unbiased estimator. It also gives an example (the ticket box example) of when, *FOR A VERY SMALL SAMPLE SIZE*, the MLE estimate is very poor. That last statement gets to the second question: how confident do you need to be in your estimates. This gets to the heart of what much of statistics is all about. That is, given a sample, what is the probability that the sample is a good representation of the population -- that someone can use the sample to make (accurate) statements about the population. It's way too big a subject for me to address here, other than to say that small samples lead to low confidence in your answers (parameter estimates). See en.wikipedia.org/wiki/Confidence_interval Hope this helps.
David Qualls Thank you very much for your thorough response. In my case, the issue was that these 7 data points were not normally distributed. I did not want to assume normal distribution in order to avoid uncertainty. For this reason, I used MLE in the hope of finding the best fitting distribution. I was trying to use Solver add-ins for optimization. Unfortunately, the results turned out pretty disappointing. It's great to know that MLE estimate is poor when it comes to a small sample size. Thanks again for your advice.
Long Nguyen I wasn't trying to say that the MLE method produces poor results when the sample size is small: I was trying to say that it COULD produce poor results -- the confidence in the final answer is LOW. The ticket-box is an example of when the method will likely fail badly for a tiny data set. Regarding using the Solver add-in, yes, I have had similar results. Seems like unless you pick pretty good starting values, solver quickly wanders off into la-la-land. My macro seems to be a little more robust, and give slightly more accurate answers, although it still needs pretty decent starting values. If you know that your data is not normally distributed, try a different distribution. That's the nice thing about the worksheet I provided: you can experiment fairly easily by just replacing all the references to "normal.dist" with "gamma.dist", or "weibull.dist". You'll still need to do a visual fit FIRST, to get an answer that's in the ballpark, before you hit the macro run button, or use solver. Note however, that with only 7 data points, your uncertainty is going to be high no matter what distribution you use. Even if you get perfect correlation (the data CDF line overlays the calculated CDF line perfectly) you STILL have significant uncertainty. How do you KNOW that you drew a sample that perfectly represents your population? You don't! You may have randomly drawn data values that perfectly match a distribution that is NOT the true population distribution. As I've explained elsewhere, it's a matter of likelihood, or probability (kind-of). The more data points you draw, the less likely that you'll get the wrong answer, and thus, the more certain you can be in the parameters you estimate. Hope this helps. Good luck.
Ravi: That is a good question. Unfortunately, I haven't investigated it, and don't know the answer. Perhaps someone else could share a good answer. Please... The question of confidence intervals is extremely important, and one should definitely recognize that the MLE method itself (as detailed in my discussion and the spreadsheet) ONLY provides the MAXIMUM LIKELIHOOD parameter estimates for the provided data-set -- not confidence bounds on those parameter estimates. When dealing with small data-sets, it is important to remember that every addition of a new data-point can move the estimate quite a bit. One possible numerical approach is to remove one of your data-points and perform the MLE analysis. Then replace that point, and remove a different point and repeat the analysis. Do that for each data-point, and you'll have as many parameter estimates as you do data points. From these values you can rank-order them and pick the 5% value and the 95% value as candidates for your (90%) confidence interval. If you have a LOT of data, you would have a lot of estimates, and you could pick the 2.5% and 97.5% values for 95% confidence. This sounds like a macro I'd like to have in my tool-chest, if someone wants to write it for us! Alternately, since the scale and shape parameters should follow the central limit theorem, you should just be able to use the standard formulas for confidence intervals. But, I'm less confident about that answer -- I haven't researched it.
This was pointless. the introduction was good and in the intro you said you would show how MLE works but all you did was to show a macro that you had written. This does nothing to help sanyone understand what MLE does.
Sorry. I can sort-of see what you're saying. I know the title is somewhat misaligned from what is actually in the video. I'll probably try to come up with a better title. The target audience was individuals who had NO experience with parameter estimation, so I went thru a lot of things quite unrelated to MLE proper. And because of that it went way long, so I had to shut it off before I really said what needed to be said. Nevertheless, I do think that the segment beginning at minute 13:45 explains pretty well how to set it up. The macro just uses the goal-seek to individually seek each parameter to an optimum value (derivative close to zero), and loops until both parameters are optimum. You have to loop because each time you switch parameters, it messes up the other parameter. You just keep looping until they're both good. Do you have any suggestions for what would make it better for you? By the way, there are probably better videos already on RUclips for what you want. I haven't viewed any of them. I may do some looking and see if I can point viewers to better videos than mine.
OriginalJoseyWales you are quite rude. The teacher put in effort to do this, there are better ways to have said what you said instead of saying "pointless". I doubt you'd be happy if someone said what you did was rubbish, after taking out time to do it. Just be careful with your words as they can make or break someone. Cheers.
You're pointless OriginalJoseyWales. If you don't know how to open up the macro to discover exactly what this guy shared FOR FREE, then maybe consider not getting into it. It might be over your head. Otherwise, do some pre-read on Mean Time To Failure equations using a Weibull distribution to understand that you need to find the best fitting variables to the equation given a data set which is done through an iterative function to test parameters until you get as close as possible to the values that are the MAXIMUM LIKELIHOOD of the variables. It's not that complicated.
Hi there, Thanks for posting this video - very helpful for beginners! ... I have question... I followed all of the exposé until 'epsilon' was mentioned... Might you be able to explain where it came from / what it's use is? Is it specific to the function you tried modelling?
Marie a.Londres Epsilon is standard notation in calculus. See en.wikipedia.org/wiki/%28%CE%B5,_%CE%B4%29-definition_of_limitI may have used it slightly backward in that I used epsilon to represent a small change in the independent variables (the parameters of the distribution) rather than the dependent variable. Perhaps I should have called it delta instead. But regardless of that, the point is that "epsilon" was a small change in one of the parameters: which parameter was determined by the value (1 through 5) in cell U8 of the spreadsheet. To form a partial-derivative, you vary one of the parameters by a small amount, and note the associated change in the dependent variable (the likelihood function in this case). By looking for the value of the parameter where the partial derivative goes to zero (that is, the place where small changes in the parameter produce no change in the likelihood), we know we're at a flat spot on the curve, or at a local-maximum. With a little luck, we can find a single place (a single set of parameters) where ALL the flat-spots coalesce (that is, all the partial derivatives are zero), and we have indeed found a (local) maximum. With even more luck, that local maximum will be the actual global maximum: the single set of parameters we're looking for. Hope this helps.
Dear David, Thanks so much for your thorough and generous response. I have downloaded the spreadsheet and will continue to study this - Really helpful - thanks very much!
I'm not sure exactly what you're asking, but please see my response to Kevin Taylor's question. I think it may answer your question. Thanks for watching.
Hi David! Thanks for your explanation... I'm still struggling to do it with suspensions. actually I have thousands of success for few failures. My data is sorted by months in use. i have a warranty system which gives me the plot F(x) vs MIS. I really would like to simulate it in excel and get to the same beta and alpha parameters. I prepared a excel sheet with the data from this system... would you mind to take a look and give me some hint I could use to make it work? thanks again!
First, read the Google+ post on the subject. The link is in the "about" section just below the video. That should answer your questions. By way of a "spoiler" for the post, I'll go ahead and say that: each data point (each failure, AND each success) will contribute one term to the Likelihood value. Each failure will appear as the probability density (PDF) of the failure time, and each success will appear as the actual probability of that success (that is, CDF(infinity) - CDF(termination time)). But do go read the post. It should be clear after that. If not, ask specific questions, and maybe I can help.
I uploaded it to google drive. drive.google.com/open?id=0Bwt0k22D0tFGSWlPcnlheXhZWWc&authuser=0 The displayed link seems to be an image. When I clicked on it, I got a message about no preview being available, but just below the message I saw a "download" button. I tried it and it downloaded. See if it works for you. I've saved it as a "macro enabled" worksheet. If you're unsure about the macro, just open the macro in the editor and read through it before running it.
Hi David, thank you very much for this detail explanation of MLE using Excel. I need to fit Power law distributions to the data could you please give me an excel macro to do that - like you have for Weibull distribution. Sangeeta
As far as the logic of the method, and of the spread sheet goes, modeling your data with a Power Law distribution is no different than modeling it with a Weibull distribution, or with a Normal distribution. In terms of actually performing the modelling using the spreadsheet I provided, you will probably need to write a function in VB for the Power Law PDF, and possibly the CDF -- depending on whether you are using the MLE method, or the Least Squares method. In my version of Excel, no Power Law distribution functions are built in. If you have data suspensions you want to model, then you will need to use the MLE method, but you will also need the CDF of the distribution for the formulas for the suspensions. See the actual formulas within the cells of the spreadsheet (roughly, columns G through Q). If the formula ends with 'TRUE', it is the CDF; otherwise, it is the PDF. Hope this helps.
The basic idea is still the same: you create the scalar "likelihood" value from the chosen parameters and the full data set, then form the partial derivatives of (the logarithm of) that likelihood value with respect to each parameter, and numerically try to discover the values of those parameters that cause each partial derivative to go to zero. The difference from the examples I used in the video will be that the Weibull distribution doesn't accept negative values. A number of distributions you might want to try don't make any sense for negative values. In this case, you basically have two options: 1) Use a different distribution. The normal might work for you. You'll just have to experiment to see what distribution best fits your data. 2) Use some kind of axis-shift. That is, try adding 3000 to each data point so that your lowest value becomes zero, instead of negative. Depending on the context, this isn't really cheating (as it might appear at first). When we say that hardware always fails at non-negative times, it's only because of when we start the clock. If we started the clock NOW, we would have to say that all failures that happened in the past occurred at negative times. To prevent that, we just get smarter about when we start the clock. Alternately, you may actually want to form a distribution about minus-x. That is, invert all the signs before you try to fit a distribution (assuming ALL your data is negative, of course. This won't work if some is negative, and some is positive -- use an axis shift for that). Hope this helps.
David Qualls My data fits to a wakeby distribution(or "the distribution of headaches" as it should be called) which has 5-parameters. My data is 13 datasets each which have a positive goodness-of-fit test with the Wakeby Distribution. That means that I have 13 datasets with 13 Wakeby Fits with 5 Parameters per fit all with different values which equals to 65 paramaters(13x5). for alpha, beta, gamma, psi, zeta ~~ I am trying to create an estimation equation of these 5-parameters. I was trying to build an MLE for each parameter using your method but I think this would be incorrect. I've read about the use of Multi-parameter Maximum Likelihood Equation and was wondering if you think that MLE MPMLE are the right way to go in this situation. I hope this isn't too confusing.
I'd need to study the specifics of your problem to offer any actually helpful advice. That's probably best done off-line. But it does sound like an interesting problem, even if it is a "headache". Good luck!
David Qualls Right now, I'm in my finals exams but I'm still trawling the internet looking for if anyone has a likelihood equation of the wakeby function. So far hasn't been positive but once this week is over I'd be able to put more time into the search. Is there a definitive method on creating a likelihood equation? Is it built from the PDF of the distribution or CDF? I'm a civil engineering student with no background in Statistics so far I'm teaching myself step by step.
Great video, and thank you for sharing your work. I am wondering if you can provide a definition of a Left suspension. Is it the same as a truncation? I am trying to determine weibull parameters for a population of assets where failure records do not exist prior to a certain year. Thank you
In reliability, where "operating age at failure" is modeled by a random variable, a Left Suspension would be: examining the asset and discovering that it has ALREADY FAILED, but you don't know when the failure happened. You only know that it happened sometime between age = zero, and age = present-age.
This scenario happens frequently in complex machines. The machine is disassembled, or examined, and some component within the machine is found to have "failed", but it may not be clear when the failure actually occurred -- unless of course, the failed component CAUSED the machine to fail, and you know the age when this happened.
A "truncation" is normally a Right Suspension. Again, in reliability, if an asset is pulled out of service BEFORE it fails, but for some reason it is never returned to the population (i.e. to operation) to experience failure, then that asset counts as a truncated data point, and serves as a Right Suspension in the parameter estimation process.
To complete the picture, an Interval Suspension is a failure that occurs at some unknown time BETWEEN two ages. Suppose a complex machine is disassembled at 100 operating-hours, and component X is examined and seen to be fine. Then at 200 operating-hours, the machine is disassembled again, only this time component X is examined and found to have failed. Since you don't know when the component failed, only that it failed between 100 and 200 hours, its failure is an interval suspension.
Turns out, left and right suspensions are actually only specific cases of the more general Interval suspension. In a left suspension, the left-end of the interval is time = zero. In a right suspension, the right-end of the interval is time = infinity.
In your case, where no failure data exists before a certain point in time, I'm not sure there is any data there to harvest. But that's the question to ask yourself: "Is there ANYTHING I know about the assets operating before failure data was kept?" If you know they succeeded (and you don't end up double counting them because they operated into the time where failure data is kept), you MIGHT have some good suspension data.
I just want to say thanks for posting this and sharing your spreadsheet. I had never heard of MLE until a couple of weeks ago, and your explanations as well as my digging into the details of your spreadsheet really helped me put it all together, and it's likely I will expand what you've done for reliability engineering work in the future. Good stuff.
I also appreciate you keeping up with the comments as I also didn't understand the purpose of epsilon until you explained it. While I thought that's what you were doing, I wasn't 100% sure.
Thanks for your kind words. I'm interested in your expanded work, if you don't mind sharing it.
Where I worked before, "rank regression" was the more common method of parameter estimation when using a Weibull distribution. While the rank regression method isn't necessarily difficult, I just found the MLE method to be much more elegant and intuitive, especially because it can be used with ANY distribution (as long as you can form the PDF and CDF via a formula). This feature allowed me to experiment with alternative distributions, and I ultimately settled on a 5-parameter distribution for modeling most jet-engine reliability. A 2 or 3-parameter Weibull just could not adequately model the infant mortality we frequently saw in reality.
Check out this link: www.weibull.com/hotwire/issue16/relbasics16.htm
for some validation of my leanings. I actually had not done the comparisons they have, so I'm glad to see someone has put in the work to compare the methods.
Wonderful video. Thanks for sharing the spread sheet.
That is very great explanation. 👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻👍🏻
Great video David, very helpful. many thanks
Worthful video demonstrate steps to reach better way.
I laughed when you "we get a tornado" (5:34). Thanks.
Hi David, thank you very much for posting this. I wonder where can I add interval censor for failures? It seems that I could only add interval censor for suspensions in interval suspension right and left column. Thank you.
Sorry, I don't know what "interval censor" means.
I suspect you *may* be using this term the same way I use "Interval Suspension". If so, then please see my response to Kevin Taylor where I discuss this. The spreadsheet is actually setup to input interval suspensions. Take a look at my response to Kevin Taylor and see if I'm addressing your situation. If not, please explain you situation more precisely, with examples, and maybe I, or someone else can figure out how to modify the spreadsheet for your situation.
OK, so I looked up "interval censored data", and it appears to be exactly what I described as "interval suspensions". I'm not sure why the spreadsheet can't be used with your data. Again, please explain (after reading the response to Kevin Taylor) why it can't be used with your data.
thank you, you sved the day
Nice template. I do have a question though. Why didn't you use the solver to just maximize log likelihood by changing alpha and beta? In that case, you didn't have to compute the derivative of loglikelihood and hence could have done away with the epsilon parameter in the likelihood column. It would have made the template less complicated. But hey, that's just me .
It's not just you.
I spoke about it briefly when replying to Long Nguyen. But, in full disclosure, I only learned of the free availability of solver after I had published the video. But when I applied solver to finding best parameters, it seemed like it wasn't quite as robust as the more brutish technique I used. Nevertheless, as you point out, it would have made the template MUCH simpler.
Also, as a teaching device, I believe there is utility in NOT hiding too many of the details. Solver is like magic. I am confident that most undergrads will have absolutely no clue how solver is doing its magic in the background. The approach I used will (or should) help students understand how to solve this type of problem, and be able to apply the concepts to other problems. Or, maybe they should just use solver on all other problems! ;-)
Thanks for your input.
I commend the academic purpose behind the method that you have employed in the video.
You say that solver is not as robust as goal seek. Aren't these using the same kind of optimization technique? I tried solver with your dataset and I got the same set of parameters as you did. Is there any way that you could share the dataset that you used when you got different results?
But it was really informative. Whenever i wanted to estimate parameters for non-linear regression, I always went for R or python, but not once thought of the ubiquitous MS Excel. I will definitely try this method next time to verify my results.
Great work, Sir!!! (y)
Thanks for the compliment.
No, Solver gives the same results, as I recall: it's been a while since I did the comparison. The issue I was having was that the starting points had to be closer to the correct answer when using Solver. Otherwise, Solver was more inclined wander off to infinity than my approach was. Again, I refer to my previous responses to Emiel van Maurik, and Long Nguyen.
Yes, I imagine Solver uses a modified Newton's method, just as I imagine Goal-Seek does. I mean, other than guessing at a solution (like the evolution method), what other way is there to hone in on the zeros of a function. The references I've studied for solving non-linear vector functions have been consistent in this regard. You (numerically) form the Jacobian at some starting point in the solution space, invert it, and solve for the vector of differences between the starting point, and the zeros. If the function is perfectly linear within the region, that vector of differences nails the solution. If not linear, then it over- or under-shoots the zeros. Just like with Newton's method. So, while Solver is inherently vector (and must use the Jacobian), Goal-Seek is inherently scalar, where the Jacobian reduces to just the derivative of the function.
But I'm getting way far off from the MLE topic... Perhaps some other video addresses the pros/cons of using Solver as a vector solver, versus using Goal-Seek in a round-robin, scalar-solver approach, like I did it. But definitely, thanks for asking. Your question caused me to revisit and reconsider some things. Thanks again.
Hi David, Thanks a lot for your tutorial and further explanation here. Still I have questions of understanding your Excel spreadsheet, I hope you can help me with that.
First I will explain what I am working on for better understanding.
I'm also doing a project on failure data and what would be the optimal interval for replacing a component. This could be from any component, so any distribution could be possible. Therefore I was asked to create an Excel spreadsheet just like you did. In this sheet, users could then insert their own data and determine easily step by step which distributions fits the data best (Normal/Exponential/Weibull). Then they should get a MTBF with a margin of error at a chosen confidence interval. This was not the biggest problem. I was also asked to take censored data into account, again just like you did. Unfortunately, this was the point where I got stuck.
I analyzed your Excel spreadsheet first and I have various questions. In the first place I am a bit confused on your epsilon value. What it exactly means and how you use it in your eventual calculation. I only want to use the MLE since I mostly consider having random data. Then I also do not understand the 'Param #' cell, what happens when making this choice and why do you inserted 5 parameters. Could you easily add one if you want?
Since it is also possible to use the exponential distribution for for example Electronic parts, did you also made such an Excel spreadsheet for that.
Finally I was trying to insert a macro just like you did but It failed, unfortunately I have never worked with macro's.
I still have till June 17 to finish my Excel spreadsheet. Hope you can help me. If you want I can send you my spreadsheet so you can see what I have done.
Emiel
The epsilon value is internal to the solver mechanism. You don't really have to mess with it. I spent some time in the video explaining how it worked, but understanding it is not really necessary to performing an MLE analysis. You just have to have SOME mechanism for finding the parameters that maximize the "Likelihood" scalar value. I initially kind-of "maximized" it (in the video) by GUESSING at the parameters and trying to do a best-fit by visually comparing the data-CDF curve with the computed CDF curve. You can do that too. It's just not very elegant to do it that way, and it's very difficult to hone in on the BEST value(s) when doing it with guess-work.
I set up the spreadsheet to use the "Goal Seek" function, and as such, had to have a value that indicated "maximum". This is the reason for forming the derivative value:
"d(Log_Likelihood)/d(parameter)"
Recall that maximums or minimums are found by setting the derivative to zero. The spreadsheet numerically computes the derivative via the "epsilon" terms, and the GoalSeek attempts to find values of the parameters that make that derivative value zero.
It would have been simpler to just use the "Solver" add-in routine since you can just ask Solver to look for value-sets (parameters) that maximize a given scalar value (the likelihood value). Using Solver would allow us to skip all that epsilon stuff, and forming the derivative manually. You just tell solver which value you want maximized, which values (the distribution parameters) to change to do it, and hit the go button. It takes care of everything else.
I spoke of this briefly when answering a question by Long Nguyen. As I noted to him, in my experience, Solver tends to wander off into never-never land more often than the approach I used.
I setup the spreadsheet to use 5 parameters because the distribution I was experimenting with had 5 parameters in it: 2 parameters for the "wear-out" failure mode, and 3 parameters for the "infant mortality" failure mode. I wrote an article on the distribution but never published it. If I get around to publishing it, I'll come back here and put a link to it.
Hi, I used LSM to find shape parameter then used gamma function and Maximum likelihood method to find scale parameter. I want to know whether my approach is correct and reliable. Seems like I'm getting the closet values to what I supposed to get. Thank you.. Will be waiting for your reply.
+Lizy Ae: I can't really understand your question. Probably best to just ask your professor. He or she can see exactly what you're doing, and can help guide you.
Sorry I should be clearer on the question. Anyway, thanks for your suggestion.
Hello David. So my average value is in negative value. I can't execute Weibull's Distribution. Can you please tell me what's the alternative to Weibull's?
No, not without seeing the data, or understanding the physical meaning of the data. Sorry, I don't know how to recommend any distribution without those things.
But you can perhaps do it yourself by first plotting a histogram of the data using the technique I used beginning at about minute (3:20) of the video running through minute (4:15).
Once you have the histogram in front of you, compare the shape of it to any of the standard shapes like normal, log-normal, gamma, Weibull, beta, etc.
If the shape looks similar to one of the standard shapes, it might fit if you can transform the independent variable (let's call it "time" so I don't have to keep typing "independent variable") in some way. For example, you say that you have time-values that are less than zero, which immediately rules out a gamma or Weibull curve. But what if, instead of computing the PDF or CDF of time, you computed PDF(-1*time). Does that make all values positive? If not, would a shift on the x-axis do it. That is, if you just added a single constant to every value of time, would that make them all positive and begin the curve at time=zero.
You'll just have to be creative and try different things.
Hope this helps.
Hi, you mentioned failures referring to machineries. How did you get these data?Are these coming from the MBTF?
+Marcello Cusma The data can NOTIONALLY represent "time to failure". Since I work in reliability, that makes the most sense to me, but the data could be from any process modeled by a random variable. For example, it could be the length of molecules grown in a polymerization process. Or it could be the weight of items received at a shipping node. The data I used in the video was not actual data taken from any real thing. I generated the data sets using random-number generation techniques, and Excel.
Not all data sets will have suspensions. In reliability, we get all the data-suspension variants: left-suspension, interval-suspension, and right-suspension. That is why it made sense to me to demonstrate the MLE method using reliability-type data. In the video, I only demonstrated analyzing a data-set that included right-suspensions, but the technique works for all suspensions.
Finally, let me repeat: the data in the video notionally represents TIME TO FAILURE of each item, NOT MTBF. MTBF is one single parameter that you would use to characterize a set of time-to-failure data. It only gives the mean (or average) value of the data set, and says nothing about the "spread" of the data.Hope this helps.
Thank you a lot, David. I was going to use your spreadsheet, but I couldn't. I just replace my data in column A (Failures). I used Normal data to see how your macro works.
I already knew that mean and Standard Deviation are 38.8 and 11.4 respectively. But after click on (MLE Estimate Parameters) I got following results for mean and SD:
Mean: 1327.8
SD: 1086.82
My data are: 26, 33, 65, 28, 34, 55, 25, 44, 50, 36, 26, 37, 43, 62, 35, 38, 45, 32, 28, 34
Why does this happen?
Like virtually all solving routines (and the built-in Excel solver is no different in this regard), it needs a reasonable starting point.
My suspicion is that you only replaced the failure data (and possibly only the first screen-full) then hit the ".. Estimate" button. You need to make sure all the suspension data is cleared out of column E, and all the left-over data is cleared from column A. Then you need to put REASONABLE starting point values into the estimates of mean and SD in cells T4 and U4. If you don't, it'll begin with the mean and SD from the original example data, which is way off, and probably won't converge to the right answer.
The reason this can happen is that the solver routine "falls into a dimple" on the solution landscape. That is, in well-behaved problems, the true answer lies at the bottom of a giant canyon, or a crater (or in our case, since we're seeking a maximum, at the peak of a large mountain). If you begin looking for that maximum or minimum far enough away from the true peak or outside of the crater, the solver gets tricked into falling into a small dimple, or resting on top of a mole-hill instead of the top of the mountain. These are called "local" maxima or minima, and they're NOT what you want. But, start off close enough to the mountain (or inside the crater), and the solver always recognizes which way is up, and always climbs to the "global" maxima.
Just to test it, I tried just clearing out the original values from columns A & E, inputting your data, then pressing the button. Sure-enough, I got garbage answers. Then I tried putting in reasonable starting-point-estimates into cells T4 and U4, (I used 50 and 5) and it quite quickly gave the expected results of 38.8 and 11.4.
How far off your starting points can be from the correct values is related to how well-behaved your function is, and to some degree, on your data. For the set you provided, when I put in a value of 20 for SD, the process diverged and ran off to some point way in the distance (414 million). So, apparently, a good starting point for the SD is pretty important! On the flip-side, I tried it with starting points as low as 0.7, and it still worked. Lower than 0.7, however, caused the macro to crash.
Hope this helps.
Thanks. It definitely helped. Yes, the starting point for SD is very important, but it seems the starting point for mean has less sensitivity.
I have another question. I have 300 data and I know the normal distribution will be fit on them. Is it reasonable to find the distribution parameters (mean and SD) from MLE, or It would be better if I calculate them based on data values (I mean calculate mean and SD using data based on their formula)? In fact I was wondering if there is any advantageous in obtaining distribution parameters using MLE when you have the values of the distribution? For instance, about previous example, My data are: 26, 33, 65, 28, 34, 55, 25, 44, 50, 36, 26, 37, 43, 62, 35, 38, 45, 32, 28, 34. Is it reasonable to obtain the parameters of normal distribution by using MLE?
No, I think using MLE is overkill in cases like yours, where you know the population is Normal, and you don't have any suspensions. In these cases, just calculating the mean and standard deviation by text-book formulas is DEFINITELY the way to go.
But, how do you estimate the parameters if there are suspensions? Maybe there are ways, I just don't know what they are. Or, what about if you are working with distributions that don't have closed formulas for the parameters? These are the cases where the MLE can become very helpful.
Dear David,
I was wondering if MLE is efficient for a small dataset (6-7 data points). If it's not, which method is more suitable for fitting distribution to such a small dataset. I can give you these data points as follows:
6.8, 13.9, 14.6, 16, 17.5, 18.6, 17
Your advice would be much appreciated. Thank you.
Regards,
"Efficiency" is really not the right question to ask. The questions you need to ask yourself are:
1) Does the MLE method produce a good and "unbiased" estimate for the parameter in question?
2) How confident do I need to be in my parameter estimates?
I didn't address either of these issues in the video, nor in any prior question responses.
If you are unfamiliar with "bias", think about the way we compute standard deviation. We know that the definition of standard deviation ends with a division by 'N': the number of data-points in the set (population). See en.wikipedia.org/wiki/Standard_deviation#Basic_examples.
BUT, if we are using the conventional formula to estimate the *POPULATION* standard deviation, we don't divide by N, but rather by N-1. This is because the standard formula applied to a SAMPLE does in fact give us the standard deviation of the sample, but it gives us a *biased estimate* of the standard deviation of the POPULATION. We need to make that distinction -- the standard deviation of a SAMPLE is NOT the *best* estimate for the standard deviation of the POPULATION (assuming a normally distributed population).
Turns out, we get the same behavior with MLE for standard deviation (sigma) for a normal distribution. The MLE for sigma produces the conventional formula, where the final division is by N, not N-1. Thus, the MLE parameter estimate for sigma for the population is slightly biased. In your case (7 data points), assuming your population is normal and that you are trying to estimate the standard deviation, the MLE parameter will be off about 14% from the unbiased estimate. In much larger data sets the difference between the MLE value and an unbiased estimate may become negligible.
This doesn't mean MLE is bad. The article en.wikipedia.org/wiki/Bias_of_an_estimator gives an example of when an MLE estimate is actually better (for Poisson distribution) than the unbiased estimator. It also gives an example (the ticket box example) of when, *FOR A VERY SMALL SAMPLE SIZE*, the MLE estimate is very poor.
That last statement gets to the second question: how confident do you need to be in your estimates. This gets to the heart of what much of statistics is all about. That is, given a sample, what is the probability that the sample is a good representation of the population -- that someone can use the sample to make (accurate) statements about the population. It's way too big a subject for me to address here, other than to say that small samples lead to low confidence in your answers (parameter estimates). See en.wikipedia.org/wiki/Confidence_interval
Hope this helps.
David Qualls
Thank you very much for your thorough response. In my case, the issue was that these 7 data points were not normally distributed. I did not want to assume normal distribution in order to avoid uncertainty. For this reason, I used MLE in the hope of finding the best fitting distribution. I was trying to use Solver add-ins for optimization. Unfortunately, the results turned out pretty disappointing.
It's great to know that MLE estimate is poor when it comes to a small sample size. Thanks again for your advice.
Long Nguyen I wasn't trying to say that the MLE method produces poor results when the sample size is small: I was trying to say that it COULD produce poor results -- the confidence in the final answer is LOW. The ticket-box is an example of when the method will likely fail badly for a tiny data set.
Regarding using the Solver add-in, yes, I have had similar results. Seems like unless you pick pretty good starting values, solver quickly wanders off into la-la-land. My macro seems to be a little more robust, and give slightly more accurate answers, although it still needs pretty decent starting values.
If you know that your data is not normally distributed, try a different distribution. That's the nice thing about the worksheet I provided: you can experiment fairly easily by just replacing all the references to "normal.dist" with "gamma.dist", or "weibull.dist". You'll still need to do a visual fit FIRST, to get an answer that's in the ballpark, before you hit the macro run button, or use solver.
Note however, that with only 7 data points, your uncertainty is going to be high no matter what distribution you use. Even if you get perfect correlation (the data CDF line overlays the calculated CDF line perfectly) you STILL have significant uncertainty. How do you KNOW that you drew a sample that perfectly represents your population? You don't! You may have randomly drawn data values that perfectly match a distribution that is NOT the true population distribution. As I've explained elsewhere, it's a matter of likelihood, or probability (kind-of). The more data points you draw, the less likely that you'll get the wrong answer, and thus, the more certain you can be in the parameters you estimate.
Hope this helps. Good luck.
David Qualls
Thank you very much for your advice.
How can one build the 0.95 confidence interval of MLE for weibull parameters and plot the bound line on chart? Please help me..
Ravi: That is a good question. Unfortunately, I haven't investigated it, and don't know the answer. Perhaps someone else could share a good answer. Please...
The question of confidence intervals is extremely important, and one should definitely recognize that the MLE method itself (as detailed in my discussion and the spreadsheet) ONLY provides the MAXIMUM LIKELIHOOD parameter estimates for the provided data-set -- not confidence bounds on those parameter estimates. When dealing with small data-sets, it is important to remember that every addition of a new data-point can move the estimate quite a bit.
One possible numerical approach is to remove one of your data-points and perform the MLE analysis. Then replace that point, and remove a different point and repeat the analysis. Do that for each data-point, and you'll have as many parameter estimates as you do data points. From these values you can rank-order them and pick the 5% value and the 95% value as candidates for your (90%) confidence interval. If you have a LOT of data, you would have a lot of estimates, and you could pick the 2.5% and 97.5% values for 95% confidence.
This sounds like a macro I'd like to have in my tool-chest, if someone wants to write it for us!
Alternately, since the scale and shape parameters should follow the central limit theorem, you should just be able to use the standard formulas for confidence intervals. But, I'm less confident about that answer -- I haven't researched it.
This was pointless. the introduction was good and in the intro you said you would show how MLE works but all you did was to show a macro that you had written. This does nothing to help sanyone understand what MLE does.
Sorry.
I can sort-of see what you're saying. I know the title is somewhat misaligned from what is actually in the video. I'll probably try to come up with a better title.
The target audience was individuals who had NO experience with parameter estimation, so I went thru a lot of things quite unrelated to MLE proper. And because of that it went way long, so I had to shut it off before I really said what needed to be said.
Nevertheless, I do think that the segment beginning at minute 13:45 explains pretty well how to set it up. The macro just uses the goal-seek to individually seek each parameter to an optimum value (derivative close to zero), and loops until both parameters are optimum. You have to loop because each time you switch parameters, it messes up the other parameter. You just keep looping until they're both good.
Do you have any suggestions for what would make it better for you?
By the way, there are probably better videos already on RUclips for what you want. I haven't viewed any of them. I may do some looking and see if I can point viewers to better videos than mine.
OriginalJoseyWales you are quite rude. The teacher put in effort to do this, there are better ways to have said what you said instead of saying "pointless". I doubt you'd be happy if someone said what you did was rubbish, after taking out time to do it. Just be careful with your words as they can make or break someone. Cheers.
You're pointless OriginalJoseyWales. If you don't know how to open up the macro to discover exactly what this guy shared FOR FREE, then maybe consider not getting into it. It might be over your head. Otherwise, do some pre-read on Mean Time To Failure equations using a Weibull distribution to understand that you need to find the best fitting variables to the equation given a data set which is done through an iterative function to test parameters until you get as close as possible to the values that are the MAXIMUM LIKELIHOOD of the variables. It's not that complicated.
Hi there,
Thanks for posting this video - very helpful for beginners!
... I have question... I followed all of the exposé until 'epsilon' was mentioned... Might you be able to explain where it came from / what it's use is? Is it specific to the function you tried modelling?
Marie a.Londres Epsilon is standard notation in calculus. See en.wikipedia.org/wiki/%28%CE%B5,_%CE%B4%29-definition_of_limitI may have used it slightly backward in that I used epsilon to represent a small change in the independent variables (the parameters of the distribution) rather than the dependent variable. Perhaps I should have called it delta instead. But regardless of that, the point is that "epsilon" was a small change in one of the parameters: which parameter was determined by the value (1 through 5) in cell U8 of the spreadsheet. To form a partial-derivative, you vary one of the parameters by a small amount, and note the associated change in the dependent variable (the likelihood function in this case). By looking for the value of the parameter where the partial derivative goes to zero (that is, the place where small changes in the parameter produce no change in the likelihood), we know we're at a flat spot on the curve, or at a local-maximum. With a little luck, we can find a single place (a single set of parameters) where ALL the flat-spots coalesce (that is, all the partial derivatives are zero), and we have indeed found a (local) maximum. With even more luck, that local maximum will be the actual global maximum: the single set of parameters we're looking for.
Hope this helps.
Dear David, Thanks so much for your thorough and generous response. I have downloaded the spreadsheet and will continue to study this - Really helpful - thanks very much!
Hii...nice video
Could you please share your thoughts on the right suspended data type, like how to use, what is formulas, would be a great help!
I'm not sure exactly what you're asking, but please see my response to Kevin Taylor's question. I think it may answer your question. Thanks for watching.
Hi David! Thanks for your explanation... I'm still struggling to do it with suspensions. actually I have thousands of success for few failures. My data is sorted by months in use. i have a warranty system which gives me the plot F(x) vs MIS. I really would like to simulate it in excel and get to the same beta and alpha parameters. I prepared a excel sheet with the data from this system... would you mind to take a look and give me some hint I could use to make it work? thanks again!
First, read the Google+ post on the subject. The link is in the "about" section just below the video. That should answer your questions.
By way of a "spoiler" for the post, I'll go ahead and say that: each data point (each failure, AND each success) will contribute one term to the Likelihood value. Each failure will appear as the probability density (PDF) of the failure time, and each success will appear as the actual probability of that success (that is, CDF(infinity) - CDF(termination time)). But do go read the post. It should be clear after that. If not, ask specific questions, and maybe I can help.
This is great. Thanks. Is there any way for us to get the spreadsheet?
I uploaded it to google drive.
drive.google.com/open?id=0Bwt0k22D0tFGSWlPcnlheXhZWWc&authuser=0
The displayed link seems to be an image. When I clicked on it, I got a message about no preview being available, but just below the message I saw a "download" button. I tried it and it downloaded. See if it works for you.
I've saved it as a "macro enabled" worksheet. If you're unsure about the macro, just open the macro in the editor and read through it before running it.
David Qualls Thanks so much for sharing this. You're very kind.
Hi David, thank you very much for this detail explanation of MLE using Excel. I need to fit Power law distributions to the data could you please give me an excel macro to do that - like you have for Weibull distribution. Sangeeta
As far as the logic of the method, and of the spread sheet goes, modeling your data with a Power Law distribution is no different than modeling it with a Weibull distribution, or with a Normal distribution. In terms of actually performing the modelling using the spreadsheet I provided, you will probably need to write a function in VB for the Power Law PDF, and possibly the CDF -- depending on whether you are using the MLE method, or the Least Squares method. In my version of Excel, no Power Law distribution functions are built in.
If you have data suspensions you want to model, then you will need to use the MLE method, but you will also need the CDF of the distribution for the formulas for the suspensions. See the actual formulas within the cells of the spreadsheet (roughly, columns G through Q). If the formula ends with 'TRUE', it is the CDF; otherwise, it is the PDF.
Hope this helps.
what if my distribution was below 0?
from -3000~1000
The basic idea is still the same: you create the scalar "likelihood" value from the chosen parameters and the full data set, then form the partial derivatives of (the logarithm of) that likelihood value with respect to each parameter, and numerically try to discover the values of those parameters that cause each partial derivative to go to zero.
The difference from the examples I used in the video will be that the Weibull distribution doesn't accept negative values. A number of distributions you might want to try don't make any sense for negative values. In this case, you basically have two options:
1) Use a different distribution. The normal might work for you. You'll just have to experiment to see what distribution best fits your data.
2) Use some kind of axis-shift. That is, try adding 3000 to each data point so that your lowest value becomes zero, instead of negative. Depending on the context, this isn't really cheating (as it might appear at first). When we say that hardware always fails at non-negative times, it's only because of when we start the clock. If we started the clock NOW, we would have to say that all failures that happened in the past occurred at negative times. To prevent that, we just get smarter about when we start the clock. Alternately, you may actually want to form a distribution about minus-x. That is, invert all the signs before you try to fit a distribution (assuming ALL your data is negative, of course. This won't work if some is negative, and some is positive -- use an axis shift for that).
Hope this helps.
David Qualls
My data fits to a wakeby distribution(or "the distribution of headaches" as it should be called) which has 5-parameters. My data is 13 datasets each which have a positive goodness-of-fit test with the Wakeby Distribution.
That means that I have 13 datasets with 13 Wakeby Fits with 5 Parameters per fit all with different values which equals to 65 paramaters(13x5).
for alpha, beta, gamma, psi, zeta ~~ I am trying to create an estimation equation of these 5-parameters.
I was trying to build an MLE for each parameter using your method but I think this would be incorrect.
I've read about the use of Multi-parameter Maximum Likelihood Equation and was wondering if you think that MLE MPMLE are the right way to go in this situation.
I hope this isn't too confusing.
David Qualls Hi I think you deleted your entire comment.
I'd need to study the specifics of your problem to offer any actually helpful advice. That's probably best done off-line. But it does sound like an interesting problem, even if it is a "headache".
Good luck!
David Qualls Right now, I'm in my finals exams but I'm still trawling the internet looking for if anyone has a likelihood equation of the wakeby function.
So far hasn't been positive but once this week is over I'd be able to put more time into the search.
Is there a definitive method on creating a likelihood equation?
Is it built from the PDF of the distribution or CDF?
I'm a civil engineering student with no background in Statistics so far I'm teaching myself step by step.