That’s been there since at least 1.2 and I love it: every time I geek out a little and thank Scott for getting me into Kerbal Space Program through his videos.
One element to this story which is present in Lovell's book, but which you didn't mention - and is worth highlighting as it's yet another of the coincidences and minor failures that contributed to the explosion. When they were attempting to drain the tanks using the heaters whey were also monitoring the temperature within the tanks - this should have meant they would have noticed they were baking it at 500C... But they temperature sensors and monitor they were using was calibrates only to go up as high as the 27C they expected the thermostats to cut out at. So when the thermostats failed, and burned off the insulation - as far as the technicians could tell, the temperatures were precisely the 27C they were expecting - not the catastrophic 500C. If their thermometers could go higher than 27C, they'd have immediately know the tank was faulty. Just makes me thing of the Chernobyl series - "It's ... the maximum reading on low-limit dosimeters. They gave us the number they had...."
The temperature monitor only went up to 80F. They knew the temperature would be a little higher because they were using the heaters to remove the O2 but yeah, if they had a wider (higher) range on the monitor, they'd have seen that something was wrong with the tank shortly after the pad test.
Yes. I was 9 years old when this happened and watched and covered any info that I could find at the time even though I was to young to fully understand it all.
The official report is really interesting. Any engineer should read it to understand just how simple things can add up to a huge problem. I always tell people they should read it but it’s pretty dry. This video is an excellent summary of events!
If you like that kind of thing, check out some of the reports for marine and air accident investigations. I've read some from most english speaking countries, and they are enjoyable for the same reason, and the writing is usually very good, if formal.
@@svchineeljunk-riggedschoon4038 Or rail accidents as well ... it's astonishing how many are just a sequence of error after error after error. Of course, what that means is that just one error fewer, and the accident wouldn't have happened.
@@KaiHenningsen The US chemical safety board have some amazing videos here, like above, it's interesting how one small thing can quickly snowball into a disaster.
SV Chineel, Junk-Rigged Schooner Sailing Hi! it’s funny your name is similar to one of the sailing channel i follow, i seem to follow a lot of sailing channels! Yes I quite often read the AAIB bulletins. The Air Safety Institute channel is very good.
Remember kids, redundancy is crucial in systems that are truly vital. Just remember to route all three of your redundant hydraulic control lines right next to each other so the occasional stray turbofan disc going trough your aircraft has a chance of slicing through all three of them in a single go!
@@jjohnston94 It"s not funny. Hundreds of people died because of it. Worse than 737 MAX. I worked with one of the people responsible on a later project and it didn't seem to have hurt his career much.
I worked for the company that built that thermostat that welded. The design engineer for the thermostat line had the same picture you showed on the wall of his cube as a reminder that limits are not to be exceeded. That example was used when customers would call and ask if they could be used at higher voltages/currents. Nearly all of the people who were working at that time had retired when I started, I wish I could have gotten first hand accounts of the aftermath.
But, the cryotank in Apollo 13's SM was originally installed in Apollo 10 and removed for test anomalies. When it was refurbished and installed in 13, North American Rockwell neglected to swap out the thermostat. The voltage tolerances had changed since 10. I wonder if Lee Atwood tore that technician a new asshole.
dramatizations are like that. the movie "margin call" portays the 2008 financial crisis as if it happened overnight, when it reality it took months, I think.
It happened during 2007, there were a lot of articles detailing the house of cards that was starting to fall. It was the first time I read "trillion dollar loss"
I can only imagine what the astronauts would have been thinking when the service module blew out! Disasters in space are scary stuff. Kudos to NASA for bringing everyone back safely! This mission is probably the most incredible point in the entire space program. Thank you Scott for explaining how it all happened in a way that's so easy to understand!
Well, their immediate response was, "Huh? What was that?" There's a few different things that can cause jolts in the spacecraft and in reality, it took them quite a while to begin recognizing the severity of this issue. They had been dealing with the problem for about 90 minutes before they realized that the command module was beyond saving and started powering up the Lunar Module.
I’m surprised ground wasn’t monitoring heater current when they decided to use the heater to drain O2 tank 2. If they had been they would have seen that the heater was not cycling as it should have been, and probably caught the problem there. Alternatively, as has been mentioned elsewhere in the comment section, if the tank thermometer read above 80 degrees that most likely also would have worked to catch the problem. Of course with the benefit of hindsight it’s easy to be critical. The steps the crew and ground support took after the accident occurred were nothing short of incredible, and really spoke to how well they worked together and knew their spacecraft.
@@TheCrackedFirebird Why would anyone need a thermostat to go higher in a cryogenic tank?? Of course, what's not really been said by others is that it's remarkable that the tank didn't halt and catch fire duing the draining and refilling. It's a real testimate to just how astonishingly well built the damn thing was that it managed to get as far as going into space. It's just a bummer that's the time when it choose to go kaboom. Equally astonishing is how well built the rest of the CM/SM that it survived the explosion and that it was able to be restarted after several days in a deep cold state - frankly the most frightening line in the movie after "we're venting something into space' is 'like driving a toaster through a car-wash'.
@Bobb Grimley Thanks Bobb, glad you're out there checking the internet for mistakes. Funnily enough, my spellchecker tells me that there is no such word as 'Bobb' either. Just in case you want to let the person who named you that they made a spelling mistake too.
There was a temperature gauge on the launch pad to display the temperature inside the O2 tank heater assembly. During the boil off process the temperature inside the tank heater got up to more than 500F but unfortunately the temperature gauge only went up to 80F. The gauge was pegged all the way up to the top at 80 and so no one realized that the temperature inside the tank heater was up so high that it was damaging the insulation on the wires.
@@opl500 It wouldn't have exploded that early in the mission anyway. It was too early to stir the tanks. From what I've read the latest in the mission that it could have exploded and the crew might still have had a chance to survive was the beginning of powered descent. Once a sizable portion of the fuel in the LM's descent stage was used up they wouldn't have had any way to get back out of lunar orbit.
8:50 "Houston, we've had a problem here" Movie: Catastrophic explosion, shock from crew, drama Real life: 80's freeze frame and festive steel drum flourish
I was a kid, was all over ever aspect of Apollo. Kepy an eye on it over the years. I am now 60. NEVER have I heard any detailed explanation of what happened, how and why. Also. You are scary smart. You smoothly went through every engineering detail like you built the tank and system. You have filled in an important blank of what happened in my youth. Very Impressed.
The LOX tanks were made at a Beechcraft plant north of Boulder ,Colorado. I knew one of the engineers and he gave exactly the same explanation. Well done it is rare these days for any media to get a story right by doing their homework such as you have. Well done Sir !
@@georgeemil3618 As far as I know: No. The Apollo hardware was a steady flow of already built hardware. You do not want to include major structural changes as long as you absolutely do need them. A13 has shown that it is possible to build an adaptor even when in flight. If I had to make the decision, I would say: good enough.If you absoultely insist, construct a prefessional adaptor and put that on the packing list, but otherwise: don't change a working system unless there is a good reason to do so. After all this was in accident, which should not have happend in the first place.
One design deficiency was also that both oxygen (and hydrogen) tanks fed the fuel cells over a common manifold. No isolation valves, no check valves. You have one leak anywhere in the system, you will loose everything. If there had been the possibility to isolate the faulty tank, the outcome would've been less dangerous. Yes, the moon would've been lost. But there would've been enough oxygen in the good tank to return home. They later changed that for the J- missions and AFAIK the space shuttle not only had check valves between the tanks and the manifold, but also the possibility to switch to 'split plant' operation, i.e. feeding each FC from its 'own' tank and isolating them from each other. I still don't get why they didn't implement that in the first place. They used statistical methods to determine how often a system would fail in a given number of times, but overlooked that interconnecting independend units increases the failure rate. Strange...
i never thought all those years ago when i was trying to get into orbit on the free version of KSP that i would still be watching your videos in the next decade
What I don't understand is why these tanks were even used at all. You said they were removed from Apollo 10's Service Module and replaced with upgraded tanks. Why, then, weren't "upgraded" tanks used on Apollo 13? Were the original tanks "upgraded," then installed in 13's Service Module?
The cause was bloody stupid penny pinching. When the damage to the tank was discovered, it should have immediately been tagged as defective then cut apart to see what all had been affected by the impact.
I have no engineering background at all and found this engaging and fascinating, not to mention well-explained. Scott, thank you for your service to the rest of us who love space (and space history) but didn’t go to MIT.
I already knew what caused the explosion on Apollo 13 but I also knew that I'd learn a bunch of new and interesting details by clicking on this video because it's Scott Manly!
This channel might end up being my favorite channel on space and one of my top 3 channels on RUclips in all categories. Definately the best walk thru of Apollo 13's tank issue. Thanks.
Just started watching but had to get my up vote and comment in to help you in the algos since the video is only 30 minutes old. .... and this is one of my favorite of your videos. Love this deep dive!
About 30 years ago, when I was working for IBM in the UK, they had an all-site meeting at a local convention center, and the guest speaker was none other than Jim Lovell, commander of Apollo 13. Whatever IBM had paid him to be there, it was money well spent. He gave an enthralling and inspiring account of the mission and the entire audience was hanging on every word.
I challenge you to find an orkier space repair job than Apollo 13. "Oh no, vital systems of the ship blew up!" "Don't worry, we have some cardboard, plastic bags, and duct tape! We will be fine!"
You must not care about "precision" at all, given you misspelled "orcs", not to mention, you have them consulting with NASA on how to fix engineering problems during spaceflight.... Lame attempt at humor... Try again.
Lovell/Kluger's explanation of this in the book of Apollo13 is almost as complete but contained one extra titbit you didn't mention: The pressure gauge. When the thermostats stuck shut (ohm's law - double the voltage, double the current, quadruple the power) to boil the LOx off, the pressure gauge should have alerted them that it was way high. The problem was the gauge topped out in normal use meaning the most basic way they could have detected this failure eluded them. The lesson here stands true in anything with a gauge: Always make sure the gauge can read higher than you expect it ever to need to! Thus endeth lesson 37 in the gospel of engineering according to me....
Lovell did also mention they duplicated the whole thing on one of the test flight articles. I hope they didn't throw away the recordings of that event.
Exactly right. Michael Crichton actually took inspiration from Apollo 13, I think, in the Jurassic Park novel: The motion sensors were programmed to make sure none of the dinosaurs had escaped so it topped out at the number there was supposed to be, but they also should’ve programmed them to make sure there weren’t more.
Doesn't seem like a highly unusual partial pressure for oxygen by itself. Diving equipment has a partial oxygen pressure (gas mix pressure x concentration) of 40bar when filled with air. But running electrical equipment in that environment is a different thing, of course...
In the 80’s I attended a 5 day Configuration Management course for military hardware. Th presenter was American who was a CM manager on Apollo. He claimed that the burning of the insulation was effectively a CM failure, as the tanks had been changed, but the test procedure was one version behind, and had put too much voltage in. He claimed the design change made the insulation thinner. Your explanation is much more detailed than his!
I never really thought about the fact that the failure on the O2 tank was in fact caused by a piece of hardware that was later determined to be unnecessary.
Excellent.....no waffle, no padding....just concise, clear explanation. At the time, I listened/watched every minute of the drama. At one time, real doubts of survival were intimated, and we all held our breath. Mission Control's finest hour.
Yes, that's the most comprehensive account of the catalogue of failures that led to the catastrophic explosion of that tank on Apollo13. Well presented.
@get to the Choppaa And the flight manager that didn"t want to bother her boss at home on a Saturday when they thought they might have a problem from the external tank foam hit on the wing of Columbia and then sleep on it over the weekend while neglecting to inform the flight crew of their concern.
The faults of man. I believe political and financial pressure on the launch had a role to play in all this. Given the incredible complexity of the build, I imagine there were only a few that could do it without damaging something else.... But then again most engineers, managers and others think those of us who turn a wrench are replaceable; "whatever you were doing can be done by somebody else" attitude. What's clear to me is someone knew there was a problem with the tank and chose to rely upon the backup procedure for the primary.
I've been following the Apollo 50 account on Twitter (definitely recommended by the way) who tweet in real time about the missions from 50 years ago. It's been fun watching their updates recently about all the O2 tank issues during the build up to launch
It"s reminiscent of the decision to ground test with the crew in the Command Module on 100% oxygen and at the same time not use fire resistant materials in the CM since a fire could be extinguished quickly in space by dumping the oxygen. Both decisions by high level NASA managers, possibly even the same one.
The issue was not, that it was 100% oxygen. In order to perform the leakage test, the pressure inside the capsule had to be raised way over 1 bar. Something that would have never happend in space. In fact when in space the pressure in the capsule was way lower then 1 bar, giving roughly the same amount of oxygen per volume as it would be, would there be ordinary air. So with the regular pressure in flight, the risk is more or less exactly the same as on earth under regular atmospheric conditions. It only happend because the leak test was done with the capsule at higher pressure then normal. The issue was solved by not filling the capsule with pure oxygen at the Cape but instead staying with air but add an additional pressure valve which would close during the ascent, when the dropping pressure reached the intended capsule pressure. From them on, the air was steadily replaced with pure oxygen.
@@dalecomer5951 No. The crew would have been ok, if the test would have been performed with ordinary air. But this was not possible, since the capsule was not equipped with air containers. The capsule only had the facilities to be filled with oxygen. That is why they used it to perform the test. It was a leakage test. Thus they needed to pressurize the capsule with a higher pressure then the surroundings. Just using 1 bar (same as the surroundings) for the test would not have been enough. They NEEDED the higher pressure for the test.
It's easy to criticize in retrospect, not so much when you have an entire multi billion dollar government program and agency breathing down your neck and the eyes of the world watching your every move with bated breath.
Thank you for a great explanation of something I have wondered about my whole life. I was a little boy when it happened. Everything I have heard about it since then has left me with more questions than answers. Thank you for finally clearing things up.
@@621Lafayette Yes, agreed this is an excellent podcast. Season 2 is Apollo 13. Season 1 is the moon landing. Both are very good and highly recommended.
Imagine damaging life-critical equipment and then putting it into crewed space vehicle anyway. What is this, soviet union QA standards? Pretty sure they had better. Geez.
Thanks Scott, a nice chewy video to get your teeth into. I can't believe they didn't do more when they found they couldn't empty the tank. If they'd investigated, the whole disaster would have been avoided. What on earth did they think - that it didn't matter, or that fairies had fixed it?! Mind you, I also can't believe that I sat at the tea table, with the whole family dead quiet, listening to the static on the radio, waiting (it seemed like forever) for Apollo 13 to reply, after re-entry. 50 years ago!! I cannot possibly be that old....
I was in an audience with where Jim Lovell was the speaker. True or not he said that they found that someone had dropped one of the components during construction and thats what caused the problem.
Never put Tom Hanks on something involves flight, or you might end up stranded on a deserted island, having to land your plane on a river, getting stuck on the airport because your home country no longer exists or almost dying in outer space.
@@GoldSrc_ i was thinking of Gus Grissom/Apollo 1 . i'm not even American but that death was pure horror. he avoided drowning when the hatch to mercury capsule blew off... only to die in the gas that keeps people alive while on the launchpad
@@eisenklad Yeah, Apollo 1 was horrible, and the capsule had lots of bad design choices, like a hatch that opens inwards and held close with latches, which made impossible to open with the higher pressure inside as well.
At the beginning of the video I thought "I have seen enough documentary about Apollo 13 - you can't tell me more". Yeah, well ... the hubris is strong with me :-) Thanks for the video and congratulations for one million subscribers!
Sometimes it's hard to find the right level of technicality for a public explanation. I submit there should be different levels you can choose from, depending on your own estimate of your education/competence. This one just a tiny bit too much in a couple of spots (the chemistry), but otherwise just right.
Thanks Scott for explaining this failure. When it occurred my Father-in-Law was a Physicist at NASA (Moffet Field). He told me he almost blew up his lab. Although he explained to me the failure modes, I didn’t really understand totally. I think I remember him saying the explosion should of separated the Command Module from the Service Module. It was a long time ago. Thank You for explaining it to me and and letting me relive a time I spent with him. Regards Howard
Methods that can measure mass is quite impractical in space I would presume. To use gas formulas would implies that one needs to measure the density directly?
Temperature and pressure are commonly used to determine operating mass even in situation where there are mixtures of vapor and liquid. Or even mixtures of differing gasses.
Yeah I got a brainfart there, totally got the ideal gas law backwards. Given a fixed volume, one DO get to calculate density directly from temperature and pressure. My thermaldynamics teacher would probably strangle me for that lol.
Something you didn't mention but which was in an article I read many years ago is that there was a temperature gauge to show the oxygen tank temperature during that long attempt to remove the excess oxygen from the damaged tank. The problem was that someone had decided that since there was a thermostat which would not allow the temperature to go above 78ºF (25ºC), there was no need for the temperature gauge to go any higher than that so it was deliberately "pegged" so it could not show a higher temperature. This meant that there was no way for anyone to know that the tank had ever overheated during the attempt to remove the oxygen. With only one point of failure and no double-check when the thermostat switch "welded" itself closed, there was no way of either controlling the temperature or of seeing that the temperature was uncontrolled.
I worked at an engineering firm in 1971 and I personally tested a device that I was told was being added to future Apollo missions to stop the oxygen tank failure from ever happening again. The device was an accelerometer that would be placed on a cross beam to detect a rocket engine premature shutdown. People are never mentioning, that when rocket engine #5 failed during liftoff, it shook the space craft with a low frequency vibration of around 130G force. I was told that is what caused the oxygen tank failure. Yes, parts might have already been damaged in the oxygen tank, but it was still functioning. It was the violent shaking of the tank that caused the final failure. I suspect that the temperature sensor was dislodged from its mounting and caused a false temperature reading inside the tank. This is a classic temperature feedback failure and runaway. A temperature sensor can be easily checked for malfunction and the system failure reported and the tank turned off. If the temperature sensor is working and dislodged loose from its mounting, the temperature control loop will start putting maximum heat into the system to control the temperature at its setpoint. The sensor not being in good thermal contact with the temperature zone will not see the true temperature rise from the heater and the system will run away and over heat. The real cause of the oxygen tank failure was the #5 rocket engine premature shutdown causing the intense vibration and dislodging the temperature sensor from its mounting. I have worked with temperature control loops for 35 years in mission critical areas where a sensor failure would cause massive damage. It was not possible to easily detect this failure in the 1960's. The only way to prevent that kind of failure was with a mechanical means and a thermal fuse. The mechanical means prevented the senor from being dislodge and the thermal fuse would break power to the heaters if the temperature rose about a certain point. The oxygen tanks obviously did not have a thermal fuse. The temperature in that tank should never been allowed to rise to 500 degrees. Yes, we did solve this problem much later when advanced computers were introduced. The solution was adding a thermal profile of the temperature zone into the control system software. If the heater was turned on full and the temperature rise was not within the thermal profile of the temperature zone, after a fixed about of time, the sensor must not be in proper contact with the thermal zone. This is easily done with computers today to prevent thermal runaways. What happens when the computer crashes with the heater on, a fail safe device would detect the lack of computer update and would remove all power to all temperature zones.
Several people have discovered this the hard way with various cheaper 3D printers. There's a 3D printer firmware version out there that has a setting turned off by default. It's to shut down the hot end heat if the thermal sensor reads a temperature that's obviously far too low and isn't going up as power to the heater is increased. The result is thermal runaway until something catches fire. There are ways to enable that setting on that revision, and later revisions have that enabled by default. Some companies, like Prusa, would make sure to enable the setting, but many of them would just download the firmware, change whatever was required to fit their printer's mechanical design, then start shipping printers. But the root cause of the problem was the people writing the open source printer firmware didn't think "Maximum safety, 100% of the time." and have the code have every safety setting enabled by default. What should have happened with the Apollo 13 tank was after it was damaged it should have been tagged as defective, not to be used for flight. Then it should have been cut apart to see what all had been damaged. Then the wiring problems would have been found, spurring a check of other wiring to ensure that the changes in voltage specification had been actually and properly addressed by really upgrading wires etc rather than either "pencil whipping" bogus change paperwork or as mentioned in the video, assuming that contacts that can handle higher than spec power can handle *switching* it. There's a documentary from 2003 titled "Space Shuttle Garage" (It was on either Discovery or History Channel) which covered all the work done to the Shuttles between flights. What it revealed was that most of the processes and procedures were frozen circa 1981. I only got to see part of it, but one part was in hindsight a bit horrifying. It showed a crew *failing* to get either one of the two special wheel nut torque wrenches to work properly. Did they keep at it until they got it right? Nope! End of shift they put a note in the procedure manual so the next shift could have a go. Here's this massive binder full of sheets of paper, looked like tabs and bookmarks and who knows what sticking out the sides. At the start of each shift the crew would have to go through that binder to figure out what had to be done, and what was supposed to have been done but wasn't and why, and hope the people who left problem notes had described things accurately. I was thinking "Oh. My. God. Why?" It's 2003 (or possibly the docu video shooting was done in 2002) why haven't they upgraded... anything? That binder should have years ago been replaced with a computer on a cart with all the material from the binder on it, fully indexed and searchable. It should also have note taking features and a scheduling and alerting system so that at the start of a shift the crew can login and what needs done is *right there* and any issues popped up with a cannot miss it alert, visual and auditory. Just as awful and outdated was the process for making replacement tiles. I thought they would have used CNC machines to carve new ones. Nope. 100% manual process. There was a warehouse with full size physical templates for every tile on every Shuttle, with their locations stored on index cards in an old wooden card catalog, exactly like the ones most libraries had abandoned for a computer index by at least the early 90's. Find the tile number (or figure out out from surrounding tiles if all the numbers are gone from the damaged one) then pull its card to find the template. Read measurements on the card to rough cut a block of silica foam on a bandsaw. Mount the foam block and template in a 3D carving pantograph and manually run the probe all over the template so the diamond cutting burr in the enclosed box side (with dust vacuum) shapes the foam. Then the foam gets a freehand dip into a white ceramic slurry, dried then fired. Black tiles get a second dip in the black coating. Of course there are Mark 1 Eyeball inspections after every step. The last step is pulling the tile's number stencil from a second old fashioned wooden card catalog and using it to airbrush the number onto the tile in several spots. I wonder what's become of all those tile templates? NASA could raise a good chunk of change by selling the templates as sets with their index and stencil cards. How about $500 a set times how many thousand tiles? Have an Ultra Premium higher priced set with a newly made tile to match. Supplies for "UP Grade" sets limited by how much leftover silica foam and coating remains. I figured I'd be able to catch a subsequent showing in full, since Discovery and History repeat everything multiple times. But a couple of months later Columbia broke apart and that warts and all documentary has been very effectively memory holed. I wouldn't be surprised to find the master recordings and every copy held by everyone involved was ordered deleted. It never appeared for sale on the Discovery or History web stores. History Channel had another documentary on spies that was a one time only showing and has never been seen again, I suspect due to a couple of inconvenient revelations it had about Chinese spies and Vietnam, but that's a subject for a different thread.
Basically, a well paid engineer had the perfect idea to put the only two oxygen tanks on the same shelf, to be sure that if one explodes, it will kill the other one in the same time. In aeronautics we try to separate critical systems, mainly to prevent this type of dangerous troubles. That's surprising no one thought about this for a space related mission =/ (sorry if my english sucks)
A great further (more detailed) insight into something I've been "studying" on & off for about a year now, out of total fascination for the Causation Factor(s) of the explosion. Thanks Scot. Great work.
I don't remember Apollo 11 very well, but I do remember as a 5 year old talking about Apollo 13 with my mother, even back then I was being the engineer and trying to solve the problem. I remember asking my mom why the Russians cannot go and rescue them. When I was growing up in the early 60's everything was space and the moon, for me, it was just something that was normal and that we always did!
There are a lot of people and they put in a lot effort to create videos about space and space industries. Person can say who is making a video with understanding and who is there just to show some content. Great video....
Recently picked up KSP and today I saw for the first time that they actually put "flying safe" into the loading screen. Just the best
I like your name as well as “stirring oxygen tanks”
That’s been there since at least 1.2 and I love it: every time I geek out a little and thank Scott for getting me into Kerbal Space Program through his videos.
The updates to the loading screen over the years have been great..Nust little Easter eggs to streamers and RUclipsrs
One element to this story which is present in Lovell's book, but which you didn't mention - and is worth highlighting as it's yet another of the coincidences and minor failures that contributed to the explosion. When they were attempting to drain the tanks using the heaters whey were also monitoring the temperature within the tanks - this should have meant they would have noticed they were baking it at 500C... But they temperature sensors and monitor they were using was calibrates only to go up as high as the 27C they expected the thermostats to cut out at. So when the thermostats failed, and burned off the insulation - as far as the technicians could tell, the temperatures were precisely the 27C they were expecting - not the catastrophic 500C. If their thermometers could go higher than 27C, they'd have immediately know the tank was faulty.
Just makes me thing of the Chernobyl series - "It's ... the maximum reading on low-limit dosimeters. They gave us the number they had...."
Dave F Ouch! The law of unintended consequences.
I know.... I knew this but forgot to mention it
Always design your stuff as if it will be hacked by somebody.
The temperature monitor only went up to 80F. They knew the temperature would be a little higher because they were using the heaters to remove the O2 but yeah, if they had a wider (higher) range on the monitor, they'd have seen that something was wrong with the tank shortly after the pad test.
A good case for when testing things, choose a gauge with a range high enough to tell you if things have gone catastrophically wrong.
That is the most detailed explanation I've ever heard for the Apollo 13 failure. Great job, as usual Scott.
How about one million + parts flying in close formation.. supplied by the lowest bidder 🐒
Yes. I was 9 years old when this happened and watched and covered any info that I could find at the time even though I was to young to fully understand it all.
@@puremaga17 >>> A helicopter? 😜
You should read Jim Lovell's book Lost Moon/Apollo 13.
It explains in great detail, what happened with the tank, on the ground.
Before this, I thought I knew what happened. Turns out I am not too old to learn after all.
The official report is really interesting. Any engineer should read it to understand just how simple things can add up to a huge problem. I always tell people they should read it but it’s pretty dry. This video is an excellent summary of events!
WDE46 I love reading reports like that! Dry or not they are the true source of facts!
If you like that kind of thing, check out some of the reports for marine and air accident investigations. I've read some from most english speaking countries, and they are enjoyable for the same reason, and the writing is usually very good, if formal.
@@svchineeljunk-riggedschoon4038 Or rail accidents as well ... it's astonishing how many are just a sequence of error after error after error. Of course, what that means is that just one error fewer, and the accident wouldn't have happened.
@@KaiHenningsen The US chemical safety board have some amazing videos here, like above, it's interesting how one small thing can quickly snowball into a disaster.
SV Chineel, Junk-Rigged Schooner Sailing Hi! it’s funny your name is similar to one of the sailing channel i follow, i seem to follow a lot of sailing channels! Yes I quite often read the AAIB bulletins. The Air Safety Institute channel is very good.
i feel kinda proud for the humanity that we hit a million subscribers on this channel
Thats what happens when you provide quality content !
Gradatim Ferociter is very appropriate in this case
I agree with you, although I miss the old rocket intro.
Yes !
I feel like this channel deserves at least 100 million subs for such quality content, I find the fact that it only has one million very disappointing.
Remember kids, redundancy is crucial in systems that are truly vital. Just remember to route all three of your redundant hydraulic control lines right next to each other so the occasional stray turbofan disc going trough your aircraft has a chance of slicing through all three of them in a single go!
*DC10 references intensity*
Damn, that's exactly what I thought when I read the technical data books about all this stuff ! Happy to see I'm not hte only one !
@@Cby0530 lol
@@jjohnston94 It"s not funny. Hundreds of people died because of it. Worse than 737 MAX. I worked with one of the people responsible on a later project and it didn't seem to have hurt his career much.
@@dalecomer5951 It's called gallows humor and it's funny
I worked for the company that built that thermostat that welded. The design engineer for the thermostat line had the same picture you showed on the wall of his cube as a reminder that limits are not to be exceeded. That example was used when customers would call and ask if they could be used at higher voltages/currents. Nearly all of the people who were working at that time had retired when I started, I wish I could have gotten first hand accounts of the aftermath.
so it happened much slower then the movies shows it happening🤔🤔
But, the cryotank in Apollo 13's SM was originally installed in Apollo 10 and removed for test anomalies. When it was refurbished and installed in 13, North American Rockwell neglected to swap out the thermostat. The voltage tolerances had changed since 10.
I wonder if Lee Atwood tore that technician a new asshole.
@@dalethelander3781 if not he should have
dramatizations are like that. the movie "margin call" portays the 2008 financial crisis as if it happened overnight, when it reality it took months, I think.
It happened during 2007, there were a lot of articles detailing the house of cards that was starting to fall. It was the first time I read "trillion dollar loss"
"Sir, we dropped the highly critical module, lots of brittle components in there."
"Nah, it'll be fine."
I can only imagine what the astronauts would have been thinking when the service module blew out! Disasters in space are scary stuff. Kudos to NASA for bringing everyone back safely! This mission is probably the most incredible point in the entire space program. Thank you Scott for explaining how it all happened in a way that's so easy to understand!
Well, their immediate response was, "Huh? What was that?" There's a few different things that can cause jolts in the spacecraft and in reality, it took them quite a while to begin recognizing the severity of this issue. They had been dealing with the problem for about 90 minutes before they realized that the command module was beyond saving and started powering up the Lunar Module.
Trumann sent north American. Rockwell a gag bill for towing the CSM back from space
I’m surprised ground wasn’t monitoring heater current when they decided to use the heater to drain O2 tank 2. If they had been they would have seen that the heater was not cycling as it should have been, and probably caught the problem there. Alternatively, as has been mentioned elsewhere in the comment section, if the tank thermometer read above 80 degrees that most likely also would have worked to catch the problem. Of course with the benefit of hindsight it’s easy to be critical. The steps the crew and ground support took after the accident occurred were nothing short of incredible, and really spoke to how well they worked together and knew their spacecraft.
Current is not that easy to monitor when your ADCs weight at least 300 grams.
The thermostat wasn't able to read above 80.
@@TheCrackedFirebird I feel like I have heard this before but with radiation... Not great, not terrible.
@@TheCrackedFirebird Why would anyone need a thermostat to go higher in a cryogenic tank?? Of course, what's not really been said by others is that it's remarkable that the tank didn't halt and catch fire duing the draining and refilling. It's a real testimate to just how astonishingly well built the damn thing was that it managed to get as far as going into space. It's just a bummer that's the time when it choose to go kaboom. Equally astonishing is how well built the rest of the CM/SM that it survived the explosion and that it was able to be restarted after several days in a deep cold state - frankly the most frightening line in the movie after "we're venting something into space' is 'like driving a toaster through a car-wash'.
@Bobb Grimley Thanks Bobb, glad you're out there checking the internet for mistakes. Funnily enough, my spellchecker tells me that there is no such word as 'Bobb' either. Just in case you want to let the person who named you that they made a spelling mistake too.
There was a temperature gauge on the launch pad to display the temperature inside the O2 tank heater assembly. During the boil off process the temperature inside the tank heater got up to more than 500F but unfortunately the temperature gauge only went up to 80F. The gauge was pegged all the way up to the top at 80 and so no one realized that the temperature inside the tank heater was up so high that it was damaging the insulation on the wires.
That's something I forgot to mention
@@scottmanley Giving credit where credit is due - nice Scott👍🏼😄
I guess look on the bright side - the o2 tank could've exploded during launch to orbit instead of afterwards when it did
@@opl500 It wouldn't have exploded that early in the mission anyway. It was too early to stir the tanks. From what I've read the latest in the mission that it could have exploded and the crew might still have had a chance to survive was the beginning of powered descent. Once a sizable portion of the fuel in the LM's descent stage was used up they wouldn't have had any way to get back out of lunar orbit.
@@joevignolor4u949 I'm confused...what has the amount of fuel in the LM got to do with the O2 Service Module tank exploding???
8:50
"Houston, we've had a problem here"
Movie: Catastrophic explosion, shock from crew, drama
Real life: 80's freeze frame and festive steel drum flourish
wow that was A lot of detail that I have never heard before. Great video Scott thank you and stay safe.
I was a kid, was all over ever aspect of Apollo. Kepy an eye on it over the years. I am now 60. NEVER have I heard any detailed explanation of what happened, how and why. Also. You are scary smart. You smoothly went through every engineering detail like you built the tank and system. You have filled in an important blank of what happened in my youth. Very Impressed.
The LOX tanks were made at a Beechcraft plant north of Boulder ,Colorado. I knew one of the engineers and he gave exactly the same explanation. Well done it is rare these days for any media to get a story right by doing their homework such as you have. Well done Sir !
I can’t believe fire in space look exactly the same as that in Minecraft
Where do you think Minecraft got their fire animation from?
@@privateer_am That's a stupid statement there kid.
@@CaseyFinSF
are you joking?
@@Killbayne Dont feed the troll :)
@@CaseyFinSF woosh
Can't you go over all the changes in Apollo 14 that never gets talked about
See @11:15 - he mentions these briefly.
++Apollo
Did they redesign the CO2 filter assemblies to be the same shape and size?
@@georgeemil3618
As far as I know: No.
The Apollo hardware was a steady flow of already built hardware. You do not want to include major structural changes as long as you absolutely do need them.
A13 has shown that it is possible to build an adaptor even when in flight. If I had to make the decision, I would say: good enough.If you absoultely insist, construct a prefessional adaptor and put that on the packing list, but otherwise: don't change a working system unless there is a good reason to do so. After all this was in accident, which should not have happend in the first place.
@@kallewirsch2263 Major structural changes were made to the SM for the J missions to accomodate the SIM bay.
One design deficiency was also that both oxygen (and hydrogen) tanks fed the fuel cells over a common manifold. No isolation valves, no check valves. You have one leak anywhere in the system, you will loose everything. If there had been the possibility to isolate the faulty tank, the outcome would've been less dangerous. Yes, the moon would've been lost. But there would've been enough oxygen in the good tank to return home. They later changed that for the J- missions and AFAIK the space shuttle not only had check valves between the tanks and the manifold, but also the possibility to switch to 'split plant' operation, i.e. feeding each FC from its 'own' tank and isolating them from each other. I still don't get why they didn't implement that in the first place. They used statistical methods to determine how often a system would fail in a given number of times, but overlooked that interconnecting independend units increases the failure rate. Strange...
You would think NASA would have learned about planning for equipment isolation after Gemini 8. That has always bothered me.
10:08 everyone gangsta til service module 106 gets hit by a ghast
I thought I was the only one to notice that the fire is from minecraft textures lol
i never thought all those years ago when i was trying to get into orbit on the free version of KSP that i would still be watching your videos in the next decade
I just finally watched the movie yesterday, because of you mentioning it. And I was really curious about this, of course. Excellent timing.
I was literally just looking for more info on this yesterday. Like a video of what stirring the tanks was like.
Ya, I thought they just shook them up and down.
Scott is a valuable human resource.
Live Safe
mVm
mVm MotoVlogMusic You mean “fly safe”
Scott you're just awesome. Thanks for going into such detail and helping us forget our current situation.
What I don't understand is why these tanks were even used at all. You said they were removed from Apollo 10's Service Module and replaced with upgraded tanks. Why, then, weren't "upgraded" tanks used on Apollo 13? Were the original tanks "upgraded," then installed in 13's Service Module?
Well, I never really understood what was the cause of the explosion. Now I know, after 20 yrs! Thank you Mr. Manley!
The cause was bloody stupid penny pinching. When the damage to the tank was discovered, it should have immediately been tagged as defective then cut apart to see what all had been affected by the impact.
Най-краткото и най-точно обяснение на случилото се с Аполо 13 .
Както винаги страхотно видео .
Respect.
Fantastic explanation, Scott. It's good to hear what really happened; that it wasn't just a 'spark' in the cryotank.
A spark in the cryo tank would been catastrophic?…
I have no engineering background at all and found this engaging and fascinating, not to mention well-explained. Scott, thank you for your service to the rest of us who love space (and space history) but didn’t go to MIT.
Bad teflon ruined my morning eggs so many times...no wonder they had problems with it on Apollo.
Not supposed to use metal utensils. And I’m only guessing that’s what caused it.
I already knew what caused the explosion on Apollo 13 but I also knew that I'd learn a bunch of new and interesting details by clicking on this video because it's Scott Manly!
This channel might end up being my favorite channel on space and one of my top 3 channels on RUclips in all categories. Definately the best walk thru of Apollo 13's tank issue. Thanks.
Just started watching but had to get my up vote and comment in to help you in the algos since the video is only 30 minutes old.
.... and this is one of my favorite of your videos. Love this deep dive!
About 30 years ago, when I was working for IBM in the UK, they had an all-site meeting at a local convention center, and the guest speaker was none other than Jim Lovell, commander of Apollo 13. Whatever IBM had paid him to be there, it was money well spent. He gave an enthralling and inspiring account of the mission and the entire audience was hanging on every word.
I challenge you to find an orkier space repair job than Apollo 13.
"Oh no, vital systems of the ship blew up!"
"Don't worry, we have some cardboard, plastic bags, and duct tape! We will be fine!"
You do realize those are two separate problems they had to deal with, right?
@@yes_head I was also talking about orks. How much do you think I cared about precision in that instance?
Nazamroth oi yu dum gitz! Da air fingy blowed up! Git tha fixer tape n baggies
You must not care about "precision" at all, given you misspelled "orcs", not to mention, you have them consulting with NASA on how to fix engineering problems during spaceflight....
Lame attempt at humor...
Try again.
@@codymoe4986I believe they’re referring to the Orks in Warhammer…
Thanks, Scott. I really enjoyed the detail and clarity.
Lovell/Kluger's explanation of this in the book of Apollo13 is almost as complete but contained one extra titbit you didn't mention: The pressure gauge.
When the thermostats stuck shut (ohm's law - double the voltage, double the current, quadruple the power) to boil the LOx off, the pressure gauge should have alerted them that it was way high. The problem was the gauge topped out in normal use meaning the most basic way they could have detected this failure eluded them.
The lesson here stands true in anything with a gauge: Always make sure the gauge can read higher than you expect it ever to need to!
Thus endeth lesson 37 in the gospel of engineering according to me....
Lovell did also mention they duplicated the whole thing on one of the test flight articles. I hope they didn't throw away the recordings of that event.
Captain G should take an engineer to realize this tidbit. But it takes an engineer to reason his/her way out of using an overly capable gauge
So "Not bad, not terrible" strikes again.
Exactly right. Michael Crichton actually took inspiration from Apollo 13, I think, in the Jurassic Park novel: The motion sensors were programmed to make sure none of the dinosaurs had escaped so it topped out at the number there was supposed to be, but they also should’ve programmed them to make sure there weren’t more.
Let's not forget the bean counter, down stream, who gets a deal on a cheaper gauge...
This is great thanks - feel like I’ve been waiting 50 years for a good explanation like this.
"But at 50+ atmospheres of pure oxygen..." yeesh
Doesn't seem like a highly unusual partial pressure for oxygen by itself. Diving equipment has a partial oxygen pressure (gas mix pressure x concentration) of 40bar when filled with air. But running electrical equipment in that environment is a different thing, of course...
@@Kenionatus Air & oxygen are not the same though.
I'm not exactly familiar with the explosive properties of pure oxygen in liquid form but I imagine it's extremely dangerous.
50+ atmospheres would not be 50+ psi but 50 x 14.7 psi, right? Somebody explain.
@@lawrencequave8691 1 atmosphere =1 BAR = 14.7 PSI
In the 80’s I attended a 5 day Configuration Management course for military hardware. Th presenter was American who was a CM manager on Apollo. He claimed that the burning of the insulation was effectively a CM failure, as the tanks had been changed, but the test procedure was one version behind, and had put too much voltage in. He claimed the design change made the insulation thinner. Your explanation is much more detailed than his!
I never really thought about the fact that the failure on the O2 tank was in fact caused by a piece of hardware that was later determined to be unnecessary.
Jason Gibbs Wrong, the fan/agitator wasn’t necessary, the heating element was still required to maintain the pressure at super critical equilibrium
Saw the Apollo 13 capsule at Hutchinson Kansas. Absolutely insane detail
I don't always test my rockets. But when I do, I do it in space.
George D. Marsack hopefully you mean in ksp
Excellent.....no waffle, no padding....just concise, clear explanation. At the time, I listened/watched every minute of the drama. At one time, real doubts of survival were intimated, and we all held our breath. Mission Control's finest hour.
Thanks for the great videos in are extremely boring lockdown. fly safe people
Is it only me who's Scott's 'Fly safe' - really makes me comforting ;)
when he adds the minecraft fire you know hes cultured
Yes, that's the most comprehensive account of the catalogue of failures that led to the catastrophic explosion of that tank on Apollo13. Well presented.
I guess it's like aviation - fail to dot enough i's or cross enough t's and you're hating life.
but it's always the missed dot on the lower case j that causes the REAL problems
@@bumrocky Or the missed bar over the R
@@moejoe987654321 which rocket was that? I forget..
@@igvc1876 The Mariner 1, I think Sotty boy has a vid on it
I wrote a white paper in college on this subject. I also spoke to Jim Lovell about it. Thanks for posting on this. Great video!! Thanks Scott!!
Ground work: Empties to 92%
History: Close enough, its not like this is rocket science.
😏
They should have replaced the tank right now and there. Who ever made the decision to proceed with that tank is a retard.
@get to the Choppaa really?!? wtf
@get to the Choppaa And the flight manager that didn"t want to bother her boss at home on a Saturday when they thought they might have a problem from the external tank foam hit on the wing of Columbia and then sleep on it over the weekend while neglecting to inform the flight crew of their concern.
The faults of man. I believe political and financial pressure on the launch had a role to play in all this. Given the incredible complexity of the build, I imagine there were only a few that could do it without damaging something else.... But then again most engineers, managers and others think those of us who turn a wrench are replaceable; "whatever you were doing can be done by somebody else" attitude. What's clear to me is someone knew there was a problem with the tank and chose to rely upon the backup procedure for the primary.
I've been following the Apollo 50 account on Twitter (definitely recommended by the way) who tweet in real time about the missions from 50 years ago. It's been fun watching their updates recently about all the O2 tank issues during the build up to launch
It"s reminiscent of the decision to ground test with the crew in the Command Module on 100% oxygen and at the same time not use fire resistant materials in the CM since a fire could be extinguished quickly in space by dumping the oxygen. Both decisions by high level NASA managers, possibly even the same one.
The issue was not, that it was 100% oxygen.
In order to perform the leakage test, the pressure inside the capsule had to be raised way over 1 bar.
Something that would have never happend in space. In fact when in space the pressure in the capsule was way lower then 1 bar, giving roughly the same amount of oxygen per volume as it would be, would there be ordinary air.
So with the regular pressure in flight, the risk is more or less exactly the same as on earth under regular atmospheric conditions.
It only happend because the leak test was done with the capsule at higher pressure then normal. The issue was solved by not filling the capsule with pure oxygen at the Cape but instead staying with air but add an additional pressure valve which would close during the ascent, when the dropping pressure reached the intended capsule pressure. From them on, the air was steadily replaced with pure oxygen.
@@kallewirsch2263 So you are stating the crew would have been okay if the capsule pressure had only been 1 bar or so?
@@dalecomer5951
No.
The crew would have been ok, if the test would have been performed with ordinary air.
But this was not possible, since the capsule was not equipped with air containers. The capsule only had the facilities to be filled with oxygen. That is why they used it to perform the test.
It was a leakage test. Thus they needed to pressurize the capsule with a higher pressure then the surroundings. Just using 1 bar (same as the surroundings) for the test would not have been enough. They NEEDED the higher pressure for the test.
@@kallewirsch2263 Out of curiosity, when were you born?
@@dalecomer5951
1963
I keep re-watching videos that I know I've liked, and finding that I have to like them again. All your videos too.
Wait so the tank failed a test on the ground and they just said "send it"?
and even before that they were fine with using a suspected damaged ox tank
.. and it failed multiple tests. lol
Same could be said for the Challenger. These folks are under such immense pressure to launch or pure ignorance, they jeopardize everyone's lives.
VaporheadATC Yeah NASA has way too many managers in management!
It's easy to criticize in retrospect, not so much when you have an entire multi billion dollar government program and agency breathing down your neck and the eyes of the world watching your every move with bated breath.
Thanks Scott for making it clear what probably happen on Apollo 13
What a coincidence, just recently watched the 90's film starring Tom Hanks about this incident.
You mean the alleged pedophile?
Thank you for a great explanation of something I have wondered about my whole life. I was a little boy when it happened. Everything I have heard about it since then has left me with more questions than answers. Thank you for finally clearing things up.
The BBC are doing a podcast series on this called ‘13 minutes to the moon’.
www.bbc.co.uk/programmes/w13xttx2/episodes/player
Cool, that will keep me occupied for a few hours, thank you!
@@621Lafayette Yes, agreed this is an excellent podcast. Season 2 is Apollo 13. Season 1 is the moon landing. Both are very good and highly recommended.
probably available in other places (?), but this is the first time anyone has explained just what happened. Thanks Scott.
Imagine damaging life-critical equipment and then putting it into crewed space vehicle anyway. What is this, soviet union QA standards? Pretty sure they had better. Geez.
It was nothing personal, just business
EXACTLY
Well, Soviet Union/Russia had the same number of deadly incidents in their crewed space program with an almost order of magnitude more launches.
Thanks Scott, a nice chewy video to get your teeth into.
I can't believe they didn't do more when they found they couldn't empty the tank. If they'd investigated, the whole disaster would have been avoided. What on earth did they think - that it didn't matter, or that fairies had fixed it?!
Mind you, I also can't believe that I sat at the tea table, with the whole family dead quiet, listening to the static on the radio, waiting (it seemed like forever) for Apollo 13 to reply, after re-entry.
50 years ago!! I cannot possibly be that old....
I was in an audience with where Jim Lovell was the speaker. True or not he said that they found that someone had dropped one of the components during construction and thats what caused the problem.
John Cheresna Lots of holes lined up for the problem to occur!
Scott, your video is the best technical explanation of the Apollo 13 oxygen tank failure I've ever seen. Thank you.
Its because Tom Hanks was on that mission
I thought he was landing an airliner in the Hudson River that day.
Never put Tom Hanks on something involves flight, or you might end up stranded on a deserted island, having to land your plane on a river, getting stuck on the airport because your home country no longer exists or almost dying in outer space.
@@jackmcslay don't forget cargo ships. In fact, don't let Tom Hanks be in charge of anything.
What happens if we put Tom Hanks and Matt Damon in the same movie?? (Again?)
Wasn't he in Australia filming a movie on a corona-like virus when he caught the corona virus?
An excellent rundown that I've been waiting to hear since the words "We've got a problem" were spoken. Thank you filling that long standing gap.
I love how you say mun instead of moon.
Clearest explanation of this problem I have ever heard.
Ah yes, enslaved oxygen breaking free.
Thank you, for the laugh.👍
"The only thing we have to lose is our carbohydrate chains!"
I love when Scott posts videos about the real events from Apollo 13 and explains what really happened.
are you going to do all the tragedies that NASA/Roscosmos had on their respective anniversaries?
eisenklad That would be great!
Challenger is still too much for me, it makes me sad and it pisses me off because people could have been saved that day, just like Columbia ;_;
@@GoldSrc_ i was thinking of Gus Grissom/Apollo 1 . i'm not even American but that death was pure horror.
he avoided drowning when the hatch to mercury capsule blew off... only to die in the gas that keeps people alive while on the launchpad
@@eisenklad Yeah, Apollo 1 was horrible, and the capsule had lots of bad design choices, like a hatch that opens inwards and held close with latches, which made impossible to open with the higher pressure inside as well.
At the beginning of the video I thought "I have seen enough documentary about Apollo 13 - you can't tell me more". Yeah, well ... the hubris is strong with me :-)
Thanks for the video and congratulations for one million subscribers!
Sometimes it's hard to find the right level of technicality for a public explanation. I submit there should be different levels you can choose from, depending on your own estimate of your education/competence. This one just a tiny bit too much in a couple of spots (the chemistry), but otherwise just right.
@@jjohnston94 I found it perfect - especially *because* of the chemistry. This one had been left out in the other documentaries.
10:10 Minecraft fire animation
Thanks Scott for explaining this failure. When it occurred my Father-in-Law was a Physicist at NASA (Moffet Field). He told me he almost blew up his lab. Although he explained to me the failure modes, I didn’t really understand totally. I think I remember him saying the explosion should of separated the Command Module from the Service Module. It was a long time ago. Thank You for explaining it to me and and letting me relive a time I spent with him.
Regards
Howard
My wife is supercritical, she puts me under immense pressure...
hey, as long as there's no surface tension you're good
Thanks for that Scott, superb detail as ever!
So if the oxygen is in a supercritical state, how do you measure the fullness of it?
Both pressure and temperature measurements work as usual, leading to the usual routes to calculate how much gas is present.
And mass.
Methods that can measure mass is quite impractical in space I would presume. To use gas formulas would implies that one needs to measure the density directly?
Temperature and pressure are commonly used to determine operating mass even in situation where there are mixtures of vapor and liquid. Or even mixtures of differing gasses.
Yeah I got a brainfart there, totally got the ideal gas law backwards. Given a fixed volume, one DO get to calculate density directly from temperature and pressure. My thermaldynamics teacher would probably strangle me for that lol.
Great video with detailed explanation. Thanks!
Comments: 8
I can count at least 23
Prove it.
It's called caching. RUclips doesn't keep all stats up to date for everyone equally.
Something you didn't mention but which was in an article I read many years ago is that there was a temperature gauge to show the oxygen tank temperature during that long attempt to remove the excess oxygen from the damaged tank. The problem was that someone had decided that since there was a thermostat which would not allow the temperature to go above 78ºF (25ºC), there was no need for the temperature gauge to go any higher than that so it was deliberately "pegged" so it could not show a higher temperature. This meant that there was no way for anyone to know that the tank had ever overheated during the attempt to remove the oxygen. With only one point of failure and no double-check when the thermostat switch "welded" itself closed, there was no way of either controlling the temperature or of seeing that the temperature was uncontrolled.
First!
*second
VERY well researched Scott. GOOD JOB!
I worked at an engineering firm in 1971 and I personally tested a device that I was told was being added to future Apollo missions to stop the oxygen tank failure from ever happening again. The device was an accelerometer that would be placed on a cross beam to detect a rocket engine premature shutdown. People are never mentioning, that when rocket engine #5 failed during liftoff, it shook the space craft with a low frequency vibration of around 130G force. I was told that is what caused the oxygen tank failure. Yes, parts might have already been damaged in the oxygen tank, but it was still functioning. It was the violent shaking of the tank that caused the final failure. I suspect that the temperature sensor was dislodged from its mounting and caused a false temperature reading inside the tank. This is a classic temperature feedback failure and runaway. A temperature sensor can be easily checked for malfunction and the system failure reported and the tank turned off. If the temperature sensor is working and dislodged loose from its mounting, the temperature control loop will start putting maximum heat into the system to control the temperature at its setpoint. The sensor not being in good thermal contact with the temperature zone will not see the true temperature rise from the heater and the system will run away and over heat. The real cause of the oxygen tank failure was the #5 rocket engine premature shutdown causing the intense vibration and dislodging the temperature sensor from its mounting. I have worked with temperature control loops for 35 years in mission critical areas where a sensor failure would cause massive damage. It was not possible to easily detect this failure in the 1960's. The only way to prevent that kind of failure was with a mechanical means and a thermal fuse. The mechanical means prevented the senor from being dislodge and the thermal fuse would break power to the heaters if the temperature rose about a certain point. The oxygen tanks obviously did not have a thermal fuse. The temperature in that tank should never been allowed to rise to 500 degrees. Yes, we did solve this problem much later when advanced computers were introduced. The solution was adding a thermal profile of the temperature zone into the control system software. If the heater was turned on full and the temperature rise was not within the thermal profile of the temperature zone, after a fixed about of time, the sensor must not be in proper contact with the thermal zone. This is easily done with computers today to prevent thermal runaways. What happens when the computer crashes with the heater on, a fail safe device would detect the lack of computer update and would remove all power to all temperature zones.
Several people have discovered this the hard way with various cheaper 3D printers. There's a 3D printer firmware version out there that has a setting turned off by default. It's to shut down the hot end heat if the thermal sensor reads a temperature that's obviously far too low and isn't going up as power to the heater is increased. The result is thermal runaway until something catches fire. There are ways to enable that setting on that revision, and later revisions have that enabled by default. Some companies, like Prusa, would make sure to enable the setting, but many of them would just download the firmware, change whatever was required to fit their printer's mechanical design, then start shipping printers.
But the root cause of the problem was the people writing the open source printer firmware didn't think "Maximum safety, 100% of the time." and have the code have every safety setting enabled by default.
What should have happened with the Apollo 13 tank was after it was damaged it should have been tagged as defective, not to be used for flight. Then it should have been cut apart to see what all had been damaged. Then the wiring problems would have been found, spurring a check of other wiring to ensure that the changes in voltage specification had been actually and properly addressed by really upgrading wires etc rather than either "pencil whipping" bogus change paperwork or as mentioned in the video, assuming that contacts that can handle higher than spec power can handle *switching* it.
There's a documentary from 2003 titled "Space Shuttle Garage" (It was on either Discovery or History Channel) which covered all the work done to the Shuttles between flights. What it revealed was that most of the processes and procedures were frozen circa 1981. I only got to see part of it, but one part was in hindsight a bit horrifying. It showed a crew *failing* to get either one of the two special wheel nut torque wrenches to work properly. Did they keep at it until they got it right? Nope! End of shift they put a note in the procedure manual so the next shift could have a go. Here's this massive binder full of sheets of paper, looked like tabs and bookmarks and who knows what sticking out the sides. At the start of each shift the crew would have to go through that binder to figure out what had to be done, and what was supposed to have been done but wasn't and why, and hope the people who left problem notes had described things accurately.
I was thinking "Oh. My. God. Why?" It's 2003 (or possibly the docu video shooting was done in 2002) why haven't they upgraded... anything? That binder should have years ago been replaced with a computer on a cart with all the material from the binder on it, fully indexed and searchable. It should also have note taking features and a scheduling and alerting system so that at the start of a shift the crew can login and what needs done is *right there* and any issues popped up with a cannot miss it alert, visual and auditory.
Just as awful and outdated was the process for making replacement tiles. I thought they would have used CNC machines to carve new ones. Nope. 100% manual process. There was a warehouse with full size physical templates for every tile on every Shuttle, with their locations stored on index cards in an old wooden card catalog, exactly like the ones most libraries had abandoned for a computer index by at least the early 90's. Find the tile number (or figure out out from surrounding tiles if all the numbers are gone from the damaged one) then pull its card to find the template. Read measurements on the card to rough cut a block of silica foam on a bandsaw. Mount the foam block and template in a 3D carving pantograph and manually run the probe all over the template so the diamond cutting burr in the enclosed box side (with dust vacuum) shapes the foam. Then the foam gets a freehand dip into a white ceramic slurry, dried then fired. Black tiles get a second dip in the black coating. Of course there are Mark 1 Eyeball inspections after every step. The last step is pulling the tile's number stencil from a second old fashioned wooden card catalog and using it to airbrush the number onto the tile in several spots. I wonder what's become of all those tile templates? NASA could raise a good chunk of change by selling the templates as sets with their index and stencil cards. How about $500 a set times how many thousand tiles? Have an Ultra Premium higher priced set with a newly made tile to match. Supplies for "UP Grade" sets limited by how much leftover silica foam and coating remains.
I figured I'd be able to catch a subsequent showing in full, since Discovery and History repeat everything multiple times. But a couple of months later Columbia broke apart and that warts and all documentary has been very effectively memory holed. I wouldn't be surprised to find the master recordings and every copy held by everyone involved was ordered deleted. It never appeared for sale on the Discovery or History web stores.
History Channel had another documentary on spies that was a one time only showing and has never been seen again, I suspect due to a couple of inconvenient revelations it had about Chinese spies and Vietnam, but that's a subject for a different thread.
Wow!
The most correct and complete explanation of the oxygen tank explosion on Apollo 13 that I have ever seen.
Thank you Mr. Manley,
Anthony
Scott you are a legend in the game
Basically, a well paid engineer had the perfect idea to put the only two oxygen tanks on the same shelf, to be sure that if one explodes, it will kill the other one in the same time. In aeronautics we try to separate critical systems, mainly to prevent this type of dangerous troubles. That's surprising no one thought about this for a space related mission =/ (sorry if my english sucks)
Scott, thanks for this video (and all of your others) a great description of the events. Well done and easy to understand
Good Breakdown of the Sequence of Events that led to the Tank Failure.
Wow! Never found an explaining that was so detailed and well presented! Great job as always!
Scott, congratulations on 1 million subs, you deserve it. Thank you for this outstanding channel!
Thanks. I''ve heard so many inaccurate and conflicting explanations; I appreciate a definitive analysis.
A great further (more detailed) insight into something I've been "studying" on & off for about a year now, out of total fascination for the Causation Factor(s) of the explosion. Thanks Scot. Great work.
I always love the pictures/videos you're able to find to compliment what you're explaining!
I don't remember Apollo 11 very well, but I do remember as a 5 year old talking about Apollo 13 with my mother, even back then I was being the engineer and trying to solve the problem.
I remember asking my mom why the Russians cannot go and rescue them. When I was growing up in the early 60's everything was space and the moon, for me, it was just something that was normal and that we always did!
Such a fascinating recount/explanation. Like any accident- there are almost always a myriad of small errors leading up to the incident. Very cool
ahhhhhh I see now!!! Thanks for explaining it nice and clearly!!
Brilliant summary Scott!
I like how "the most replayed" part of this Scott Manley's video is the 3 second intro being skipped lol
We all watched the whole video
There are a lot of people and they put in a lot effort to create videos about space and space industries.
Person can say who is making a video with understanding and who is there just to show some content.
Great video....
Scott, thank you for keeping me entertained in these times of isolation! Fly safe.