Afternoon Sal, Retired USCG Marine Engineer Investigator here. Having investigated a similar occurrence on board an LNG ship headed into port back several years ago, there are some similarities here. That ship, an all steam propulsion system, lost steam pressure which shut down all electrical and propulsion systems. During the crews attempt to restore power, the ships high voltage and low voltage buses kept tripping off after being reset. The ship was dead except for the emergency bus for 5 days offshore while being towed around in a circle. What we found after downloading the automation data, was that all the main breakers and bus ties are connected by a PLC (Computer) that tells it the position of all the other breakers and will prevent you from closing the wrong one. The engineers realized that each time the breaker tripped, the PLC had to be reset for every breaker every time it tripped. So if it trips 6 times, you had to press the reset 6 times to clear it on all the breakers. The engineers did not know this since they had never operated these manually before. Once that was found, the systems operated normally. A 100k ton ship dead due to something so small is what I'm expecting them to find.
Seems like an odd choice to have the Diesel engine rely on external generators and power its pumps. From an engineering perspective it just seems odd to rely on external sources and busses in order to keep something as important as the engine running. I guess they tried to make up for it by having multiple breakers and multiple power sources.
Since quite a few minutes passed between the two trip events and if the PLC requires a reset procedure, wouldnt it immediately trip the breaker(s) again when they are manually closed? Thats not what happened here. Power came back for a while and then went out again. And since a reset procedure doesnt explain the first trip event, the second was most likely for the same reason and I suspect an actual electrical reason like a power surge when all systems are running (which could explain the delay between the two blackouts) or faulty sensor readings that would cause a trip event. Well, its all speculation. Hyundai will find the reason, I'm pretty sure about that.
PLC, very little info from those they just followed the lines of code, hidden reset was a bum bit of logic, that was a bad week for you, the PLC reset should have been in Std Ops manual, who got their are kicked ?. As for this ship I am puzzled at the flakey operation of the HV / LV control logic as it through teddies out of the pram after losing one gen from low RPM. Also why was the EGT not in alarm giving advanced warning of a closed damper. Have I read this wrong was there only one Gen on the board?. The system should have been able to pick up the loads. Still not enough technical data. HV disconnects today are very complex and it is that logged data I am interested in. Best.
Transformers can take time to initiate a flash over discharge after each energisation. The fault can appear to have cleared before it flashes again. Looks like check lists akin to those on aircraft of employ in emergencies. Switching over to the redundant circuits might have been useful. Time will tell.
Electrical engineer specializing in a relevant field. Obviously there's a lot of detail missing from that electric plant one-line diagram, and I'm not familiar with CONOPS on container vessels, but it looks like Dali was running an unprotected vital bus lineup, where both port and starboard buses are powered by a single source. For the purposes of the lineup, DG3 and DG4 form a single logical source. This is a dangerous lineup in restricted waters. I suspect what's going on here is operators were trying to "wear level" their equipment by running one side of the plant (port or starboard) for X months at a time before switching to the other side. The problem is, you can't do that in restricted waters, because *by tying the port and starboard buses together, any single casualty will result in a complete loss of power.* You're always supposed to be operating in a split plant lineup, with 1 port and 1 starboard source energized (HVR and LVR breakers open) unless a casualty prevents you from doing that. This is where I might be wrong for container ships out at sea; when they're in deep water they have tons of time to recover propulsion in the event of a casualty. However, this should NEVER be the lineup in port. Changing subject slightly, it's clear that whoever was operating the electric plant didn't have a clue what they were doing. If HR2 and LR2 trip, it's not a problem with TR2--there's either an overload or fault on the LV bus. When they closed the EG output breaker into the LV bus and the EG stayed up, it indicates that an overload tripped HR2/LR2. The source of that overload should've been identified and isolated before closing the HR1/LR1 breakers. The fact that HR1/LR1 tripped after 30+ seconds indicates the trip was on long time delay protection. This means that somebody or something (like an automatic relay) turned on a huge load that the forward feeder cabling or transformer wasn't rated to handle. That's why I say they don't have a clue. Excessive/bad automation could also be a contributing factor like Sam Brown said.
If HR1 and LR1 trip, one takes a look at the TR1 differential relay flag. If TR1 is overloaded, then LR1 should be set to trip first. TR1 windings can handle overload for minutes or even hours. Operations of HR1 should be limited because every time a power transformer is energized the inrush current seriously stresses the windings. If the fault is in TR1, then HR1 has to trip. If the HR1 opened, then the system should have transferred to TR2 automatically. Emergency generator should be online in 10 seconds or before main engine auxiliary systems trip out. This looks like incorrect operation of the protective relay system. It also looks like TR2 might have an issue. If the generators were overloaded, then their own breakers should have tripped on overload. Generators can tolerate overload for seconds for motor starts. HR1 overload relays might reasonably be set to 150% of rated capacity. The windings lose life but they do not generally fail.
The splitting breakers on the HV and LV busses are normally closed and only used for maintenance purposes, only time i've seen those opened was in dry dock. Its usual to run on one transformer, and change over as part of the planned maintenance program. these planned maintenance items are recorded in the company PM systems. The comment that got me was that the 3rd alternator , was said to be running but not on line. Its usual on st by to have one more running than required for stby loads, so if one fails the others just keep going without tripping on overload. seen that happen a few times.
Should not have crew like the "native crew on the USS San Pablo" operating a ship in US waters. I could use a non-pc term like the Chinese who built the railroads in California, but of course the YTAIPC censor would not like that. The "Jones Act" should be expanded to include all ships that come into US waters, not just "between" US ports. A USMM crew would never have left the dock with electrical problems. IMO Also , I cannot find fault with harbor pilots. They did their best, and did warn the harbor authority, when they knew they had lost control with 1st power failure, nevermind the 2nd power failure. All the BS about the ship being "steered " into the bridge, is a bunch of BS. Some people love/live on cow-shit, I guess.
Fully agree; especially considering the position of all the breakers as shown in the schematic block diagram as per status of departure March 26. Plus that all 4 four generators almost have same power here. If this is not standard and compulsory anyway, I could imagine one of the first “containment” procedures in restricted waters to be raised by the NTSB (and authorities world-wide) is that the cross-tie breakers HVR and LVR have to be open, thus starboard and port power supply systems being supplied and running independently. This assumes there’s also redundancy in vital functions like e.g. fuel pumps for main engine and diesel generators, of course. In any case it helps to reduce total load on each bus. That would also apply for emergency procedure re-connecting the big diesel generators, too; rather than having all load again on a single bus only, where some single fail could cause overall overload and blackout of complete system accordingly. Let’s use allegory from aviation with B-747 or A380 take-off going for long-haul flight. Nobody would think off having 1 engine completely switched off, another just on idle speed (especially both on same side) while the 2 remaining ones on the other wing to provide all the thrust plus power for hydraulics, electricity etc. (even if the engines have this capability) …
As I understand your description of the EBUS… The one-line is simplified to the point of being unusable for troubleshooting at this level. When LV power is lost to the EBUS, the EBUS is automatically disconnected from the LV bus and the EG starts/supplies the EBUS. The EG must be synchronized to the “normal” LV bus before the transfer back can take place. The EBUS supplies only essential loads; and the EG is sized to supply only those loads.
Did he? Do 110 and 220 V come from different "fuse boxes"??? Do we even excuse "fuses" in residential buildings any more? The guy made a ton of excuses for why Engineers allowed Accountants to Build this Ship. Period.
It's hard work explaining why Dali speeded up and turned right to hit the pylon. It's as hard as explaining why building 7 collapsed by itself with a "normal" explanation.
I wonder why the FBI had to confiscate all the cell phones of the crew members and had all the crew members confined on the ship for months even though some of the containers on the ship were known to be leaking hazardous materials so that the NTSB could come up with this "normal" explanation of why the ship lost power. 🙄
I don’t care if this vessel lost power. All I want to know is why the vessel turned sharply to the starboard. If she was heading strait up the channel between the Chanel markers to the open area and the rudder was amidships. Some body did something to cause the vessel to turn. They were only 3 ship lengths away from passing under the bridge, her momentum would have easily carried her to more open water. There she could have dropped her anchors and the figure out the power issue. I have yet heard this change of course explained yet? What are they hiding?
Sal -- Ex Navy electrician mate and retired automation controls designer here. After reading many of the comments and rereading the report, I'd like to add my 2 cents. Why was the normal lineup to have 1 transformer in used and the other powered down? I would have expected that both transformers would have their high voltage breakers closed at all times and that there would be a schedule to switch the transformer in use on a regular basis. The way electrical system was actually being operated, one transformer was completely disconnected, often for extended periods of time. Leaving electrical equipment sitting unused and unpowered for long periods of time in a ship at sea is a recipe for disaster. Changing temperature and humidity levels on a ship plays h*ll with electrical connections and insulation. In addition equipment sitting idle will always manage to gather dust bunnies from somewhere. The end result is usually a short circuit or a ground when the equipment is turned back on, always occurring at the worst possible time. With this in mind, the following sequence of events may have occurred. 1. Until the day before, TR1 had not been used for months. Until the MV Dali pulled away from the pier, it had been lightly loaded as no large motors were in use. Once underway TR1 experienced normal at sea loads, the conditions in the transformer start to change. I suspect this caused a fault in the transformer. 2. The electrical control automation system detects the fault on TR1 and trips both HR1 AND LR1 breakers to isolate the fault. This is the 1st power failure. 3. The crew overrides the automation and closes TR1 and LR1 to restore power. 4. The fault in TR1 is still there, but the first level of automation trips for HR1 and LR1 have been overridden. The automation system most likely tries to alert the crew to the problem (by setting off more alarms), but they are too busy trying to get propulsion back. This is indicated by the heavy black smoke seen after the first power restoration. 5. After some programmed time delay (and not nearly long enough from the crews viewpoint), when no action has been taken, the automation safety program proceeds to it's next level of protective measures, and trips the DGR3 and DGR4 breakers, shutting down power to the High Voltage bus to protect the generators as well as tripping HR1 and LR1 for a second time to isolate the fault. 6. With the fault isolated, the automation now connects DG2 to the High Voltage bus by closing DGR2. 7. Now forced to deal with the problem, the crew leaves TR1 deenergized and manually close HR2 and LR2 to bring TR2 on line and power the low voltage bus. They have power back, but by now it's too late, and MV Dali hits the bridge. I understand that ship owners want to cut costs, and automation is cheaper than trained people. But I have never seen or designed a piece of automated equipment that was smarter than a well trained operator.
Very comprehensive and logical hypothesis about the sequence. Wrt 4. and `heavy black smoke´ (which is important piece of the puzzle), I wonder whether that just comes from significantly reduced load of DGR 3 & 4 after first power-trip but then "immediately" need to go to high load again (following the hypothesis with defect in TR1 causing extra power demand). Same also for DGR2 later. Or, like you've outlined, possibly also from attempts to get main engine up & running again, too. Also would like to use this opportunity here since you've mentioned `Ex Navy´: during your service, did you also perform kind of `blackout drill exercises´ to be able to react as fast as can in such emergency situation then (would expect so in the military)? Dali crew was under extreme pressure, knowing they’ve more or less just left the pier, being in harbour area still. So besides expertise and analytic skills, it also needs training to quickly get things under control again, partly “even without thinking about but simply know what to do next”. To be clear: it's not at all about kind of assessing or judging the crew. I'm much more interested in procedures.
@@NRZ-3Pi10 My time in the Navy was spent in "Rickover's Nuclear Navy" (yes - he was still running the show back then). The drills, tests and practice was all part of our life. It was pretty rare when we were not preparing for some type of test or inspection. In addition, all qualified operators assisted in drilling the "new guys" coming on board from Nuclear Power school. So if we weren't being drilled, we were drilling the new guys. In addition, every 2 years, we had to "requalify", meaning we had to go through the same routine as the newcomers, getting our books signed and taking the tests and oral boards. There's a reason retention rates for nuc's are not the best.
@@johntrottier1162 Alright, no surprise and exactly what I would have expected especially for nuclear powered vessels. "Rickover's Nuclear Navy" sounds like `your favorite drill instructor´ 😅
"I suspect this caused a fault in the transformer." I was guessing the same. If they are controlling all breakers from a central PLC, that is a part of the problem. Breakers should be locally controlled, by a smart local controller or local manually. These two breakers should have been locked out, requiring manual reset, but not disabling the protective relays to trip again quickly. Then my "automatic reconfiguration at the relay level (smart grid)". No PLC needed, so faster. PLCs should only be back up to smart breakers, as recommended by my automation engineer. The power could have been repowered using the left transformer quickly, if done automatically. www.pes-psrc.org/kb/report/087.pdf Retired electrical engineer, power protection, ship design (after 2003), maximum 13.8 kV. Before that, up to 69 kV.
Fascinating report. As a retired USN officer, who served on the USS Lexington and was qualified as OOD-Underway, I listened to your descriptions attentively and stopped the video so that I could peruse the electrical diagram in detail. Same for the text excerpts from the NTSB report. What I was surprised by was the seemingly inherent fragility of that electrical system, and the fact that the emergency generator could only power some of the critical systems. Sure, a warship like mine is designed to take a lot of abuse and keep going - but any ship should have multiple ways of restoring engine propulsion, steering (the Lex had 5 ways to steer her), and full electrical supply from emergency generators to all critical systems. Sounds like the designers never anticipated that the high-volt bus would ever be unpowered. // Since this is not the first collision of a large vessel with a bridge and associated loss-of-life, I think that perhaps there should be the requirement for the tugs to stay close aboard until the ship is thru the bridge. Expensive and inconvenient - but that's exactly what we have now after this collision.
Just for my curiosity I also place this question here: during your service on the USS Lexington, did you also perform kind of `blackout drill exercises´ to be able to react as fast as can in such emergency situation then (would expect so in the military)? Dali crew was under extreme pressure, knowing they’ve more or less just left the pier, being in harbour area still. So besides expertise and analytic skills, it also needs training to quickly get things under control again, partly “even without thinking about but simply know what to do next”.
As a Captain of merchant vessels this is one of my biggest concerns. As technology progresses, more and more ship's systems that were previously of the electromechanical variety are switched to intergrated systems with chips and circuits running them. This is important for many reasons, including fuel economy, performance optimisation and longevity of components. However, more and more maintenance work is diverted from the crew to external contractors, with specialised knowledge and tools. Only problem - they are never there when you need them the most...
THAT is true with the maritime industry AND the aviation industry Many times agents from the company who did the construction have to be called in to figure out what is going on I remember one time a SAAB 340 aircraft left the maintenance hangar and took off for a test flight and their propellors feathered automatically while in the air for NO obvious reason. Somehow they got them back and landed. Had to get people from SAAB SWEDEN to come over to figure it out. Turned out to be some chaffed wiring down in the belly of the airplane
Unmanned shipping on the radar for the future, I do not think in my lifetime. With many companies already manning ship's according to minimum manning regulations this is not good for seafarers. I am an ex Master now Marine Pilot and yes, with all the automation and electronics onboard nowadays it seems to me that it can be a good idea but also having it's own problems as you mentioned in your comment. I have had a couple of situations onboard during Piloting when a critical item stopped working and nobody knew why. During Piloting through narrow navigational channels is not the time to contact external contractors for help. Seconds and minutes seem like hours for the Pilot and Master, luckily in my cases with good bridge and engine room team work we managed to get through the navigational channel even with the critical item not working.
I hate to be that guy, but most people think this contractor thing is just small devices and for consumers. You are now looking at an issue that have been progressing since the 90s and it isn't technology that is the fault, but the manufacturers. Want change? Better be ready to support legislation related to "Right to repair" Bad news is if you get such legislation is that your crew will most likely have to have more training and have to learn a really new way of troubleshooting compared to what they have been used to. But currently it is all a black undocumented box with no schematics and when SHTF happens, you are really far from help.
That is the truth. My world was a decision of keep it simple we could work on or uber complex only a factory rep could work on. Any time I had influence we went with simple and it paid off.
As an Industrial electrician I am very familiar with the switch panels in your diagram. I can tell you that in a land based factory, during normal operation, the Tie Breakers are left open. Normally, one even numbered generator and one odd number generator is used on both sides of the bus. Both transformers are used as well. This way, if there is a fault , only half of the loads lose power. It also makes it quicker to isolate the faulty circuit. The purpose of a Tie Breaker is to feed the entire bus if there is a problem with one of the transformers or the feeds. Your diagram may show the position of the breakers as they were found on the Dali , But I can tell you this is not the way these panels are designed to be operated under normal conditions. One more additional point. Usually an overload will trip the branch Breaker and the feed breakers will not trip, however if there is a low resistance short in a branch circuit then all of the breakers in series can trip.
Fully agree; especially considering the position of all the breakers as shown in the schematic block diagram as per status of departure March 26. Plus that all 4 four generators almost have same power here. If this is not standard and compulsory anyway, I could imagine one of the first “containment” procedures in restricted waters to be raised by the NTSB (and authorities world-wide) is that the cross-tie breakers HVR and LVR have to be open, thus starboard and port power supply systems being supplied and running independently. This assumes there’s also redundancy in vital functions like e.g. fuel pumps for main engine and diesel generators, of course. In any case it helps to reduce total load on each bus. That would also apply for emergency procedure re-connecting the big diesel generators, too; rather than having all load again on a single bus only, where some single fail could cause overall overload and blackout of complete system accordingly. Let’s use allegory from aviation with B-747 or A380 take-off going for long-haul flight. Nobody would think off having 1 engine completely switched off, another just on idle speed (especially both on same side) while the 2 remaining ones on the other wing to provide all the thrust plus power for hydraulics, electricity etc. (even if the engines have this capability) …
That is what I was thinking. The next question is, was this an automation design error, an automation function error or an operator training error. (I'm not looking for blame. I'm asking to prevent future problems.)
In my experience dealing with hospital emergency power the tie breakers would normally be open as well. In fact you were not going to close that tie breaker with out a particular key and in a real emergency.... On a different note, in a hospital setting, the emergency generator does not kick on if the main breaker trips, it only kicks on if you lose normal power. I don't know if it is the same on a ship, but based purely on my own experience I would guess the emergency generator did not kick on because the normal generators were still providing power. I am thinking ships crew had to force on the emergency generator and that is why there was a delay. But I don't know how much my Hospital background carries over to ships.
Wow, Sal. I'm impressed how clear and concisely you explain the electrical generation and mechanical systems of this ship. Understanding it is one thing, but communicating it to a layman so it's understandable is true mastery of knowledge. Kudos !!
I trained with basic Electrical/electronics before going into Computer Technician, and you explained it perfectly. The design of the system was poor though, because if one side trips out, there is ZERO backup for 45+ seconds... and this just proved that it can be Critical. Wanna bet that One - tugs have to escort ships; Two - bigger dolphins are put around bridge pylons all over; and Three (probably most important) the control system regs are changed for new compliance specs? Too Many Failure Points!!!
Thank you for the education. Your point on the Pilot and pilot dispatcher is well taken on the lives saved. Very respectful to those who lost their lives and their families left behind.
I think you did a great job of explaining it myself. I have a decent very fundamental understanding of electrical systems but not on ships, but I was perfectly able to follow along with your explanation. That's why I love your channel! You explain things in easy to understand ways but also manage to bring in so much more information than other sources provide.
I have a little experience with electrical distribution, 20 year substation electrician with a public utility and ex Coast Guard MK. We keep all our transformers energized whether they are loaded or not. This allows us notice if either one faults. It seems to me that that the Transformer that was not being used for months had an internal issue that took a while to manifest itself. If both transformers we're energized as a regular procedure a potential fault would have been detected. I'm just guessing that they do not run the whole bus energized at all times. Electrical equipment does not like to sit idle and deenergized for long periods of time. On land, moisture is a huge deal inside a Transformer, I would imagine at sea even more so. By running everything energized you will also keep heat in the equipment and moisture will be less of an issue. Just a guess.
Well and cautiously said. It seems the crew just did not want to try out TR2. The relays should have automatically transferred the power flow to TR2 or get the emergency generator online before the main engine auxiliary systems tripped out.
I'm curious if it was a fault, what protective element was picked up. I'm suspecting a differential trip on the TR1 transformer since both the high and low side breakers opened. If that's the case, there would likely be evidence of a fault or proof through apparatus testing of the transformer.
I was guessing idle/ de-energized transformer absorbing moisture also. I have heard of problems with transformers and capacitors failing due to absorbing moisture when they were not energized. [My experience has been related to variable frequency drives (VFDs) failing. And also the "conditioning" of VFDs that have been de-energized for an extended period of time. But that is more details than needed.]
@@erniecolussy1705 The 2 biggest enemies of electrical and electronic equipment are moisture (or water) and heat. I have seen few failures caused by other things except a backhoe cutting buried cables.
Former U.S. Navy Engineering Officer Of the Watch (EOOW) here. Sal, your explanation makes perfect sense they way you described it. You claim your no Engineer, I beg to differ. You did a masterful job of breaking this down. Perhaps the merchant fleet needs to adopt a "Sea & Anchor" detail similar to what the U.S. Navy does. Splitting the electrical bus during critical maneuvering operations may have saved the Dali and others from catastrophe.
A big YES on Sea & Anchor detail. I watched on a video and saw the Canadian Navy on one of the smaller patrol/ training ships, they had let a guy on to film them cruising around Vancouver Island, he slept on board. Anyway even going in a very narrow passage ( Vancouver Island has many narrow and ragging river currents at peck tidal flows) the Navy vessel had a whole detail of crew on anchor emergency STANDBY. All the safety locking hooks were taken OFF, one sailor standing by the brake control wheel. 😊
Looking at the electrical diagram, would that even be possible with the way they set up the bus bars and transformers? There is no way to directly power each bus (High Voltage & Low Voltage) separately, the LV bus pulls directly off the HV bus through transformers. The only thing I can see that would have provided *some* redundancy is to require both sets of supply breakers closed during this time and the interconnect (HVR) open. However, this makes things difficult as well since the big powerplants are on one side and the smaller ones on the other, which is not great either. From my limited electrical and larger aviation background, this layout has very little redundancy and so it's not surprising that it failed like it did.
Former EMC here, yes the most reliable line-up would be 'split plant'. Still, not clear how the EDG was supplying loads and they closed in the transformer feed as well without syncing or something. Probably some more details we don't see here.
Thank you Sal for great an accurate information.There are a lot of pundits who don’t know what they are talking about and are to lazy to get correct information 👍
@@vilelive I don't know about others, but I call it that to avoid calling it an accident or incident, which each implies something about the scale and/or source of the event. It's objectively terrible enough, and fiasco captures that.
Hey there Sal, great video! I was highly anticipating it! There is a CORRECTION to be made though. The second blackout was NOT caused by HR1 and LR1 breakers tripping again. According to the report it was actually caused by breakers DGR3 and DGR4 both tripping (the main breakers for Gen3 and Gen4). This is why DGR2 closed to provide power to the HV Bus which was lost this time. GEN2 was in standby mode so it had spun up. I’m a bit confused at this point because the report then states that the crew closes HR2 and LR2, but there’s no mention of HR1 and LR1 being opened or tripped. Not sure why they decided to energize the TR2 side if TR1 was already in operation. Regardless, the main correction I wanted to point out is that the Main Breakers for DG3 and DG4 did indeed trip, which is what caused the second outtage. According to the report at least 🤷♂️🤷♂️ Thank you again Sal!
Great report Sal, seen similar blackouts a few times in my seagoing career (fortunately not when manoeuvring!) - often caused by the Emergency Generator control system “sensing” there was a power failure and trying to put the Emergency.Generator on the board causing the main breakers to open when in fact there was no black out on the main power supply side.
Did Sal say or has the cause of the breaker tripping been found? Did someone plug in their hair dryer? I understand there was issues at the dock with the same system. So did they ever find what was tripping the breaker?
@@Mr.Unacceptable ... That's what the Title of the Video asks. Why would someone interested in being transparent follow through with answering their own question?
@@alan6832 I wouldn't be surprised if it's "possible," but it doesn't sound like an attack vector anyone in industry is very worried about. Makes me think that if it is "possible" to remotely/semi-remotely overload the HV bus, it's a "so involved you'd be better off hacking something else on the boat"- type thing.
Electrical Switchboard are very simple and yet complex. Electrical Engineering degree is not easy to get. I got an Electrical Officer position in the US NAVY, however, my degree is also a hard degree to get. The switchboard breaker could be thought of a very big piece of metal....and that piece of metal has a lot of auxiliary sensors and conditions to cause it to trip. There is also maintenance that has to occur which includes opening the switchboard. Dust, sooth, oil vapor build up, fuel oil and etc can build up behind the switchboard...mmm closing those vents....
Sal, you are a treasure! Not only do you give us the clearest explanations, but you point us to the issues we should be focused on and not the diversive, paranoid drivel that so many try to feed us.
Excellent detailed explanation. As a diesel mechanic it all makes sense. And to six people who lost their lives R I P and condolences to the families. Gerry
Thank you Sal. I’ll tell you how good your reporting/channel is… Look at the comment section! High level discussion from a wide selection of Knowledgeable people. Keep it up Sal, you’re doing great.
Fantastic, clear explanation. I've conducted many safety investigations and it's always difficult to present results in a way that folks (who are not familiar with the system) can understand. Well done!
Main Power Breakers don't trip unless there is some electrical fault somewhere in the system causing them to trip. When a Breaker is reset and trips again that is a sure indication of an ongoing electrical fault. Those breakers are going to continue to trip until the electrical fault causing the breaker trip is cleared. "This is an electrical or control problem" Thanks for the clear and thorough explanation. Staying tuned in for further updates.
I believe the trip units in the LV and HV breakers were not from short circuits or overcurrent but from a signal via the PLC /SCADA. Perhaps the scrubber incident and switching transformer the previous day required a few releases to the system that were overlooked as they were scrambling to transit. They were looking at the Breakers when they should have focused on the HMI screen and what releases were causing the trip!
Yeah, i also reckon there was an overcurrent situation- causing the main breakers to trip. I very much doubt that the PLCs ordered the main breakers to open. It'd be a VERY unusal situation for a PLC to command a disconnect- of primary main power.
Something to ponder: it is strange that HR1 AND LR1 both tripped. If a fault occurred in the transformer itself HR1 should trip. If the fault occurred further downstream (LV panel or LV loads) then (in a system presumably designed with proper breaker coordination) only LR1 should trip. An at-home analogy is that if you plug in something shorted the 15A or 20A branch circuit breaker trips but the 100A main breaker does not, even though the instantaneous fault current is likely >100A.
@@sootikins Spot on. Keep in mind the magnitude of the fault can be well above 100 amps. As those CBs are thermal, it take time for the trip to occur. Lower amp rated CB hopefully trips first and in a house system, yea.
The 220 V and 110 V boards are combined in almost every installation. You have the house or office fed by a 220 V transformer with a neutral center tap, which gives you two hots that have 220 V between them, and 110 V each to neutral. The breaker board has two hot bus bars that alternate positions down the centre, so a single breaker gives you 110 V, and a double breaker gives you the full 220 V. Then there's also bus bars for neutral, and for earth, though in the main board those would typically be tied together.
Shipboard electrical system are usually ungrounded three phase high voltage to be step down 220 v to step down 110 v. Most land base in facilities are single phase 220 V transmitted step down to 110 v...in the US. Ungrounded is a simple concepts that trips most people up.
@@rgloria40 220v isn’t technically “stepped down” to 110v. As mentioned above, it’s single phase 220v center-tapped to provide separate 110v buses referenced to the center tap of the 220 coming in to your power distribution box.
He was just trying make an analogy so that his audience could perhaps relate to the electrical system design of the ship. I don't think he meant it to be a fully accurate representation of household power design.
@@alliejr You should never make an analogy when you don't know what you are talking about. He didn't over simplify a home breaker box, he obviously didn't know what he was talking about. He may be an expert on ships but doesn't seem to know squat about his home breaker box.
Being a retired engineering sailor with a LOT of experience in this area (NAVY ships operate on pretty much the same principles), the way they had the electric plant aligned was a recipe for disaster. No backup should any either HR1 or LR1 trip and the other generator running "unloaded" on the other side of HVR. Had they had HR2 and LR2 closed (which I would think to be typical in a "restricted maneuvering" situation) this more than likely never would have happened. The generators should have been split better as well. Assuming this is an accurate portrayal of the generators relative location to each other, DGR 1 or 2 and DGR 3 or 4 should have been used with one of the others running unloaded. Poor plant management, awful SOP's, Lack of foresight and bad planning were huge factors here.
Well said. I was stunned that the online delay for the emergency generator was 45 seconds. It needs to be online before the main engine auxiliary systems are set to trip out. Code requires hospital standby systems be online in 10 seconds or less. It can be done. Transfer to TR2 should have been automatic. HR1 and LR1 trip out sounds like a differential relay operation due to a transformer fault. Not something one recloses on. This ship should have had at least two red X's on its engineering log and never left the dock.
Not my area of expertise. But isn’t the question what happens on the loss of a critical pump? How long to until the main engine shuts down? How long to restart the main engine? How long to start the standby pump and for the flow control valves to operate. I don’t disagree with your point in having the tie breaker open and the bus split during restricted maneuvering. Had they had the emergency EG running they would have never have lost rudder control.
@@fountainvalley100 All those were important clocks ticking. And those clocks are known and can be designed for. The ship was leaving port with too many problems.
I was thinking the same basic thing. I would just add that both HVR and LVR should be open to prevent a failure on one half to cause a failure on the other half. Then my next thought, was the system operating the way it was an engineering design error, an automation design/ function error or an operator error?
Just watched your blog. Terrific explanation for us non-maritime folks. WOW, what a education we're getting from you. Thank you for the detail. Not boring at all. Very well presented. Keep up the good work.
It does strike me that the emergency back up systems never envisaged a power failure whilst in a confined waters. I'm so impressed with the Pilot dispatcher and the Maryland Police in getting the bridge closed, incredibly impressive.
The cruel irony for the workmen lost is that if it wasn’t for the work being done the traffic could not have been stopped so quick. They had people on both ends, slowing down and merging traffic for the repairs and they were able to quickly stop the traffic. I believe all the bodies were found in trucks so it seems like they were attempting to evacuate but didn’t have enough time to get off the bridge.
@@daveh9755 Aparently, the actual workers were on a break and had trturned to their vehicles to sit down and rest/eat snack. The inspector was likely inspecting. ( curiious if he had radio and was told to get off the bridge ASAP). One worker in a truck survived because it had manually operated window and was able to get out after the truck plunged into water.
Yeah, that seems like a major, glaring deficiency to me. Seems like there should be a "port" mode that permits the captain/pilot to defeat automatic shutdown features while in confined waters.
One time a contractor hit the "emergency power off" for our facilities UPS. It took several minutes to get power back on. It took days to get all the systems working again. Many things are controlled by your basic PCs and they often don't like it when they loose power without a proper shutdown.
It surprised me when the breakers either side of the TR1 tripped the breakers for TR2 did not close automatically, or as a variation the crew close those TR2 breakers. Closing breakers that have tripped because of a fault is asking the breakers to reconfirm that a fault exists - it did - so why not use TR2 and its breakers immediately? I'm sure the CG will be asking this.
Excellent analysis. From all appearances that is exactly what the crew did. If the fault still existed on the affected bus then the breakers will open again. I’m not an expert, but I’m going to assume that there’s an SOP for what equipment must be on-line for entering/leaving port/restricted maneuvering. And, why was the emergency generator unable to pick-up the load?
Yep, it seems it would be more logical to restore power by close line through TR2, there seems to be a problem on the line going through TR1, which was not used for a while, and perhaps was not checked before departure.
I too have some Q’s about why there wasn’t automatic closing of the TR1 breakers on failure of TR2. Also the emgen wasn’t mentioned in the blackout alongside just that the standby main genset started and put itself on the board and .. then failed due to low fuel flow….
Exactly. And hospital standby power systems are online in 10 seconds or less by code. Standby power needs to be online BEFORE main engine auxiliary systems trip out or those systems should have their own smaller APU online with a transfer switch ready to close on the emergency bus. And was TR2 locked out of service? Ship should not leave the dock. It's a red X.
Sal, thank you for explaining detailed systems and their use so clearly and in a way that us non-engineers can grasp. Keep up the excellent work. Thank you ,
It's been many years since I've been a sailor, but as I recall, when we were at Sea and Anchor (maneuvering) detail, all systems were up and running. All the generators and as I recall, the Engineers (Snipes) would split the load so one branch would never have to shoulder the entire ships electrical load....from the simple diagram that I see here, it looks like that should have been done.
HR1 and LR 1 won't trip the HV bus, they are both downstream of that. The HV bus is protected by the breakers between the generators and the bus. I also don't think a lube oil pump will run on the HV bus, 6600 volts is way too much for the motor needed for a lube oil pump (and yes, I understand it's a big pump). Ergo, I think you will find it is only the LV bus that failed in the first blackout. The second blackout appears to have tripped breakers upstream of the HV bus so DGR3 and 4 must have tripped. I feel sorry for the poor trainee pilot, what a thing to have happen on a training trip.
Sal did say that the L.O. pump was on the HV buss, seems unlikely as a 440V 3phase buss would easily support a 250-400 hp motor with a soft starter. There's probably a dozen different trips associated with a protective relaying scheme for a 6.6kV buss tie breaker, not the least of which is "under-voltage" which trips all high voltage breakers if the control voltage drops below a specific minimum. This might explain why they couldn't clear the fault and re-close. This situation needs analysis from a qualified electrical engineer.
From the NTSB document: "The generators were connected to a 6,600-volt high-voltage (HV) main electrical bus by the vessel’s power management system (see figure 3) that powered various shipboard equipment, including the main engine lubricating oil pumps, the bow thruster (a propulsor on the ship’s bow that that assists with ship maneuverability), and reefer containers" "A 440-volt low-voltage (LV) electrical bus was connected to the HV bus via redundant step-down transformers (TR1 and TR2 in figure 5). The LV bus powered vessel lighting and other equipment, including steering gear pumps and the main engine cooling water pumps."
Iron sharpens iron just as excellence begets excellence and now Sal’s high quality reports attracts the highest quality comments a community engagement. Such an amazing global resource, I wish there was medal or national honor we could bestow in recognition of all the hard work and dedication to the industry and lives it supports.
I read the preliminary report last night. The only item which surprised me was when NTSB stated that M/V Dali was only required to shift to LSMGO at 12 nm. I never sailed on a ship with a scrubber and completed shifting over by 200 nm. That was required for the North America ECA.
I was an electrician for over 35 years and never saw separate panels in a house for different voltages. That's common in commercial and industrial applications, but never in a house.
I refurbished a house built in 1945 in 2011-2013 and it had, believe it or not 6 fuse boxes of different voltages. I still have the one light fixture from the top of the laundry room that looks like a plate of spaghetti that was up in the attic (had knob and tube, 12-2 paper and tar wiring). Everything was stripped out of the house and new panel and new wiring installed. I often wondered what someone was thinking; not safety that's for sure.
He was just trying make an analogy so that his audience could perhaps relate to the electrical system design of the ship. I don't think he meant it to be a fully accurate representation of household power design.
As a nuclear engineer and expert in redundancy and safety systems, I can see two design flaws and administrative shortfalls in the Dali. The engineers designed everything to protect the main engine and the equipment assuming that protects the ship and crew. Not necessarily. Sometimes we may have to sacrifice in a grave emergency to save the ship and crew, which could have been the case with a design of "run to destruction" option. A safety interlock "lockout" feature may have been helpful to avoid the second trips and blackout that may have allowed saving the ship. If an electrical component starts overheat that is better than risking ship and crew. A fire watch in key places can be posted when used. Second the main engine design has apparently little in the design of self support. Why is the oil and cooling not pumped off the engine itself? "On" then means "on" and not "conditional on." "Well they don't come that way." Time to rethink main engine drive on large ships. Third exercising breakers and loads is a routine PM that should be performed in dock and at sea. Also a blackout test procedure should be performed before sailing.
Seems to me that you are correct; why aren't the oil and water pumps mechanically driven or at least the minimum requirements? To me that seems to me that that is the only logical way, like things were on engines in automobiles before all the computerized crap. Glad I still have my 1996 pickup where you can actually see the exhaust manifolds because of the lack of engine compartment garbage. Have a 1954 Ford NAA tractor that is even more simple.
Good points. As an electrical engineer with some experience in protective relaying systems, I was surprised to hear the system did not automatically transfer to TR2 after TR1 was isolated. Generally, closing in again manually (overriding the protective system) on a transformer fault is a bad idea. And TR1 appears to have failed again in this case. Also, having experience with standby power systems, I was surprised to hear that the delay to the emergency generator getting online was 45 seconds, and that that exceeds the delay time for main engine auxiliary systems to trip out. Hospital standby systems are set for 10 seconds or faster. Thus there were at least two opportunities to tune the existing system into a state that would have promptly restored power and avoided the loss of the main engine. Of course, if TR2 was out of service, that should be a condition that keeps a ship at the dock.
@@timtrewyn453 Yes you have a better solution to have an automatic switchover. But undervoltage trips could be overridden and perhaps there could be rather than a safety lockout, an emergency configuration that shuts off refrigeration loads or non-essential loads for emergency operation. I am thinking more of emergencies during heavy seas or major ship threatening struggles, there should be an "essentials mode" including non-safety related trips that could be overridden. Yes I thought about the transformer after the comment. It looks like a transformer problem.
I believe it was a release in the PLC logic that sent a trip signal to each breaker. It seems the scrubber incident is the only "unusual occurrence" here so we should look at that. If the HMI can bring up the PLC program or a SCADA graphic user interface with a screen that has reales on it we may see what is causing the trip. Seems like something is timing out-so the screen would show us. On tunnel boring machines Ive designed 25 years ago I had HMI's
I'm surprised that the shipping industry standard uses only electric driven pumps, in the electric generation industry, standard is shaft driven oil pumps with electric auxiliary pumps and additional battery driven emergency oil pumps as a fail safe. Your description of events is well done.
In all steam turbine Navy ships I worked on there were shaft driven lube oil pumps with electric emergency backup pumps. The electric primary pumps must be a feature of diesel propulsion plants.
Electric pumps have big advantages. For one thing, you can pre-oil an engine with an electric pump, minimizing wear at engine start. Not a marine diesel guy, but I can see why they would prefer electric pumps. Ditto for coolant pumps, where you can vary the pump speed to allow faster warmup of the engine and more precisely control operating temperature.
Sounds like these critical main engine systems need their own transfer switches with an automatic set of options to get power before they have to trip out.
@@dougrobinson8602 Electric pumps are still used to start up and shut down, but when at speed the shaft driven pumps take over, much more reliable and additional redundancy. There are a myriad of ways this could have been avoided by redundant systems, I'm just surprised at how lax standards are. We now see the cost of these lax standards.
First time hearing about the one crew member running for his life and another one getting injured. Also bridge worker running for his life to safety. WOW!
Positive alcohol content 14 hours after the incident could be challenged in court. "I was so rattled AFTER the crash, I had a few swigs to calm my nerves!"....
It could be challenged, but a savvy person could shoot that down. I was once in a jury for a drink driving case. The accused refused a test on scene, so we had BAC results showing they were drunk hours later. The defense said the question was "what was the BAC at the time of arrest, which we don't have." I asked for data about how alcohol is metabolized so we could try to extrapolate the BAC at time of arrest. I do not understand why they were not tested immediately here.
As far as I know, any USCG regulated ship cannot have alcohol on board & crew cannot bring ANY alcohol on board, at least while operating in US territorial waters. You would lose your TWIC clearance very quickly if you just had alcohol in your possession on the ship. There's no legal way they could've drank unless they crew were allowed to deboard to shore before the urinalysis
@@j_taylor If the alcohol was below 0.08 hours later, you can't just assume it was higher based on an average person's metabolism. Also jurors don't get to ask for new evidence; you CAN ask for words to be defined, to examine exhibits and to rehear testimony. You also can't convict based on legal theories you dreamed up in the jury room.
Thank you for this comprehensive and understandable interpretation of the preliminary report. With all the sensational/dramatic and speculative commentary flying around on social media plus the news coverage it's so refreshing to hear someone , from the industry, who actually knows what he's talking about. I have enjoyed your coverage since the event occurred.
Watched a similar video but it did not make sense because the presenter did not explain that the bow is filled with ballast to stop it shifting unexpectedly after the cutting detonation. Good job explaining this overlooked but critical detail.
I'm NOT an engineer nor a seafarer but I'm grateful for this explanation. Although not yet comprehensive the NTSB report with your breakdown was most interesting. Thank you. (Tony from Melbourne Australia.)
Finally… someone who has knowledge on what they’re looking at, and not just spewing random articles and talking nonsense to fill a 15 min video… great explanation of the data for those of us with mariner experience as well as for those who don’t
Thanks for taking the time out on your busy life sal we really appreciate you. Thanks for keeping us in the loop. So what you're saying is something or somebody trip the breakers 🤔 sounds like really bad luck that all things go perfectly wrong or very good planning. I wonder which odds are higher ! It's because they speak Marine engineering they don't speak human lol
I sailed for years and was always interested those big ships. Sailing in Boston harbor before taking her out for the season buddie and I were close to the oil docking area skirting through (short cut) this tanker was being controlled by two tugs. Ever look how fast they move ? I tried to sail past as tugs pull in closer port side. 600 feet 500 ft. then one of the tug pilots open the window and Yell, "Can you put that little boat over there so we can put this Big! boat over there" Tacked got some wind and slowly moved.. I miss sailing..
@@richkennedy326 no rich my buddie norman panicked , he was learning how to be comfortable and safe on the water again. His dad drowned diving when he was 8. but he was willing to learn sailing. So many boat close calls.. thx R
I am a retired senior engineering officer. ships i sailed on had a separate distribution board for all items to be fed from the emergency generator(emergengy lighting ,nav lights and 1 steering gear pump. This ships system where the emergency generator is just connected in to the low voltage bus is totally different . when the first blackout happened the emergency generator sharted and connected to the lv bus , crucially it didn't need to syncronise its ac waveform to the bus as there was no other ac wave present , but at the point that the main generator were reconnected it would be necessary to syncronise the ac wave form with the emergency generator this appears not to have been done (or it may not have been possible) connecting unsyncronised AC generator together is not allowed as a syncronising torque acts on the alternators which is powerful enough to shear drive shafts , and /or destroy alternators. i think that the emergency generator needed to be disconnected before the main generators were put back on line.
@DavidHamby-ORF-48 had auto paralleling on several ships but it was very slow to work, and chasing the frequency down to synchronise was certainly quicker done manually.
@@DavidHamby-ORF-48 yes....but auto paralleling is slow...that is probably why the third generator was on standby...running, and synced freewheeling but not carrying a load.
Incoming is slightly faster than running to insure incoming doesn’t reverse power trip. Can set speed by hand then enable auto closing. If running wobbles, it won’t close out of limits.
With commercial building electrical panels, major breakers aren't really supposed to be tripping. Smaller breakers are supposed to be tripping when there is a problem. The only time a big breaker trips is when one of the big parts fails.
Sounds like the differential relay protecting TR1 operated. Sounds like they manually closed back in on a damaged TR1 and it tripped again. They needed to close on TR2. Indeed, the relay system should have automatically closed on TR2, unless TR2 was out of service, in which case the ship should not have left the dock.
@@timtrewyn453 Yep, reclosing after a trip is a bad idea when the other transformer is available. I'm just curious why the standard configuration is to have the tie closed and one transformer out of service? It would be more reliable to have the 440V tie open and each transformer supplying half of the 440V engine room loads.
@@timtrewyn453 It could be something as simple as a loose terminal bolt on the breakers or something more major like a developing short in a transformer. Once they get the Dali back in a dock and can shut it down completely, they can do a proper forensic analysis on all of the parts until they find the core problem.
Former submariner electrical officer here. Most of my electrical operators had 1-2 years of experience and we followed class procedures for operating the electrical plant. Starboard HV-LV breakers tripped. That suggests a poorly transformer, perhaps compromised insulation. Was crew logging ground readings? Are they in the automation logs? Running LV plant off one transformer for several months seems unusual. Idle equipment cools below the dew point and develops grounds. Operating and warm is good. Six month crew contracts and 6 mont engineer officer contracts don’t give time to develop deep experience with the plant and its operation.
@@DavidHamby-ORF-48 Would HR1 and LR1 be connected in a way that would ensure mutual triggering? It would seem to me it would be a smart thing to do that to electrically isolate the transformer if it's developing a short.
When one works in a specialized field, regardless of what it is is, is sometimes hard to get someone to understand you when you are talking about apples when all they hear is oranges. Been there; Worked for a huge national company and we had so many acronyms there was a book that listed all of them. Even so when talking about some of the specialized machinery we used, you had to be very generic and leave out all the details when talking to a friend or relative or non co-worker or an employees that did not work in your area.
Hi Sal Many thanks for a very detailed and easily digested summary (specially for the "non bofins"). Love your "ship hits the fan"!!! Congrats on your recent 50M views and note you are fast approaching another milestone - 300K subscribers. Cheers David
Sal I can’t explain the fist blackout but the second I think I can. I’m assuming ships power works the same as land based generators and an engineer can explain this better but I will give it a try! There is a macro level of detail in the generators when you have multiple power sources connected they have to be synchronised ie the voltage and sine-wave from each generators have to be synchronised to be connected i imagine the normal startup processes applies electrical load in stages when you compress that timeframe the generators wander out of the voltage and frequency limits for synchronisation and cause a trip! If they did a rapid recovery and dumped a ships worth of load on generators that hard barely restarted I can easily see a trip
Re the comments about delayed alcohol & drug testing: Comments about alcohol are correct, but drug testing (for metabolites) comes back positive for weeks after consumption (for most common recreational drugs). So those 3rd party negative drug tests for the entire crew provide useful knowledge.
erm- depends on the drug, and the test. Lot tests will miss cocaine or heroin thats more than 72 hrs ago, but may show traces pot 2 months after "exposure".
I probably need to dig into the NTSB report, but thank you so much for this summary. The Electrical system schematic is the key to this incident. However what is not shown on the schematic is where various loads are. The opening of HR1 and LR1 should not cause a total loss of power to both buses, but only a loss of power to the LV bus. The HR bus should still be powered, supplying power to steering pumps and lube oil pumps. It appear the LV bus had control circuits, cooling water pumps and lighting. So we saw a complete blackout of all the lighting, but as long as the DGs kept running, there would have been power on the HV bus. One thing I learned operating submarine electrical systems, is that automatic control is always better than manual control. Letting the system work automatically, is better than getting in there and shutting breaker manually. When you said the operators started manually shutting breakers, I thought, thats when the real problem started. If they had done nothing, and let the EG come up and supply the LV bus, while the DGs continued to supply the HV bus, they may have been able to recover without incident.
It would indeed be sad if their re-energizing TR1 temporarily reset the time delay on the EG, further delaying the EG coming online. Emergency power needs to online before main engine auxiliary systems trip out. You are right, the system should have switched over to TR2. For some reason that just did not happen, and they did not seem to even try to make it happen. Was there a known problem with TR2? It should have been energized in the first place, with loads logically sectionalized or the protective relays set to protect a looped bus.
My understanding was not correct. The Cooling water pumps are 440V powered. Which means failure of the low voltage side would cause the main engine shutdown due to loss of cooling water. Yeah if my understanding of the schematic is correct, the only thing HV1, LV1, HV2, and LV2 isolate are the transformers. Which makes sense from a systems design standpoint. You want to be able to isolate those transformers from both sides. But each of the other systems being fed High Voltage 6600V power, lube oil pumps, steering pumps, etc, would have their own disconnects so a fault or short in one of those systems doesn't take down the whole HV system. In the schematic shown, the only HV load shown is the Bow Thruster but we can see the HV circuits trail off on either side of the generator. That's where I would expect to see those other HV loads shown. Something isn't adding up in this explanation. My only idea is those lube oil and cooling water pumps have Low Voltage control systems that shut them down despite High Voltage Power being available.
Kind of surprised that after the breakers on both sides of transformer 1 tripped, they reset them instead of connecting the buses through the backup. I would have thought that after a trip, you’d treat that leg as suspect and avoid it in favour of the backup route until the reason for the trip was known. Hindsight and all that, but this is the sort of thing that I would have expected to be written down as standard procedure.
Exactly. Transfer to TR2 should have been automatic. Reclosing on what appeared to be a transformer differential relay operation was a not so good idea, but maybe they knew TR2 had an issue.
Ditto both of you. I am reading the comments ahead of the video -- to have flags set for what other people are seeing as key points -- and I look forward to any explanation of something like, "There was a faulted state logged for Transformer 2, so all power flow was shunted away from it. That is a major service, and it was scheduled under warranty at _X_ harbor in _two months_ due to shortages of critical components."
Concur, especially with the initial reports that the cargo refrigeration loads had caused the breakers to trip twice while still pierside. Human beings are inordinately *bad* at changing their habits, and instead we rely on past practice and old, out of date data.
On the nimitz we had rules against performing electrical maintenance during flight ops or restricted maneuvering. they wanted zero chance of operator error accidentally impacting the ships electrical distribution or propulsion during critical times. Do commercial vessels have prescribed configurations for their distributions systems during restricted maneuvering? is there extra manning in the propulsion spaces during these times?
Just for my curiosity: during your service on Nimitz class, did you also perform kind of `blackout drill exercises´ to be able to react as fast as can in such emergency situation then (would expect so in the military)? Dali crew was under extreme pressure, knowing they’ve more or less just left the pier, being in harbour area still. So besides expertise and analytic skills, it also needs training to quickly get things under control again, partly “even without thinking about but simply know what to do next”. To be clear: it's not at all about assessing or judging their performance; i'm much more interested to learn about procedures.
@@NRZ-3Pi10 we had a real blackout a few days after I reported aboard, but it wasn't during flight ops. I wasn't even in my division then though, as you spend the first two weeks going though FAM&I (familiarization and indoctrination) to life aboard a carrier. Very basic stuff, like learning where to go and where not to go, safety procedures, administrative things (like where the ships stores, dentists and doctors, chapel, etc. are). after that nukes go though training division, where you learn the class- and ship-specific details and procedures of the reactor and propulsion of the nimitz, this takes a few months. then you get to go to your division proper.
The second time was likely caused by the emergency generator backfeeding the high voltage buss, when the breakers near the transformer were closed, the restarted generators 3 and 4 were out of frequency or phase. Hugh current flows occurred and the diesel breakers opened to protect them. EG1 should have been disconnect first.
That's the title of this video, but he only discusses the "how" question. Next question could be " Why and how did the ship change heading while rudder was centered and ship moving forward at 8 knots?" This should be addressed in a subsequent report. What were the steering inputs applied, was the bow thruster engaged? What effect did the anchor drop contribute?
@@JeffBartlett-kj6sq Backfeed is unlikely. The diagram is very likely wrong. Usually the emergency generator has its own bus. The normal source and generator breakers are interlocked, prohibiting backfeed upstream.
@@moregrouchy Currents and wind are responsible for the course change. With the breakers tripped, the bow thruster can't operate, it is also useless at that speed. Did you listen to the video? The anchor was basically useless, as the brakes weren't engaged, per crew report.
As a power plant and electrical distribution tech for new ships I have a lot of experience in testing power plant blackout restoration. Sounds like 10hrs prior and the incident itself, they both had two blackouts in a row; which leads me to believe they forgot to restart fuel pumps or seawater cooling pumps and the generators shutdown on high temperatures most likely.
All this makes a tug escort past navigation hazards like bridges look like a bargain, not an expense. In Japan LNG ships have attached tugs 15 miles or so from the berth when loaded, and escorted out into deep water, clear of hazards in ballast. Worth a thought.
Where do you draw the line for tug escort? Even for this ship's case, you had two more bridges to clear - never mind other possible hazards - after the Key bridge before it's in open ocean and the last one to clear - Cheasapeake Bay Bridge - is 150 nm from the Key bridge. No one is gonna do/pay for the tug escort for 150+ nm.
@@dertythegrowerwould it, actually? I mean compared to insurance cost tugs would probably be negligible, especially if that made insurance less expensive...
@@kain0m You also need more tugs and more crews. Plus, it will make Baltimore less competitve to other ports where you do not have long and expensive mandatory tug escorts
As a former power plant EE and manager at several power plants (coal, NG, Oil, diesel), i am not a marine engineer but a couple of things stood out in your video. I agree with Sam this sounds like a protection scheme issue, the system if properly maintained was trying to protect TR1. Its normal in transformer protection relaying to open both the HV and LV breakers when a fault is detected on the related buswork, associated components (CTs/Pts), and of course the transformer itself. The ICMS should be key here as if it is similar to what we used in the utilities i worked for, the data should show which signal initiated the relays on the LR1 and HR1 breakers to open. Work upstream from there and you will find most root causes. My 2nd item is more operational in nature. When the crew attempted to restore power after the first power failure, why did they go back to the same TR1 and its breakers when the system was telling them 'Hey i am detecting a problem in this section of the buss"? I can only say in the utility world when we had dual transformers and associated breakers on standby, our procedure was always to use the standby (in this case TR2) and isolate the tripped portion of the buswork for later review before attempting to re-energize. Now if TR2 and that buswork had known maintenance issues well then that's another discussion. Great video!!
Why the data storage went out with the blackout does not make sense. Any data system could easily be battery powered as backup for this exact situation.
@@howardsimpson489 I agree. In our utility systems the control systems were always on (1) backup gen set; (2) backup batteries especially assigned to the control systems and data.
Thank you. I was wondering why both the LV and HV breakers were open. This implies to me the fault was in the LV area. Are you saying it may be normal for both to open when only one has an over-current condition?
@@LarryCook1960Let's say the temperature monitoring indicates a possible failure in the transformer cooling system. You'll want to completely de-energize the transformer. You definitely don't want a transformer fire on a ship at sea! Even on land at a substation, it may require evacuating nearby homes and businesses.
@@wramsey2656 The lack of backup power for data has also puzzled me from when I first heard about the gap. It only makes sense if none of the systems had any data to report due to the loss of power, but I'd expect a few hundred milliseconds of bad power/loss of power followed by power restored/power good, not to mention continuous switchboard statuses for both switchboards, and any fault indications they may show. A frequency/voltage drop could be an important clue as to cause.
Something went wonky in the control voltage of those breakers which most likely had UVRs under voltage relays that require power to keep the the breakers closed. With breakers of that size there is usually a mechanical charging handle that enables closing of the breaker against a dead buss. These were likely used by crew to bring one of the generators back on line. Synching a second genset also takes time. Theres also load shedding to consider along with switching logic circuits. Reefer loads on board are huge consumers of electricity. As this was a leased vessel, i suspect lack of switchgear maintenance and cleaning, relay/breaker fatigue. As a retired journeyman marine electrician i can only speculate without looking at the prints, maintenance logs, and what was done at the last yard period?
Excellent summmary of NTSB report. What is of particular interest is that only 2 generators were running under load. Load share between generators is easier on 2 working generators than on 3 or more. All the information points to a problem with the load share circuitry. The traditional method of managing load share is by the Droop Curve Method. When load share is not working trip outs are the result. I am sure tthe NTSB is digging into this. This could be a design prroblem or an operator problem.
Transfer to TR2 should have been automatic. Periodic protective relay testing and documentation needed. Load should have been close to one generator's standby rating. Underloading diesel generators can lead to engine problems. Caterpillar has a white paper on this.
@@koneofsilence5896 I'm sorry for upsetting the pedantry police. Point being, he's on travel, and brings a professional rig with him "just in case" he has to make a video like this from his hotel room. In particular, SM7B requires (well, sounds much better with) a preamp like an air lifter, then you've got whatever he's using for XLR with phantom power, and all the cabling, and of course the mic stand - not to mention whatever he's using for his camera, possibly a tripod, etc. It's a healthy bit of kit to take with you, especially when there are plenty of direct to usb solutions that take up a lot less space and are "good enough" for many RUclipsrs.
@@PsRohrbaughthank you for the audio explanation. Those of us who travel regularly on business are touchy about our work being referred to as vacation. Kind of like telling the sound guy, must be nice to get to see all of those free concerts.’ Be safe.
The Shure SM7 is a dynamic mic, it doesn't need phantom power. Also, you mean a "cloud lifter", not an "air lifter". Not a big deal, really. Any simple USB mic/line audio interface can suffice On the road. And for youtube, there's not.much need for a cloud lifter (which are great, BTW). I suspect he brings his whole video and audio rig with him wherever he travels. And it's not pedantic to differentiate between vacation and work trips. They can be very different and usually are.
Thank you. Together, two generators should deliver almost three million Watts of power. Those are not small circuit breakers, but highly specialized equipment capable of disconnecting serious power. The breakers probably have compressed air or some sort of ballistic system to extinguish arcs which would have tried to keep carrying current as the breakers opened. It seems like a dedicated electronic control unit would be required to properly sequence the breaker operation; no doubt that circuitry is going to get a very close examination to determine why it triggered a disconnect. A dead short would have caused some sort of explosion, given that three million Watts were in play. To me that suggests a control fault.
If the high side and low side breakers on TR1 both tripped, that probably indicates TR1's differential relay operated and a blocking relay would also trip to prevent a push button reclosing of the breakers. Thus the "manual" closing of the breakers, which is a not so good idea and turned out to be a not so good idea. The system should have automatically transferred generator output to TR2. For some reason, the crew felt that had to try to make TR1 work again. The design appears to have sufficient redundancy, except for the 45 second delay for the emergency generator. That unit needs to be online before the main engine auxiliary systems trip out.
@@wtmayhew Protective relays that apply a trip voltage to the trip circuit of a circuit breaker have indicators or "flags" that change state to show the particular relay detected an abnormal condition and closed contacts on the trip circuit to open the breaker. It would help if we knew which relays were showing flags in tripped state. It is likely that TR1 was the point of failure. If the crew can see the fault was caused by a temporary condition like a snake or rat causing a short, then they are justified in attempting a reclose. If they do not see that temporary condition, then they should consider TR1 out of service. TR2 should have already been energized and the load should have automatically been transferred to it. The schematic provided shows that is possible to do. Electric utility systems do that successfully, and we are talking utilities having ship scale loads like 10 megawatts being successfully transferred to the paired transformer in the substation. Perhaps there is some maritime exception, but I doubt a single point of failure should get things so out of hand by design. There are two transformers there for just this type of scenario. My main source for these assertions is the Westinghouse Electric Corporation classic text "Applied Protective Relaying". I am confident that the NTSB has access to very good protective relaying expertise and utilization equipment expertise. There is also more information for them than has been shared with us, the relay flags being an example.
Sal thank you for making this make sense. At the end of the day no matter how big or how small the problem was does not bring back the 6 lives lost or the key bridge. What a terrible peice of history to have to be apart of. Thank you for always make it easy for us to understand. Please as you always do in your videos keep these 6 family’s in your prayer may god give peace to the families and a very huge shout out to NTBS for putting the report out after the final victim was recovered. For the respect to those family’s as bad as we all wanted answers I am glad the last victim was out. Tufs.
Sal, is it possible that the breakers trip because of excessive power draw from the refrigeration containers. In other words to many refer containers. As always fascinating videos, thank you.
No. The reefer containers are fed from the HV switchboard (through separate transformers) and would not affect the HV-LV transformers. Further, if the HV switchboard is overloaded, the installed "Preferential Trip" system would engage, and the first consumers kicked would be the reefers.
As a former USCG federal on scene coordinator the rescue operations should be completely separate from the crew investigation. I wonder why the investigators were not on board within an hour of the incident. I was in NYC and we had to have responders ready to go. I spent many nights on barges that ran a ground in the East River and responded within an hour to the scene. I am curious as to why the drug and alcohol testing took so long
Afternoon Sal, Retired USCG Marine Engineer Investigator here. Having investigated a similar occurrence on board an LNG ship headed into port back several years ago, there are some similarities here. That ship, an all steam propulsion system, lost steam pressure which shut down all electrical and propulsion systems. During the crews attempt to restore power, the ships high voltage and low voltage buses kept tripping off after being reset. The ship was dead except for the emergency bus for 5 days offshore while being towed around in a circle. What we found after downloading the automation data, was that all the main breakers and bus ties are connected by a PLC (Computer) that tells it the position of all the other breakers and will prevent you from closing the wrong one. The engineers realized that each time the breaker tripped, the PLC had to be reset for every breaker every time it tripped. So if it trips 6 times, you had to press the reset 6 times to clear it on all the breakers. The engineers did not know this since they had never operated these manually before. Once that was found, the systems operated normally. A 100k ton ship dead due to something so small is what I'm expecting them to find.
Programmable Logic Controllers (PLC) - Yes, that is a good suspect, but also what caused the initial breaker tripping?
Seems like an odd choice to have the Diesel engine rely on external generators and power its pumps. From an engineering perspective it just seems odd to rely on external sources and busses in order to keep something as important as the engine running.
I guess they tried to make up for it by having multiple breakers and multiple power sources.
Since quite a few minutes passed between the two trip events and if the PLC requires a reset procedure, wouldnt it immediately trip the breaker(s) again when they are manually closed? Thats not what happened here. Power came back for a while and then went out again. And since a reset procedure doesnt explain the first trip event, the second was most likely for the same reason and I suspect an actual electrical reason like a power surge when all systems are running (which could explain the delay between the two blackouts) or faulty sensor readings that would cause a trip event.
Well, its all speculation. Hyundai will find the reason, I'm pretty sure about that.
PLC, very little info from those they just followed the lines of code, hidden reset was a bum bit of logic, that was a bad week for you, the PLC reset should have been in Std Ops manual, who got their are kicked ?.
As for this ship I am puzzled at the flakey operation of the HV / LV control logic as it through teddies out of the pram after losing one gen from low RPM. Also why was the EGT not in alarm giving advanced warning of a closed damper. Have I read this wrong was there only one Gen on the board?. The system should have been able to pick up the loads. Still not enough technical data. HV disconnects today are very complex and it is that logged data I am interested in. Best.
Transformers can take time to initiate a flash over discharge after each energisation. The fault can appear to have cleared before it flashes again. Looks like check lists akin to those on aircraft of employ in emergencies. Switching over to the redundant circuits might have been useful. Time will tell.
Electrical engineer specializing in a relevant field. Obviously there's a lot of detail missing from that electric plant one-line diagram, and I'm not familiar with CONOPS on container vessels, but it looks like Dali was running an unprotected vital bus lineup, where both port and starboard buses are powered by a single source. For the purposes of the lineup, DG3 and DG4 form a single logical source. This is a dangerous lineup in restricted waters. I suspect what's going on here is operators were trying to "wear level" their equipment by running one side of the plant (port or starboard) for X months at a time before switching to the other side. The problem is, you can't do that in restricted waters, because *by tying the port and starboard buses together, any single casualty will result in a complete loss of power.* You're always supposed to be operating in a split plant lineup, with 1 port and 1 starboard source energized (HVR and LVR breakers open) unless a casualty prevents you from doing that. This is where I might be wrong for container ships out at sea; when they're in deep water they have tons of time to recover propulsion in the event of a casualty. However, this should NEVER be the lineup in port. Changing subject slightly, it's clear that whoever was operating the electric plant didn't have a clue what they were doing. If HR2 and LR2 trip, it's not a problem with TR2--there's either an overload or fault on the LV bus. When they closed the EG output breaker into the LV bus and the EG stayed up, it indicates that an overload tripped HR2/LR2. The source of that overload should've been identified and isolated before closing the HR1/LR1 breakers. The fact that HR1/LR1 tripped after 30+ seconds indicates the trip was on long time delay protection. This means that somebody or something (like an automatic relay) turned on a huge load that the forward feeder cabling or transformer wasn't rated to handle. That's why I say they don't have a clue. Excessive/bad automation could also be a contributing factor like Sam Brown said.
If HR1 and LR1 trip, one takes a look at the TR1 differential relay flag. If TR1 is overloaded, then LR1 should be set to trip first. TR1 windings can handle overload for minutes or even hours. Operations of HR1 should be limited because every time a power transformer is energized the inrush current seriously stresses the windings. If the fault is in TR1, then HR1 has to trip. If the HR1 opened, then the system should have transferred to TR2 automatically. Emergency generator should be online in 10 seconds or before main engine auxiliary systems trip out. This looks like incorrect operation of the protective relay system. It also looks like TR2 might have an issue. If the generators were overloaded, then their own breakers should have tripped on overload. Generators can tolerate overload for seconds for motor starts. HR1 overload relays might reasonably be set to 150% of rated capacity. The windings lose life but they do not generally fail.
The splitting breakers on the HV and LV busses are normally closed and only used for maintenance purposes, only time i've seen those opened was in dry dock. Its usual to run on one transformer, and change over as part of the planned maintenance program. these planned maintenance items are recorded in the company PM systems. The comment that got me was that the 3rd alternator , was said to be running but not on line. Its usual on st by to have one more running than required for stby loads, so if one fails the others just keep going without tripping on overload. seen that happen a few times.
Should not have crew like the "native crew on the USS San Pablo" operating a ship in US waters. I could use a non-pc term like the Chinese who built the railroads in California, but of course the YTAIPC censor would not like that.
The "Jones Act" should be expanded to include all ships that come into US waters, not just "between" US ports.
A USMM crew would never have left the dock with electrical problems. IMO
Also , I cannot find fault with harbor pilots. They did their best, and did warn the harbor authority, when they knew they had lost control with 1st power failure, nevermind the 2nd power failure.
All the BS about the ship being "steered " into the bridge, is a bunch of BS. Some people love/live on cow-shit, I guess.
Fully agree; especially considering the position of all the breakers as shown in the schematic block diagram as per status of departure March 26. Plus that all 4 four generators almost have same power here.
If this is not standard and compulsory anyway, I could imagine one of the first “containment” procedures in restricted waters to be raised by the NTSB (and authorities world-wide) is that the cross-tie breakers HVR and LVR have to be open, thus starboard and port power supply systems being supplied and running independently. This assumes there’s also redundancy in vital functions like e.g. fuel pumps for main engine and diesel generators, of course. In any case it helps to reduce total load on each bus. That would also apply for emergency procedure re-connecting the big diesel generators, too; rather than having all load again on a single bus only, where some single fail could cause overall overload and blackout of complete system accordingly.
Let’s use allegory from aviation with B-747 or A380 take-off going for long-haul flight. Nobody would think off having 1 engine completely switched off, another just on idle speed (especially both on same side) while the 2 remaining ones on the other wing to provide all the thrust plus power for hydraulics, electricity etc. (even if the engines have this capability) …
As I understand your description of the EBUS… The one-line is simplified to the point of being unusable for troubleshooting at this level. When LV power is lost to the EBUS, the EBUS is automatically disconnected from the LV bus and the EG starts/supplies the EBUS. The EG must be synchronized to the “normal” LV bus before the transfer back can take place. The EBUS supplies only essential loads; and the EG is sized to supply only those loads.
Thanks Sal! As a retired engineer, I can tell you that you did a great job explaining. Keep up the good work!
Did he?
Do 110 and 220 V come from different "fuse boxes"???
Do we even excuse "fuses" in residential buildings any more?
The guy made a ton of excuses for why Engineers allowed Accountants to Build this Ship.
Period.
It's hard work explaining why Dali speeded up and turned right to hit the pylon.
It's as hard as explaining why building 7 collapsed by itself with a "normal" explanation.
I wonder why the FBI had to confiscate all the cell phones of the crew members and had all the crew members confined on the ship for months even though some of the containers on the ship were known to be leaking hazardous materials so that the NTSB could come up with this "normal" explanation of why the ship lost power. 🙄
I don’t care if this vessel lost power. All I want to know is why the vessel turned sharply to the starboard. If she was heading strait up the channel between the Chanel markers to the open area and the rudder was amidships. Some body did something to cause the vessel to turn. They were only 3 ship lengths away from passing under the bridge, her momentum would have easily carried her to more open water. There she could have dropped her anchors and the figure out the power issue. I have yet heard this change of course explained yet? What are they hiding?
Sal -- Ex Navy electrician mate and retired automation controls designer here.
After reading many of the comments and rereading the report, I'd like to add my 2 cents.
Why was the normal lineup to have 1 transformer in used and the other powered down?
I would have expected that both transformers would have their high voltage breakers closed at all times and that there would be a schedule to switch the transformer in use on a regular basis.
The way electrical system was actually being operated, one transformer was completely disconnected, often for extended periods of time.
Leaving electrical equipment sitting unused and unpowered for long periods of time in a ship at sea is a recipe for disaster. Changing temperature and humidity levels on a ship plays h*ll with electrical connections and insulation. In addition equipment sitting idle will always manage to gather dust bunnies from somewhere.
The end result is usually a short circuit or a ground when the equipment is turned back on, always occurring at the worst possible time.
With this in mind, the following sequence of events may have occurred.
1. Until the day before, TR1 had not been used for months. Until the MV Dali pulled away from the pier, it had been lightly loaded as no large motors were in use. Once underway TR1 experienced normal at sea loads, the conditions in the transformer start to change. I suspect this caused a fault in the transformer.
2. The electrical control automation system detects the fault on TR1 and trips both HR1 AND LR1 breakers to isolate the fault. This is the 1st power failure.
3. The crew overrides the automation and closes TR1 and LR1 to restore power.
4. The fault in TR1 is still there, but the first level of automation trips for HR1 and LR1 have been overridden. The automation system most likely tries to alert the crew to the problem (by setting off more alarms), but they are too busy trying to get propulsion back. This is indicated by the heavy black smoke seen after the first power restoration.
5. After some programmed time delay (and not nearly long enough from the crews viewpoint), when no action has been taken, the automation safety program proceeds to it's next level of protective measures, and trips the DGR3 and DGR4 breakers, shutting down power to the High Voltage bus to protect the generators as well as tripping HR1 and LR1 for a second time to isolate the fault.
6. With the fault isolated, the automation now connects DG2 to the High Voltage bus by closing DGR2.
7. Now forced to deal with the problem, the crew leaves TR1 deenergized and manually close HR2 and LR2 to bring TR2 on line and power the low voltage bus.
They have power back, but by now it's too late, and MV Dali hits the bridge.
I understand that ship owners want to cut costs, and automation is cheaper than trained people. But I have never seen or designed a piece of automated equipment that was smarter than a well trained operator.
Very comprehensive and logical hypothesis about the sequence.
Wrt 4. and `heavy black smoke´ (which is important piece of the puzzle), I wonder whether that just comes from significantly reduced load of DGR 3 & 4 after first power-trip but then "immediately" need to go to high load again (following the hypothesis with defect in TR1 causing extra power demand). Same also for DGR2 later. Or, like you've outlined, possibly also from attempts to get main engine up & running again, too.
Also would like to use this opportunity here since you've mentioned `Ex Navy´: during your service, did you also perform kind of `blackout drill exercises´ to be able to react as fast as can in such emergency situation then (would expect so in the military)?
Dali crew was under extreme pressure, knowing they’ve more or less just left the pier, being in harbour area still. So besides expertise and analytic skills, it also needs training to quickly get things under control again, partly “even without thinking about but simply know what to do next”. To be clear: it's not at all about kind of assessing or judging the crew. I'm much more interested in procedures.
@@NRZ-3Pi10
My time in the Navy was spent in "Rickover's Nuclear Navy" (yes - he was still running the show back then). The drills, tests and practice was all part of our life. It was pretty rare when we were not preparing for some type of test or inspection. In addition, all qualified operators assisted in drilling the "new guys" coming on board from Nuclear Power school. So if we weren't being drilled, we were drilling the new guys.
In addition, every 2 years, we had to "requalify", meaning we had to go through the same routine as the newcomers, getting our books signed and taking the tests and oral boards.
There's a reason retention rates for nuc's are not the best.
@@johntrottier1162 Alright, no surprise and exactly what I would have expected especially for nuclear powered vessels. "Rickover's Nuclear Navy" sounds like `your favorite drill instructor´ 😅
Dust bunnies: I think you have found the problem!
"I suspect this caused a fault in the transformer."
I was guessing the same. If they are controlling all breakers from a central PLC, that is a part of the problem. Breakers should be locally controlled, by a smart local controller or local manually. These two breakers should have been locked out, requiring manual reset, but not disabling the protective relays to trip again quickly.
Then my "automatic reconfiguration at the relay level (smart grid)". No PLC needed, so faster. PLCs should only be back up to smart breakers, as recommended by my automation engineer. The power could have been repowered using the left transformer quickly, if done automatically. www.pes-psrc.org/kb/report/087.pdf
Retired electrical engineer, power protection, ship design (after 2003), maximum 13.8 kV. Before that, up to 69 kV.
Fascinating report. As a retired USN officer, who served on the USS Lexington and was qualified as OOD-Underway, I listened to your descriptions attentively and stopped the video so that I could peruse the electrical diagram in detail. Same for the text excerpts from the NTSB report. What I was surprised by was the seemingly inherent fragility of that electrical system, and the fact that the emergency generator could only power some of the critical systems. Sure, a warship like mine is designed to take a lot of abuse and keep going - but any ship should have multiple ways of restoring engine propulsion, steering (the Lex had 5 ways to steer her), and full electrical supply from emergency generators to all critical systems. Sounds like the designers never anticipated that the high-volt bus would ever be unpowered. // Since this is not the first collision of a large vessel with a bridge and associated loss-of-life, I think that perhaps there should be the requirement for the tugs to stay close aboard until the ship is thru the bridge. Expensive and inconvenient - but that's exactly what we have now after this collision.
Just for my curiosity I also place this question here: during your service on the USS Lexington, did you also perform kind of `blackout drill exercises´ to be able to react as fast as can in such emergency situation then (would expect so in the military)?
Dali crew was under extreme pressure, knowing they’ve more or less just left the pier, being in harbour area still. So besides expertise and analytic skills, it also needs training to quickly get things under control again, partly “even without thinking about but simply know what to do next”.
As a Captain of merchant vessels this is one of my biggest concerns. As technology progresses, more and more ship's systems that were previously of the electromechanical variety are switched to intergrated systems with chips and circuits running them. This is important for many reasons, including fuel economy, performance optimisation and longevity of components. However, more and more maintenance work is diverted from the crew to external contractors, with specialised knowledge and tools. Only problem - they are never there when you need them the most...
THAT is true with the maritime industry AND the aviation industry
Many times agents from the company who did the construction have to be called in to figure out what is going on
I remember one time a SAAB 340 aircraft left the maintenance hangar and took off for a test flight and their propellors feathered automatically while in the air for NO obvious reason. Somehow they got them back and landed. Had to get people from SAAB SWEDEN to come over to figure it out. Turned out to be some chaffed wiring down in the belly of the airplane
Unmanned shipping on the radar for the future, I do not think in my lifetime. With many companies already manning ship's according to minimum manning regulations this is not good for seafarers. I am an ex Master now Marine Pilot and yes, with all the automation and electronics onboard nowadays it seems to me that it can be a good idea but also having it's own problems as you mentioned in your comment. I have had a couple of situations onboard during Piloting when a critical item stopped working and nobody knew why. During Piloting through narrow navigational channels is not the time to contact external contractors for help. Seconds and minutes seem like hours for the Pilot and Master, luckily in my cases with good bridge and engine room team work we managed to get through the navigational channel even with the critical item not working.
I hate to be that guy, but most people think this contractor thing is just small devices and for consumers.
You are now looking at an issue that have been progressing since the 90s and it isn't technology that is the fault, but the manufacturers.
Want change? Better be ready to support legislation related to "Right to repair"
Bad news is if you get such legislation is that your crew will most likely have to have more training and have to learn a really new way of troubleshooting compared to what they have been used to.
But currently it is all a black undocumented box with no schematics and when SHTF happens, you are really far from help.
When a floating computer pretents to be a ship, ship happens...! Also Training ,training, training of ship's engineers, technicians....!
That is the truth. My world was a decision of keep it simple we could work on or uber complex only a factory rep could work on. Any time I had influence we went with simple and it paid off.
As an Industrial electrician I am very familiar with the switch panels in your diagram. I can tell you that in a land based factory, during normal operation, the Tie Breakers are left open. Normally, one even numbered generator and one odd number generator is used on both sides of the bus. Both transformers are used as well. This way, if there is a fault , only half of the loads lose power. It also makes it quicker to isolate the faulty circuit. The purpose of a Tie Breaker is to feed the entire bus if there is a problem with one of the transformers or the feeds. Your diagram may show the position of the breakers as they were found on the Dali , But I can tell you this is not the way these panels are designed to be operated under normal conditions. One more additional point. Usually an overload will trip the branch Breaker and the feed breakers will not trip, however if there is a low resistance short in a branch circuit then all of the breakers in series can trip.
That is what I was thinking.
Well said. When HR1 and LR1 trip, one takes a look at the TR1 differential relay flag. Transfer of all load to TR2 should have happened automatically.
Fully agree; especially considering the position of all the breakers as shown in the schematic block diagram as per status of departure March 26. Plus that all 4 four generators almost have same power here.
If this is not standard and compulsory anyway, I could imagine one of the first “containment” procedures in restricted waters to be raised by the NTSB (and authorities world-wide) is that the cross-tie breakers HVR and LVR have to be open, thus starboard and port power supply systems being supplied and running independently. This assumes there’s also redundancy in vital functions like e.g. fuel pumps for main engine and diesel generators, of course. In any case it helps to reduce total load on each bus. That would also apply for emergency procedure re-connecting the big diesel generators, too; rather than having all load again on a single bus only, where some single fail could cause overall overload and blackout of complete system accordingly.
Let’s use allegory from aviation with B-747 or A380 take-off going for long-haul flight. Nobody would think off having 1 engine completely switched off, another just on idle speed (especially both on same side) while the 2 remaining ones on the other wing to provide all the thrust plus power for hydraulics, electricity etc. (even if the engines have this capability) …
That is what I was thinking.
The next question is, was this an automation design error, an automation function error or an operator training error.
(I'm not looking for blame. I'm asking to prevent future problems.)
In my experience dealing with hospital emergency power the tie breakers would normally be open as well. In fact you were not going to close that tie breaker with out a particular key and in a real emergency.... On a different note, in a hospital setting, the emergency generator does not kick on if the main breaker trips, it only kicks on if you lose normal power. I don't know if it is the same on a ship, but based purely on my own experience I would guess the emergency generator did not kick on because the normal generators were still providing power. I am thinking ships crew had to force on the emergency generator and that is why there was a delay. But I don't know how much my Hospital background carries over to ships.
Wow, Sal. I'm impressed how clear and concisely you explain the electrical generation and mechanical systems of this ship. Understanding it is one thing, but communicating it to a layman so it's understandable is true mastery of knowledge. Kudos !!
I trained with basic Electrical/electronics before going into Computer Technician, and you explained it perfectly. The design of the system was poor though, because if one side trips out, there is ZERO backup for 45+ seconds... and this just proved that it can be Critical.
Wanna bet that One - tugs have to escort ships; Two - bigger dolphins are put around bridge pylons all over; and Three (probably most important) the control system regs are changed for new compliance specs? Too Many Failure Points!!!
Since they still had HV power, they could have tried closing the No. 2 side first. But we still don't know where the fault was.
@@liam3284you are correct sir.
Hasn't been removed yet? OMFG!
Whatever happened to East Palestine Ohio, Jack? What about Lahaina? Why is Baltimore so much more special?...
Thank you for the education. Your point on the Pilot and pilot dispatcher is well taken on the lives saved. Very respectful to those who lost their lives and their families left behind.
I think you did a great job of explaining it myself. I have a decent very fundamental understanding of electrical systems but not on ships, but I was perfectly able to follow along with your explanation. That's why I love your channel! You explain things in easy to understand ways but also manage to bring in so much more information than other sources provide.
I have a little experience with electrical distribution, 20 year substation electrician with a public utility and ex Coast Guard MK.
We keep all our transformers energized whether they are loaded or not. This allows us notice if either one faults. It seems to me that that the Transformer that was not being used for months had an internal issue that took a while to manifest itself. If both transformers we're energized as a regular procedure a potential fault would have been detected. I'm just guessing that they do not run the whole bus energized at all times. Electrical equipment does not like to sit idle and deenergized for long periods of time. On land, moisture is a huge deal inside a Transformer, I would imagine at sea even more so. By running everything energized you will also keep heat in the equipment and moisture will be less of an issue.
Just a guess.
Well and cautiously said. It seems the crew just did not want to try out TR2. The relays should have automatically transferred the power flow to TR2 or get the emergency generator online before the main engine auxiliary systems tripped out.
I'm curious if it was a fault, what protective element was picked up. I'm suspecting a differential trip on the TR1 transformer since both the high and low side breakers opened. If that's the case, there would likely be evidence of a fault or proof through apparatus testing of the transformer.
I was guessing idle/ de-energized transformer absorbing moisture also. I have heard of problems with transformers and capacitors failing due to absorbing moisture when they were not energized.
[My experience has been related to variable frequency drives (VFDs) failing. And also the "conditioning" of VFDs that have been de-energized for an extended period of time. But that is more details than needed.]
Yes, I totally agree. Transformers do not like to be de-energized for long periods of time. Especially if it's a dry-type.
@@erniecolussy1705 The 2 biggest enemies of electrical and electronic equipment are moisture (or water) and heat. I have seen few failures caused by other things except a backhoe cutting buried cables.
Former U.S. Navy Engineering Officer Of the Watch (EOOW) here. Sal, your explanation makes perfect sense they way you described it. You claim your no Engineer, I beg to differ. You did a masterful job of breaking this down. Perhaps the merchant fleet needs to adopt a "Sea & Anchor" detail similar to what the U.S. Navy does. Splitting the electrical bus during critical maneuvering operations may have saved the Dali and others from catastrophe.
I was thinking the same thing......Why was the bus linked and not split with independent generators at least till the ship hits open waters?
A big YES on Sea & Anchor detail. I watched on a video and saw the Canadian Navy on one of the smaller patrol/ training ships, they had let a guy on to film them cruising around Vancouver Island, he slept on board.
Anyway even going in a very narrow passage ( Vancouver Island has many narrow and ragging river currents at peck tidal flows) the Navy vessel had a whole detail of crew on anchor emergency STANDBY. All the safety locking hooks were taken OFF, one sailor standing by the brake control wheel. 😊
Looking at the electrical diagram, would that even be possible with the way they set up the bus bars and transformers? There is no way to directly power each bus (High Voltage & Low Voltage) separately, the LV bus pulls directly off the HV bus through transformers. The only thing I can see that would have provided *some* redundancy is to require both sets of supply breakers closed during this time and the interconnect (HVR) open. However, this makes things difficult as well since the big powerplants are on one side and the smaller ones on the other, which is not great either. From my limited electrical and larger aviation background, this layout has very little redundancy and so it's not surprising that it failed like it did.
@@rp1645 That video was from the "Downie Live" channel, if anyone is interested in watching.
Former EMC here, yes the most reliable line-up would be 'split plant'. Still, not clear how the EDG was supplying loads and they closed in the transformer feed as well without syncing or something. Probably some more details we don't see here.
I pay no attention to any other source(s) of information on the Dali episode -- yours is the one I trust to be accurate and comprehensive. Thank you.
Yep, sorta like Juan Brown Blancolirio is for airplanes
What? You don't watch the MSM? All three networks released videos today saying "New information from NTSB: The Dali lost power twice." 🤦♂
Thank you Sal for great an accurate information.There are a lot of pundits who don’t know what they are talking about and are to lazy to get correct information 👍
@@vilelive I don't know about others, but I call it that to avoid calling it an accident or incident, which each implies something about the scale and/or source of the event. It's objectively terrible enough, and fiasco captures that.
Mmmmm………. No - he bluffs his way through technical information. But - in the land of the blind - the one-eyed man, is king.
Hey there Sal, great video! I was highly anticipating it! There is a CORRECTION to be made though. The second blackout was NOT caused by HR1 and LR1 breakers tripping again. According to the report it was actually caused by breakers DGR3 and DGR4 both tripping (the main breakers for Gen3 and Gen4).
This is why DGR2 closed to provide power to the HV Bus which was lost this time. GEN2 was in standby mode so it had spun up.
I’m a bit confused at this point because the report then states that the crew closes HR2 and LR2, but there’s no mention of HR1 and LR1 being opened or tripped. Not sure why they decided to energize the TR2 side if TR1 was already in operation.
Regardless, the main correction I wanted to point out is that the Main Breakers for DG3 and DG4 did indeed trip, which is what caused the second outtage. According to the report at least 🤷♂️🤷♂️
Thank you again Sal!
You did a very good job of telling us what happened in real time. Thank you.
I respect your dedication to your audience. Bet it means a lot to your fans out there as well. Thank you sir.
Great report Sal, seen similar blackouts a few times in my seagoing career (fortunately not when manoeuvring!) - often caused by the Emergency Generator control system “sensing” there was a power failure and trying to put the Emergency.Generator on the board causing the main breakers to open when in fact there was no black out on the main power supply side.
Excellent description, no blaming, no guessing just facts
that's why I like Sal's content.
Did Sal say or has the cause of the breaker tripping been found? Did someone plug in their hair dryer? I understand there was issues at the dock with the same system. So did they ever find what was tripping the breaker?
@@Mr.Unacceptable ... That's what the Title of the Video asks.
Why would someone interested in being transparent follow through with answering their own question?
@@truthsRsung ..... Wut?
@@GrimJackal ... If you want wisdom, start with the knowledge of a Dictionary and Stop asking Zero Word Questions.
Thank you! I read the report and had many questions. You answered nearly all of them. I'm grateful for your expertise.
That was the best and clearest explanation I have heard yet, I couldn't leave until I heard it all. Thank you for making these videos Sal.
Hey, engineer here! Of course we can talk to other people; they just don’t understand! Was happy to see the tugs have my name: McAllister.
Engineer here! David we only communicate well with other engineers 😂
A hacker can't trip a circuit breaker?
@@alan6832 I wouldn't be surprised if it's "possible," but it doesn't sound like an attack vector anyone in industry is very worried about.
Makes me think that if it is "possible" to remotely/semi-remotely overload the HV bus, it's a "so involved you'd be better off hacking something else on the boat"- type thing.
Absurd phrases like "beautiful Washington DC" (0:22) don't foster mutual communication.
@@alan6832 I'm pretty sure you'd need some kind of remote triggered device which would leave evidence.
"ship hits the fan" is the pun that I needed to hear this morning. 👍
Brilliant word game.
just waiting for merch with that on it, lol.
Electrical Switchboard are very simple and yet complex. Electrical Engineering degree is not easy to get. I got an Electrical Officer position in the US NAVY, however, my degree is also a hard degree to get. The switchboard breaker could be thought of a very big piece of metal....and that piece of metal has a lot of auxiliary sensors and conditions to cause it to trip. There is also maintenance that has to occur which includes opening the switchboard. Dust, sooth, oil vapor build up, fuel oil and etc can build up behind the switchboard...mmm closing those vents....
But instead, it hit a bridge!
I'm still waiting for a guy as well informed as Sal starting a similar channel on sewage. "What's going on with shitting"
Sal, you are a treasure! Not only do you give us the clearest explanations, but you point us to the issues we should be focused on and not the diversive, paranoid drivel that so many try to feed us.
Excellent detailed explanation. As a diesel mechanic it all makes sense. And to six people who lost their lives
R I P and condolences to the families. Gerry
Thank you Sal.
I’ll tell you how good your reporting/channel is…
Look at the comment section!
High level discussion from a wide selection of Knowledgeable people.
Keep it up Sal, you’re doing great.
Fantastic, clear explanation. I've conducted many safety investigations and it's always difficult to present results in a way that folks (who are not familiar with the system) can understand. Well done!
Main Power Breakers don't trip unless there is some electrical fault somewhere in the system causing them to trip. When a Breaker is reset and trips again that is a sure indication of an ongoing electrical fault.
Those breakers are going to continue to trip until the electrical fault causing the breaker trip is cleared.
"This is an electrical or control problem"
Thanks for the clear and thorough explanation.
Staying tuned in for further updates.
I believe the trip units in the LV and HV breakers were not from short circuits or overcurrent but from a signal via the PLC /SCADA. Perhaps the scrubber incident and switching transformer the previous day required a few releases to the system that were overlooked as they were scrambling to transit. They were looking at the Breakers when they should have focused on the HMI screen and what releases were causing the trip!
Yeah, i also reckon there was an overcurrent situation- causing the main breakers to trip.
I very much doubt that the PLCs ordered the main breakers to open.
It'd be a VERY unusal situation for a PLC to command a disconnect- of primary main power.
Something to ponder: it is strange that HR1 AND LR1 both tripped. If a fault occurred in the transformer itself HR1 should trip. If the fault occurred further downstream (LV panel or LV loads) then (in a system presumably designed with proper breaker coordination) only LR1 should trip. An at-home analogy is that if you plug in something shorted the 15A or 20A branch circuit breaker trips but the 100A main breaker does not, even though the instantaneous fault current is likely >100A.
@@sootikins Spot on. Keep in mind the magnitude of the fault can be well above 100 amps. As those CBs are thermal, it take time for the trip to occur. Lower amp rated CB hopefully trips first and in a house system, yea.
@@gregoryschmitz2131 Just noticed I made a typo - meant to say that the instantaneous fault current is >100A, not
Excellent job going through the NTSB report.
As a former submarine crewmember and power plant operator ,you did a good job in explaining this report
You always do a great job with these videos, Sal!
Thanks for making this understandable for the uninitiated
Great breakdown of the NTSB report. Thanks, Sal.
The 220 V and 110 V boards are combined in almost every installation. You have the house or office fed by a 220 V transformer with a neutral center tap, which gives you two hots that have 220 V between them, and 110 V each to neutral. The breaker board has two hot bus bars that alternate positions down the centre, so a single breaker gives you 110 V, and a double breaker gives you the full 220 V. Then there's also bus bars for neutral, and for earth, though in the main board those would typically be tied together.
Shipboard electrical system are usually ungrounded three phase high voltage to be step down 220 v to step down 110 v. Most land base in facilities are single phase 220 V transmitted step down to 110 v...in the US. Ungrounded is a simple concepts that trips most people up.
@@rgloria40 220v isn’t technically “stepped down” to 110v. As mentioned above, it’s single phase 220v center-tapped to provide separate 110v buses referenced to the center tap of the 220 coming in to your power distribution box.
I know all about 110/220 but don’t ask me which side is starboard/port
He was just trying make an analogy so that his audience could perhaps relate to the electrical system design of the ship. I don't think he meant it to be a fully accurate representation of household power design.
@@alliejr You should never make an analogy when you don't know what you are talking about. He didn't over simplify a home breaker box, he obviously didn't know what he was talking about. He may be an expert on ships but doesn't seem to know squat about his home breaker box.
Being a retired engineering sailor with a LOT of experience in this area (NAVY ships operate on pretty much the same principles), the way they had the electric plant aligned was a recipe for disaster. No backup should any either HR1 or LR1 trip and the other generator running "unloaded" on the other side of HVR. Had they had HR2 and LR2 closed (which I would think to be typical in a "restricted maneuvering" situation) this more than likely never would have happened. The generators should have been split better as well. Assuming this is an accurate portrayal of the generators relative location to each other, DGR 1 or 2 and DGR 3 or 4 should have been used with one of the others running unloaded. Poor plant management, awful SOP's, Lack of foresight and bad planning were huge factors here.
Well said. I was stunned that the online delay for the emergency generator was 45 seconds. It needs to be online before the main engine auxiliary systems are set to trip out. Code requires hospital standby systems be online in 10 seconds or less. It can be done. Transfer to TR2 should have been automatic. HR1 and LR1 trip out sounds like a differential relay operation due to a transformer fault. Not something one recloses on. This ship should have had at least two red X's on its engineering log and never left the dock.
Not my area of expertise. But isn’t the question what happens on the loss of a critical pump? How long to until the main engine shuts down? How long to restart the main engine? How long to start the standby pump and for the flow control valves to operate. I don’t disagree with your point in having the tie breaker open and the bus split during restricted maneuvering. Had they had the emergency EG running they would have never have lost rudder control.
@@fountainvalley100 All those were important clocks ticking. And those clocks are known and can be designed for. The ship was leaving port with too many problems.
I was thinking the same basic thing. I would just add that both HVR and LVR should be open to prevent a failure on one half to cause a failure on the other half.
Then my next thought, was the system operating the way it was an engineering design error, an automation design/ function error or an operator error?
Hate to say this but Lack of foresight and bad planning are Standard Operating Procedures in many places
Just watched your blog. Terrific explanation for us non-maritime folks. WOW, what a education we're getting from you. Thank you for the detail. Not boring at all. Very well presented. Keep up the good work.
Great synopsis. Thanks much.
Ex-Marine Engineer that spent a career in Gas & Oil.
It does strike me that the emergency back up systems never envisaged a power failure whilst in a confined waters.
I'm so impressed with the Pilot dispatcher and the Maryland Police in getting the bridge closed, incredibly impressive.
Almost like it was planned all along.
@@PD-we8vf Oh stop it.
The cruel irony for the workmen lost is that if it wasn’t for the work being done the traffic could not have been stopped so quick. They had people on both ends, slowing down and merging traffic for the repairs and they were able to quickly stop the traffic. I believe all the bodies were found in trucks so it seems like they were attempting to evacuate but didn’t have enough time to get off the bridge.
@@daveh9755 Aparently, the actual workers were on a break and had trturned to their vehicles to sit down and rest/eat snack. The inspector was likely inspecting. ( curiious if he had radio and was told to get off the bridge ASAP).
One worker in a truck survived because it had manually operated window and was able to get out after the truck plunged into water.
Yeah, that seems like a major, glaring deficiency to me. Seems like there should be a "port" mode that permits the captain/pilot to defeat automatic shutdown features while in confined waters.
One time a contractor hit the "emergency power off" for our facilities UPS. It took several minutes to get power back on. It took days to get all the systems working again. Many things are controlled by your basic PCs and they often don't like it when they loose power without a proper shutdown.
Testing. Trial and error before the real incident. People don't like the time and expense.
It surprised me when the breakers either side of the TR1 tripped the breakers for TR2 did not close automatically, or as a variation the crew close those TR2 breakers. Closing breakers that have tripped because of a fault is asking the breakers to reconfirm that a fault exists - it did - so why not use TR2 and its breakers immediately? I'm sure the CG will be asking this.
I had the same thought while watching this.
Excellent analysis. From all appearances that is exactly what the crew did. If the fault still existed on the affected bus then the breakers will open again. I’m not an expert, but I’m going to assume that there’s an SOP for what equipment must be on-line for entering/leaving port/restricted maneuvering. And, why was the emergency generator unable to pick-up the load?
Yep, it seems it would be more logical to restore power by close line through TR2, there seems to be a problem on the line going through TR1, which was not used for a while, and perhaps was not checked before departure.
I too have some Q’s about why there wasn’t automatic closing of the TR1 breakers on failure of TR2. Also the emgen wasn’t mentioned in the blackout alongside just that the standby main genset started and put itself on the board and .. then failed due to low fuel flow….
Exactly. And hospital standby power systems are online in 10 seconds or less by code. Standby power needs to be online BEFORE main engine auxiliary systems trip out or those systems should have their own smaller APU online with a transfer switch ready to close on the emergency bus. And was TR2 locked out of service? Ship should not leave the dock. It's a red X.
Congratulation, you did a great job describing the electrical issues. Thank you. Gaston
Sal, thank you for explaining detailed systems and their use so clearly and in a way that us non-engineers can grasp. Keep up the excellent work. Thank you ,
It's been many years since I've been a sailor, but as I recall, when we were at Sea and Anchor (maneuvering) detail, all systems were up and running. All the generators and as I recall, the Engineers (Snipes) would split the load so one branch would never have to shoulder the entire ships electrical load....from the simple diagram that I see here, it looks like that should have been done.
HR1 and LR 1 won't trip the HV bus, they are both downstream of that. The HV bus is protected by the breakers between the generators and the bus. I also don't think a lube oil pump will run on the HV bus, 6600 volts is way too much for the motor needed for a lube oil pump (and yes, I understand it's a big pump). Ergo, I think you will find it is only the LV bus that failed in the first blackout. The second blackout appears to have tripped breakers upstream of the HV bus so DGR3 and 4 must have tripped.
I feel sorry for the poor trainee pilot, what a thing to have happen on a training trip.
So what equipment do you think is on HV bus if not oil pump ?
Sal did say that the L.O. pump was on the HV buss, seems unlikely as a 440V 3phase buss would easily support a 250-400 hp motor with a soft starter. There's probably a dozen different trips associated with a protective relaying scheme for a 6.6kV buss tie breaker, not the least of which is "under-voltage" which trips all high voltage breakers if the control voltage drops below a specific minimum. This might explain why they couldn't clear the fault and re-close. This situation needs analysis from a qualified electrical engineer.
From the NTSB document:
"The generators were connected to a 6,600-volt high-voltage (HV) main electrical bus by the vessel’s power
management system (see figure 3) that powered various shipboard equipment,
including the main engine lubricating oil pumps, the bow thruster (a propulsor on the
ship’s bow that that assists with ship maneuverability), and reefer containers"
"A 440-volt low-voltage (LV) electrical bus was connected to the HV bus via
redundant step-down transformers (TR1 and TR2 in figure 5). The LV bus powered
vessel lighting and other equipment, including steering gear pumps and the main
engine cooling water pumps."
When The "Other Guy" evaluated the report he said the Oil and Water pumps were on the low voltage bus. A major difference between the two evaluations.
@@dmitripogosian5084 The reefer circuits for the containers draw a lot of power, and must be fed from the HV bus.
Fascinating!
Amazing story about response speed.
Thanks for sharing Your valuable time and insights.🎉😂❤
Iron sharpens iron just as excellence begets excellence and now Sal’s high quality reports attracts the highest quality comments a community engagement. Such an amazing global resource, I wish there was medal or national honor we could bestow in recognition of all the hard work and dedication to the industry and lives it supports.
Great update Sal! You made it understandable for this old retired firefighter.
Excellent! I look forward to your reporting everyday. Humor included!!
I read the preliminary report last night. The only item which surprised me was when NTSB stated that M/V Dali was only required to shift to LSMGO at 12 nm. I never sailed on a ship with a scrubber and completed shifting over by 200 nm. That was required for the North America ECA.
I was an electrician for over 35 years and never saw separate panels in a house for different voltages. That's common in commercial and industrial applications, but never in a house.
Isn't the meter mounted into a panel on the outside of the house with a shutoff? Isn't that 220? Then wires passed into the interior panel?
I refurbished a house built in 1945 in 2011-2013 and it had, believe it or not 6 fuse boxes of different voltages. I still have the one light fixture from the top of the laundry room that looks like a plate of spaghetti that was up in the attic (had knob and tube, 12-2 paper and tar wiring). Everything was stripped out of the house and new panel and new wiring installed. I often wondered what someone was thinking; not safety that's for sure.
True, but we all knew what he meant (at least most of us)
He was just trying make an analogy so that his audience could perhaps relate to the electrical system design of the ship. I don't think he meant it to be a fully accurate representation of household power design.
@@alliejr An analogy fails when the physical description of the common item is incorrect.
Sal, you are the best! Thanks for all you do!
Excellent and quantitative analysis. Clearly stated. Thank you.
As a nuclear engineer and expert in redundancy and safety systems, I can see two design flaws and administrative shortfalls in the Dali. The engineers designed everything to protect the main engine and the equipment assuming that protects the ship and crew. Not necessarily. Sometimes we may have to sacrifice in a grave emergency to save the ship and crew, which could have been the case with a design of "run to destruction" option. A safety interlock "lockout" feature may have been helpful to avoid the second trips and blackout that may have allowed saving the ship. If an electrical component starts overheat that is better than risking ship and crew. A fire watch in key places can be posted when used. Second the main engine design has apparently little in the design of self support. Why is the oil and cooling not pumped off the engine itself? "On" then means "on" and not "conditional on." "Well they don't come that way." Time to rethink main engine drive on large ships. Third exercising breakers and loads is a routine PM that should be performed in dock and at sea. Also a blackout test procedure should be performed before sailing.
Seems to me that you are correct; why aren't the oil and water pumps mechanically driven or at least the minimum requirements? To me that seems to me that that is the only logical way, like things were on engines in automobiles before all the computerized crap. Glad I still have my 1996 pickup where you can actually see the exhaust manifolds because of the lack of engine compartment garbage. Have a 1954 Ford NAA tractor that is even more simple.
Good points. As an electrical engineer with some experience in protective relaying systems, I was surprised to hear the system did not automatically transfer to TR2 after TR1 was isolated. Generally, closing in again manually (overriding the protective system) on a transformer fault is a bad idea. And TR1 appears to have failed again in this case. Also, having experience with standby power systems, I was surprised to hear that the delay to the emergency generator getting online was 45 seconds, and that that exceeds the delay time for main engine auxiliary systems to trip out. Hospital standby systems are set for 10 seconds or faster. Thus there were at least two opportunities to tune the existing system into a state that would have promptly restored power and avoided the loss of the main engine. Of course, if TR2 was out of service, that should be a condition that keeps a ship at the dock.
@@timtrewyn453 Yes you have a better solution to have an automatic switchover. But undervoltage trips could be overridden and perhaps there could be rather than a safety lockout, an emergency configuration that shuts off refrigeration loads or non-essential loads for emergency operation. I am thinking more of emergencies during heavy seas or major ship threatening struggles, there should be an "essentials mode" including non-safety related trips that could be overridden. Yes I thought about the transformer after the comment. It looks like a transformer problem.
I believe it was a release in the PLC logic that sent a trip signal to each breaker. It seems the scrubber incident is the only "unusual occurrence" here so we should look at that. If the HMI can bring up the PLC program or a SCADA graphic user interface with a screen that has reales on it we may see what is causing the trip. Seems like something is timing out-so the screen would show us. On tunnel boring machines Ive designed 25 years ago I had HMI's
Driving pumps off an engine that is reversing is problematic.
I'm surprised that the shipping industry standard uses only electric driven pumps, in the electric generation industry, standard is shaft driven oil pumps with electric auxiliary pumps and additional battery driven emergency oil pumps as a fail safe. Your description of events is well done.
Those are bigger too, you can't shut them down as easily usually.
In all steam turbine Navy ships I worked on there were shaft driven lube oil pumps with electric emergency backup pumps. The electric primary pumps must be a feature of diesel propulsion plants.
Electric pumps have big advantages. For one thing, you can pre-oil an engine with an electric pump, minimizing wear at engine start. Not a marine diesel guy, but I can see why they would prefer electric pumps. Ditto for coolant pumps, where you can vary the pump speed to allow faster warmup of the engine and more precisely control operating temperature.
Sounds like these critical main engine systems need their own transfer switches with an automatic set of options to get power before they have to trip out.
@@dougrobinson8602 Electric pumps are still used to start up and shut down, but when at speed the shaft driven pumps take over, much more reliable and additional redundancy. There are a myriad of ways this could have been avoided by redundant systems, I'm just surprised at how lax standards are. We now see the cost of these lax standards.
First time hearing about the one crew member running for his life and another one getting injured.
Also bridge worker running for his life to safety. WOW!
The guy they fished out of the water reportedly had hand crank windows on his truck.
And that's why I keep a window breaker tool close at hand. No manual windows = break glass in emergency @@SteamCrane
@@SteamCrane I certainly prefer hand crank windows!
Thanks , Sal, for breaking it down so that I could understand what was going on.
Great job explaining the electrical structure and sequence of events. Your work is always top notch.
Positive alcohol content 14 hours after the incident could be challenged in court. "I was so rattled AFTER the crash, I had a few swigs to calm my nerves!"....
It could be challenged, but a savvy person could shoot that down.
I was once in a jury for a drink driving case. The accused refused a test on scene, so we had BAC results showing they were drunk hours later. The defense said the question was "what was the BAC at the time of arrest, which we don't have." I asked for data about how alcohol is metabolized so we could try to extrapolate the BAC at time of arrest.
I do not understand why they were not tested immediately here.
As far as I know, any USCG regulated ship cannot have alcohol on board & crew cannot bring ANY alcohol on board, at least while operating in US territorial waters.
You would lose your TWIC clearance very quickly if you just had alcohol in your possession on the ship.
There's no legal way they could've drank unless they crew were allowed to deboard to shore before the urinalysis
@@j_taylor If the alcohol was below 0.08 hours later, you can't just assume it was higher based on an average person's metabolism. Also jurors don't get to ask for new evidence; you CAN ask for words to be defined, to examine exhibits and to rehear testimony. You also can't convict based on legal theories you dreamed up in the jury room.
@@j_taylor They were by the Master, just not the third-party tester used by the USCG.
A civil case is on the balance of probabilities, no?
Thank you for this comprehensive and understandable interpretation of the preliminary report. With all the sensational/dramatic and speculative commentary flying around on social media plus the news coverage it's so refreshing to hear someone , from the industry, who actually knows what he's talking about. I have enjoyed your coverage since the event occurred.
Watched a similar video but it did not make sense because the presenter did not explain that the bow is filled with ballast to stop it shifting unexpectedly after the cutting detonation. Good job explaining this overlooked but critical detail.
I'm NOT an engineer nor a seafarer but I'm grateful for this explanation. Although not yet comprehensive the NTSB report with your breakdown was most interesting. Thank you. (Tony from Melbourne Australia.)
Finally… someone who has knowledge on what they’re looking at, and not just spewing random articles and talking nonsense to fill a 15 min video… great explanation of the data for those of us with mariner experience as well as for those who don’t
Thanks for taking the time out on your busy life sal we really appreciate you. Thanks for keeping us in the loop. So what you're saying is something or somebody trip the breakers 🤔 sounds like really bad luck that all things go perfectly wrong or very good planning. I wonder which odds are higher ! It's because they speak Marine engineering they don't speak human lol
I sailed for years and was always interested those big ships. Sailing in Boston harbor before taking her out for the season buddie and I were close to the oil docking area skirting through (short cut) this tanker was being controlled by two tugs. Ever look how fast they move ? I tried to sail past as tugs pull in closer port side. 600 feet 500 ft. then one of the tug pilots open the window and Yell, "Can you put that little boat over there so we can put this Big! boat over there" Tacked got some wind and slowly moved.. I miss sailing..
Funny!
@@richkennedy326 no rich my buddie norman panicked , he was learning how to be comfortable and safe on the water again. His dad drowned diving when he was 8. but he was willing to learn sailing. So many boat close calls.. thx R
I am a retired senior engineering officer. ships i sailed on had a separate distribution board for all items to be fed from the emergency generator(emergengy lighting ,nav lights and 1 steering gear pump. This ships system where the emergency generator is just connected in to the low voltage bus is totally different . when the first blackout happened the emergency generator sharted and connected to the lv bus , crucially it didn't need to syncronise its ac waveform to the bus as there was no other ac wave present , but at the point that the main generator were reconnected it would be necessary to syncronise the ac wave form with the emergency generator this appears not to have been done (or it may not have been possible) connecting unsyncronised AC generator together is not allowed as a syncronising torque acts on the alternators which is powerful enough to shear drive shafts , and /or destroy alternators. i think that the emergency generator needed to be disconnected before the main generators were put back on line.
Well said.
Automatic paralleling is available. Usually for utility scale generators.
@DavidHamby-ORF-48 had auto paralleling on several ships but it was very slow to work, and chasing the frequency down to synchronise was certainly quicker done manually.
@@DavidHamby-ORF-48 yes....but auto paralleling is slow...that is probably why the third generator was on standby...running, and synced freewheeling but not carrying a load.
Incoming is slightly faster than running to insure incoming doesn’t reverse power trip. Can set speed by hand then enable auto closing. If running wobbles, it won’t close out of limits.
Thanks for this, Sal. Will watch it a few times to get through all that tech talk.
As a marine engineer, I'm impressed with, and appreciate the accurate explanations. Well done!
With commercial building electrical panels, major breakers aren't really supposed to be tripping. Smaller breakers are supposed to be tripping when there is a problem. The only time a big breaker trips is when one of the big parts fails.
Sounds like the differential relay protecting TR1 operated. Sounds like they manually closed back in on a damaged TR1 and it tripped again. They needed to close on TR2. Indeed, the relay system should have automatically closed on TR2, unless TR2 was out of service, in which case the ship should not have left the dock.
@@timtrewyn453 Yep, reclosing after a trip is a bad idea when the other transformer is available.
I'm just curious why the standard configuration is to have the tie closed and one transformer out of service? It would be more reliable to have the 440V tie open and each transformer supplying half of the 440V engine room loads.
@@timtrewyn453 It could be something as simple as a loose terminal bolt on the breakers or something more major like a developing short in a transformer. Once they get the Dali back in a dock and can shut it down completely, they can do a proper forensic analysis on all of the parts until they find the core problem.
Former submariner electrical officer here. Most of my electrical operators had 1-2 years of experience and we followed class procedures for operating the electrical plant.
Starboard HV-LV breakers tripped. That suggests a poorly transformer, perhaps compromised insulation. Was crew logging ground readings? Are they in the automation logs? Running LV plant off one transformer for several months seems unusual. Idle equipment cools below the dew point and develops grounds. Operating and warm is good.
Six month crew contracts and 6 mont engineer officer contracts don’t give time to develop deep experience with the plant and its operation.
@@DavidHamby-ORF-48 Would HR1 and LR1 be connected in a way that would ensure mutual triggering? It would seem to me it would be a smart thing to do that to electrically isolate the transformer if it's developing a short.
Thank you for a Great explanation.
Dr. Sal
Your comment about marine engineers seems to apply to most engineer types, having known a few in my time. Thank you for such clear analysis.
When one works in a specialized field, regardless of what it is is, is sometimes hard to get someone to understand you when you are talking about apples when all they hear is oranges. Been there; Worked for a huge national company and we had so many acronyms there was a book that listed all of them. Even so when talking about some of the specialized machinery we used, you had to be very generic and leave out all the details when talking to a friend or relative or non co-worker or an employees that did not work in your area.
Hi Sal
Many thanks for a very detailed and easily digested summary (specially for the "non bofins"). Love your "ship hits the fan"!!!
Congrats on your recent 50M views and note you are fast approaching another milestone - 300K subscribers.
Cheers
David
Well done! very understandable thank you
Sal I can’t explain the fist blackout but the second I think I can. I’m assuming ships power works the same as land based generators and an engineer can explain this better but I will give it a try!
There is a macro level of detail in the generators when you have multiple power sources connected they have to be synchronised ie the voltage and sine-wave from each generators have to be synchronised to be connected i imagine the normal startup processes applies electrical load in stages when you compress that timeframe the generators wander out of the voltage and frequency limits for synchronisation and cause a trip!
If they did a rapid recovery and dumped a ships worth of load on generators that hard barely restarted I can easily see a trip
Re the comments about delayed alcohol & drug testing: Comments about alcohol are correct, but drug testing (for metabolites) comes back positive for weeks after consumption (for most common recreational drugs). So those 3rd party negative drug tests for the entire crew provide useful knowledge.
erm- depends on the drug, and the test. Lot tests will miss cocaine or heroin thats more than 72 hrs ago, but may show traces pot 2 months after "exposure".
I probably need to dig into the NTSB report, but thank you so much for this summary. The Electrical system schematic is the key to this incident. However what is not shown on the schematic is where various loads are. The opening of HR1 and LR1 should not cause a total loss of power to both buses, but only a loss of power to the LV bus. The HR bus should still be powered, supplying power to steering pumps and lube oil pumps. It appear the LV bus had control circuits, cooling water pumps and lighting. So we saw a complete blackout of all the lighting, but as long as the DGs kept running, there would have been power on the HV bus. One thing I learned operating submarine electrical systems, is that automatic control is always better than manual control. Letting the system work automatically, is better than getting in there and shutting breaker manually. When you said the operators started manually shutting breakers, I thought, thats when the real problem started. If they had done nothing, and let the EG come up and supply the LV bus, while the DGs continued to supply the HV bus, they may have been able to recover without incident.
It would indeed be sad if their re-energizing TR1 temporarily reset the time delay on the EG, further delaying the EG coming online. Emergency power needs to online before main engine auxiliary systems trip out. You are right, the system should have switched over to TR2. For some reason that just did not happen, and they did not seem to even try to make it happen. Was there a known problem with TR2? It should have been energized in the first place, with loads logically sectionalized or the protective relays set to protect a looped bus.
My understanding was not correct. The Cooling water pumps are 440V powered. Which means failure of the low voltage side would cause the main engine shutdown due to loss of cooling water.
Yeah if my understanding of the schematic is correct, the only thing HV1, LV1, HV2, and LV2 isolate are the transformers. Which makes sense from a systems design standpoint. You want to be able to isolate those transformers from both sides. But each of the other systems being fed High Voltage 6600V power, lube oil pumps, steering pumps, etc, would have their own disconnects so a fault or short in one of those systems doesn't take down the whole HV system.
In the schematic shown, the only HV load shown is the Bow Thruster but we can see the HV circuits trail off on either side of the generator. That's where I would expect to see those other HV loads shown. Something isn't adding up in this explanation.
My only idea is those lube oil and cooling water pumps have Low Voltage control systems that shut them down despite High Voltage Power being available.
I have no doubt that most marine engineers don't know how to explain this stuff to the general public as well as you do. I'm glad you are doing it.
Fascinating information well presented, thanks!
Kind of surprised that after the breakers on both sides of transformer 1 tripped, they reset them instead of connecting the buses through the backup. I would have thought that after a trip, you’d treat that leg as suspect and avoid it in favour of the backup route until the reason for the trip was known.
Hindsight and all that, but this is the sort of thing that I would have expected to be written down as standard procedure.
Exactly. Transfer to TR2 should have been automatic. Reclosing on what appeared to be a transformer differential relay operation was a not so good idea, but maybe they knew TR2 had an issue.
Ditto both of you.
I am reading the comments ahead of the video -- to have flags set for what other people are seeing as key points -- and I look forward to any explanation of something like, "There was a faulted state logged for Transformer 2, so all power flow was shunted away from it. That is a major service, and it was scheduled under warranty at _X_ harbor in _two months_ due to shortages of critical components."
Concur, especially with the initial reports that the cargo refrigeration loads had caused the breakers to trip twice while still pierside. Human beings are inordinately *bad* at changing their habits, and instead we rely on past practice and old, out of date data.
On the nimitz we had rules against performing electrical maintenance during flight ops or restricted maneuvering. they wanted zero chance of operator error accidentally impacting the ships electrical distribution or propulsion during critical times. Do commercial vessels have prescribed configurations for their distributions systems during restricted maneuvering? is there extra manning in the propulsion spaces during these times?
The did maintenance and had the first accident due to it while at berth
Just for my curiosity: during your service on Nimitz class, did you also perform kind of `blackout drill exercises´ to be able to react as fast as can in such emergency situation then (would expect so in the military)?
Dali crew was under extreme pressure, knowing they’ve more or less just left the pier, being in harbour area still. So besides expertise and analytic skills, it also needs training to quickly get things under control again, partly “even without thinking about but simply know what to do next”. To be clear: it's not at all about assessing or judging their performance; i'm much more interested to learn about procedures.
excellent question
@@NRZ-3Pi10 we had a real blackout a few days after I reported aboard, but it wasn't during flight ops. I wasn't even in my division then though, as you spend the first two weeks going though FAM&I (familiarization and indoctrination) to life aboard a carrier. Very basic stuff, like learning where to go and where not to go, safety procedures, administrative things (like where the ships stores, dentists and doctors, chapel, etc. are). after that nukes go though training division, where you learn the class- and ship-specific details and procedures of the reactor and propulsion of the nimitz, this takes a few months. then you get to go to your division proper.
Now the question is, “Why did the breakers trip?”
The second time was likely caused by the emergency generator backfeeding the high voltage buss, when the breakers near the transformer were closed, the restarted generators 3 and 4 were out of frequency or phase. Hugh current flows occurred and the diesel breakers opened to protect them. EG1 should have been disconnect first.
That's the title of this video, but he only discusses the "how" question. Next question could be " Why and how did the ship change heading while rudder was centered and ship moving forward at 8 knots?" This should be addressed in a subsequent report. What were the steering inputs applied, was the bow thruster engaged? What effect did the anchor drop contribute?
@@JeffBartlett-kj6sq Backfeed is unlikely. The diagram is very likely wrong. Usually the emergency generator has its own bus. The normal source and generator breakers are interlocked, prohibiting backfeed upstream.
@@moregrouchy Currents and wind are responsible for the course change. With the breakers tripped, the bow thruster can't operate, it is also useless at that speed. Did you listen to the video? The anchor was basically useless, as the brakes weren't engaged, per crew report.
@@moregrouchy
Your questions have been answered at least 100 times. Please refer to the other hundreds of videos again.
Sal = always prepared! Glad you brought your camera gear!
You did a great job of explaining the report and the events.
As a power plant and electrical distribution tech for new ships I have a lot of experience in testing power plant blackout restoration.
Sounds like 10hrs prior and the incident itself, they both had two blackouts in a row; which leads me to believe they forgot to restart fuel pumps or seawater cooling pumps and the generators shutdown on high temperatures most likely.
All this makes a tug escort past navigation hazards like bridges look like a bargain, not an expense. In Japan LNG ships have attached tugs 15 miles or so from the berth when loaded, and escorted out into deep water, clear of hazards in ballast. Worth a thought.
Increases cost of goods for all.
Also many modern bridges have superior defense... concrete is cheaper hah
Where do you draw the line for tug escort? Even for this ship's case, you had two more bridges to clear - never mind other possible hazards - after the Key bridge before it's in open ocean and the last one to clear - Cheasapeake Bay Bridge - is 150 nm from the Key bridge. No one is gonna do/pay for the tug escort for 150+ nm.
@@dertythegrowerwould it, actually? I mean compared to insurance cost tugs would probably be negligible, especially if that made insurance less expensive...
Where are you getting all these tugs and crews? Do you know how big America is? Do you know how many ports there are?
@@kain0m You also need more tugs and more crews. Plus, it will make Baltimore less competitve to other ports where you do not have long and expensive mandatory tug escorts
Excellent!! Well done Sir, well done!
Bet the family is with you. Have a great teaching session too. Bet there is going to be some fantastic conversations. Thinking of you. ❤
I'm very impressed with the presentation you did. I thought it was easy to understand and an excellent summation/ evaluation of the report.
As a former power plant EE and manager at several power plants (coal, NG, Oil, diesel), i am not a marine engineer but a couple of things stood out in your video. I agree with Sam this sounds like a protection scheme issue, the system if properly maintained was trying to protect TR1. Its normal in transformer protection relaying to open both the HV and LV breakers when a fault is detected on the related buswork, associated components (CTs/Pts), and of course the transformer itself. The ICMS should be key here as if it is similar to what we used in the utilities i worked for, the data should show which signal initiated the relays on the LR1 and HR1 breakers to open. Work upstream from there and you will find most root causes. My 2nd item is more operational in nature. When the crew attempted to restore power after the first power failure, why did they go back to the same TR1 and its breakers when the system was telling them 'Hey i am detecting a problem in this section of the buss"? I can only say in the utility world when we had dual transformers and associated breakers on standby, our procedure was always to use the standby (in this case TR2) and isolate the tripped portion of the buswork for later review before attempting to re-energize. Now if TR2 and that buswork had known maintenance issues well then that's another discussion. Great video!!
Why the data storage went out with the blackout does not make sense. Any data system could easily be battery powered as backup for this exact situation.
@@howardsimpson489 I agree. In our utility systems the control systems were always on (1) backup gen set; (2) backup batteries especially assigned to the control systems and data.
Thank you. I was wondering why both the LV and HV breakers were open. This implies to me the fault was in the LV area. Are you saying it may be normal for both to open when only one has an over-current condition?
@@LarryCook1960Let's say the temperature monitoring indicates a possible failure in the transformer cooling system. You'll want to completely de-energize the transformer. You definitely don't want a transformer fire on a ship at sea! Even on land at a substation, it may require evacuating nearby homes and businesses.
@@wramsey2656 The lack of backup power for data has also puzzled me from when I first heard about the gap. It only makes sense if none of the systems had any data to report due to the loss of power, but I'd expect a few hundred milliseconds of bad power/loss of power followed by power restored/power good, not to mention continuous switchboard statuses for both switchboards, and any fault indications they may show. A frequency/voltage drop could be an important clue as to cause.
Something went wonky in the control voltage of those breakers which most likely had UVRs under voltage relays that require power to keep the the breakers closed. With breakers of that size there is usually a mechanical charging handle that enables closing of the breaker against a dead buss. These were likely used by crew to bring one of the generators back on line. Synching a second genset also takes time. Theres also load shedding to consider along with switching logic circuits. Reefer loads on board are huge consumers of electricity. As this was a leased vessel, i suspect lack of switchgear maintenance and cleaning, relay/breaker fatigue. As a retired journeyman marine electrician i can only speculate without looking at the prints, maintenance logs, and what was done at the last yard period?
Excellent summmary of NTSB report. What is of particular interest is that only 2 generators were running under load. Load share between generators is easier on 2 working generators than on 3 or more. All the information points to a problem with the load share circuitry. The traditional method of managing load share is by the Droop Curve Method. When load share is not working trip outs are the result. I am sure tthe NTSB is digging into this. This could be a design prroblem or an operator problem.
Transfer to TR2 should have been automatic. Periodic protective relay testing and documentation needed. Load should have been close to one generator's standby rating. Underloading diesel generators can lead to engine problems. Caterpillar has a white paper on this.
As a layman I found your explanation concise and complete. Time will tell and get to the root cause so that this does not happen again. Thank you.
What really great and informative comments. Great video.
Sal this is an excellent channel.
Dude brings a SM7B and associated kit on vacation. That's dedication.
what you mean - vacation
its a work trip
@@koneofsilence5896 I'm sorry for upsetting the pedantry police. Point being, he's on travel, and brings a professional rig with him "just in case" he has to make a video like this from his hotel room.
In particular, SM7B requires (well, sounds much better with) a preamp like an air lifter, then you've got whatever he's using for XLR with phantom power, and all the cabling, and of course the mic stand - not to mention whatever he's using for his camera, possibly a tripod, etc.
It's a healthy bit of kit to take with you, especially when there are plenty of direct to usb solutions that take up a lot less space and are "good enough" for many RUclipsrs.
@@PsRohrbaughthank you for the audio explanation. Those of us who travel regularly on business are touchy about our work being referred to as vacation. Kind of like telling the sound guy, must be nice to get to see all of those free concerts.’ Be safe.
@@PsRohrbaugh :)
Jup
Real professional
And i will note down 'pedantry police ' for future use
The Shure SM7 is a dynamic mic, it doesn't need phantom power. Also, you mean a "cloud lifter", not an "air lifter". Not a big deal, really. Any simple USB mic/line audio interface can suffice On the road. And for youtube, there's not.much need for a cloud lifter (which are great, BTW). I suspect he brings his whole video and audio rig with him wherever he travels. And it's not pedantic to differentiate between vacation and work trips. They can be very different and usually are.
Thank you. Together, two generators should deliver almost three million Watts of power. Those are not small circuit breakers, but highly specialized equipment capable of disconnecting serious power. The breakers probably have compressed air or some sort of ballistic system to extinguish arcs which would have tried to keep carrying current as the breakers opened. It seems like a dedicated electronic control unit would be required to properly sequence the breaker operation; no doubt that circuitry is going to get a very close examination to determine why it triggered a disconnect. A dead short would have caused some sort of explosion, given that three million Watts were in play. To me that suggests a control fault.
If the high side and low side breakers on TR1 both tripped, that probably indicates TR1's differential relay operated and a blocking relay would also trip to prevent a push button reclosing of the breakers. Thus the "manual" closing of the breakers, which is a not so good idea and turned out to be a not so good idea. The system should have automatically transferred generator output to TR2. For some reason, the crew felt that had to try to make TR1 work again. The design appears to have sufficient redundancy, except for the 45 second delay for the emergency generator. That unit needs to be online before the main engine auxiliary systems trip out.
@@timtrewyn453 Thanks very much for the excellent narrative. I still wonder what caused the initial disconnect. Transformer fault?…
@@wtmayhew Protective relays that apply a trip voltage to the trip circuit of a circuit breaker have indicators or "flags" that change state to show the particular relay detected an abnormal condition and closed contacts on the trip circuit to open the breaker. It would help if we knew which relays were showing flags in tripped state. It is likely that TR1 was the point of failure. If the crew can see the fault was caused by a temporary condition like a snake or rat causing a short, then they are justified in attempting a reclose. If they do not see that temporary condition, then they should consider TR1 out of service. TR2 should have already been energized and the load should have automatically been transferred to it. The schematic provided shows that is possible to do. Electric utility systems do that successfully, and we are talking utilities having ship scale loads like 10 megawatts being successfully transferred to the paired transformer in the substation. Perhaps there is some maritime exception, but I doubt a single point of failure should get things so out of hand by design. There are two transformers there for just this type of scenario. My main source for these assertions is the Westinghouse Electric Corporation classic text "Applied Protective Relaying". I am confident that the NTSB has access to very good protective relaying expertise and utilization equipment expertise. There is also more information for them than has been shared with us, the relay flags being an example.
@@timtrewyn453 Thanks for sharing your expertise and research. Much appreciated.
Excellent video with the right amount of detail presented in a manner that was easy to understand.
Sal thank you for making this make sense. At the end of the day no matter how big or how small the problem was does not bring back the 6 lives lost or the key bridge. What a terrible peice of history to have to be apart of. Thank you for always make it easy for us to understand. Please as you always do in your videos keep these 6 family’s in your prayer may god give peace to the families and a very huge shout out to NTBS for putting the report out after the final victim was recovered. For the respect to those family’s as bad as we all wanted answers I am glad the last victim was out. Tufs.
Sal, is it possible that the breakers trip because of excessive power draw from the refrigeration containers. In other words to many refer containers. As always fascinating videos, thank you.
No. The reefer containers are fed from the HV switchboard (through separate transformers) and would not affect the HV-LV transformers. Further, if the HV switchboard is overloaded, the installed "Preferential Trip" system would engage, and the first consumers kicked would be the reefers.
NTSB did another detonation on power outage, but what I find odd is Chief Engineer clears ship on power outage when it’s not clear if it was safe.
As a former USCG federal on scene coordinator the rescue operations should be completely separate from the crew investigation. I wonder why the investigators were not on board within an hour of the incident. I was in NYC and we had to have responders ready to go. I spent many nights on barges that ran a ground in the East River and responded within an hour to the scene. I am curious as to why the drug and alcohol testing took so long
Great report! You made it all very easy to understand. Thank you.
Great episode Sall!! Your awsome!! Thankyou!!😊