Okay, I'm with the I.T. guy. You can be very confident in your backup and still highly reluctant to actually test them. I know, at some point, you have to do it. But, also, there's always a risk that you missed something. There's a valid use case for actually testing your disaster recovery plan, true. If you can do it non-destructively, then, yes, by all means. But. Putting any system that is actively operating into an abnormal state should never be done without preparation. You pick your time, and you have all hands in case something goes horribly wrong.
yeah im absolutely with you - that was one hell of a stupid CEO and the consultant sounds like an absolute con artist to me... a professional might suggest doing a test and lead with a question like "alright what if we shut this off right now", but any actual testing wouldnt happen... and showing up with a chainsaw is just downright unprofessional heck intentionally destroying your primaries to prove a point is a monumentally stupid position to begin with and it does not matter whether the secondaries work or not i would have called security on that hack the way you actually do a test like that is: step 1) you make a hard backup of everything and announce a maintenance period to your users 2) you have as many people ready to do damage control as possible in case something goes wrong 3) you start with a controlled failover and if that works fine then 4) you can go for a hard shutdown or even a power cut on the primaries, essentially simulating an uncontrolled failover and then 5) the engineers have a field day getting the original primaries back up and running properly while the secondaries run business as usual
There is actually a good rationale for testing disaster recovery on a live production system. If the IT guy knows that an outage is coming, then he can put precautions in place that he wouldn't be able to put in place in a true emergency, intentionally or otherwise. This is why Netflix uses a "chaos monkey" to randomly terminate running processes in their streaming service, and Amazon has even tested their disaster recovery by cutting power to live datacenters. Obviously, if you aren't fully confident in your disaster recovery plan, you need to work up to this gradually.
The answer to this one is obvious. He's the Canadian guy who chopped down the power lines, pitching a business idea for a new wilderness survival kit. EDIT: Wow, I was closer than I expected.
@@lucbloom Nah, he is almost right, it's the Canadian chainsaw guy, he is running a business where he checks for... survival of things. He's all about electricity and chainsaws.
As a programmer, I can 100% understand that. Even after testing something a lot of times (including temporarily turning the "main server" off for testing), the backup can still fail when it's needed most.
5:50 They were proving that the building's security was not adequate by sneaking a chainsaw into the meeting without anybody noticing, thus proving to the company that their security was not up to scratch and needed to be improved.
Even if you have 100% confidence in the backup the problem is that, once you intentionally and irretrievably take out your live server, the backup IS now the live server - there is NO backup. You then tell the CEO that if you want a backup backup, he/she will have to pay approximately double what they are paying at the moment, in addition to whatever it cost to replace the hardware you just nuked. Now you watch *them* sweat.
That was my guess aswell 20 seconds into the video. Might have been because I binge-watched White Collar lately, and the episode where Neal is tasked with thesting that bank's security is one of my favourites.
Same here. My only other guess was that he'd stolen a rival company's chainsaw so they could reverse engineer it or prove it violated a patent they owned.
I had the CEO & head buyer of a private hospital viewing new surgical tools for their proctology dept! (Pre-Python song, tiny chainsaws WERE used for surgery : )
My first idea was the guy who brought the chainsaw was a sales rep and he got paid because they decided to buy equipment from them. My second was penetration testing. “Look, I got in here with this chainsaw, it could have been anything! You need to work on your security…”
Literally my line of thinking too. 'Thought it was a prototype product on offer, then potentially a very silly pen test showing their defense-in-depth was lacking enough for a guy with a chainsaw to just waltz in. XD
Alright, to be fair, I'd probably be nervous even if the backup was reliable. The reason is that they're not _completely_ reliable. In situations where companies have failover datacenters and stuff, there are usually annual disaster recovery tests specifically to check if the failover works as intended. Basically there are usually enough risks to make it undesirable to demolish a production rack for giggles.
I read a story that a newly elected US President was being briefed by head of secret service that in case of emergency they can have a helicopter on white house lawn in 2 minutes and the president said "OK let's see. Bring the helicopter right now"
My first thought was that OSHA had come to crack down on some violations, and said “you can either pay your fees or we cut down this dangerous object right now”
I remember reading Jon's article back in the late 80's/early 90's in Personal Computer Workd or similar magazine. First lateral highlight I knew from the start!
The first thing that popped into my head was a professional chainsaw juggler. The next thought was a pen tester proving that there was a flaw in a company's physical security in a theatric way.
my first thought was a judgement of Solomon cut the baby in half situation, like the two people were arguing about who owned something and called the guy to solve the dispute and he went "I cut it in two with this chainsaw and you both get half" and then decided who got the thing based on how they felt about that.
My first assumption was that they were on Shark Tank, and chainsaw guy had made some novel new startup involving a brand-new form of chainsaw or something.
My guess was "he's a chainsaw salesman". 2:18: Chainsaws were invented to assist in childbirth. Obviously they were smaller than your tree-cutting saw, but there are lots of people who use them for purposes other than logging or massacre.
Honestly thought, like a lot of people that it was a pentest trick to show that if someone can smuggle a whole chainsaw into the CEO's office, they could do much worse. 6:30 (Spoilers Below) I feel like the nervousness could also be due to the looming paperwork headache that would come with having to replace the chainsawed servers, and set them all up correctly, (in addition to the general antsiness of having someone wield a chainsaw and threatening to break your stuff with it), rather than any lack of confidence in the backups not being as good as they should be.
I think I watched too much White Collar lately. My first thought 20 seconds into the video: The guy with the chainsaw was tasked to test the company's security for flaws and he managed to find one big enough to smuggle a chainsaw into the building. The CEO gladly paid because now they could close that hole in their security, before somebody with really bad intentions misused it.
Am I the only one that went to CEO getting divorced and the threat was to cut a precious item like a table or a car in half. Should I be thinking about becoming a mediator??
I figured it might’ve been a guy selling bulk chainsaws for the company but the two guys having a role threw that out the window. Also who would pay someone for bulk chainsaws without demonstrating them. Lol this was a good one
But, like, even if the backups worked, that'd still be an expensive piece of IT hardware, with a chainsaw through it. IT guy was right to object! I was actually thinking, this was literally the first chainsaw ever invented, but you'd want a demonstration, I'd think.
My first thought was that he was selling chainsaws and at the end of the pitch they decided to buy some. I'm not sure that this approach actually says anything about the quality of the backups. I'm pretty sure that the IT person would have been pretty stressed out about someone wielding a chainsaw at his lovely expensive server and thinking about the hassle of replacing it. Also, you can never have 100% confidence in anything IT related, you just have to minimise the risk as much as possible.
I wonder if the IT Consultant - Jon Honeyball is the person of the same name who contributes to PC Pro magazine. Fairly unusual name and in the same industry also.
My first guess was that someone invented a superior version of a chainsaw and showed the prototype to the CEO and the R&D guy, but since it had a combustion engine, he wouldn't test it indoors.
3:51 someone plugged in their apple device. It's not relevant to anything but I just thought I'd point that out. Perhaps for some likes, possibly a heart.. meh
Huh. I was wrong on this one, but I thought I recognised the story. Some years back a company had a nice big custom boardroom table made with some beautiful wood, inlaid this and detailed that, a real work of art - and the company decided to stiff the manufacturer as they figured it would too expensive to dismantle and remove from the building and there was no resale value for something that custom made. The manufacturer turned up with a chainsaw and threatened to just chop it in half. The company quickly decided to pay up.
(My immediate thought was uncharacteristically dark for me, but I’ll mention it anyway.) He threatened to chainsaw them unless they paid him “a large fee.”
about halfway in I was like. when Tom said that it was not used I was wanting them to ask was it Reved/turned on and idling. (sound reason). like I was thinking was they testing sound in a room and someone went its easier for me to get the chainsaw and use that for the Measurement then waiting on the 10K special sound equipment. or some CEO going Chainsaws are not loud. some one whit a chainsaw shows up revs it and puts it into idle and CEO goes Never mind its Loud as heck.
CAPTION ISSUE: There's a serious accessibility problem with the captions on these videos. There's a white on black, a light cyan on black, yellow on black, and … very dark green on black which is so low contrast that it's kind of unusable. Sometimes there have been a light green and a very dark green. The only control I have over this (per video) is to force them all white. If there's a way to better control the caption colors on the editing end vs. just saying "use another color" … the dark green isn't working well and I don't know how to fix it so it would.
Thanks for reaching out. That "very dark green" is actually quite a light green that's so bright that it's close to yellow. So I don't know if that's just your personal perception or you have a monitor or graphic card setting that has your green channel turned down extremely low? -- David
They were testing that safety feature that stops the chainsaw instantly if it touches skin, and they were able to demonstrate it without actually using the chainsaw by showing the mechanism triggering when it touched someone's skin.
Ummm... It's one thing to test and verify your backups work, but it's an entirely different thing to do it by forcing the issue and requiring them to work that very instant, not to mention the property damage involved... D:
@@darekpower any head of IT would be nervous in that situation and the consultant is an absolute idiot - the only thing the consultant proved there was that they have an ego problem and love pointless grandstanding over gotcha questions
Maybe just unplug the back up servers?!? 🤔 Save yourself a consultant fee. Plus, using a chainsaw would likely cause live cables and internal consonants to short-circuit, possibly causing damage to files you wouldn't get with just power cut.
Consultant: "What would happen if I went into the datacenter and..." IT guy: "...the fire suppression system would then activate and you would asphyxiate..."
Malcolm solves his problems with a chainsaw/ Malcolm solves his problems with a chainsaw/ Malcolm solves his problems with a chainsaw/ And he never has the same problem twice!
I was thinking it was a lawyer for someone who got hurt using the chainsaw, he got it out to show a design fault that the company should have spotted so they settled.
I thought it was to test security and say “look I managed to get to the office of the CEO with a chainsaw”
I thought that as well🙁
That was my second idea - penetration testing, basically.
That was my exact idea.
Me too
Guess a lot of us thought that 😅
Okay, I'm with the I.T. guy. You can be very confident in your backup and still highly reluctant to actually test them. I know, at some point, you have to do it. But, also, there's always a risk that you missed something. There's a valid use case for actually testing your disaster recovery plan, true. If you can do it non-destructively, then, yes, by all means. But. Putting any system that is actively operating into an abnormal state should never be done without preparation. You pick your time, and you have all hands in case something goes horribly wrong.
also it is really hard to predict what would happen to the flowing data if a server get's chipped apart bit by bit but not going down at once.
Also, I'd be a little afraid of the guy would cut the wrong thing and electrocute himself regardless of the backup situation.
yeah im absolutely with you - that was one hell of a stupid CEO and the consultant sounds like an absolute con artist to me...
a professional might suggest doing a test and lead with a question like "alright what if we shut this off right now", but any actual testing wouldnt happen... and showing up with a chainsaw is just downright unprofessional
heck intentionally destroying your primaries to prove a point is a monumentally stupid position to begin with and it does not matter whether the secondaries work or not
i would have called security on that hack
the way you actually do a test like that is: step 1) you make a hard backup of everything and announce a maintenance period to your users 2) you have as many people ready to do damage control as possible in case something goes wrong 3) you start with a controlled failover and if that works fine then 4) you can go for a hard shutdown or even a power cut on the primaries, essentially simulating an uncontrolled failover
and then 5) the engineers have a field day getting the original primaries back up and running properly while the secondaries run business as usual
There is actually a good rationale for testing disaster recovery on a live production system. If the IT guy knows that an outage is coming, then he can put precautions in place that he wouldn't be able to put in place in a true emergency, intentionally or otherwise. This is why Netflix uses a "chaos monkey" to randomly terminate running processes in their streaming service, and Amazon has even tested their disaster recovery by cutting power to live datacenters.
Obviously, if you aren't fully confident in your disaster recovery plan, you need to work up to this gradually.
@@dovos8572 You'd lose whatever the last packet was, it'd discard any incomplete packets.
The answer to this one is obvious. He's the Canadian guy who chopped down the power lines, pitching a business idea for a new wilderness survival kit.
EDIT: Wow, I was closer than I expected.
Ok, you expected to be at the opposite of the answer then 😂
@@lucbloom Nah, he is almost right, it's the Canadian chainsaw guy, he is running a business where he checks for... survival of things.
He's all about electricity and chainsaws.
As a programmer, I can 100% understand that.
Even after testing something a lot of times (including temporarily turning the "main server" off for testing), the backup can still fail when it's needed most.
5:50 They were proving that the building's security was not adequate by sneaking a chainsaw into the meeting without anybody noticing, thus proving to the company that their security was not up to scratch and needed to be improved.
Even if you have 100% confidence in the backup the problem is that, once you intentionally and irretrievably take out your live server, the backup IS now the live server - there is NO backup. You then tell the CEO that if you want a backup backup, he/she will have to pay approximately double what they are paying at the moment, in addition to whatever it cost to replace the hardware you just nuked. Now you watch *them* sweat.
My guess was a security consultant that smuggled a chainsaw through security to prove a point. Close enough.
That was my guess aswell 20 seconds into the video. Might have been because I binge-watched White Collar lately, and the episode where Neal is tasked with thesting that bank's security is one of my favourites.
Yeah, I was also thinking chainsaw person was a pentester or such.
Same here. My only other guess was that he'd stolen a rival company's chainsaw so they could reverse engineer it or prove it violated a patent they owned.
I had the CEO & head buyer of a private hospital viewing new surgical tools for their proctology dept!
(Pre-Python song, tiny chainsaws WERE used for surgery : )
Chainsaws were actually invented for surgery, tree chopping came later.
My first idea was the guy who brought the chainsaw was a sales rep and he got paid because they decided to buy equipment from them.
My second was penetration testing. “Look, I got in here with this chainsaw, it could have been anything! You need to work on your security…”
Literally my line of thinking too. 'Thought it was a prototype product on offer, then potentially a very silly pen test showing their defense-in-depth was lacking enough for a guy with a chainsaw to just waltz in. XD
Alright, to be fair, I'd probably be nervous even if the backup was reliable. The reason is that they're not _completely_ reliable. In situations where companies have failover datacenters and stuff, there are usually annual disaster recovery tests specifically to check if the failover works as intended. Basically there are usually enough risks to make it undesirable to demolish a production rack for giggles.
I wouldn't be nervous even if there was no backup. It's not *my* data after all...
It would also mean that you then no longer have a backup system (because one of your two systems was just destroyed)
@@Anaicyl That's exactly why the ideal state is 3+ backups. They definitely do this for storage.
I very much liked the guests on this episode. Great dynamics.
5:10 But it can’t be a drill; it’s a chainsaw!
I read a story that a newly elected US President was being briefed by head of secret service that in case of emergency they can have a helicopter on white house lawn in 2 minutes and the president said "OK let's see. Bring the helicopter right now"
"Malcolm solves his problem with a chainsaw...and he never has the same problem twice."
TOBY ON THE SHOW YES
Well that sounds dangerous. I can imagine the IT guy not being fond of the idea even if his backup was quite good.
My first thought was penetration testing, proving that a deadly weapon could be brought into the office through security.
Somewhere along that discussion of scaring the other person, my thoughts veered of towards "salary negotiation".
I love Julian, so glad to see him here!
Every time I see the thumbnail for a Lateral video, the theme song plays in my head before I click. Great job on that!
My first thought was that OSHA had come to crack down on some violations, and said “you can either pay your fees or we cut down this dangerous object right now”
Very happy to have Toby on an EP
The first thing I thought of was Gunnar Hansen or Mark Burnham turning up for their audition. That would have been awesome.
I remember reading Jon's article back in the late 80's/early 90's in Personal Computer Workd or similar magazine. First lateral highlight I knew from the start!
My prediction was that they sold chainsaws and were showing off the new model.
I thought he was selling chainsaw cases to a manufacturer of chainsaws.
After a while I thought it was a magician's show. Still entertaining for anyone on sidelines, I'd imagine
The first thing that popped into my head was a professional chainsaw juggler. The next thought was a pen tester proving that there was a flaw in a company's physical security in a theatric way.
Now that's penetration testing
Pen test by means of verbal threat. Brilliant.
He threatened to cut down a sycamore tree in Northern England
my first thought was a judgement of Solomon cut the baby in half situation, like the two people were arguing about who owned something and called the guy to solve the dispute and he went "I cut it in two with this chainsaw and you both get half" and then decided who got the thing based on how they felt about that.
I was thinking they sold chainsaws, here's the look of next season's model.
My first assumption was that they were on Shark Tank, and chainsaw guy had made some novel new startup involving a brand-new form of chainsaw or something.
My guess was "he's a chainsaw salesman".
2:18: Chainsaws were invented to assist in childbirth. Obviously they were smaller than your tree-cutting saw, but there are lots of people who use them for purposes other than logging or massacre.
I had completely forgotten this story, although the version I heard involved cutting down a power pole, not taking it directly to the server.
I may have to implement the chainsaw test
Honestly thought, like a lot of people that it was a pentest trick to show that if someone can smuggle a whole chainsaw into the CEO's office, they could do much worse.
6:30 (Spoilers Below)
I feel like the nervousness could also be due to the looming paperwork headache that would come with having to replace the chainsawed servers, and set them all up correctly, (in addition to the general antsiness of having someone wield a chainsaw and threatening to break your stuff with it), rather than any lack of confidence in the backups not being as good as they should be.
I think I watched too much White Collar lately. My first thought 20 seconds into the video: The guy with the chainsaw was tasked to test the company's security for flaws and he managed to find one big enough to smuggle a chainsaw into the building. The CEO gladly paid because now they could close that hole in their security, before somebody with really bad intentions misused it.
Am I the only one that went to CEO getting divorced and the threat was to cut a precious item like a table or a car in half. Should I be thinking about becoming a mediator??
I figured it might’ve been a guy selling bulk chainsaws for the company but the two guys having a role threw that out the window. Also who would pay someone for bulk chainsaws without demonstrating them. Lol this was a good one
One hour after this released project farm released a video comparing chainsaws
But, like, even if the backups worked, that'd still be an expensive piece of IT hardware, with a chainsaw through it. IT guy was right to object!
I was actually thinking, this was literally the first chainsaw ever invented, but you'd want a demonstration, I'd think.
Ha, I knew it would be something security-related.
New on Netflix this fall, King Soloman, CEO.
At the question prompt alone I assumed the man was a chainsaw salesman
My first thought was that he was selling chainsaws and at the end of the pitch they decided to buy some.
I'm not sure that this approach actually says anything about the quality of the backups. I'm pretty sure that the IT person would have been pretty stressed out about someone wielding a chainsaw at his lovely expensive server and thinking about the hassle of replacing it. Also, you can never have 100% confidence in anything IT related, you just have to minimise the risk as much as possible.
I wonder if the IT Consultant - Jon Honeyball is the person of the same name who contributes to PC Pro magazine. Fairly unusual name and in the same industry also.
My first guess was that someone invented a superior version of a chainsaw and showed the prototype to the CEO and the R&D guy, but since it had a combustion engine, he wouldn't test it indoors.
My guess is that he is from a marketing company, and they are talking about their new add for this company where they will "cut down the prices"
3:51 someone plugged in their apple device. It's not relevant to anything but I just thought I'd point that out. Perhaps for some likes, possibly a heart.. meh
Huh. I was wrong on this one, but I thought I recognised the story. Some years back a company had a nice big custom boardroom table made with some beautiful wood, inlaid this and detailed that, a real work of art - and the company decided to stiff the manufacturer as they figured it would too expensive to dismantle and remove from the building and there was no resale value for something that custom made. The manufacturer turned up with a chainsaw and threatened to just chop it in half. The company quickly decided to pay up.
(My immediate thought was uncharacteristically dark for me, but I’ll mention it anyway.) He threatened to chainsaw them unless they paid him “a large fee.”
It does say that they "happily" paid him the fee.
Blimey! I know Jon Honeyball so I've just insisted he tell this story to our mutual Facebook friends :-)
about halfway in I was like. when Tom said that it was not used I was wanting them to ask was it Reved/turned on and idling. (sound reason).
like I was thinking was they testing sound in a room and someone went its easier for me to get the chainsaw and use that for the Measurement then waiting on the 10K special sound equipment.
or some CEO going Chainsaws are not loud. some one whit a chainsaw shows up revs it and puts it into idle and CEO goes Never mind its Loud as heck.
My guess was Penetration Testing. Not quite there, but close.
CAPTION ISSUE: There's a serious accessibility problem with the captions on these videos. There's a white on black, a light cyan on black, yellow on black, and … very dark green on black which is so low contrast that it's kind of unusable. Sometimes there have been a light green and a very dark green. The only control I have over this (per video) is to force them all white. If there's a way to better control the caption colors on the editing end vs. just saying "use another color" … the dark green isn't working well and I don't know how to fix it so it would.
Thanks for reaching out. That "very dark green" is actually quite a light green that's so bright that it's close to yellow. So I don't know if that's just your personal perception or you have a monitor or graphic card setting that has your green channel turned down extremely low? -- David
I'm surprised no one suggested using the chainsaw a musical instrument
He's a chainsaw salesman.
If you work hard, you can buy yourself a chainsaw as nice as this one.
Also, I think about Ice sculptures when I think about chainsaws.
Oh no, the CEO has a chainsaw fetish…
Thought it would be something like a guy pitching a chainsaw on sharktank and being nervous about being denied.
For some reason, my mind went to Quark and Garak in DS9. But Garak never used a chainsaw.
He needed to prove to the 1700’s Time Travelers club that he was a legitimate member to received subsidies.
Was it Bruce Campbell doing a speaking gig
They were testing that safety feature that stops the chainsaw instantly if it touches skin, and they were able to demonstrate it without actually using the chainsaw by showing the mechanism triggering when it touched someone's skin.
That's what I thought! Definitely the loophole you'd expect "unused" to be implying.
At first, I thought about the chainsaw as a tool to help with childbirth (look it up!). I'm not scared but I want to save lives.
2:18 Don't look up what the chainsaw was originally invented for.
It was actually a girl, applying for a job which required the ability to juggle chainsaws. So she did - they weren't running, but.
Ummm...
It's one thing to test and verify your backups work, but it's an entirely different thing to do it by forcing the issue and requiring them to work that very instant, not to mention the property damage involved... D:
I dont think they were actually gonna do it, Im pretty sure it was a setup to test the HEAD IT's reaction, and if he would be nervous or not.
not really different
@@darekpower any head of IT would be nervous in that situation and the consultant is an absolute idiot - the only thing the consultant proved there was that they have an ego problem and love pointless grandstanding over gotcha questions
just a salesman showing his latest model to a gardening supply chain would have been too obvious
The problem with the world is that eventually Lateral will run out of unused questions
Was guessing that the guy wanted to sell a patent related to chainsaws.
Play DOOM on the job.
*imagines Decino seeing the chainsaw and acting like it's not there*
@@KernelLeakAnd of course, he can (and does) just leave.
Maybe just unplug the back up servers?!? 🤔 Save yourself a consultant fee.
Plus, using a chainsaw would likely cause live cables and internal consonants to short-circuit, possibly causing damage to files you wouldn't get with just power cut.
And I thought it was Eminem!
Someone's been watching Die Hard
Too soon to talk about chainsaws with the Sycamore Gap news >_
The IT guy stopped him because he hadn't raised a change request.
At the very least they should unplug it first
+
Consultant: "What would happen if I went into the datacenter and..."
IT guy: "...the fire suppression system would then activate and you would asphyxiate..."
Malcolm solves his problems with a chainsaw/
Malcolm solves his problems with a chainsaw/
Malcolm solves his problems with a chainsaw/
And he never has the same problem twice!
My brain went it’s an emotional support chainsaw to help them get through an interview/presentation lol
Watching this clip made it dawn on me that I've never used a chainsaw, and now I want to. Using a chainsaw seems fun, chainsaws are cool.
I was thinking it was a lawyer for someone who got hurt using the chainsaw, he got it out to show a design fault that the company should have spotted so they settled.
My god. I've never been in love with someone as much as I am now. I know we'll never meet, but all my loving and all the strength for the future Toby.
I think this was a penetration test. (Also, no comments on my screen yet)
That is this person showed that the safety screening did not work.
Not quite right there, but reasonably close.
New on Netflix this fall, King Soloman, CEO.