Oxide and Friends 5/27/2024 -- Rebooting a datacenter: A decade later

Back in May 2014 Joyent accidentally rebooted an entire datacenter (not just the handful of node as intended!). That incident--traumatic was it was--informed many aspects of the Oxide product. Bryan and Adam were joined by members of that former Joyent team to discuss, commiserate, and--perhaps--get some things off their chests.
Context: mastodon.socia...
Notes: github.com/oxi...

[@AntranigVartanian]

Funny thing is, I missed the live stream, but while the live stream was happening I was watching Bryan's talk about this during one of the conferences :D

[@silverquick32]

"at risk of breaking tradition, do you want to say what it is we're talking about?" 😂😂

[@JSDudeca]

I only have written a bespoke opts parser in powershell because the OOTB one is awful. Eg it cannot be coerced to using - - params.

[@jammehcow]

I think you guys meant "2014" instead of "2024" in the description - unless they decided to celebrate the 10th anniversary with a live recreation lol

[@ryancnelson]

i'm really bummed I missed this. The podcast, i mean. I wish I'd missed the real event.

[@RossReedstrom]

The discussion about the fat finger total outage and total restart reminds me of Practical Engineering's (Grady Hillhouse) recent video about how hard it is to do a black start of a public power grid.

[@Dygear]

Welcome everyone to the Roast of Bryan Cantrill.

[@charlotteathena]

thank you for making all these mistakes for us

[@Don__]

Everyone saying "I also almost made this mistake" screams normalization of deviance.

[@Don__]

@@lale5767 social. I'm not familiar with the statistical meaning of the term. But I also never learned statistics in English.