came to this talk before I knew what tiger beetle was - since then I have watched almost all the tiger beetle videos and looked into the code. TigerBeetle is legit - your consensus algorithm is one of the most straightforward beautiful things, the two LSM tree's built in house plus all IO optimizations plus all the memory safety are so cool. View Time Stamped replication is amazing. State replication machines and protocol aware recovery - so cool. Also, love this talk and am coming back for a second time to watch it again. Thanks!
Hum I'm not at the level required to follow this talk properly. that said, I'm glad Zig conf tackle advance topics. It's motivating to slowly understanding more about system programming.
Thanks for the video. I've worked a bit with the network API of the Zig standard library and it was somewhat confusing to me. Ended-up using a very simple poll(2) syscall and call it a day. I know it's not AT ALL efficient, but at least it's available everywhere and is pretty simple. I really hope there will be a way, someday, to make efficient network applications without having to dig into very obscure, non-portable APIs. For now, the Zig standard networking API seems at an early stage.
I always find it interesting to see that people stop at Overlapped IO when talking about non-blocking IO on Windows. APIs like Registered IO (now available for well over 10 years) and IoRing (okay, this is something new right now) are available too and are very similar to io_uring which everyone seems to talk about...
AFAIK (correct me if i'm wrong here), windows RIO requires registered buffers which wouldn't fit zig's intrusive memory model + would hit the locked memory limit on linux for general purpose use. IoRing is getting there but currently only documents/supports reads and requires windows 11.
@@kprotty6828 Thanks for the reply. I mainly wanted to make the general observation that Registered IO and IoRing were not mentioned in the talk as it seems to be the case everywhere. These APIs always seem to be overlooked. And yes, Registered IO works with physically locked buffers and I only worked with it in specific contexts where this wasn't an issue (never tried to build a library on top of it). While writes are now supported by IoRing, they are still not documented in the Win32 API (took this info from the windows-internals blog).
Hey I’m new to concurrency and I’m trying to wrap my heads about all the concepts surrounding this topic. Do you have any good books or blogs or whatever so that I can understand it deeper ?
I would like to see a runtime with dynamic amount of io reactors. a single poll based event loop can stop to scale completely when many fd given. On the other hand thread per io reactors can be waste of resource as you don't always need io throughput.
I am not nearly as deep into zig as you guys but as someone who came from outside I have to say that asyncio seems to be a big mistake imho. Zig is supposed to be "simple" but even after that shirt period of evolvment time zig seems to be more and more cluttered with optional/secondary stuff.
Great question! We were aware of it, but our philosophy is to work on commodity hardware, targeting slow/small hardware as well, and being fast like that (e.g. thinking about disk/memory access patterns). With large enough disk blocks (64 KiB) and with io_uring, performance is almost also on par, from what we have read. TigerBeetle was designed just as io_uring was out, so we were also lucky that we had the benefit of being able to target it.
![I.N "HALLUCINATION" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/n5B5q1Hwt_U/mqdefault.jpg)
![[#37] μscope - Jim Calabro](/img/1.gif)
came to this talk before I knew what tiger beetle was - since then I have watched almost all the tiger beetle videos and looked into the code. TigerBeetle is legit - your consensus algorithm is one of the most straightforward beautiful things, the two LSM tree's built in house plus all IO optimizations plus all the memory safety are so cool. View Time Stamped replication is amazing. State replication machines and protocol aware recovery - so cool. Also, love this talk and am coming back for a second time to watch it again. Thanks!
Awesome talk! Thanks for putting it together, learned a lot :)
Oh nice another lecture by King! I love hearing him talk! He as a knack for doing complex material in a very airy funny way. I love it!
Hum I'm not at the level required to follow this talk properly.
that said, I'm glad Zig conf tackle advance topics. It's motivating to slowly understanding more about system programming.
Same boat but now I have some language to use for reading :D
@@sanador2826 Great mantality !
I'm back! Great talk !
What an information bomb! 🎁 A lot of things are starting to make sense now. Keep it up.
Great talk . Thank you 😀
Can't wait for this! Sounds like with this you would have amazing performance simply by embracing async in your pograms
Thanks! Very clear explanation.
Awesome talk King!
Yey, cool to see some zig talks again
New to zig! Learned a lot.
Lots of context and knowledge. Very interesting :)
Thanks for the video. I've worked a bit with the network API of the Zig standard library and it was somewhat confusing to me. Ended-up using a very simple poll(2) syscall and call it a day. I know it's not AT ALL efficient, but at least it's available everywhere and is pretty simple.
I really hope there will be a way, someday, to make efficient network applications without having to dig into very obscure, non-portable APIs. For now, the Zig standard networking API seems at an early stage.
I always find it interesting to see that people stop at Overlapped IO when talking about non-blocking IO on Windows. APIs like Registered IO (now available for well over 10 years) and IoRing (okay, this is something new right now) are available too and are very similar to io_uring which everyone seems to talk about...
AFAIK (correct me if i'm wrong here), windows RIO requires registered buffers which wouldn't fit zig's intrusive memory model + would hit the locked memory limit on linux for general purpose use. IoRing is getting there but currently only documents/supports reads and requires windows 11.
@@kprotty6828 Thanks for the reply. I mainly wanted to make the general observation that Registered IO and IoRing were not mentioned in the talk as it seems to be the case everywhere. These APIs always seem to be overlooked. And yes, Registered IO works with physically locked buffers and I only worked with it in specific contexts where this wasn't an issue (never tried to build a library on top of it). While writes are now supported by IoRing, they are still not documented in the Win32 API (took this info from the windows-internals blog).
"I own concurrency". Only 15 secs in, a sudden tsunami of respect thrusting my way.
Hey I’m new to concurrency and I’m trying to wrap my heads about all the concepts surrounding this topic. Do you have any good books or blogs or whatever so that I can understand it deeper ?
True that. Techempower benchies is what we use 😅
20:12 Jared Sumner just hates Windows confirmed
I would like to see a runtime with dynamic amount of io reactors. a single poll based event loop can stop to scale completely when many fd given. On the other hand thread per io reactors can be waste of resource as you don't always need io throughput.
gr8 talk
I am not nearly as deep into zig as you guys but as someone who came from outside I have to say that asyncio seems to be a big mistake imho. Zig is supposed to be "simple" but even after that shirt period of evolvment time zig seems to be more and more cluttered with optional/secondary stuff.
One question, why TigerBeetle didn't go straight with DPDK. Sacrificing cross platform, but kinda makes sense in your case, no?
Great question! We were aware of it, but our philosophy is to work on commodity hardware, targeting slow/small hardware as well, and being fast like that (e.g. thinking about disk/memory access patterns). With large enough disk blocks (64 KiB) and with io_uring, performance is almost also on par, from what we have read. TigerBeetle was designed just as io_uring was out, so we were also lucky that we had the benefit of being able to target it.
If you were hoping to learn how to write some specific Zig code, you won’t see it in this talk. Zero lines of ZIP Code are displayed.
Yes, zero lines of "ZIP Code" displayed, you are correct ..
great talk 🤍