Self-Directed Research
Self-Directed Research
  • Видео 16
  • Просмотров 10 573
Compile Time Crimes
*We shouldn't have to, but it turns out we can*
James explains how to combine macros and const-fns to work around limitations of what is possible at compile time, and how to do extremely wasteful calculations at compile time to deduplicate lists of things to make embedded systems go brrr
Visit sdr-podcast.com/episodes/compile-time-crimes to see the show notes and transcript!
Sponsor: CodeCrafters is a service for learning programming skills by doing. Visit our referral link at
app.codecrafters.io/join?via=sdr-pod to start your free trial. If you upgrade, you'll get a discount and a portion of the sale will support this podcast.
## [00:00:00] Intro
## [00:01:35] Postcard RPC and needed trickery
#...
Просмотров: 2 304

Видео

Target triples - lies, damned lies
Просмотров 53919 часов назад
The Wonderful Story of Target Triples and Six More - Visit sdr-podcast.com/episodes/target-triples to see the show notes and transcript! - CodeCrafters is a service for learning programming skills by doing. Visit our referral link at app.codecrafters.io/join?via=sdr-pod to start your free trial. If you upgrade, you'll get a discount and a portion of the sale will support this podcast. [00:00:00...
What good is partial understanding?
Просмотров 46414 дней назад
*Self-describing formats, but at what cost?* An exploration of self-describing vs non-self-describing formats, and how it changes the shape of your programs more than you might think Visit sdr-podcast.com/episodes/partial-understanding to see the show notes and transcript! Descript is the fully featured, end-to-end video editor that you already know how to use. Check out our referral link (get....
Things you might not have known about clipboards (and some things I still don't understand)
Просмотров 55921 день назад
Amos and James reminisce about how weird clipboards have always been. Or is it pasteboards? Or buffers? Oh boy. Visit sdr-podcast.com/episodes/clipboards/ to see the show notes and transcript! CodeCrafters is a service for learning programming skills by _doing_. Visit our referral link (app.codecrafters.io/join?via=sdr-pod) to start your free trial. If you decide to upgrade, you'll get a discou...
Async Allocators
Просмотров 4,6 тыс.28 дней назад
A deep dive into the potential benefits, and awkward drawbacks, by making all allocations async and fallible Visit sdr-podcast.com/episodes/async-allocators/sdr-podcast.com/episodes/async-allocators/ to see the show notes and transcript! CodeCrafters is a service for learning programming skills by _doing_. Visit our referral link at app.codecrafters.io/join?via=sdr-pod CodeCrafters to start you...
Frame Synchronization
Просмотров 416Месяц назад
An overview of how devices decide how to split streams of bits and bytes into frames, and the things that can go wrong. Visit sdr-podcast.com/episodes/frame-sync/ to see the show notes and transcript! The Self-Directed Research podcast is made possible by our sponsors. We offer 30 second host-read ads at the end of every episode. Not sure how to get your message out, or what to say? Let us help...
Fixing build times with rubicon
Просмотров 322Месяц назад
Amos presents rubicon, which through terrible dynamic linking crimes, brought joy again into developing their website Visit sdr-podcast.com/episodes/dynamic-linking/ to see the show notes and transcript! The Self-Directed Research podcast is made possible by our sponsors. We offer 30 second host-read ads at the end of every episode. Not sure how to get your message out, or what to say? Let us h...
What Are You Syncing About?
Просмотров 492Месяц назад
An introduction to how the 'plumbing' of async tasks and drivers wait, sleep, and are notified for efficient cooperative action; and a glowing overview of the maitake-sync crate's main primitives Visit sdr-podcast.com/episodes/syncing/ to see the show notes and transcript! Today's episode is sponsored by the postcard crate (crates.io/crates/postcard), which is looking for sponsors for the 2.0 r...
You might not need Arc
Просмотров 207Месяц назад
How global is your context? And does it really need atomic reference counting? Visit ladybird.org (ladybird.org) for more information and to join the mailing list. Check out sdr-podcast.com/episodes/arc/ for more information on this episode (like show notes, transcript, more information on the sponsor) *Show Notes* - American Psycho business card scene (ruclips.net/video/T4UQf7zb3Pk/видео.html)...
Talking to Microcontrollers with Postcard-RPC
Просмотров 1232 месяца назад
A conceptual introduction to structured communication protocols, and the design decisions behind the postcard-rpc crate Visit ladybird.org (ladybird.org) for more information and to join the mailing list. Check out sdr-podcast.com/episodes/postcard-rpc/ for more information on this episode (like show notes, transcript, more information on the sponsor) [00:00:00] Start [00:00:56] You can get a l...
Thread-locals galore
Просмотров 1452 месяца назад
An overview of Thread Local Variables, and the challenges they pose for experimental dynamic linking support. Visit ladybird.org (ladybird.org) for more information and to join the mailing list. Check out sdr-podcast.com/episodes/thread-locals/ for more information on this episode (like show notes, transcript, more information on the sponsor) [00:00:00] Start [00:00:43] Pre-presentation chat [0...
Direct Memory Access for the Uninitiated
Просмотров 1522 месяца назад
An introduction to DMA, including what it is commonly used for, and a model of how to think about what the hardware does. You can read the Inside Rust Survey Announcement (blog.rust-lang.org/inside-rust/2024/08/22/embedded-wg-micro-survey.html) for information about the Embedded Rust Community Micro Survey, or you can take the survey now by clicking here (www.surveyhero.com/c/uenp3ydt). Check o...
How Usable is the Rust Cranelift Backend Today
Просмотров 1672 месяца назад
An experience report of attempting and MOSTLY succeeding in using the Cranelift backend for real MacOS applications Thank you to fasterthanlime for sponsoring this episode. Check out fasterthanli.me/ to see how you can support them! Check out sdr-podcast.com/episodes/cranelift/ for more information on this episode (like show notes, transcript, more information on the sponsor) [00:00:00] Start [...
BBQueue - Going Just Far Enough with Generics
Просмотров 1193 месяца назад
A dive into abstracting unusual behavior differences using generics to manage different storage and async usage styles Thank you to OneVariable (onevariable.com/) for sponsoring this episode - Get in touch by emailing contact@onevariable.com. Check out sdr-podcast.com/episodes/bbqueue/ for more information on this episode (like show notes, transcript, more information on the sponsor) [00:00:00]...
I Was Wrong About Rust Build Times
Просмотров 1653 месяца назад
An update to previous research about speeding build times, informed by unexpected increased cost of maintenance Thank you to Tweede golf (tweedegolf.nl/en) for sponsoring this episode! Check out sdr-podcast.com/episodes/wrong-build-times/ for more information on this episode (like show notes, transcript, more information on the sponsor) [00:00:00] Start [00:00:39] Slow Rust build? I was wrong [...
Trailer
Просмотров 823 месяца назад
Trailer

Комментарии

  • @SoapSoapCrayon
    @SoapSoapCrayon 6 часов назад

    23:59 this is exactly what happened to me when i started trying to use compile time rust. I really like the language, i like a lot of the features, and I think learning it has made be better, but I switched back to C++ for my day-to-day because of garbage like this. It felt like the Rust STL and compile time stuff is extremely not-fit for production quite yet, and I'll be following the changes, but I'm still not switching over to it for large projects just yet.

  • @rez188
    @rez188 10 часов назад

    Important clarification: zig currently does not allow allocations in comptime

    • @sdr_pod
      @sdr_pod 4 часа назад

      Oh, good to know! Neither of us are Zig programmers, so we were definitely guessing.

  • @MarekKnapek
    @MarekKnapek 11 часов назад

    Why using the O(n^2) loop if you could use sort instead. Implementing merge sort is easy. After you sorted your array, it is easy to extract unique values into result array. Easy in C++14 constexpr.

  • @sebschrader
    @sebschrader 18 часов назад

    What about the dead simple approach Ethernet takes: 8b10 or 64b66 for newest faster variants

  • @KabukiStarship
    @KabukiStarship 22 часа назад

    Interesting. I made the Serial Chinese Room, Interproces, and Telemetry Script, AKA Script2 and the SCRIPT Protocol for RPC, which is hybrid byte encoded and human readable based on contiguous data types.

  • @beyondcatastrophe_
    @beyondcatastrophe_ День назад

    16:25 I know this is not very rust-like, but do you even need the maybe-uninit? Just initialize everything with zero, you already unwrap everything that is below the current output index, so you don't even use the fact that it's an Option (you also return the number of `Some` values, so it doesn't even help on the consumer side)

    • @sdr_pod
      @sdr_pod 19 часов назад

      For this example, no! For the slightly more complex data types in the real code (or for the general case), it would be nice to have!

    • @RichardPringle9
      @RichardPringle9 17 часов назад

      ​@@sdr_pod can we get a link to the real code put in the show notes?

    • @sdr_pod
      @sdr_pod 13 часов назад

      Sure! We'll add it there soon. For now, this is most of the code that inspired this episode: github.com/jamesmunns/postcard-rpc/blob/main/source/postcard-rpc/src/uniques.rs

  • @Bobbias
    @Bobbias День назад

    "It's O(n^2) but that's the compiler's problem" its not even a problem if your n is small anyway.

  • @duvallj
    @duvallj День назад

    20:32 oh no typo on the slides, second loop needs an increment... very good episode tho! I'm in the midst of some compile-time crimes myself, w/ a proc_macro tho, neat to see const fn crimes too :)

  • @ChlorieHCl
    @ChlorieHCl День назад

    Wow this is just like C++ constexpr but more painful. I thought C++ is already too restrictive for when you use vectors in constexpr code but not being able to even use for loops seems annoying as hell to me. Props to you for getting it working.

    • @Jplaysterraria
      @Jplaysterraria День назад

      It is helpfully getting better

    • @oserodal2702
      @oserodal2702 День назад

      C++ 🤮

    • @stefanalecu9532
      @stefanalecu9532 День назад

      ​@@oserodal2702 such insightful critique, you clown

    • @oserodal2702
      @oserodal2702 День назад

      @@stefanalecu9532 C++ 🥰

    • @KabukiStarship
      @KabukiStarship 22 часа назад

      I don't have this problem. For my C++ API Script2 I replaced the C++ std lib and unicode engine with modern data driven design and I found that it was easiest for me to remember and avoid abstract argument collisions if I prefix the function with T if it's a template, and C if it's constexpr. constexpr actually works on non-const data too, it's an optional compile implementation. Search for constexpr in Script2 and you'll see where it comes in useful. For my API I don't define the print code until the top layer using operator overloads, which allows me to print to any destination like ring buffers, blocks of RAM, streams, etc. It might not be the nicest to look at but it doesn't take much thought.

  • @humanbeing2730
    @humanbeing2730 6 дней назад

    I dont think most people in my life would und erstand why someone would watch this for and reason but I enjoeyed it nonetheless :)

  • @alextalker5979
    @alextalker5979 7 дней назад

    I'd say, while having async allocator for general purpose might be questionable(since if you plan to schedule other work.while waiting for the allocation, the work might need an allocation too and it all turns onto itself perpetually), you can have a niche applIcation where you want to use Huge Pages but say implement your own allocation on top of them with whatever guarantees yoh like, since bigger page means lesser virtual page table(trivially speaking) and while Linux kernel for example can us free huge pages for general purpose also, if you really want you can and then the global allocator can still do most of the general purpose lifting while you have a clear interface to cooperatively manage "the new thing"

    • @alextalker5979
      @alextalker5979 7 дней назад

      The same, more for kernel level, actially goes for stuff like CMA(continious memory allocation) reserved of predetermined size of memory at the system startup, which you could request as a big chunk at once and then cooperatively navigate arounds its usage for the data structures you require, as well as care for address alignmemt & all

  • @capability-snob
    @capability-snob 8 дней назад

    The reason aaui for the file uri is the os-x capability sandbox. Sandboxed applications can't just enumerate the filesystem, instead, to gain access to a file at the user's request they need to perform an NSLookup on the request. This prevents applications from gaining access to files not explicitly shared with the app by the user.

  • @capability-snob
    @capability-snob 8 дней назад

    This was super fun! On FP registers: besides the fact that these are sometimes different sizes to gp registers, another reason they are often separate is that adding additional ports to a register file is very costly in space. There's additional circuitry per-register-per-port. Add to that, many implementations duplicate register files to get more fast read ports, but you still need to write to all of the register files whenever you update. Separating out the FP path, then, makes sense for FP heavy workloads. Also, I think POWER predates PowerPC, but I could be misremembering. The thinking was that when IBM, Apple, and Motarola came together to design the RISC they could all standardise on, they ended up starting from IBMs ISA and pruning out COBOL-specific instructions.

  • @dunkyl
    @dunkyl 8 дней назад

    This episode had all so many little details that give great perspective on what I was already familiar with. Thanks!

  • @pinealservo
    @pinealservo 8 дней назад

    There are a bunch of Power architecture CPU cores with a variable length encoding extension (VLE) in the automotive embedded space. These are like the thumb-only ARM cores, except based on PPC rather than Arm. When I started using them, you couldn't even get an upstream GCC release that supported them. There's also an arch named Coldfire which is the name under which the 68k arch lives on.

  • @Dygear
    @Dygear 8 дней назад

    6:18 - Saying Siri turned on all of the lights in my house for some reason. LOL. Thanks Amos!

  • @IgnoreMyChan
    @IgnoreMyChan 8 дней назад

    That's quite a lot of advertisement in the first minutes for such a small channel. 👋

  • @eduardgi7780
    @eduardgi7780 12 дней назад

    Seems brilliant

  • @izd4
    @izd4 19 дней назад

    eliza mentioned, hell yeah

  • @Cyclically
    @Cyclically 20 дней назад

    Only 280 subscribers? This guy's underrated, and I bet he's gonna blow up in popularity in a few months.

  • @remrevo3944
    @remrevo3944 21 день назад

    I don't know how it works on other platforms, (I imagine it is similar though), but at least on wayland, clipboards are implemented as IPC. For example if you run `wl-copy`, it actually forks into the background and *hosts* the specified data under a mime type. (Which is per default determined by xdg-mime.) The `wl-copy` source code is actually very readable and is great for figuring out such details. So any export to a specific mime type can be done at the time of the request of the actual content, instead of at time out the copy.

  • @totto164
    @totto164 22 дня назад

    Recently I discovered how the clipboard works on gnome, no matter if you run wayland or x11. It is really similar to Mac OS It only has those two "clipboards", clipboard and primary, since wayland copies that from x11 and gnome just uses that same format. But it also has different MIME types, it doesn't have the ability to store multiple items, but some mime types like "text/uri-list" just say that URIs are seperated by " ", so in the end you have all files you e.g. copy stored in that format. It also behaves like Mac OS, if you e.g. copy an image, it has a "image/png" mime type, and if it has an associates name also the "text/plain" of the name. the URIs are actual files in case of copied files. So after I heard that what you said about Mac OS, I am not sure if mac invented that, or linux / gnome 🤔 P.S: I use linux since 5 years now and never encontered any clipboard bug, the clipboard is just awesome and if you are a programmer or copy / paste often, you should get a application that serves as a clipboard history, windows has thateveb built in.

  • @Dygear
    @Dygear 22 дня назад

    Surely, SURELY someone you guys know works at apple and is close to or on the Safari team. I used to be able to just twitter DM a few people, but they all left twitter.

  • @urban6989
    @urban6989 22 дня назад

    each episode brings something interesting! 😉

  • @mrpocock
    @mrpocock 26 дней назад

    So using a u8 pointer or slice has always confused me a bit. You can only free the exact u8 pointer that was allocated, right? Not part of it? So should it not be something like Allocation(NonNull([u8]))?

    • @elijahshadbolt7334
      @elijahshadbolt7334 24 дня назад

      Maybe it's for implementations that want a safe free-after-free?

  • @Qallyx
    @Qallyx 26 дней назад

    I read allocators as alligators and was VERY curious

  • @VACatholic
    @VACatholic 26 дней назад

    Solving the wrong problem most likely. Would be better if the code could be restructured such that arena allocators could be used and freed all at once, rather than worrying about the lifecycle of every object individually.

  • @anon_y_mousse
    @anon_y_mousse 27 дней назад

    My unpopular opinion is that we shouldn't have async anything. Just use a proper thread and stop dealing with ambiguity. It would also eliminate the viral nature of async functions. This is really just another N-cost abstraction where you have to know the internals of how it works to know the actual cost.

    • @SimonBuchanNz
      @SimonBuchanNz 24 дня назад

      Async is literally just the compiler writing a state machine for you. If that is something you need, then having to do it by hand is *normally* way more confusing and annoying. Reification in general is the best way to deal with confusing concepts: push the complexity onto the type system so at least if you have the wrong idea of what's going on you find out about it. Async language support is a pretty good option for doing that for parallelism, in comparison to other options like, say, observables.

    • @anon_y_mousse
      @anon_y_mousse 23 дня назад

      @@SimonBuchanNz The implication inherent in what you're saying is both insulting and wrong. I'll forgive your word choice here as you being ignorant to its etymology and not that you're evil, but your attempt to describe how the mechanism works is factually incorrect. Saving state between calls doesn't make a state machine and they're not equivalent just because they both use the word state. The whole point of asynchronous functions was to act as coroutines. If you'd ever manually implemented your own coroutines, you'd know that you don't even need threading to implement them, which is sadly how modern implementations handle them, with threads.

    • @SimonBuchanNz
      @SimonBuchanNz 23 дня назад

      @@anon_y_mousse firstly, that's really annoying and dumb to call a counterargument "wrong and insulting". You don't have to say I'm wrong if you're arguing, that's implicit in the arguing, so you're just pointlessly being a jerk; and it's very silly to feel insulted that somebody dares to disagree with you. Regarding state machines, the way every language lowers async involves introducing a state variable with a value for each await point, and the continuation will transition to a new state, this is consistently referred to as a state machine in implementation literature. And if you do manually implement an async function, you do end up building an essentially equivalent implementation. Literally the only practical difference to the traditional definitions from CompSci courses is those normally have explicitly enumerated concrete inputs that transitions are defined in terms of; in the real world it's quite rare for things explicitly named state machines implementations to bother to separate collecting the input from selecting a transition. Perhaps you shouldn't be assuming the other person knows less than you, and back up your arguments with actual reasons *why* it's wrong rather than simply claiming you know better. And yes, you don't need to use threads for async, and most languages and runtimes will anyway (notably, *not* with what's possibly the most common usage in JavaScript!). There's lots of reasons, both more obvious (it's faster, spawn ergonomics are far nicer) and not as obvious (async FS is garbage on many platforms, so you need a blocking thread anyway) for that, but I don't really know why you're bringing this up as a reason to not use async when your best alternative is to spin up threads yourself. Now you both have handling real concurrency *and* annoying expensive thread management to deal with and the best fix for the latter is to just use a thread pool and dispatch jobs to it and... now you have the main runtime cost and blocking hazards of an async runtime but with worse ergonomics. There's a *reason* things are the way they are. If you want to use threads, sure, go nuts. But you sure are nuts if you think that lets you "stop dealing with ambiguity", that's just an inherent property of both parallelism and concurrency.

    • @funkdefied1
      @funkdefied1 23 дня назад

      Async isn’t about threading. Single-threaded, IO-limited processes benefit immensely from async. How would you use a “proper thread” to make slow, concurrent network calls?

    • @anon_y_mousse
      @anon_y_mousse 23 дня назад

      @@funkdefied1 It's implemented with threads in the backend. Learn how things work then get back to me.

  • @slava6105
    @slava6105 27 дней назад

    9:15 And now i want to allocate SIZE_MAX (or rust equivalent) bytes of contiguous memory.

  • @ojasmishra3487
    @ojasmishra3487 28 дней назад

    allocations need to be atomic right. So if async alloc is called twice you might have cases where one of the allocations gets invalidated. eg. freeBlock->next gets set twice. If you're using mutexes for async, might as well block the whole time which is the current interface

  • @foxwhite25
    @foxwhite25 28 дней назад

    How are you allocating a future when you get a allocation error that means oom.

  • @---..
    @---.. 28 дней назад

    If you are out of memory, then the executor is likely unable to allocate the memory needed to track the future returned from the async allocator. That kind of thing is the big challenge with failable allocation: the fail code path needs to be allocation free and (like with panic mentioned in the video) we don't have a good way to ensure a given code path is allocation free in Rust.

    • @sdr_pod
      @sdr_pod 28 дней назад

      At least in Rust, futures don't necessarily HAVE to allocate, and in the experimental allocator we were discussing, the allocation future doesn't. The future is stored in the task (so you would have allocated the task when you spawned it), meaning that the allocation path has no allocations of its own.

    • @Turalcar
      @Turalcar 27 дней назад

      > we don't have a good way to ensure a given code path is allocation free in Rust. I encountered this when I needed to ensure a code path is async-signal-safe :). Almost all wakers potentially allocate.

    • @Erhannis
      @Erhannis 26 дней назад

      Perhaps you could preallocate a small pool you maintain? And then if you fail to allocate another spare the next allocation fails.

  • @switchblade6226
    @switchblade6226 28 дней назад

    My personal opinion is that async alloc and/or drop should be an optional library thing rather than the default behavior of the language. Async drop for example would complicate RAII a lot since that would mean that now the lifetime of objects must be async, and since the async state machines arent strictly guaranteed to ever complete, any time you drop something you have an issue of "destructor running somewhere on an undetermined scheduler at an undetermined point in time". Which kinda defeats the point of RAII that says that objects are dropped when they exit scope. It would also mean that you have to implicitly schedule a drop, otherwise you cant drop things once they exit a scope. And for a zero-overhead language where scheduling is an explicit action its a definite no-no.

    • @MatthijsvanDuin
      @MatthijsvanDuin 25 дней назад

      async drop gets implicitly awaited by the implicit drop at end of scope, so RAII is actually no problem at all

    • @MatthijsvanDuin
      @MatthijsvanDuin 25 дней назад

      Also just to clarify, nobody is proposing changing any default behaviour. Async drop is something a type has to opt into by implementing AsyncDrop, and it actually _extends_ the ability to use RAII to cases where the cleanup needs to await.

  • @leonss2356
    @leonss2356 28 дней назад

    not async, but Zig's allocators are fallible

    • @theevilcottonball
      @theevilcottonball 28 дней назад

      But what if overcommit is turned on. Also what is the program really supposed to do on OOM? If I would write an OOM safe program it would be one that allocates everything upfront so it cannot fail during execution, one could even punt that job to the loader and allocate out of a static char array (with strict aliasing turned off),

    • @leonss2356
      @leonss2356 28 дней назад

      @@theevilcottonball i'm kinda new to this, but i'd guess the following: if overcommit is on, you're out of luck. as for what to do on OOM, it'd depend on the program and how critical the allocation is, most programs probably can't do much with it, but, for example, it could enable implementing allocation retries. about allocating everything upfront, if your program allows for that then sure, not usually the case though.

  • @Jesus228FBI
    @Jesus228FBI 28 дней назад

    17:38 basically, C/C++ programming

  • @VixieTSQ
    @VixieTSQ 29 дней назад

    this is a cool video

  • @Dygear
    @Dygear Месяц назад

    Ah, ETX my old friend. Never heard of COBS tho, and that's pretty awesome!

  • @dyniecc
    @dyniecc Месяц назад

    13:46 Polyglot files?

    • @sdr_pod
      @sdr_pod Месяц назад

      That's the term we were looking for, thanks!

  • @duvallj
    @duvallj Месяц назад

    I liked the visual gag :)

  • @hatchibomb
    @hatchibomb Месяц назад

    Don't know much about rust but this was very relaxing, thanks!

    • @sdr_pod
      @sdr_pod Месяц назад

      Glad you enjoyed! We have new episodes every week :)