Jonathan Blow on Drivers

Поделиться
HTML-код
  • Опубликовано: 28 авг 2024
  • The operating system comments:
    • Jonathan Blow on how a...
    If you have questions, you can come to one of Jon's streams:
    / j_blow

Комментарии • 66

  • @davidjohnston4240
    @davidjohnston4240 2 года назад +52

    This is a real problem with Random Number Generators. We put the hardware in the chip to provide fast, secure random numbers available through the instruction set - one instruction the same on all OSs. Then the OS wants to be arbiter of random numbers and puts in a truly crappy RNG layer of software in the kernel that doesn't meet international standards and the security is questionable and the 'secure' random numbers have to pass through drivers, file system, kernel and ioctls to get to your running program, rather than being delivered by the hardware directly into a register in your running program.

    • @erikitter6773
      @erikitter6773 2 года назад +4

      It might be a bit of a problem if you put it into hardware and it is demostrated that it has weaknesses that can be exploited -- ontop of making it practically impossible to check that a solution in hardware is secure (which you certainly can do for a few critical lines of code only making very secure assumptions about what is below [as the instructions have to work for other stuff too, while the special highly security critical instruction does not have to - plus additional problems, a bit of search for the debate about it will reveal a true shit show. But it is useful for games and that is where we are after all, right?]).

    • @davidjohnston4240
      @davidjohnston4240 2 года назад +18

      @@erikitter6773 That's why we put a whole boatload of effort into making the HW sidechannel, fault injection and cryptographically secure and we use formal methods and extensive pre and post silicon testing to make sure. After over a decade of doing this for the same RNG in out products we can be pretty confident that it's ok.

    • @franciscofarias6385
      @franciscofarias6385 10 месяцев назад

      @@davidjohnston4240 Also I think we have every reason to trust the HW industry more nowadays lol

  • @Liam-pf7ih
    @Liam-pf7ih 2 года назад +38

    people clearly don't understand that if they personally don't want to implement what the driver implements, they can borrow someone else's code that does it in the form of a lib or header or anything, having drivers in the way just means that no one gets to implement what the driver implements even when they want to.

    • @jaysistar2711
      @jaysistar2711 2 года назад +6

      Exactly. Even the file system could be implemented as block level storage, have all of the file abstraction function in process. Isolation is a great thing. Containers come close, but the OS still implements the file system and network stack.

  • @JediIsLegends
    @JediIsLegends 3 года назад +6

    Thanks for the follow up to the the OS discussion you added awhile back.

  • @SirReberal
    @SirReberal 3 года назад +29

    Just as I was trying to figure out what that lamp was, the camera focused on it.
    The magic of Jon Blow, man.

    • @RWW124
      @RWW124 2 года назад +3

      Its a Moroccan lamp

  • @jaysistar2711
    @jaysistar2711 2 года назад +11

    I don't know if anybody realizes it, but on x86 and x86_64 most of what we think about as an OS is in the hardware when in protected mode.

  • @fluffy_tail4365
    @fluffy_tail4365 2 года назад +17

    so what happens when two processes try to access the video card or any other hardware with state and one gets interrupted and the other starts DMAing over the half finished state of the other process?? Who mixes sounds from different processes???

    • @erikitter6773
      @erikitter6773 2 года назад

      What I hope you expect exactly. We don't have OS and drivers because having them is funny or the Illuminati. We have them because we need them.

    • @thatonegoblin7051
      @thatonegoblin7051 Год назад +1

      packet id's solve this?

    • @youtubeenjoyer1743
      @youtubeenjoyer1743 Год назад +9

      i do

  • @dorbie
    @dorbie 2 года назад +28

    Improved thinner drivers would be a better answer than no drivers (Vulkan etc. attempts to do this at least in part). No drivers means you must mandate a common hardware interface to every vendor's graphics card for example. The last time we saw that was VGA cards. You do actually need some level of abstraction from a common ABI to allow for innovation, progress, compatibility, interoperability, security etc. It's one thing to complain but solutions are harder. Details matter a lot, and glib assertions about how to solve hard problems might be deep in the Dunning-Kruger effect hole. This kind of thinking has lead to billion dollar failures like Larrabee (I remember game programming "experts" circle jerking over the access "to the metal" that massively parallel coding in shitty x86 assembly would bring). This no driver stuff is equally well informed. Add a GPU HAL and shader compiler to the TODO list... ⌛
    P.S. the biggest challenge and compatibility problem is proprietary bullshit from OS vendors who don't even make GPUs but insist upon how the interface to them should work in insipid attempts to manipulate the market. OpenGL, D3D, Metal, Vulkan etc. If I could do one useful thing it'd be mandating Vulkan everywhere. No drivers is not an option. However we can't even mandate a common ABI which is the real root of many problems. It might not solve all the problems but it'd be damned good start. Everything else would get a wrapper around Vulkan for legacy code and the short bus riders.

    • @miikavihersaari3104
      @miikavihersaari3104 Год назад +5

      There is no driver for the x86/64 ISA, and that seems to be kinda successful. Having an ISA doesn't take anything away from the people who want abstractions, because you can still have them. Even if all GPUs had a common, standardized ISA we would still have Vulkan, just that then anyone who wanted to speak directly to the hardware would actually have that option. In the current situation you can't. The first step toward this would be vendors releasing documentation for their HW, and I'm glad both AMD and NVidia have open sourced their Linux drivers.

    • @dorbie
      @dorbie Год назад +4

      @@miikavihersaari3104 There's more to a graphics driver than the ISA. There's already the SPIR, That's used for shaders, not the hardware interface. And folks aren't hand coding shaders directly in that. It's intent is portable compilation across hardware architectures to reduce runtime compilation. This isn't AMD & Intel with compatible ISA. It's NVIDIA , ATI (AMD), & others. The architecture of Vulkan is innately more powerful and lighter weight than previous interfaces but will do less validation so it's less forgiving if you do bad things.

    • @miikavihersaari3104
      @miikavihersaari3104 Год назад +1

      @@dorbie I'm not sure we understand each others' points. My point was that having an open and documented ISA (and of course video BIOS) wouldn't mean that we couldn't have APIs like Vulkan. The added benefit would be being able to directly command the hardware, but it wouldn't force anyone to do that. I wasn't arguing that GPUs have or necessarily should have a standardized ISA, simply that even if this were so, current rendering APIs wouldn't go anywhere. I'm coming from the angle that it should be possible to command the hardware directly if you wanted to do that, for example because you want to know and examine how your computer works. Right now you need to run elaborate experiments on these black boxes and do reverse-engineering, because these companies don't want you to do, well, forward-engineering. I'm not saying you have to agree with me, just that I'm not sure I understood what you meant.

    • @dorbie
      @dorbie Год назад

      @@miikavihersaari3104 no, the problem is you don’t understand how GPU hardware interfaces work and are arguing for ISA and video bios and I tried to accommodate the spirit of your post. If you are proposing solutions to a complex technical problem the details matter. Your remarks don’t even make sense if they were applicable. For example if you said ARM and Intel should have a common ISA you’d immediately be dismissed as a crank.

    • @miikavihersaari3104
      @miikavihersaari3104 Год назад

      @@dorbie Funny, I'm getting the same impression of you, that you don't know how GPU hardware interfaces work 🙂 Have you for example studied the linux amdgpu kernel module or read AMD's RDNA 2 ISA documentation? If you really understand how the interface works then tell me exactly and in detail how the following is done:
      - switching to the proper video mode and resolution
      - writing data into the GPU memory, containing vertices and a simple vert and fragment shader
      - sending the instructions to execute these shader programs in a pipeline that writes to the framebuffer
      - presenting the framebuffer onto the screen
      Also, if you bothered to read my post correctly, you would see I never argued for a common ISA for anything. Just documented and open ones. My intention wasn't to pick a fight with you. I wanted to understand what you meant.

  • @hcutter9802
    @hcutter9802 Год назад +13

    What I love about Blow is some things he says I %100 agree with, others I %100 disagree with (this driver rant has been done many times before, including by me, and it's wrong - drivers have to exist and they have to be kernel mediated, for reasons I can fill a couple of pages with). With very little 'maybe' room in the middle! :D If I was conducting a conference of "Let's fix all the SHIT in IT, NOW!" I'd totally have Blow at the table.

  • @WGlass-ws5pe
    @WGlass-ws5pe 5 месяцев назад +7

    "I don't know what pipewire is, but anytime something has a name like that, my reflex is that it's gonna be terrible"
    A man after my own heart 😂

  • @casey206969
    @casey206969 2 месяца назад +1

    Jon's right; people write some pretty nasty stuff with printf. There should be a driver that filters out harmful messages.

  • @CianMcsweeney
    @CianMcsweeney 6 месяцев назад +1

    For reference, the PipeWire software that someone mentioned as a positive has a 270,000 line count....

  • @AnonEMoose-mr8jm
    @AnonEMoose-mr8jm 6 месяцев назад

    I have a feeling that JBlow would like Plan9 and the Exokernel. If you ask me, that's where the sweet spot is. You get your exokernel and you can have your library operating systems. If we modified the exokernel to ONLY implement capabilities-based security and namespaces then you can safely do everything in userspace.

  • @HiddenUsename
    @HiddenUsename 7 месяцев назад +4

    So, how would a network card work without a driver? Say some bytes came down the wire. Whose are they? How would the correct process get them?

  • @FrankHarwald
    @FrankHarwald 2 месяца назад

    I never had problems with my audio system on linux - but that's because I never use pulseaudio. In fact the first thing I do when I install a need software which uses audio is I completely disable pulseaudio & just use kernel kernel audio (alsa) which has existed for over a decade & works without any problem. (theoretically you'll loose cross-unix portability because alsa is a linux-only thing & pulseaudio exists on bsd too but that's never a practical concern because every audio program I've ever seen also has support for bsd specific kernel audio too so you can just change it).

  • @KyleLuce
    @KyleLuce 3 года назад +25

    This is fine in a single process operating system or embedded system with singular focus/use case, but the operating system does quite a bit more with scheduling and I/O management. If all processes had unmitigated hardware access (DMA etc), there'd be quite a bit of conflicts for resources. Starvation and all sorts of state issues...
    This is not what we want in modern systems with many concurrent tasks. Unfortunately hardware has very specific nuances that don't have uniform interfaces. Reasons for this stem from multiple causes, including proprietary IP/patents. Spinning up of some type of mapping hardware would indeed be more expensive. Also kind of impossible as feature sets are hardly ever uniform across devices. You probably wouldn't want that though, as how would innovation happen?
    There's probably a better way than what we have for sure. There always is. But going the route of less abstraction, is not likely what the majority of people want.
    I feel the pain though. It certainly would be great if there were uniform interfaces.

    • @swapode
      @swapode 3 года назад +6

      I don't buy the innovation argument for a couple of pretty major reasons. Mainly that we there has been very little in terms of innovation in the PC space and the innovation we've got is generally pretty terrible designed, almost necessarily terribly underused by software (even games, despite all the effort put into them - not even touching the whole "buy more expensive hardware to run our ever more shitty software") and allows for incredibly shady business practices. On the flipside if manufacturers were forced to comply with well thought out open standards in which they have an advisory but not deciding role they'd still have plenty of room for innovation: Better manufacturing, energy efficiency and so on.

    • @KyleLuce
      @KyleLuce 3 года назад +8

      @@swapode how would you propose a uniform standard and still add features that are unique, while still following the standard? I think you may have missed my point. It's contradictory a bit.
      Say company Z invented feature Y... Without exposing feature Y in a "standard interface" they'd be limited in terms of usability for feature Y. Exposing feature Y in the "standard interface" and suddenly you have a feature gap until all hardware can get parity for that feature. Which may never happen.
      So option 1: is abstract the feature and use in most common cases without exposing anything new in the interface
      Option 2; expose the interface to the new feature Y allowing programmability, at the expense of breaking the standard interface.
      Option 3: ideal case... Everyone is able to both implement feature Y and update the "standard interface". (Or if not, the company implements a no op and updates to match the interface... less ideal and confusing).
      Assuming option 3 can happen in all cases is not likely
      Additionally, In this theoretical restrictive world: Without hardware Z having any path to unique features, what incentives do they have to create them? Just to keep parity with another company?
      This is the friction to innovation.

    • @swapode
      @swapode 3 года назад +1

      ​@@KyleLuce It's a complex topic and I certainly don't have all the answers.
      But those problems exist already anyway. The vast majority of features has to become some kind of defacto standard delivered by most if not all manufacturers before it can really take off.
      And for as to what incentive is there for innovation. The same as now: Sell new hardware.

    • @KyleLuce
      @KyleLuce 3 года назад +3

      @@swapode but drivers provide all the mappings to custom features and in some cases there are very specific exclusive things exposed based on the particular hardware you're using.
      I'd think "selling more hardware" than your competition would be a bit intractable when you're forced to have feature and interface parity.
      But honestly you're focusing on a very small section of my original comments. I'm mainly getting at the practicality of having drivers to map complex nuanced state in the hardware to a more standard interface. (Which he is against)
      I worked in hardware design and verification space earlier in my career and the spec sheets are enormous. Drivers provide sane mappings and hardware bug workarounds (yes these happen, quite a bit).
      The other big section was the impracticality of unmitigated low level access to hardware on a multi process operating system. (Which he is suggesting, to allow).
      So many conflicts.

    • @swapode
      @swapode 3 года назад +1

      ​@@KyleLuce Yes, I deliberately focused on the innovation part. I can't see the "just let's not have drivers" idea working in practice for a multitude of reasons. I just don't think that innovation is a good one.
      I said "selling new hardware" - which kinda requires new standards which kinda requires features which kinda have to come from the manufacturers. Selling more than the competition is done by delivering it better or cheaper (or marketing of course).

  • @isaac_shelton
    @isaac_shelton 6 месяцев назад +2

    John has some decent opinions about some things, but none of what he says here makes any sense if you actually understand what OSs/drivers do and why direct hardware access is usually not allowed

  • @mac2105
    @mac2105 Год назад +5

    What Jon wants is a Microkernel. Give GNU/Hurd a try.

    • @FrankHarwald
      @FrankHarwald 2 месяца назад

      Yes, he want's a microkernel, a well designed one, but no, unfortunately GNU/Hurd is badly designed microkernel.

  • @blipojones2114
    @blipojones2114 2 года назад +5

    "there need to be impetus for cleanup" - i think the companies making money off this lack of standardisation are hoping there will be no cleanup.
    Why do all johns talks, i think, end up very, i wont say anti-capitalist, but not realistic in a capitalistic environment.

    • @strelok5661
      @strelok5661 2 года назад

      he bullshits alot

    • @erikitter6773
      @erikitter6773 2 года назад

      The people screeming for cleanup also shitting on c++ standard processes all the time -- and in between telling you that you should not really need an OS at all or something. Yes we need standard options. But they will always be trailing developments slowly making sure the world still works pretty well after standardization. This does not solve anything floating around these silos here.

  • @declansnyder2281
    @declansnyder2281 3 месяца назад +3

    What should there be instead of drivers?
    Johnathan Blow: "some kind of protocol that is standardized where you just dma to the hardware and the hardware DMAs back"
    Does he not realize this is what a driver is?

  • @4.0.4
    @4.0.4 Год назад +3

    Some abstraction is good. Imagine how often a program would crash your machine.

    • @danight33
      @danight33 Год назад +1

      lmao, processes could write whatever they want to the network card, no moderation. Sounds awesome

  • @user-ov5nd1fb7s
    @user-ov5nd1fb7s Год назад +3

    Is there something that Jon likes? I've watched like a dozen videos and there isn't a single thing he said that he likes. If he thinks everything is screwed up in our industry, he is probably wrong.

    • @swifton
      @swifton  Год назад +2

      Nah, it's just more fun to make videos where he's dunking on things. He does praise things sometimes, usually select computer games, like Stephen's Sausage Roll. He also praised LLVM in one of the first programming language videos.

    • @user-ov5nd1fb7s
      @user-ov5nd1fb7s Год назад +5

      @@swifton Maybe that is true. But you have to admit it doesn't look good. What he is complaining about can't be fixed by him or anyone else. At least, most of it. He was also complaining about LLVM and all operating systems. And the reason he doesn't present an alternative to those is because he can't write one that would provide anything even remotely close to what we have. What he is really complaining about is that these tools are not targeting and optimizing for his use case but doing so for a wide variety of use cases. He can't make an OS that would run on every hardware and be small at the same time. He can't make a compiler backend that would produce quality code like LLVM and have it small at the same time, and not break everybody else's code.
      I have to admit that his rants are often funny. But it does get old and mostly illogical. Yes Jon, most software is bloated, we know that because we are not idiots. It is bloated, in most cases, because we need to support everybody's use case.

  • @wheezybackports6444
    @wheezybackports6444 2 года назад +4

    I've noticed John is getting at something similar to Plan 9, but worse in design. His concepts have already been done, but better. I'm not sure if he knows of Plan 9's existence.

  • @putinstea
    @putinstea 3 года назад

    Nice lamp

  • @NopeNopeNope9124
    @NopeNopeNope9124 4 месяца назад

    I would never drive a car

  • @rubberduck2078
    @rubberduck2078 3 года назад

    true

  • @boshi9
    @boshi9 2 года назад +31

    This doesn’t make any sense at all.

    • @dorktales254
      @dorktales254 2 года назад +7

      ""Communicate to hardware in a standard way", good luck with that

    • @rocapbg9518
      @rocapbg9518 2 года назад +5

      What he is saying isn't really complicated, maybe you don't really need that much experience to understand it, maybe you're a beginner? Idk

    • @erikitter6773
      @erikitter6773 2 года назад +6

      @@rocapbg9518 maybe someone is understanding that it does not make any sense at all (for general purpose computers).

    • @miikavihersaari3104
      @miikavihersaari3104 Год назад +1

      Makes a lot of sense. Besides, we're already talking to hardware in a standardized way. Between Vulkan and a desktop GPU there's very little driver, which suggests GPU hardware is converging. Having closed ISAs is the problem. Having fully documented ISAs, even if they were different for each vendor, would not cause harm to anyone. On the contrary, it would only give more options to several parties, including game companies and universities. I'm not sure I understand why some are so strongly against this.

    • @jazzfool
      @jazzfool Год назад +1

      @@miikavihersaari3104 No. Vulkan drivers are not as lean as you think. In fact, explicit graphics API drivers are still quite hefty, just not as hefty as, say, an OpenGL driver.

  • @dogphoe
    @dogphoe 4 месяца назад

    lol