Kernel Developer Tries To Solve Every Overflow Bug
HTML-код
- Опубликовано: 29 сен 2024
- Dealing with numbers in a computer isn't as simple as just add them together, eventually you hit the limit either on the bottom or top and this creates an overflow or underflow but this developer is trying to resolve every single bug caused by this.
==========Support The Channel==========
► Patreon: brodierobertso...
► Paypal: brodierobertso...
► Liberapay: brodierobertso...
► Amazon USA: brodierobertso...
==========Resources==========
Kernel Discussion: lore.kernel.or...
=========Video Platforms==========
🎥 Odysee: brodierobertso...
🎥 Podcast: techovertea.xy...
🎮 Gaming: brodierobertso...
==========Social Media==========
🎤 Discord: brodierobertso...
🐦 Twitter: brodierobertso...
🌐 Mastodon: brodierobertso...
🖥️ GitHub: brodierobertso...
==========Credits==========
🎨 Channel Art:
Profile Picture:
/ supercozman_draws
🎵 Ending music
Track: Debris & Jonth - Game Time [NCS Release]
Music provided by NoCopyrightSounds.
Watch: • Debris & Jonth - Game ...
Free Download / Stream: ncs.io/GameTime
DISCLOSURE: Wherever possible I use referral links, which means if you click one of the links in this video or description and make a purchase I may receive a small commission or other compensation. 
Наука
I don't know if I'm being trolled, why are people telling me that "underflow" isn't a term that people use, there is floating point underflow but that is not what I am talking about, I am discussing integer underflow, this is literally a term that is in use in the Common Weakness Enumeration CWE-191
Check the CWE listing, that is not the only correct description of the event
When someone uses the term underflow most people understand what they mean.
@@BrodieRobertson
In a CS course of my uni, my prof said integer wraparound is a sub category of integer overflow.
Given CWE says opposite, I believe the term is spread and used ambiguously.
Are you going to recycle/throw away that empty Dare bottle and clean your mug or not? It's been days (from our point of view) XD
It's almost like English is a weird language that adapts naturally
So, basically, Linus is saying "convince me", but in a way that makes most people back down and only the ones really prepared to do their homework have a chance.
I kinda got that impression, too. It's like, "I can't see this fixing things without it probably breaking these other likely more valid scenarios, prove to me this approach works without being a pain".
yes what he is saying is "see if you can invent a way to actually detect when wraparound _is part of a bug_ , with ZERO false positives. and then your warning message should dance around the term 'wraparound' without saying it to make sure you print the actual catalyst of the bug rather than lumping it together with wraparound" .
linus knows that's the root cause or core of the hard problem in this topic so he doesn't want to waste time talking about it until an impressive solution is created and presented
To be fair this is how changes to a project of this scale should be approached. You should prove that what you're proposing is actually beneficial/needed and doesn't cause new issues elsewhere.
Being responsible for the most important software project on the planet means the job comes with special requirements.
Linus is absolutely right that you can't put more load on someone who is already overburdened with the world on their shoulders. If they fail, the world breaks.
The tooling needs to step up and bear that load.
His point is overflow is predictable it's just that overflow is unexpected by the developer. Don't make it harder to develop for the kernel to cater for something that I don't have a problem with. At least that was my take on it.
If you tried to count every issue caused by an integer overflow or underflow, you'd probably get an integer overflow.
Theres so many changes going on in the kernel.
Alot of people are no longer associated w/ the components/areas that they once were, and many new things are being acknowledged and accomodated for... i think i mentioned theres over 300,000 lines of commits in 6.9.0, and youd be surprised how many brands, makes and models there are for things youve never even heard of and how many different people are responsible for it.
For a Linus rant this was pretty civilized. I mean he's never gone full Theo de Raadt on people, but when people have misunderstood that it was time to back down, do more hard analysis and come back with a fresh sane approach, Linus has been known to be borderline abusive.
Majority of people have thin skin, Linus is there to create a stable working Kernel. Damn your feelings!
If you wanted to count all of the issues caused by integer overflow, you'd cause integer overflow
For F's sake, just yesterday we were having a conversation with Daniel Stenberg about how Torvalds seemed to get better since he took a step back, and then the very next morning, I'm greeted with this. Sadge.
Toughen up princess, the Kernels correct functioning takes priority over all of our feelings. Especially yours.
@@sean7221 Thank you for your contribution, Sean. I'm sure that the kernel will work so much better thanks to the lead using personal attacks to defend his uncomfortable feelings about a technical proposal.
@@sean7221 Thank you, Sean, for your contribution. I'm certain that now that we are all safe knowing that the leading maintainer is still willing to respond with personal attacks to any technical proposal, regardless of how rigorous or substantiated it is, the kernel will run so much better, as long as his feefees are hurt enough.
My thoughts are over a long period of time more and more bits will be adapted to rust. It's not something that will happen quickly nor should it since stability is a prime concern, but eventually this sort of thing might get phased out. What your trying to do above is change aspects of the C language to make it more automated in terms of finding those types of bugs, but that means changing the language itself in some cases or at least using annotations.
Absolutely agree they are that common and need to be fixed.
I would wager a solid _half_ of all windows patches is actually just for "overflow bug that allows attacker to execute random code/read random parts of memory". It is _that_ common.
And I totally agree, there is no hope expecting the average programmer to "do better" without compiler aid. I like handling naked pointers as much as I like juggling live handgrenades, but C in part leaves you little options to avoid it. Every damn array is a pointer with no sanity checks, unless you force them onto it.
To top it off, they are working with C and on a Kernel level. So any runtime checks are just not feasible. Another reason this has to be compiler compiler support.
And with all that, the whitelist approach outlined is probably the best option. Enforcing checks at compile time. Phasing it in over years, with warnings first. Giving people ample time to migrate the code.
No idea why Linus had a rant here. This seems perfectly thought out.
You should add your own wraparound test that returns, by itself, if you are using wraparounds. Maybe we need a new type that supports wraparounds? Or a keyword or compiler comment that enables wraparound.
I've seen it before it was popular. I am happy now.
It's actually incredibly draining for the contributors to deal with Linus's rants like this. It's good that the discussion eventually got productive but the thick skin required initially (and initially each time a thread starts over many years) takes a toll on some very smart contributors.
How about an over-persnickety debug flag which may be used by those who wish to use it, but it doesn't bother anyone else because it is disabled by default. Linus may speak for some, even most kernel developers -- but certainly not all.
@@SeekingTheLoveThatGodMeans7648 If Linus wasnt the BDFL then you'd be right but he literally does speak for the entirety of the project sooooo
@@SeekingTheLoveThatGodMeans7648 It doesn't work like that. Linus's opinions determine what's acceptable in the vanilla kernel. Individual developers can enable extra lints, but they'll just get bombarded with noise from code they don't maintain. It ultimately isn't a technical problem, it's a political problem. We devs know how to turn on lints. But if the project leadership doesn't care about what those lints mean, and won't devote resources to fixing them, there's no point.
Microkernel? Micropython? Microcompiler? Unstoppable compiler warnings?
Just rewrite Linux in Free Pascal
in perl.
No, in COBOL
No, in holyc, yall are wholy unholy for even suggesting somethinf diffrent
@@grisu1934 There is no need to rewrite another operating in the divine HolyC programming language, for the perfect and only truly holy one already exists. I cannot believe that you would even suggest something so utterly stupid. May God have mercy on you, for I will not.
@@atemoc ah yes punish me papi >~
Just use floating point for everything.
The answer to this problem is simple, just use rust.
after 30 years of kernel development, Linus has almost always been right.
Nice Verisatium reference
C is the best option for many reasons, one of them is that it's honest and simple,
if someone feels like they can do a better job in their favorite, modern language, they should do it and create their own modern OS
after all, if linus could do it in this very obsolete C and old computers, and they are much smarter and modern than he was back then or even is today, it should be a breeze, since many claim that everything older than 5 years is useless and obsolete
C is the worst option, except for all the others. The trouble is that C just doesn't capture all the relevant information. Actually the Linux toolchain is not a pure C toolchain. They assume GCC, for example. They have meta-data encoded in comments to manage kernel resources like mutexes, for example. Linus already understand that integer overflow is a systemic problem. He just hasn't accepted that mitigation is necessary.
It's typical "rational Linus."
Forceful and adamant Linus is so much better than ad hominem Linus.
I'm not impressed by Linus here. This knee-jerk reaction to a reasonable, and well-written request is exactly why nothing has been done about a fundamental issue in the kernel. Unfortunately, Integer Overflow bugs are bit like sorting goods for recycling. The *best* time to sort your recycling is before you throw the refuse in the bin. The *best* time to fix your integer overflow bug is immediately after you wrote it, because toolchain found it. Linus is being dumb about this.
based torvalds strrikes again
SQLite team has a similar stance on automated checkers.
SQLite also happen to have a lead maintainer who "only" cares about speed, security and not breaking existing applications.
@@andersjjensen If they cared about security, they wouldn't write it in C.
@@brandonlewis2599 Rust/Zig didn't exist back when SQLite was made, if you wanted both security and speed, then you needed to use C or C++ with sanitizers and static analysis tools.
How easy would it be to convert from C to C++
I don't think that'd happen in the kernel with Linus given his views on C++
Basically a ground-up rewrite. You just wouldn't *do* things in the same way, if you were writing in C++. Look at Haiku (and BeOS) if you want to see what a C++ looks like.
7TH DAY ASKING FOR A REVIEW OF:
- YAZI (FILE MANAGER>RANGER,VIFM,NNN,LF)
- WEZTERM (TERMINAL)
This is what happens when you allow over-socialised, paranoid, safety fanatics into your project.
Linus is the reason the Linux kernel is so valuable.
Not sure what this is supposed to mean. If it's tongue-in-cheek, it's went over my head. If it's sincere, then I think maybe you need to get some fresh air, and maybe spend some time getting up-to-speed on computer security.
@@brandonlewis2599 a terse comment made on the internet is no way to express complex ideas.
I'm very aware of the dangers of buffer overruns. My point is adding more abstractions and complexity to the tools to overcome the problems caused by 'senior developers with 2 years experience' doesn't make better software. It makes worse programmers.
@@Toleich worse programmers how? do u think the way C does things is some universal truth that everyone should deal with?
@@Toleich Integer overflow isn't just a concern with indices into memory: it may also alter decisions made by the code in an unexpected way. There are integer overflow exploits that utilize this, getting control of a crucial if (...) by carefully crafting input data causing a suitable integer overflow. And hackers go to length to achieve that if they spot an opportunity.
IT security isn't a simple topic, but an important one. Especially for the Linux kernel running the Internet. A successful kernel exploit may easily cost billions of dollars. Dismissing people caring about that as paranoid and fanatics tells me that you don't know or care much about IT security and its value. That's fine as long as someone does that for you. Like those fanatics.
Oh boy, the "rust is safer discussion"
No, it is not. You can still write unsafe code and you eventually will and that's the problem. Users feel false safety and then you have the same security issue but without anyone thinking about it.
Just stay with C end of story
Rust IS safer, it is damn near impossible to get a buffer overflow in Rust, the compiler simply won't allow it.
@@liquidsnake6879 he does. please just shut the fuck up you never wrote a single line if code in your life and are bothering me
Yes, you _can_ mark every single function in Rust with "unsafe", but no human would ever agree to merge that (unless it was actually written in C, which basically is equivalent to tagging every single function with "unsafe"). The point of "unsafe" is to limit the area that needs to be examined for severe vulnerabilities to just the tagged functions and blocks, as opposed to needing to scan the entire program with a fine-toothed comb. You should still scan the entire program with a wide-toothed comb, but code not marked as unsafe doesn't require nearly as much scrutiny to accept.
You can, and maybe you personally would abuse it just to keep the C style of doing things. However, unsafe isn't necessary to write efficient Rust code. I had one occasion yet where to beat C in performance I needed to use an unsafe operation (modifying bytes of a String in place, unsafe as it may break the UTF8 encoding of the string - C does not even know what UTF-8 is). But the thing with unsafe in Rust is: it's local, thus you can reason why that unsafe is safe. At least that's the way it is used in Rust. You can opt in to do unsafe operations. In C you cannot even opt out. extern "C" is unsafe by definition - of the C language.
I am working with legacy multi-threaded C code in my current project, and boy how I wish that thing would have been written in Rust. Whereas it is indeed security software (an IPSEC VPN stack), where memory safety is a important concern, the benefits of the Rust features for sound multi-threading alone would be worth it.
C is a terrible language.
C is a very good language for writing operating systems. But writing operating systems is an endeavor only suited for the best in the field. C is a terrible language for writing monkey code. Java and C# are excellent for that.
@@andersjjensen Java and C# are not good for anything, not even monkey code. And C was a good enough systems programming language back when the PDP-11 was around. The C abstract machine assumes a flat memory space and sequential execution which is not how modern machines work. Not that you'd want to actually expose the physical machine but C's abstraction is just terribly outdated. The only reason it's even usable anymore is bc HW architects bend over backwards to make C run well on modern machines. But it's an objectively terrible language that's holding back CPU design. Still, I'd pick it over Java, C#, or any OO garbage even for "monkey code" any day.
@@ratfuk9340what's an alternative then? Honest question. Thankfully I work with embedded systems, which I love because I love working close to the bare metal, and C is great here.
@@kwastek Rust or something like it. Realitically though, C can't be dethroned in an instant (which doesn't make it a good language). C's strength is its realtive simplicity when compared to Rust for example. Still, I agree with what LLL said about Rust vs. C: in Rust the difficulty is learning the language but in C it's writing good and safe code. The difficulty should be in the language bc that leads to fewer bugs in released software. As an embedded systems dev, which of the current modern systems langauges do you think is the most interesting/useful?
@@ratfuk9340C has nothing to do with how the hardware executes the instructions. I don't see how it's holding anything back really. There is no alternative.
stop deleting my comments brodie.
it isn't brodie doing that it's a silly automated youtube thing
@@balala7567 yep i know. the youtube AI hates me.
I have just turned my computer on for the first time today lol
@@BrodieRobertson Deleting comments without even using your computer is next level power tripping
Aside from the initial aggression, subsequent follow up messages seem very reasonable in which they work through some examples and reach mutual agreement to deal with truncating assignments.
IMO this video should have included some of the follow ups. Reading some of the comments here it’s pretty clear that people are walking away without really seeing the more concrete motivations behind Linus’s complaints, and the subsequent resolution.
And I don't know if those cases where overflows/underflows are expected can be automatically detected. Usually what happens is some use of the final result of some arithmetic can lead to a "out of bounds memory access" or other problem with the value itself, so this final result has to be proven by some sort of automated theorem prover that it was never the result of accidental overflow/underflow. This seems to imply that you need to know what are the expected bounds for the inputs or the outputs, or some other extra information that can't easily be obtained from the code.
Maybe this kind of information can be obtained from runtime data for certain machines (which is an incomplete kind of information but better than nothing).
@@vitordelima " This seems to imply that you need to know what are the expected bounds for the inputs or the outputs, or some other extra information that can't easily be obtained from the code."
Well, yeah, Linus is putting the responsibility for knowing that on the coder, which is squarely where it belongs.
@@horusfalcon And it fails a lot.
@@giusdb I don't see the point in only complaining and not discussing the problem like the people from those communities (and here) do all the time. Everyone knows what you are going to say and it adds nothing.
Yeah, I was rather surprised the video ended when it ended. There was a whole discussion. What was that discussion?
It seems most people are agreeing with Linus. While I understand the concern of making the codebase accessible for lots of people to understand and edit, it makes me very uncomfortable seeing code where we aren't sure if it will have a problem or not. This strikes me along the lines of the people who create max-size arrays and say that it "probably" will never overflow. This is sloppy. I've been in code reviews for C code containing obvious race condition bugs where the developers allowed it as it probably won't happen. This attitude is bad engineering. "Probably will work" is not good enough. As long as the kernel is developed in C and C doesn't help with overflows, it means the onus has to be on the developers. Maybe the whitelist system thing can find a better solution. But having a mix of expected and unexpected overflows doesn't instill confidence. But I've seen the Linux kernel code and there is a lot there that doesn't instill confidence.
I think it ultimately comes down to the issue that you can't make C safe without turning it into something that isn't C, which would break backwards compatibility with existing code.
I don't think Linus sounded particularly angry in that reply... maybe a bit stern, but it was the necessary amount of sternness. The main thing about Linus' reply is that he's absolutely right. For a project the age and magnitude of the kernel at least, the wraparound is expected, and changing expected behavior is going to create more problems then it solves. The kernel has come way too far for that sort of change to be taken lightly - so any tools to help with that type of change need to be taken extra carefully to ensure they aren't flagging stuff that aren't actually bugs. Basically, the solution proposed was far too broad.
Wrap-around is expected .... until it isn't. Now there are typical constructs in C where you expect wrap-arounds and use them for efficiency. But given all the integer overflow related CVEs for the Linux kernel this still isn't something every kernel programmer is always aware of. Even Linus doesn't think this is a non-issue. He does not agree with the given proposal for a mitigation.
Look at the kernel mailing list thread and see how even Theodore Ts'o does get it wrong with C. The perfect C programmer is a unicorn.
I like the most that he's talking to Google people like that. So many others would just bow and oblige.
Linux is going to implode, once mean man Linus is gone. 😕
I'd disagree completely.
1 : citing the size and age of the linux kernel is basically an appeal to tradition. "we've done it this way for so long and so much that it's a lot of work to change now" is true, but it ignores the fact that those are only upfront costs, all *_future_* code (which will almost certainly go on for decades more) will only reap rewards with only minor inconveniences. Further, all of the work spent upfront will also yield upfront returns to, since all existing overflow bugs should be spotted during the adoption processes.
2 : flagging known overflow cases as bugs and forcing people to mark them is a minor annoyance, incurred to wipe out every other overflow, truncation, underflow, etc. bug that is currently in the kernel or ever would be in the kernel. These are not even remotely the same magnitudes of issues. (to be clear, yes it obviously wouldn't *_actually_* be 100%, I'm sure at some point someone somewhere will mark a case that shouldn't be overflowed as overflowable, but it would dramatically reduce the cases since someone accidentally writing out, explicitly, "yes, I'm allowing overflows here" is far less likely to them not even realizing they're causing one.) One is a minor annoyance at most, requiring a tiny bit of extra syntax before doing something dangerous, and the other almost certainly costs millions and millions in issues annually not to mention the countless dev-hours that are dumped into fixing these bugs as they crop up instead of preventing them in the first place.
Essentially, if I offered you "Give me all of your life savings and all of your future income, and I'll give it back to you with 50% more" these exact same arguments apply. "But I've spent my whole life saving up my life savings, that's a lot of money!" Yes, and 50% more than that is more. "You're asking me to give you money for the rest of my entire life" Yes, and 50% more than that is still more. The cost isn't what matters (so long as it's affordable at all) what matters is the return that you get for that cost.
So, writing a few extra characters in known-cases vs never dealing with another overflow bug... yeah I can agree that's not a fair trade but I don't think it leans the way you seem to think it does. That's an obviously amazing trade, at least, if it can be implemented. Implementation is it's own debate but, if you can implement a system like this, then you're left with a cost of people who intentionally are using overflows needing to demarkate that "yeah, I'm using overflows here" and the benefits of stopping every other overflow bug. Again, I really fail to see how that's a trade that anyone could even concieve as being a poor deal.
It's a lot of work sure, but what matters is the return on investment and it returns more than a CS student who just learned about recursion.
@@olafschluter706 good to know, do they use Linkdin or do you need to go hunting to nab one? Is the power in the horn itself, or do you need the horse that comes with it too?
This problem needs a solution, but Linus is right in that there's already A LOT of clutter in the kernel src. A fair amount of it comes from Sphinx documentation stuff (needed and arguably not "clutter"), but a good amount of it is undocumented macros and flags for tools/scripts. The majority of the docs are already years out of date. I'm not against this idea but the onboarding process for a new kernel developer is already difficult. Between the undocumented macros, flags, /scripts/ dir, and docs years out of date; I don't think adding any new flag is a good idea.
I got -100 problems and an integer overflow is -1
Linus almost has a point... properly written rust code that wants numbers to overflow looks like crap. But it's also a bs response. It's a common source of bugs and vulnerabilities. We cannot ignore these things anymore. Also, tone it down, Linus. That's not the way to have productive conversation.
This is a case where Zig does have an edge over Rust. Zig: a +% b, Rust a.wrapping_add(b). That said, Rust also has the Wrapping types, that can use the usual operators, but require extra syntax to wrap and unwrap the values.
@@angeldude101 I really wish it was just a top level type as a compromise, but then you get into all sorts of unpleasantness with the nuances for how compilers work IRL. Perhaps zig found a good balance, but I'm unfamiliar.
@@brettfo I'm not certain what exactly you mean by "top level" type, but for a type in general, that's what the Wrapping wrapper type is for.
@@angeldude101 a first class primitive built into the language. The syntax is too verbose in rust. Also, experimental. I think it's because I come from an embedded systems background where it is an extremely common pattern.
I feel the urge to reply to Linus now lmao
This is a type system issue or language design issue. We have two mathematically distinct concepts, but only single in-language construct (which roughly fits one case and has to be forced for the other). You're not fixing that without re-designing how we write code. And for Linux, which is stuck with C, we have to look at all the options and decide which one gets us the closest to the ideal for the least long-term cost.
Reading the thread further only makes me more convinced I should lol
Overall it seems that Linus believes that types are a hindrance and that they get in the way. I guess he never really interacted much with a good type system (heck, he even means something entirely different by the very term "type system").
@@mskiptr "it seems that Linus believes that types are a hindrance and that they get in the way"
If that were true he wouldn't have accepted Rust
@@mskiptr: Your perception is mostly related to the limited number of available systems programming languages. It's not that popular of a field to design a language for, so malformations in language design have a much larger percentage-wide impact than in other fields.
Or, to put it another way, the most C-like option with improved type support is C++, which Linus shot down quite some time ago.
14:33
This is actually kind of a funny example, because the behaviour depends on the type of a.
Here a is unsigned, so everything is fine, but if a would be signed, pretty much every optimizing C compiler (read: all the commonly known ones) will optimize to a point where there isn't an if anymore.
A compiler may even do that, b is signed too, if the compiler can prove, that b is always 0 or positiv (e.g. via the code above that if).
I don't know if the same syntax meaning two different things is that much better than overzealous compiler warnings.
I am far too removed from low level hardware stuff to have a properly educated opinion, but in my limited capacity, I would think "sum these and wrap the result" looking different from "sum these and complain if they're too big" is a good thing.
As any long time user of languages like C++ can tell you: Same syntax meaning different things is a bad thing most of the time. At a glance you don't realize that there's overloading going on and the code doesn't do what you think it does and you make wrong "fixes" and/or you spend a noticeable amount of time checking every variable type (have fun when everything is "auto"). It provides a ton of chances to introduce more bugs and it also noticeably increases the mental complexity that a developers needs to handle - distracting from the actual task.
To be workable for larger teams in non-trivial-size code bases, operator overloading really needs to be restricted to only a few things: Extending the operators to mathematical types that aren't in the language (vectors, matrices, complex numbers, ...) where what the operator does is blatantly obvious, or doing some background compile time checks (or statistics) while the result is not affected and nobody needs to know about this overload.
You are correct. Anyone who thinks that it is a good thing to allow `+` to do an arbitrary user-defined amount of work based purely on context is absolutely insane. Its relatively okay for higher-level languages, where the amount of work needed to do literally anything is already quite high, and the time it takes to do things isn't as important, but in lower-level programming, you often really want to know exactly what everything is doing, and hiding arbitrary control flow in your operators is a really good way to make that task infinitely more annoying.
@@JansHeikkinen On the one hand its true that overloading can get confusing but on the other it's not like any other function is semantically guarenteed to be what it says so calling a function plus or the character + doesn't actually change much especially in the non mangled way of overloading per scope or defined by the type. a string plus a string will be a specific meaning for the specific type in the same way that a float plus a float is a specific meaning despite being a overloading of the integer plus integer default
@@TurtleKwitty: This doesn't counter your argument, but your string case is technically wrong- string addition is often treated as concatenation, but it actually corresponds to concatenation _and_ per-element integer addition (and as a variation, to arbitrary length numbers of various formats), so just like quaternions (which have multiple versions of multiplication), strings shouldn't implement the addition operator.
Thank you for coming to my TED talk.
@@absalomdraconis Not sure how per element addition of characters makes sense?
If you're using a special integer type that uses a string as data model then that's exactly the case where you'd want to overload the plus to work with that, so if anything you're bolstering rather than countering at all
Quaternions having multiple ways to multiply means it's a bad candidate to use the feature but that doesn't mean it's not a good feature to have for the cases where it does fit in easily. That's exactly my point though, just because you can overload doesn't mean you should in the same way that just because you can use a name that makes no sense on a function doesn't mean you should and that doesn't make functions being named a bad festure
100% yes , Linus is the type of benevolent dictator we need. Says no, prove it to me, and when proven, changes his mind.
Except when it comes to C++.
@@Spartan322 nobody has proven anything to Linus with C++
Maybe a similar dictator without the attitude.
@@Spartan322 He's right about C++
"Just do it correctly has not worked" is genuinely true though. I see this argument all the time in the form of "X is bad because of often results in bad things" - "No, you can do X good. Git gud".
That isn't what Linus is saying here though. He is saying, what we do works well enough that changing it requires it to not only work better but DEMONSTRABLY work much better to overcome the friction of having to deal with more kernel abstraction. He does not want it to simply trade one problem(the current implementation) for a new problem(a complex abstract system over top of the kernel). Its two different priorities. Linus job is to protect the Kernel and its developers and make their job manageable, while providing the highest quality product. The proposal is suggesting a fix to an admitted problem, but their goal is to fix the problem at any cost. Maybe the problem doesn't need to be solved, or can't be solved in a way that will actually improve the Kernel. Time will tell.
@@senoraraton This is referring to a sentence in the original proposal, not Linus' response. Linus wouldn't ever say such a thing, but there are still many C defenders who do and think they are smart enough to just never make mistakes.
Poor Tux on thumbnail 😢
If only there was a language 99% compatible with well written C that had proper abstractions and tools to deal with these kind of issues! 😴
If you're talking about Rust, even that is political when trying to integrate it into the kernel
Unfortunately ego is a big thing in FOSS projects
If only ~
:P
There is also zig kind of…
it’s easy to call C from zig, and the opposite too sort of with small extra work,
zigs also compiles C out of the box too…
but I doubt kernel devs would consider it as of now.
Zig has been discussed before but it has the same issue rust had a few years ago, the ecosystem was still too immature to be adopted by the Linux kernel
The kernel will not be rewritten in any language... never. So, even if zig or rust is forced for all new stufff... the problem will still remain.
Even if he changes his tune later, it sounds like working with that guy would be toxic as fuck. Kudos on being able to change your mind, but I wouldn't want to work with someone who's had the time to fully read a written proposal and respond with his own written proposal and can't back down from saying he'll ruin your reputation for expressing strong opinions he doesn't like. Touch some grass and run it by someone first, you've got all the time in the world to edit that.
I have to admit, I love angry Linus.
i hope they don't force him to take a vacation because of this
Honestly, after 10+ years in the industry and meeting a lot of various developers of various level of expertise, the more the time passes and the more I agree with Linus takes. I used to be on the side of "oh that's unnecessary" and slowly, over time, came around... well, the legendary "should be retroactively aborted" while still making me chuckle, is still too much imo
Linus overflow caused by implementation defined behavior
read through all the follow-up and they go through examples and what the focus should be. more baby steps more granular and specific with the focus of getting false noise down to nearly zero. From it the implicit type promotion or implicit type casting seems to be more of a concern. I don't see a rant, it is to the point and It is a good read.
Don't quote me on this, just something I've heard over the years, but for integers, it's called overflow both ways, when going below and above the minimum, underflow is for floating point numbers, where incrementing a large value by a small value doesn't change it
Yup
Source: en.m.wikipedia.org/wiki/Arithmetic_underflow
Also: can't believe i never realized this was the case lol
And it makes sense, after all if you go under int min or if you go over int max, you are still just doing the exact same thing: an addition
So it makes sense those are both overflows
Chat, is this real?
@@grisu1934 no, nothing is real
I use the two interchangeably with integers
This whole proposal has UINT_MAX + 1 value.
0:03 As soon as I saw it was Kees Cook, I knew what was coming. Linus and Kees have history.
I was not aware of that
The "C type system" using type system very loosely is a major strength and significant weakness of C.
I don't know why the kernel project hasn't added to the language to address this at compile time. It seems like a better solution than to have a billion checks and workarounds in the code. Ditto for many other classes of bugs.
I don't even remember what an integer is.
I guess it's a liquid, since it can flow.
Clearly this means Linux needs to switch to Zig
rust.
I forget what my objection to Zig was, but Zig and Rust (and at least Go version 1) both have their problems. Both are meant to be improvements on what came before, but both have their defects (in the case of Rust, retention of * and & to declare pointers (it was an experiment in C, but _not_ a successful one), and the ability to add arbitrary syntax extensions (I realize some people think it's great, I do not: if I wanted it, I'd be using FORTH all over the place) ).
@@absalomdraconis The main problem IMO that could involve what you mentioned in Rust is lack of ergonomy.
So, I'm a little surprised by Linus's response to this, considering how he's responded to similar transitions in the past, although at the same time, I do see his reasoning as valid. He does have to design the metaphorical trash can to allow humans to use it while keeping bears out.
Linus wants to make sure it's good before any merger happens, the exact same things happened to rust adoption, they made made tons of changes then came back a few months later and the initial patches for accepted
@@BrodieRobertson Right, and I definitely understand the logic, but on the flipside, I'd recently watched a talk about how there was a proposal to minimize runtime allocation size whatyamacallits (forgive me if I don't remember all the fine details), which is the closest analog to overflow protection that I was aware of in terms of precedent, and had gotten fairly quick BDFL sign-off. If, before the knockdown, you'd asked me to predict how Linus would have reacted, I would have guessed closer to the runtime allocation size whatyamacallits than what happened with Rust.
I would just take a reply like that as a challenge to do it better.
Exciting news. Anyrhing that improves memory safety is a win in my book. Shame Torvalds is so hellbent against it, I do kind of get where is coming fron but I don't agree. I like when my compiler asks me wether I really want to use a potential footgun
Adding numbers is not a footgun. If you’re a bad programmer then maybe Rust is good for you, but most kernel developers are not bad programmers
@@thebatchicle3429Ridiculous, there have been plenty of bugs in the Linux codebase, they are not gods.
@@jshowao Make a totally optional tool to ferret out such problems and let those with the patience play around with it to try to identify more bug patterns. There'd be no guarantee in the kernel development process that the tool will be used, but "might use it" is still better than "won't ever use it." Even a dynamic debugger which, of course, will slow operation way down, but used with randomized test cases by those who geek out on such things... it's better than nothing. Don't panic the kernel when it happens -- just do a save of state, create a notification, then keep going.
He's not against memory safety though, he's against tooling that can't recognize very normal c code that is purposefully correct as being purposefully correct; he said "come back when you make it work better" not "fuck memory safety"
@@thebatchicle3429 Last I checked "good programmers" don't actually exist and that _all_ programmers are varying degrees of "bad".
Argue as you will - the first time I came across the term "underflow" was reading Rodney Zaks' book "Programming the Z80". That was back in 1980! The example he shows is adding two negative numbers where the result is too large in the negative direction; hence underflow, for the desired destination.`
I hope Linus lives and continues working on Linux for 1000 years. Though he may disagree with me on that.
I must admit I hate overzealous linters or compiler warnings as much as the next guy but...Kees does have a very valid point. Integer safe code is hard and the only way to get it right is to suffer a bit until all the undefined behaviors dealing with integers are found. The problem is very much that even if you have a sanitizer that doesn't halt control flow (an option for userspace, I've not used the kernel sanitizers to know how it works with that), you're going to be bombarded with warnings everywhere for things that are intentional and handled properly. The only way to get around this is really to annotate things in your type system or to take an approach similar to swift where every addition can branch to an error because it assumes a possible overflow (this is obviously terrible for performance). What Linus is suggesting may be outside of the scope of what a compiler is capable of, as it's a form of static analysis that has to check to make sure the overflow is handled prior to every place it gets used in the control flow graph. To do that, it kind of needs to have an infinite look behind distance.
Integer safe code is actually ready fucking easy, it’s only ‘hard’ because almost nobody does it. It’s especially easy with the new stdckdint.h header in C23
From here on, assign all integer overflow bugs to Linus.
Lol, I find myself constantly clicking off right before the outro starts.
This wasn't a rant Linus was dropping knowledge! I code in Rust BTW...
Maybe C needs an extension that states whether a variable is wrapable or not.
Some form of compile-time stuff perhaps.
I remember when there was a nifty c extension called c with objects. It's gotten really good and renamed c++. Maybe Linus should give it a look.
I think Linus is correct. It seems that in almost all cases, clever software should be able for find most troublesome cases. This could be a preprocessor that works like good old lint.
In C, the runtime is machine code so it doesn't do errors, the compiler and kernel does.
Sure the proposal could do some work, but Linus is sadly getting back into angrily shutting down ideas that would be beneficial for the kernel, and all the while making himself look like an ass. I don't know how I would take it if someone shot me down in my own work the way Linus does, especially when the topic is well meaning, good-faith, and generally *good practice in every other context*.
You clearly haven’t actually read the email thread then. The following emails after this are very calm and they come to a consensus on how to approach the problem
I feel like this is really sensitive lol Most bosses I've had are pretty straightforward and aggressive like this. If the project means something to you, then you just adjust accordingly and do it again. You're crying about your own literal fear of rejection and that says more about you than anyone else
@@thebatchicle3429 That doesn't excuse the knee-jerk reaction. His initial reaction is not cool. I'm glad they got there in the end, but it's just a bad look for Linus in the end.
@@brandonlewis2599 so what? It’s Linus’ project. If you don’t like his reactions you don’t need to work with him.
@@brandonlewis2599 It never ceases to amaze me the amount of people in these comments that seem to think that not only is this good behaviour, that it's needed to run major projects. I think it's pretty obvious most of these people have never worked on any major project.
I love Linus "Techtip" Torvald rants.
Linus "kerneltip" Torvalds
Linus impregnated your moms
Linus "diktip" Torvalds
Do as I say.
Any static analysis tool up to the task?
You could get most of the way there with KLEE Symbolic Execution.
Also yes, this is the right answer. Rather than making everyone's job harder, use a tool to find genuine problems.
It’s so refreshing seeing Linus being Linus again!
That's the nicest Linus content I've ever seen.
I know it's still early in the video but "just do it correctly" should be the default stance imho. Then again a patch to harden the Kernel against this wouldn't be too bad, considering that in reality there are a lot of cases where it just isn't done correctly... I hate this question since this is one of those things where ideally you should not have to do this, but the real world shows that SOMETHING is necessary, at the same time this could be one of those mitigations that just encourage people to be more careless when coding, which happens so often. Similar to how Proton made it so that devs ignore Linux even more than before.
"Just do it correctly" doesn't work and never will. C is over 50 years old and people are still writing the same kinds of bugs.
"Being careful doesn't scale." Bjarne Stroustrop
The funny thing, when you read the kernel mailing list thread on this: Theodore Ts'o, who is a name in Linux kernel development, complained about being forced to write some ugly code to silence a compiler and UBSAN warning while commenting the original code he was getting warned about as "which is obviously (to a human) correct". Just to get immediately corrected by Linus Torvalds himself, who explained: a) why the warning is valid and b) the code complained about is not correct and c) the fix to silence the warning isn't correct either and d) how to deal with that stuff correctly in a C idiomatic way - there are even helper macros in the kernel doing the stuff right. So if guys like Ts'o get it wrong...
@@SaHaRaSquad look, you two, all I'm saying is that I am more on Linus' side, although really I stand somewhere between both viewpoints.
Watching this brings back memories from the early 80s. I wrote a C compiler for the CDC machines. All interger types are 60 bits. Addresses were only 18 bits, so if you wanted to you could use the upper bits of a pointer as flags etc. But the real kicker was the 1's compliment arthmetic, so there was 0 and -0 so 0 - 1 == -0 and -0 + 1 == 0 , I also wrote a backend for the 8086 and did just use the 8 bit registers for chars. I saw code at the time that on some machines the addresses were only even and used the LSB as a flag. But then machines were much smaller and needed such "optimizations". On the 8086 it was still segmented memory so the compiler had to normalize the seg+address before they could be compared.
funny how the developers in linux seem to be overwhelmed. since they work on everything on HOME COMPUTERS, remotely. you know, unlike those giant corporate like microsoft that actually have "farms" of machine framework infrastructures. i mean, equipments does determine the quality of results.
To be honest, I always acknowledge that wrap-around is the defined behaviour on any unsigned type in C. The problem is, well, size_t type. It is unsigned just because you can have double the values than it's exact copy but treated like signed. This is the main problem - people but unsigned everywhere they want double the value. Don't do that. Use long long, enable wrap-around on signed as default in ANY code you compiling yourself and abuse the hell out of checking if (a + b < a). This is the only fix there is exist without making everything slow because someone refuses to aknowledge that they are dumb.
Good. Shows that Linus is healthy and happy. 🙂 Now, if he had said, "Fine, just do it and go away and stop bothering me.", I'd have been worried, and would be thinking "Someone should call Linus on the phone and see if he's OK!".
While I get that Linus is generally (and justifiably) protective here, but I feel like his reasoning for this is rather dubious. If you are a developer using overflow intentionally, you've already done the mental work to know that you need to use overflow, that's why you're using it. The only additional work is typing it in such a way that you can use overflow which, well, that's kinda just called programming. A rule like this *_would_* add additional mental work however. It would force the people who *_aren't_* thinking about overflows, to think about overflows... which they already *_should_* be doing, they just aren't. The people that are using overflows intentionally need to make tiny alterations to their code that do not affect behaviour at all, and only the people who *_aren't_* intentionally using overflows need to deal with any additional work... because they're unintentionally risking serious bugs and now they need to worry about fixing them. This proposal wouldn't add any more mental load to people writing good code (especially since the warning/error would just say "hey, if you mean to use an overflow here, say so") it would just make people who *_should_* be paying attention, pay attention.
It's honestly a little weird that he's even against this given he's spoken pretty positively about Rust and this is the exact sort of methodology Rust is built on as well, "don't let the user do something dangerous, and make them explicitly awknowledge what dangerous thing they are doing if they need to and, if they do, don't stop them."
His argument here is essentially "No! we can't make murder illegal, if we do that crime rates will go up!" which, yes, it's correct, making a thing illegal *_will_* increase crime rates since a once legal thing is now considered a crime, but what matters are the actions themselves. Yes, adding additional constraints will mean that acceptable uses need to be altered and some people will need to think more, but the alterations would be entirely non-functional (i.e. : nothing about the code actually has to change, you just have to add a flag or switch a type or something). This is functionally 0 work, at most a bit annoying) and the only people who actually are forced to think about more things are the people who aren't already thinking about overflows when they should be. Going back to the analogy; the murderers already *_should_* be considered criminals, this isn't overhead it's just how things should already be. It's not increasing crime, it's just recording things that already should have been recorded as crimes. If you're writing kernel code you already should be thinking about overflows and such so the only people left actually doing noticably more work are the people who *_aren't_* doing what they already should be doing. (for the record here, this analogy is not an endorsement of over-legislation, there are definitely cases, very likely the majority of cases, where laws >disproportionately< increase crime rate. Al Capone ran an entire criminal empire *_because_* a once legal thing was made illegal. This is an entirely different conversation, but it is worth clarifying that, no, this principle is not a constant, not even close.)
Is it annoying to need to explicitly declare when you're doing things like this? Sure, I could see that easily, but if kernel developers being slightly annoyed stops countless minor and major bugs (which themselves are probably a bit more than "annoying") then I have a hard time seeing how that's not a fair trade. The easiest thing for a developer to do is quit. The easiest code to write is the code you don't write. If avoiding any and all inconvenience is the end goal, why is anyone here? Well, they're here because they want to solve problems, and overflows are kinda a massive bloody problem to be solved. In contrast, "you need to use an overflow_uint8 instead of a uint8 if you want to allow overflows, it says so right here on the compiler warning" seems like it'd rank pretty low, at least to me.
I love a good Linus rant. They are even better when he is right.
Honestly, I agree with Linus, and I saw this coming 100%
"Change the existing behavior of every arithmetic operation, and also change the existing behavior of all implicit assignments" doesn't seem tenable to me, not if you also want to avoid a "Flag day".
I believe the correct solution would be to leave old functionality with wrapping being default, and to overload assignments to 'blacklist' wrapping as the new functionality.
I don't like the blacklist/whitelist analogy. Once we have two options for whether to allow or not allow wrapping, we don't have a blacklist or a whitelist system, we have a system that forces you to pick to either blacklist or whitelist every usage. It's effectively a system with both lists. There is no functional difference here between these names to say that the system is only using one or the other - with respect to all future code written in the new system
But, a non-breaking change that leaves all old operations as they were, but grants the ability to have future code not allow wrapping would at least be helpful, it'd add in the tools necessary to start combatting overflow bugs - and could be followed up by later going through all of the old code to implement non-wrapping alternatives over time, without a "flag day".
The way you were building it up, I was expecting a torrent of profane vitriol from Linus. What I saw was very firm, but reasonably polite given that it's Linus Torvalds.
I see, I see, I totally get it.
(
That's pretty bad behaviour how it can just wrap around without notice. This also seems to happen in other languages like Kotlin, while in Python the number just grows larger instead of wrapping. Linus should definitely take him serious, he has a point.
Linus has a real NIH problem.
His answers to proposals are more ego driven than need be.
Perhaps he should learn to be more open minded and less about attacking others in a childish manner.
He didn't even attack the other person lol He didn't name call or anything. Was he aggressive? Maybe, sure. But he wasn't attacking and he wasn't being a childish lol
It's good direction for Linux VM. Not if you use Linux as minimal environment for your application to run. MIttigation=off for gaming and fridge controllers.
So No one has thought of a punishing auto input application/library with logging to map your work? If anything ai should be used to verify error mapping.
a sporadic kernel dev here (usually fixing problem on esoteric hardware or porting to never seen before hardware).
Let’s not forget that that dude is a google engineer and is possibly a quota hire - which at google means a an inteligent soft eng that unfortunately is not smart and is driven by some vendetta / ideology.
If Linux develpment was Stellaris, Linus would be an end-game crisis.
Mild clickbait, implies buffer and stack over/underflows too.
Having nothing to do with any of it, I think the proposal was fair. I also think Linus makes me want to switch to Linux more.
Linus: This is objectively wrong, because I don't like it.
Linus' Rant: 12:57
I love angry Linus. /me flips the bird * 😂
Can't help but be reminded of the talks Linus has done in the past about the lack of new people contributing to the kernel. Linus might be a benevolent dictator, but the project still needs fresh blood to keep going. While not all modern development techniques would be beneficial or appropriate for kernel development, the hold the old ways have on the project needs to loosen if not for longevity
Well isnt this why a reason rust is supported? Like if u are worried about footguns why not just make rust parts of the kernel, like as
Asahi?
unfortunately rust in linux is a huge mistake
@@mister-ace it isn't, it's like its own separate code chunk so the C guys are happy, and as frustrating it can be to set up it allows so many drivers to get quickly made, it's hard to criticize rust in Linux when I use hardware (Mac mini M2) that has drivers based in rust, and being rust based is how development was pretty rapid...
We typically C this, but we are considering to C++.
Linus is right, again. Don't make proposals like this until you have completed with the basic rules of engagement. Don't break drivers, don't break userspace, make less work for developers.
The justifications in this proposal make Zig appealing, I wonder when that proposal (as short sighted as this one) will be made.
are you going to talk about the 200 CVEs for linux that dropped last weekend?
thats great, means theres a bunch of bugs being fixed
Like no. 636.
Well, Linus's reply seems rather calm and forthright compared some of his previous.
Mr. Torvalds is Finnish, so his first name is pronounced "LEE-nus" not "LYE-nus". I'm surprised that you don't seem to be aware of that.
He also has lived in the US for a long time and probably doesn't care anymore when its mispronounced lol
First
If you tried to count... You would probably over flow the buffer in your 🧠
He uses some aggressive language that seems entirely unnecessary for a first reply. However, the spirit of what he's saying is absolutely correct.
Isn't the spirit that we shouldn't address one of the most common sources of software defects and exploits because it makes the development experience less good?
@brettfo I suppose you could look at it that way.
Or you can say that the project has been around for a long time and many devs have gotten used to seeing it as a feature, not a bug. You cannot ask everyone involved in such an established project to change their methods even if your way is technically safer. So if you want to fix the bugs, make sure you flag it when it actually is a bug.
Perhaps it is best to not allow overflow if you're starting a new project.
Overflow bugs are potential CVEs. They're all potential *catastrophe* for an organization that uses Linux in their infrastructure. The spirit of what he's saying is absolutely wrong.
@@rightwingsafetysquad9872 see the entire history of computing for why that's not defensible.
@@brettfo Objectively, the complaint is correct. And if you're starting a new project, you should adhere to best practices - only allow overflow if it's specifically flagged as intended. But you do not get to step to the Linux kernel and tell Linus, Greg, Sasha, etc that they're doing things wrong.