that white house press release at the end mentions rust by name for those who dont know :) great video! i watched both and you explained what the problem is and why the problem is very well
This is just a sidebar, but I honestly wish they would decouple lifetimes from types in a way that is similar to what ocaml did. The lifetime system should act like its own typing system orthogonal to the normal type system and this would allow them to also expand the rules for lifetimes to allow for explicit cases like this without effecting the type system itself and visa versa.
That’s something I really do agree with. We ended up with kind of a worse version of colored functions- with infinite colors. It’s just like here’s a variable. It has a type which is described what you can do with it. It also has where it is and how long it exists. I think of those as pretty much orthogonal. Just my thoughts tho, I don’t even know much ocaml (yet).
@@tee1532 I learned about them from the "Oxidizing OCaml" posts: blog.janestreet.com/oxidizing-ocaml-locality/ I'm not super familiar with them tho but I do understand the basic idea.
Note that, from what I understand about OCaml, Local variables are the equivalent of only being able to have lifetimes at argument position, and disallowing all lifetimes on return types. This would make a LOT of Rust code impossible to represent using that system. I think Local variables are a good fit for OCaml because they're trying to make a GC managed language more efficient by using the stack more, and lifetimes on arguments are perfect for that use case. However, if we tried to retrofit Rust with Local variables, it would have the opposite effect; people would start cloning and using the heap more in order to erase the lifetimes on the return types.
@@jedel1043 Obviously you couldn't just take ocamls implementation and run with it in rust, though I dont think it would be as bad as it seems to have only two types of explicit lifetimes; global and local in rust, with global having a scope beyond the original scope of the variable etc. Instead what I'm suggesting is to make the current system of rust lifetimes detached from the types. Right now a 'static T is a different type from a 'a T which leads to some silly implications. It adds needless complexity to a type system which is already somewhat complicated in strange ways. For example, rust's generics are not really true generics, they are trait arguments and this also ends up creating some weird edge case problems. I've used rust for 6 years in prod and I still run into trouble at times when it comes to the type system and the lifetime system because it's not as intuitive as you'd expect in certain corner cases. This wouldn't be impossible and it would also simplify the way the compiler checks and expands the code. The other nice side effect is that you could rewrite the lifetime system without having to worry about the type system.
@@jameswise9171 oh I totally forgot about that lmao. Yeah I’m actually in legal trouble for using their logo. Watch the video again so I can get to monetization and pay my legal costs pls.
Damn what a rabbit hole this comment led me into. Seems Rust is ran by a bunch of tyrannical political activist weirdos. Why do ppl bother with this language? Also no wonder the White House loves it so much.
I consider the error to be the conversion of only one 'x to 'static, and leaving the other 'x to be 'x. The function definition makes use of two lifetimes, 'x and 'y. Buy in step2 we make use of three lifetimes, 'x, 'y and 'static. The compiler lets us change the definite lifetime of 'x into 'x and 'static. Of course it then later gets confused which of the two actually applies to the output of the function. @connemignonne already mentioned it as well.
There’s absolutely nothing wrong with switching out only one 'x for static. The type `&'static T` is a subtype of `&'x T` just like `Cat` is a subtype of `Animal`. It’s just like converting a `fn(Animal, Animal)` to a `fn(Cat, Animal)`. We’re not substituting one for the other like when we substitute 'a for 'x and 'b for 'y in the last step. It’s just subtyping. Hopefully that makes sense.
@@xenotimeyt To me this feels strange. When I use the same name of lifetime ('x) in multiple places, it should mean that the lifetime behaves in the same way; all 'x have the same relationship to all other lifetimes. When the function has type `for fn(&'y &'x T, &'x T) -> 'y T` it implies that `'x: 'y`. When I than change the lifetime 'x to different things on different places, I break the implicit relations. In the examples with animals, it is same as having `Type` and `fn(Type, Animal)` here `Animal` is implied to be `Bird` because it is used with `Type`. Changing it to `fn(Type, Animal)` is than wrong because the implied constraint is no longer there. For this I see two solutions: - Don't allow conversion from `fn(Type, Animal)` to `fn(Type, Animal)`. When subtyping you must subtype everything consistently in the same way. - Don't allow `Type`. All constrains must be explicit. In the case of the rust code, that would mean that the constraint `'a: 'b` on the function `weird` would be part of the type of the function, which would than disallow the conversion of `weird` to type `for fn(&'y &'x T, &'x T) -> 'y T` because it doesn't have the constraint.
@@bonny4 I see. The first case is totally legal and should be because borrows are covariant. Just like any `Vec` is a `Vec`, any `&'y Pidgeon` is an `&'y Animal`. This is a very fundamental rule loads upon loads of existing code depends upon. If any `S` is a `T` any `&S` is an `&T`. As for the “you must subtype everything consistently in the same way” part this just doesn’t make sense- there’s absolutely nothing wrong with treating a `fn(&Animal, Animal)` as an `fn(&Pigeon, Animal)`. Why would there be? All we’re doing is making the first argument more restrictive- it can’t just be any animal now it has to be a pidgeon. Your second solution is the actual solution mentioned in the video- kind of. Right now doing that (disallowing the broken conversion from `weird` to `step1`) would break all existing code that uses nested references in functions getting passed around. Want to `map` on an iterator nested reference? Well now you have a `for fn(&’x &'y S) -> T` that you need to create and then pass around. Because you can’t put the constraint `'x: 'y` anywhere it’s not legal to construct this. Loads and loads of code is again broken. :( But if you implement the solution from the video where you can add constraints to `for …` expressions, now you can make the constraint explicit (basically do what you said in solution #2) and existing code will be fine. Hopefully that clears it up. :)
@@xenotimeyt I don't think there is a way to fix it without any breaking changes, in the end fixing this would break at least cve-rs :). The main focus of my comment wasn't on how to fix it without breaking changes, but why conversion from `for fn(&'y &'x T, &'x T) -> &'y T` (`step1`) to `for fn(&'y &'static T, &'x T) -> &'y T` (`step2`) may be bad. I'm still not convinced that the conversion is ok. The problem I see is that in `step1` the two occurrences of 'x are connected - the first 'x constraints what the second 'x can be; it makes the second 'x more restrictive. By converting to `step2`, we are making first 'x more restrictive (that is ok), but we are breaking the connection of the first 'x that was restricting the second 'x. So after conversion to `step2`, the second 'x is less restrictive, and that is the problem I see. By "you must subtype everything consistently in the same way" i meant that you cannot restrict (subtype) something in one place, when it would cause some other restriction to be lost. In animal example, `fn(Animal, Animal)` is not representing `step1` very well. It doesn't show the connection between the two animals. I think that more accurate way would be to show it as `fn(A, A) where A: Animal`. It is ok to convert `fn(A1, A2) where A1: Animal, A2: Animal`, to `fn(Cat, A) where A: Animal`, but it is not ok to convert `fn(A, A) where A: Animal` to `fn(Cat, A) where A: Animal`. Another way to show that the problem may be in the conversion from `step1` to `step2` is that calling `step1` at the end of the `extend` function still gives error but calling `step2` does not. This can mean that some restrictions are lost in this conversion. I think that there are two equally valid ways to look at where the problem is: in the conversion from `weird` to `step1` or in the conversion from `step1` to `step2`. This is partially speculation on my side: I don't think that just disallowing the conversion from `weird` to `step1` would be break as much code because in many cases (probably majority (including `map`)) traits are used to pass functions around instead of types 'pointer to function', so there are no conversions. And if I understand it correctly the problem may occur only when there is conversion. But I very much agree that the best way to fix the problem is what you suggested in the video.
What does "subtype" mean in the context of lifetimes 'x and 'static? It seems like it's abstract vs concrete. Making it concrete in one place but not others breaks the consistency of 'x.
I think the key point that causes this is, when we call the weird function directly, the compiler checks the lifetimes properly because it knows 'b must outlive 'a in order to "cast" a &'a T to a &'b T. But that constraint is not part of the function's signature (and as such, its type), so when we store it in a variable the lifetime constraints are only the the ones in the functions signature/type. The fact that 'b must outlive 'a is "forgotten"
The way to fix it would be to disable magical inference of a lifetimes constraint from the mere presence of a parameter. If the user is forced to specify the constraint manually it will need to carry over and prevent function casting at some point.
Well yeah that would be a solution but now we have no way to store any function that uses nested lifetimes in a closure or even by just storing a variable in a function. Sure for a declaration we can make the programmer say `fn …(&'b &'a …) … where 'a: 'b …`. But now we can’t store that in a variable without having to specify a single lifetime. Which would make it impossible to use that (function stored in a variable) with arguments of different lifetimes.
@@kocsis1david Well remember that these constraints are all over existing code with nested references. Forcing any of it to be explicit would break that existing code. Also adding constraints to HRTBs is definitely not easy, it’s one thing to write down an idea for how something should work but another to go through every possible case in such a complex type system and another still to implement those into the beast that is the Rust compiler.
@@HansBezemer while my comment was ironic, your analogy is completely wrong. Rust's type system is designed around being completely safe. No exceptions. And the type system is actually safe, this thing shown in the video is an implementation bug and it must be fixed. It is not something you should be aware of and work around. If there will be a compromise, it won't be Rust anymore
@@sunofabeach9424 Yeah, I heard that about Ada, BASIC, Pascal and Java as well. You should read what Dijkstra wrote about "safe" languages. Quite enlightening.
So, this is mostly an issue in that this can end up in packages that you include in your codebase. Most people using Rust wouldn't want to do this themselves. Maybe, instead of fixing the issue in the language, they can just add some linter rules to the compiler to disallow this particular pattern, or set up some pattern checking in cargo to reject packages that use this.
The problem is fixable then like you said, by encoding the lifetime restriction as additional data (the where syntax, but the compiler can actually do this implicitly here) in the function symbol at step1. Then, when you attempt to call the function, it will see that the assumption must hold that: ``` 'a : 'static ``` which it can immediately deny the function call for in this case. The original problem by the compiler is allowing the lifetime restriction from the witness to inform the conversion of the other variable, when that information can be deleted.
Yep, although it doesn’t really matter whether you’re doing it implicitly or not. You still have to be able to construct these constrained HRTBs and adding those to the type/lifetime system is the real challenge.
6:59 now *this* is the step i dont get if the function signatur was just "(&'y&'x()) -> &'y T" then i totally understand why contravariance would allow you to change that to "(&'y&'static()) -> &'y T". likewise, i can understand why the full signature "(&'y&'x(), &'x T) -> &'y T" could be changed to "(&'y&'static(), &'static T) -> &'y T" but now, how the heck is "(&'y&'static(), &'x T) -> &'y T" also valid??? for any 'x ≠ 'static, this new signature would not be contravariant with the original one, would it? i feel like my intuition is betraying me here but i dont understand how, what am i missing?
We aren’t substituting 'static for 'x - it’s no different really from converting an `fn(Animal, Animal)` to an `fn(Cat, Animal)`. No need to change both.
@@xenotimeyt shouldnt it be more like substituting a fn(T, T) for a fn(i32,T) though? that substitution shouldnt work, as the original wouldnt accept parameters of type (i32,f64) but the second one would but yet it seems like rust accepts something like an fn
@@paulamarina04 You arent substituting- it’s just subtyping. Any `&'static T` is a `&'a T` just like any cat is an animal. The difference is that all `i32`s are not `T`s. Could be but if you choose `T` to be `f64` obviously it’s not. However the reference type `&'static` is special in that *all* references of any lifetime are subtypes of it. That is to say, for any lifetime `'q`, even one where we know nothing whatsoever about it, an `&'static T` is always an `&'q T`. Hopefully that shows you how it’s like `fn(Animal, Animal)` becoming an `fn(Cat, Animal)` and not like your `fn(T, T)` example. But to add one more thing imagine that Rust had a type `!`, which is convertible to any type whatsoever. How is that possible? Well you can never construct `!`. It’s a type you can never construct, so it’s convertible to anything. I need an `i32`. Take a `!`. This is totally type-safe because I’ll never actually get there. Functions that never return (like that infinite loop) return `!`. So you can say `let z: i32 = never_returns();` and there’s no problems because it will never return. The `!` type (which is a real thing but not stable yet) is the type version of 'static. It can freely be used as any other type the way 'static can be used as any other lifetime. What we’re doing is like converting a `fn(T, T)` to an `fn(!, T)`. No matter what `T` you choose, `!` will be convertible to it, so our first argument will be a valid `T`.
I think you're maybe (?) not right when you say the problem is in the let step1:... line, because the type signature of the witness should still implicitly enforce the correct lifetime constraint: taking a &'y &'x () still tells us everything we need to know about the constraint implicitly without the kind of explicit where-constraint you suggested It feels like the constraint is lost in the let step2:... line (because the witness was still forcing it for step1 but no longer is on step2) and I don't see why the borrow checker couldn't in principle recognise that and complain at you for contravariantly generalising just one instance of 'x in the type def to 'static where this drops a constraint that previously existed A solution that might be too heavy-handed but I think should work just fine is if it forced you to demote all instances of 'x to 'static in the definition when performing this generalisation. This fixes it, right? (e.g. we instead only allow step2: for &'y T = step1 and we complain if you try to only generalise some instances of 'x but not all of them). although I'm sure there's a less intrusive way of doing the same thing - just check whether the definition of step1 has an implicit constraint (like the one given by &'y &'x ()) that step2 doesn't when doing the type coercion. what am I missing?
6:40 I don't understand why/how "this is completely valid". The signature defines the lifetime 'x to be the same for both parameters. If we change the lifetime 'x of the first parameter for a more specific version, it should require that the lifetime 'x of the second parameter be changed in the same exact way. Is the fix could be that simple?
A ton of people have misunderstood this. We aren’t setting 'x to 'static, what we’re doing is like converting a `fn(Animal, Animal)` to an `fn(Cat, Animal)`. All cats are animals, and in the same vein, all &'static () values are &'a values (for any 'a). Hope that helps. :)
@@xenotimeytI understand that's why rustc is allowing this but from my understanding this isn't right. So without the need to express a full "where" relationship, the signature already defines that `fn(Animal, Animal)` where both `Animal` are the same, so it should not allow `fn(Cat, Animal)` but only `fn(Cat, Cat)`. I'm trying to understand if there is a way to fix the issue without a new rust edition.
@@OlivierEble It’s not that both lifetimes are the same- Rust lifetimes are sort of upper bounds- a reference having lifetime 'x means it lasts at least as long as 'x, not exactly as long as 'x. So narrowing only one to 'static is fine.
wow what are the prospects for adding constraints to the lifetime relationships in an upcoming rust release? how is this understood as a safety problem by the rust dev team?
If you go to the GitHub issue it links to a projects board with all the major unsoundness issues they’re working on ( github.com/orgs/rust-lang/projects/44 ). It seems like they’re working on a new trait solver and are waiting to address this until that gets done. They’ve clearly acknowledged it (and have since 2015). Disclaimer tho: I don’t actually work on the compiler or anything so I can’t say anything for sure.
Say we have two function pointer types: `for fn(&'a &'b (), &'b T) -> &'a T`, which implies that 'b will outlive 'a. `for fn(&'a, &'static (), &'b T) -> &'a T`, which implies that 'static will outlive 'a, but says nothing about which of 'a and 'b will live longer - so 'a might outlive 'b for this type. I would assume that in short, the problem is that Rust's type system is currently not capable of checking these implied lifetime bounds, so it permits you to cast the former into the latter despite the former having a lifetime bound that the latter does not satisfy. Right?
I mean yeah checking these implied bounds would be very very difficult. It would be *a* solution but it definitely isn’t a great one. (I don’t have an example to hand but you could probably make finding these derived bounds almost impossible using GATs also, but that’s just a thought and a whole mess I don’t want to have to explain in yt comments lol.)
The only place that checks whether 'a outlives 'b is the return of borrow in weird because there the compiler knows that borrow's lifetime is 'a and we're supposed to return 'b, where _witness adds the outlives constraint. So it's sort of a loop of 3 links: 1. _witness says 'a outlives 'b 2. we take 'a and return 'b 3. what we take and what we return is the same thing And when you assign the function you're losing the point 3
I don’t think “what we take and what we return is the same thing” is an actual constraint anywhere. Really the only requirement of a return ever is that it outlives the function. So in this case the caller chooses 'b, and because we’re returning a value that secretly has a lifetime 'a that means 'a: 'b. The fact that _witness ensures 'a: 'b just sort of implicit and not something that is checked as part of the `for fn(…) -> …` type. When we assign the function we lose the constraint 'a: 'b, since the way rust is designed the responsibility of checking that is given to callers, but here we have no callers (not until after we mess up the type enough to have that constraint be nowhere to be found). Hopefully that makes sense.
It seems to me the problem is the contravariance conversion is incorrect. The type system is saying that `'static` is a more specific lifetime. But in reality is actually a more generic lifetime
Contravariance of arguments says: > If any A is a B, then any fn(B) is a fn(A) Any `&'static T` is an `&'x T`, so any `fn(&'x T)` is an `fn(&’static T)`. It definitely feels weird because `'static` seems like the most general lifetime, but it’s actually the most specific. The `'static` lifetime is a single special case of any lifetime. The more specific lifetimes outlive the more general lifetimes. Sounds weird, but 'x *really* means (anything at least as long as 'x) which is much more general than (anything at least as long as the whole program). Hope that helps. :)
@@xenotimeytActually yes I retract my former statement. I would then suppose one of the problems is the multiple 'a and 'b become uncoupled during the conversion. Since this would make sense if all 'a and 'b were 'static. But falls apart when only some are and some aren't Im sure there is a good reason for this but I dont know it
@@thomasjhickson Glad it made some sense. The thing is that 'a and 'b and 'x and 'y and 'nyancat are all special cases of 'static. No lifetime include more lines than “the entire program”. So `'static: 'a` for *any* 'a whatsoever- and the compiler knows that.
@@xenotimeyt I found that explicitly adding `where 'a: 'b` to the original function stops this conversion. Why is the implicit bound treated differently from an explicit one I wonder. The original function has implicit requirements that are not reflected in the conversion. I would be interested in a video with an example of why this exists! (assuming there is a reason) In the video you say its an expresiveness problem, but that doesnt explain the difference in the way the bound is treated. Also if it was purely an expresiveness problem then it should just not compile. It compiling is a soundness problem. But I suppose thats why its a bug
@@thomasjhickson Rust can’t currently encode lifetime constraints in HRTBs, so there’s no where for the explicit constraint you added to go. So rust complains. On the other hand Rust doesn’t notice the `'a: 'b` constraint if you don’t say it explicitly. So it lets you convert just fine. Sure I think you could say that the fact that it doesn’t notice that constraint is the problem- but if you run with that and try to fix it you’d break totally fine existing code. Like I think you’re imagining the compiler just throwing `where 'a: 'b` at the end of these kinds of functions, but now you’re not allowed to store *any* function that takes a nested reference with generic lifetimes in a variable. That seems more specific than it is, any function that takes a nested reference where you don’t bother specifying the lifetimes, i.e. `fn blah(x: &&i32) {…}`, is treated by the compiler as `fn blah(x: &'a &'b i32) {…}`. So you couldn’t have a function pointer to `blah` with only that fix.
ive asked this in the last video, and i still dont get it: why step 2 is valid? how can you say that &'y &'static is a valid &'y &'x in the context of the function definition? clearly the second parameter must have the exact same lifetime as the second reference! i mean, &'y &'x doesnt exist by itself in vacuum, it is only a valid statement as long as the second parameter is &'x. For example, fn(T, T) must check that the first argument and the second have the same type, but here its completely ignored?
@@michawhite7613 I don't have a problem with that, I have a problem with coercing the type of step1 into the type of step2, the second is clearly more permissive than the first
Step2 is less permissive than the first. 'x only has to live for a particular period of time. 'static specifically requires that the reference lives forever.
So you lost me at the solution... You say it boils down to not being able to properly express the type of that `extend` function for the purposes of assigning it to a variable. I understand what you mean there, but I'm confused as to why that's the reason? I'm sure it's just something I'm misunderstanding, but since you have &'y &'x () as the type of the first argument in step 1, does that not automatically indicate that 'x must outlive 'y just due to how the borrow of a borrow interacts? I wouldn't expect to need extra syntax to fix this, it seems the compiler is failing to recognize this relationship between the two lifetimes, but maybe that's happening because of the HRTB and the lifetimes aren't known? But still I would think it should still have enough information to recognize this relationship.
There was another similar comment- basically the problem with that is that is now you have to have the compiler somehow search through all types and find these constraints itself. If I hand the compiler some time that somewhere deep inside it uses &'y &’x T it currently has no idea that 'x: 'y. Having it recognize this would mean every time you wanted to convert something it would have to search through and find all these constraints- and also you would have to make sure there’s no way to trick it by making types that could resolve to &'b &'a T eventually. Even if I do need an `&'y &’x T` for something there are plenty of circumstances where using an `&'static &’x T` instead is totally fine. Just because you have a reference `&'y &’x T` somewhere doesn’t necessarily mean there’s a function type signature relying on it to assume `‘x: 'y`. So you’d be being a bit too strict and break valid programs (currently accepted and should be accepted) even if you did implement it. Hopefully that clears things up a bit.
I don't understand this. If 'x: 'y is not proven then *any* instance of &'y &'x T is unsound because the underlying type of the &'y reference doesn't outlive it, and so you would be able to access dangling references. It's the basis of the lifetime model and therefore no programs with instances of these should be accepted.
@@SolarLiner That’s what I’m saying. You should have to have proof `'x: 'y` to construct the type. But unfortunately on the current state of rust there’s nowhere for that to go.
@@xenotimeyt yes, but then any program that is legal under this bug should actually be illegal and unsound, and so we shouldn't care that they'd break? I understand the need to gare this change under a new edition but there shouldn't be considerations for invalid programs that do compile due to a bug
@@SolarLiner Doing that would make it impossible to do basic things like map over an iterator or references- the function type would be `fn(&&Element) -> …` which desugars to `for fn(&’y &’x Element) -> …`. If you can’t store the constraint `'x: 'y` in the HRTB this would be illegal under that rule. Basically any function values involving nested references would be screwed.
The Rust compiler is thinking that way and treating `&'y &'x ()` like evidence that `'x: 'y`. No need to put another constraint somewhere. But this isn’t correct- because any `&'y &'static ()` is an `&'y &'x ()`. Any evidence that `'static: 'y` is clearly not evidence that `'x: 'y`. Hopefully that makes sense. :)
I don't understand why the compiler lets you do step 2, there are two generic lifetimes in the signature of weird, and for some reason it lets you pass 'static as the lifetime 'a in the witness but doesn't force the other instance of 'a (which should be the same lifetime) to match that, I think that's the actual issue here, since at this point all arguments passed with lifetime 'a are assumed to have static lifetime without any check. If there's a reason that this isn't the issue I'd love to know, I'm no rust expert.
You aren't passing in 'static or like substituting 'x for 'static. You're just making one of the parameters more restrictive. An `&'static ()` is an `&'x ()` the same way a `Cat` is an `Animal`. So what we're doing is really like converting an `fn(Animal, Animal)` to an `fn(Cat, Animal)`. No need to change both. :)
Thank you for this explanation. The type checker allows the type incorrect assignment to step1. So it's simply a bug in the type checker that in principle could be fixed. But then: Why has it not been fixed in all these years?
Because the correct type is something that you can’t express in Rust’s current type system. You *could* just disallow conversions like step1 which would plug the hole but also break all code passing around functions that take nested references.
@@xenotimeyt These type unsafe conversions must of course be forbidden in the next rust edition. Programs that use them are potentially unsafe. They will still compile for older editions. (cargo new --edition 2021)
@@holzmaurer1319 Where did you find that? Or is this something you’re saying? In which case I really *really* doubt that making it impossible to for example `map` over an iterator of references to would be worth fixing this not-that-that-serious issue.
I think the issue is with covariance of &'y: by itself 'static : 'x should not imply that &'y &'static T : &'y &'x T. More generally, R: S should only imply &'y R : &'y S if S: 'y as well (not sure I have the syntax right for the last bit).
The way things currently work the Rust compiler treats `&'y &’x ()` like evidence that `'x: 'y`. With this yes it’s correct to say that borrows being covariant is incorrect. But the real problem is the expressiveness one I mentioned in the video, if you force `'x: 'y` to be a constraint then you have nothing to worry about. Borrow covariance is an extremely fundamental rule that the type and lifetime systems rely on at several levels.
@@xenotimeyt I agree that the issue with breaking it up into steps where you preserve all the constraints (and possibly the compiler proving/failing to prove that this sequence of steps is sound) is an expressiveness problem. But I think once you've broken up it into the steps in the video the problematic coercion is in step2, where a function taking &'y &'x T is being coerced to one taking &'y &'static T without evidence that 'x: 'y. In other words, I think the correct (covariant) coercion rule is (&'y &'static T where 'x: 'y) : &'y &'x T, and not &'y &'static T : &'y &'x T, even though the first thing looks somewhat unnatural at first glance.
@@ronnodas Ok sure we can have that be the coercion rule- but how is the compiler supposed to know whether `'x: 'y`, these are generic lifetimes from the HRTB and we don’t even know what they are yet? So let’s say we’re pessimistic and just say that you don’t have proof 'x: 'y so you aren’t allowed to have nested borrows be covariant. This will break existing code- having HRTBs and nested borrows just won’t work properly. Even if by the time you actually substitute for the HRTB’s lifetimes everything is perfectly above board and the 'x: 'y constraint is obviously satisfied. The only way to actually distinguish between “you have a nested borrow and it’s fine” and “you have a nested borrow and things are about to go boom” is if we store constraints in the HRTB. Hopefully that helps. :)
Isn't the fix for that to instead of just checking the constraints on type cast, to also reintroduce any constraints that would be lost. That would be your "where x outlives y". And do that by taking the constraints of the original type, check that they hold for the target type, "BIND" the new lifetimes, created by the cast, by copying the constraints from the original types to also constrain the types they are casted to?
@@xenotimeyt Every cast. I believe the cast from a weird function to a function variable declared by a supertype of the type of the weird function is the one where the constraint was lost, if I understood the videos correctly. During the casting of `fn
@@tomasruzicka9835 Yes that’s right. So let’s say we are converting `weird` to `step1`, we need to make sure we have the constraint `'x: 'y` but we can’t put it anywhere- that’s the whole problem. In this case they don’t hold for the target type and there’s no way to make sure they do. You don’t want to break code that stores a `fn(&&i32)` in a variable (or passes it around).
I mean there honestly isn’t much to talk about for UAFs and all the ordinary vulnerabilities, if you’re really curious the code that *uses* `extend` is pretty understandable. The exception is their reimplementation of `transmute`. I might make a video about how that works tomorrow but we’ll see.
What I don't understand (I'm no expert, and lifetimes aren't intuitive to me yet) is why feeding fn(&'y &`static (), &'x T) -> &'y T into fn(&'y &`x (), &'x T) -> &'y T is allowed. To me it sounds like changing 'x to 'static in the first argument should imply also changing 'x to 'static in the second argument (and changing the first while not changing the second should be a compilation error) : only allowing to feed fn(&'y &`static (), &'static T) -> &'y T into fn(&'y &`x (), &'x T) -> &'y ,enforcing that 'x in the first argument is the same as 'x in the second, resolves this unsoundness doesn't it ? I feel like I'm thinking about lifetimes the wrong way, maybe a bit too much like regular types ? The way I see it, using actual types this time, no type checker would allow this: fn example (a: Type_A, b1: Type_B, b2: Type_B) -> Type_A let not_allowed: fn(a: A , b1: B1, b2: B2) -> A = example even if both B1 and B2 impl Some_Trait I see I'm missing something here to understand why this is allowed in the first place with lifetimes...
The two values don't need to live for the exact same amount of time. The reference just needs to live at least as long as x, and 'static lives forever.
@@michawhite7613 Yea, and I was wondering why "as long as 'x" is considered good enough in this case. To me it feels like saying: fn(&'y &`x (), &'x T) -> &'y T isn't the same as saying fn(&'y &`x (), &'z T) -> &'y T where 'x lives at least as long as 'z and saying the former really should mean (to my intuitive understanding) `x in the first argument lives just as long as `x in the second argument (same lifetime, no more no less). And if it did there would be no unsoundness here, as feeding this fn(&'y &`static (), &'x T) -> &'y T into this fn(&'y &`x (), &'x T) -> &'y T wouldn't be allowed (because it would drop the constraint that the first argument's 'x lifetime is the same as the second argument's ) Sorry if this isn't clear ^^'
We aren’t really doing anything with 'x, we’re just making the function more restrictive with its arguments by saying actually the first argument doesn’t just have to live for 'x, it has to live (in a dramatic movie villain voice) FOR-EH-VAAAAA! (Sorry about that I’m still more than a bit sleep deprived lol) But `'static` is special in that for any lifetime whatsoever `'q`, an `&'static T` is an `&’q T`. Just like a Cat is an Animal. So we’re really saying converting a `fn(Animal, Animal)` to a `fn(Cat, Animal)` which is of course safe and fine. Hopefully that clears it up. :)
@@xenotimeyt To me it feels like it makes it: - more restrictive: the first 'x cannot be any lifetime anymore, it now has to be 'static specifically but also - less restrictive: the second 'x doesn't have to live as long as the first 'x anymore, which I thought it did when written like this: fn(&'y &'x (), &'x T) -> &'y T (obviously I'm wrong since Rust doesn't complain ^^ ) To me, what you're describing is putting fn(&'y &'static (), &'x T) -> &'y T into fn(&'y &'x (), &'z T) -> &'y T where 'x: 'z , which I agree is quite logical to be allowed But I feel like fn(&'y &'x (), &'x T) -> &'y T should have the same meaning as fn(&'y &'x (), &'z T) -> &'y T where 'x: 'z and 'z: 'x , in which case putting fn(&'y &'static (), &'x T) -> &'y T into it wouldn't work unless 'x is also 'static . What I understand from your video is that even tho in both parameters the 'a lifetime is present, there is no hard constraint that both 'a lifetime are "the same" , they can be "the same" or any of them can be 'static which is where the unsoundness is. Thx for the video anyway, this is great food for thought (and thanks for trying to explain it again to me ^^' )
PS: Like I said, lifetimes aren't intuitive to me yet, and I have no real type theory background, so it might just be that I missed something about how lifetimes work in general, and ended up confused :'p
I'm not super familiar with ocaml, but as far as I can tell HRTBs sort of already behave like weak type variables. The value restriction is about generalization as in introducing a type variable (i.e. `None` is of type `'a option`), the what we're doing when we convert step1 to step2 is actually narrowing the type instead of generalizing it, and it's narrowing it in terms of a subtype relationship rather than introducing a type variable. Again, I'm somewhat of an ML noob so maybe it isn't what I think it is.
There are two things that point to step1 -> step2 looking "sus" in here: 1. if the existence of a value of type "witness" implies a nesting constraint in lifetime 'a and 'b that is missing from the type of weird. It could be implicit in the usage of &'a &'b in the signature parameter, but the nesting constraint must exist. In that case it should not be possible to convert step1 which has that constraint on the parameters to step2 which is unbound (because the &'a &'b is removed from the parameter. In other words either step1 has an implicit constraint that 'a is narrower than 'b, that should be replicated explicitly in the type of step2 that has no implicit double reference to assert that for the values to be compatible, or the restriction isn't there, in which case the argument types are not compatible (see point 2) 2. controvariance means that U->T can be converted into W->T if W is a subtype of U (that is if the values that are valid W are also valid U). In this example to allow controvariance between the types of step1 and step2 we have to require that for all possible 'x and 'y the type &'y &'static is narrower than the type &'y &'x. This is simply false if there is no restriction on the types 'x and 'y. If we take for example two scopes 'x and 'y such that 'x is not narrower than 'y the type &'y &'x is empty while &'y &'static has plenty of possible values. This problem with reference covariance is what I was pointing out in the last video, but in term of converting the argument types rather than the function type This means that either the language has to drop the assumption that the existence of a value statically typed as &'x &'y implies that 'x is narrower than 'y, and thus it should stop compiling the "weird" function until we add the constraint about the two types (or let the compiler add it for us, as it does for most of the lifetime stuff) or it has to drop the double reference covariance that allows values of &'a &'b to be converted to the type &'a &'c if 'c is narrower than 'b (and thus 'b is a subtype of 'c). The contravariance is not necessarily the problem here, unless this unsoundness is something that can only be shown in contravariant functions. I'll try to create an example with covariant values (e.g. generic traits and return values) to see if it is done properly there
1. You're correct that the types are not compatible. You shouldn't be able to say `let step1: ... = weird;`. But consider the function `fn blah(thing: &&i32) { ... }`. This is desugared as `fn blah(thing: &'a i32 &'b i32) { ... }`. If you (correctly) disallow these kinds of (unsound) conversions (like the step1 line), now there is absolutely no way to store `blah` in a variable or pass it to a function or anything. So just straight up banning this (unsound) conversion and walking away would break loads of existing Rust code. I know I've passed around functions with nested-reference-arguments a number of times. So you have to add the constraint to the type like I said in the video. Then everything would be nicely inferred and work out. 2. In a world with proper constraints you would have to prove `'x: 'y` to have the type `&'y &'x ...` in the first place. But in current Rust, without the proper fix, yes `&'y &'static ()` is not actually a subtype of `&'y &'x ()`. The problem is that you can get that it *is* a subtype only by applying some very fundemental rules the entire Rust type/lifetime system relies on. So all you need to get a paradox is: - `'static` outlives all lifetimes - 'a outliving 'b implies a borrow of lifetime 'a is a subtype of a borrow of lifetime 'b - Borrows are covariant (`S` being a subtype of `T` implies `&S` is a subtype of `&T`) - A borrow must have a lifetime that outlives the lifetime of what it holds So there are three options: a) Have a paradox and the system is unsound (where we are now) b) Break at least one of these fundamental rules (not a good idea) c) Make it so you can only have the type `&'y &'x ...` if you know `'x: 'y` (what the proposed solution would entail) So the two things you mention in the "This means that either..." paragraph are: 1. The compiler automatically adding the constraint `where 'a: 'b` to functions that take nested references like `weird`: Yes, this would fix it, but as mentioned above would also make it impossible to store, return, or pass around functions that take nested references (unless the proposed solution is implemented then everything would be fine). 2. Breaking the covariance of the borrow type would break loads and loads of existing code. So that's not a great way to go about it either. Hopefully that clears things up a bit. :)
@@xenotimeyt Maybe there is a compromise wherein you break double-borrow-covariance specifically for types _that contain lifetimes under outer "for" quantifiers?_ Only those types seem to contain the inconsistency, because they contain as-of-yet-undecided lifetimes 'x and 'y. If those were already known by this point, then the compiler could just directly check: "&'y&'x is a malformed type in this context, because 'x is less than 'y".
@@СергейМакеев-ж2н I feel like that could work but it wouldn’t be much of a compromise given the fact that most lifetimes are inferred and so end up in HRTBs. Any function with an plain old nested reference like `fn(&&i32)` is desugared by the compiler and is secretly `for fn(&'a &'b i32)`. All borrows have lifetimes- the compiler just usually figures them out for you. And if you’re doing this inside a function being stored or passed around, it’s using HRTBs and just not telling you. So that wouldn’t be a great compromise since loads and loads of code would fall afoul of that- just because you don’t type the `for` doesn’t mean it’s not there.
@@xenotimeyt Sure, but then, the only time one must actually *use* those quantifiers, is when the function is converted to a subtype (by contravariance) *without* substituting known 'a and 'b first. That is the case that, we both agree, should be straight up invalid.
@@СергейМакеев-ж2н The problem is that both HRTBs and converting nested borrows to subtypes both happen all the time. Here's a very simple toy example I made that obviously should compile, but under your compromise wouldn't: play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f74e0b996501d2fd49672553cf9e4cc3 Obviously that's a toy example but there's loads of code like this so hopefully that makes it clear why we can't just change the way borrow subtyping works all willy-nilly. :)
Isn't the problem in step2? The line step1 still has `&'y &'x ()` which implicitly says that `'x` outlives `'y` (for the same reason that `'a` outlives `'b` in weird), but step2 has `&'y &'static ()` instead which doesn't imply any relationship between `'x` vs `'y`. Ideally you would have to write `let step2: for &'y T = step1` which would have the lifetime constraint that was previously implied, but no longer, in it, which would solve the problem.
@residual-entropy I know, I was trying to ask whether the issue was actually on line 2 (like I thought) as you seemed to say it was on line 1 (8:40). Edit: additionally, if you change `step3(FOREVER, borrow)` to use step2 it compiles, but it doesn't with step1 due to lifetime not living long enough, which indicates to me that the issue is what it does after step1 in step2
@@HamishArb It’s only natural that when using only `step1` it doesn’t compile- that’s where you actually exploit the soundness bug. The only rules we apply to move from step1 to step2 are: 1. Function arguments are contravariant 2. Any `&'static T` is a `&'x T` for any 'x Both of these are pretty fundamental assertions. (Breaking them technically could be a way to solve this issue but would require a huge amount of change to everything since you’ve changed how subtyping works) Hopefully that makes more sense. :)
I'd like to expand on this topic a bit more if I can Maybe treating a 'static as a `subtype` is indeed what is the root of the problem I understand that the 'static is a valid type of `any` lifetime 'x, as the 'x exists for all lifetimes we provide, the problem is that we expect a supertype 'x to exist for any lifetime of 'x provided. As the 'static is a very special lifetime that is the supertype of all lifetimes, this premis is broken, the function that took in any lifetime of 'x and below, now takes any lifetime and 'x and below, where the compiler believes that 'x will encompass 'static As 'x will never be a supertype of 'static, we could just throw an error to force the 'x to be a 'static, No additional logic would be needed, just ensuring that in the 'x bounded to 'static, any other 'x is 'static Or we could require the 'x's lifetime to be proven to contain the 'static lifetime 'static is a special case after all. I don't think that there's any loophole other than that, but feel free to correct me TLDR; Why not just throw error when the 'x is bounded to 'static and any other lifetime
When do we say does 'x have to outlive 'static? If that were the case then yeah the compiler would be able to tell no such 'x is possible, but I don’t think we ever say that.
@@xenotimeyt We pass in 'static as 'x in step2, so now the only valid lifetime of any 'x of step1 would be 'static, that's what I meant. It's basically where the bug is manifested, in the idea that 'x could be anything other than 'static when any 'x lifetime is initialized with 'static lifetime
Is it more complex than C++? Arguable. More bazookas to shoot your legs off with? Huh? The rust compiler is constantly checking everything you do so you can’t shoot your legs off. That’s like the whole point. Shooting yourself in the foot in Rust isn’t easy, in C++ you shoot yourself in the foot by looking the wrong direction. Don’t get me wrong I’m no way saying Rust is always better than C++ or C or anything. But I really don’t think your second clause is true at all.
@@xenotimeyt You are clearly able to manufacture such bazookas in this apparently 'safe language'. The compiler passes these off as safe. Isn't this much, much worse than having the compiler give you no guarantees so you have to avoid such pitfalls yourself? In C++, you have been warned. In Rust, you might have a false sense of security. How is this better?
@@JSzitas In Rust this is a) incredibly unlikely to be triggered on accident and b) a bug. So the only real way in which Rust is worse is that if you somehow manage to trigger this accidentally, you may well have that false sense of security. But I don’t think “crazy lifetime voodoo can trigger memory unsafety - it’ll get fixed eventually” is worse than “looking the wrong direction can trigger memory unsafety”. As for the “false sense of security”- that’s a big part of why I made these videos! All you really need is to have in the back of your mind that it’s possible to break memory safety with crazy lifetime voodoo. Hopefully that all makes sense. :)
@@xenotimeyt I agree with much of what you say, but I disagree that it is so easy to create bad memory bugs in C++. If you are using modern tooling (an IDE) and follow at least some modern conventions (do not use raw pointers willy nilly without reason), you will not run into many memory errors unless you explicitly try to make them. If the case is the same for Rust, why bother? That's the gist of my argument. I write C++ professionally, in a multimillion loc codebase that does some voodoo for sake of performance. I would still say we are rather okay most of the time. I think many C++ programmers will feel the same. If the tangible benefit is 'your code might be safer maybe if you avoid voodoo', then we have been used to that for a long time. Why switch?
@@JSzitas If it’s so easy to not have memory safety error when why are there so many CVEs and security issues caused by memory safety issues? Surely the entire android team uses “modern tooling and at least some modern conventions” but they still mess up a ton. If you’re making a program to generate graphs with very well specified input only you will ever interface with, it’s probably not a huge problem (although debuggability sucks). “This code breaks if you set this config flag to eight characters where the last is a backslash” isn’t a problem. So there are some cases where this isn’t a huge deal. But for a part of your web service that processes random http requests, hippity hoppity your server is my property. Of course someday that plotting code might just be integrated into a little web dashboard and….
You can see on the projects tab its listed as "blocked on -Ztrait-solver=next": github.com/orgs/rust-lang/projects/44 I found this: rustc-dev-guide.rust-lang.org/solve/trait-solving.html but that talks about `-Znext-solver` and not `-Ztrait-solver=next` like on the project board so I'm not totally sure this is the same thing. Regardless it seems like they're waiting for the next trait solver to be stabilized before adding new behavior (such as something like where in HRTBs to it). Again though I do not actually work on the compiler and for a real answer you would have to ask someone who does. Let me know if you do more interesting details tho. :)
I mean that would just be syntactic sugar for / the same as `for fn(&'b &'a (), &'a T) -> &'b T`. Just like with for example trait bounds you can say either `fn blah...` or `fn blah... where T: Trait ...`. They're the same thing. :)
@@ghostx-z8m There's absolutely nothing wrong with a borrow `&'q &'q T` (they come up all the time tbh). Both the thing you're borrowing and the borrow itself last for exactly the same amount of time. No problems there. I sometimes say "outlives" when I really mean "lives at least as long as"- sorry if that tripped you up. With English had a single word for "lives at least as long as" lol. But yeah `'a: 'a` should always be true, if it wasn't then converting an `&'a T` to an `&'a T` would be illegal even though it obviously isn't. Hopefully that makes some sense. :)
@al-entropyMy point is that you should be able to say that a should be strictly less and not less or equal to avoid the replace a with FOREVER and break the model. Something like this... Currently for 'a
@@ghostx-z8m If you can say `for where 'a: 'b …`, with the ordinary version of of `:` meaning “lives at least as long as”, *and* you have the compiler properly infer the type of functions with nested references (like `weird`) as `for where 'a: 'b …` and the like, this is fixed and there are no problems. Are you saying that adding a “lives strictly longer than” constraint is necessary to fix the soundness issue? It isn’t. An ordinary constraint would live until step3 when you substitute 'a and 'b, and then the compiler would force you to say `'a: 'b` and everyone’s happy. Sorry but I don’t think I understand what you’re getting at (if thats not it).
Well yeah presumably it would work like trait bounds where you can say 'x: 'y where its declared or in a where clause. But you’d actually have to add that to the type/lifetime system first.
@pyyrr Depends on what you're doing. If you're only single threaded, then idiomatic C++ is indeed easy and quite safe. It's when you get into multi-threading that you realize every single language is an annoying PITA and it's a lot more difficult to reason about the code.
@@anon_y_mousse The Android developers found that the majority of bugs they had were caused by memory safety, and this issue went away when writing Rust. Is that also biased? What possible reason would the Android developers have to make their lives worse?
@@anon_y_mousse I didn’t even mention a survey so I don’t really know what you’re talking about. Memory safety bugs happen often and when they do they’re really really bad.
First off, I am not deep into rust. I have solved a few AOCs with rust and thats about it. I get the problem which is described here but I am still confused how this could have happened in terms of language design. The fundamental problem as I understand is that part of the type of the lambda is bound late at the callsite while the object represeting the lambda needs to be partially instantiated before that. But why wouldn't you craft this process in the same way normal functions work? Why leave this kind of loop-hole of lifetime checking? I mean, I wouldn't have noticed this of course but I am not a rust language designer. Shouldn't you ask yourself why you are dropping lifetime checks in the lambda case compared to a normal function immediately as a language designer? I am not trying to accuse rust language designers of being stupid or something. I just have a hard time understanding how this went under the radar. Do we know how this happened?
Can’t say for sure but it’s not like you’re dropping lifetime checks- there’s this one feature which is that if you have nested references the compiler automatically infers certain bounds and then this other feature where you can store functions abstracted over their lifetimes with HRTBs. Turns out there’s no way to keep put those constraints in the HRTB version. I doubt many people even knew that Rust auto-infers nested references => weird implicit lifetime constraints in this way. I certainly didn’t. But then someone who did had to realize wait so what happens to those when we decide to throw them in an HRTB- and remember that Rust has been evolving a lot since 2015 and the fn trait I’m pretty sure didn’t work quite the same back then. So there’s no weird exception, but just the classic not thinking of a specific way in which two features interact. And don’t forget that the whole idea of a lifetime system at all was basically completely new to the programming world (save some academic research into linear and session types and whatnot). Most of the quirks of Rust’s lifetime system are usually well-hidden, but having everything be mostly seamless for simple programs makes it quite the beast. It’s not just implied bounds from nested borrows and HRTBs but splitting borrows, complex elision rules, GATs, PhantomData, unbounded lifetimes, integration with type inference, and a thousand other things. It’s easy to look at two puzzle pieces when you see them in a video and think “why did nobody think of putting these puzzle pieces together”, when in reality it’s a 4,000 piece all-white puzzle. Not trying to say you’re being a jerk or anything, but I’d say it is always good idea to try and think about ways in which things can be more complex than you realize. ;)
@@xenotimeyt I get what you are saying of course. I wouldn't have expected anyone from catching a problem like this based on type and lifetime inference by the compiler of course. But you can write it down explicitly as you demonstrated and it still fails. Its not something the language prevents you from doing explicitly and only happens because inference does something funky. That is what I was thinking about. I simply don't know enough about the true complexity behind the scenes of course, and I can absolutely understand such a mistake creeping in in an iterative process where each individual change on its own seems fine across a long time period but they eventually end up adding up to such a complex bug. If that is kind of what happened then this is just a thing that is going to inevitably happen in any project eventually of course. Your explanation in the videos just sounded to me like lambda object arguments were flawed in this way since their inception. But I guess that was an unfounded leap.
@@lexer_ > Your explanation in the … unfounded leap. Sorry if I mislead you there, but yeah you only need to be able to stick where clauses on HRTBs to accommodate these weird inferred nested reference lifetime bounds. If those didn’t exist things would be fine (well you wouldn’t be able to pass around functions that take nested references but in terms of soundness it would be fine).
Who would have thought that having layers upon layers of unnecessary and near opaque complexity ... would lead to hard-to-find bugs and safety issues ? The blatant propaganda around Rust is getting way out of control.
I’m absolutely on team “Rust is flat out too complex and it’s a problem”. Unnecessary though? Rust lifetimes are a chaotic mess but they solve a problem. Memory safety without a runtime cost. And propaganda? There are lots of people who are happy users of Rust anyway. Just because something has its fair share of problems doesn’t mean there can’t be loads of people who like it anyway. I’m one of them.
@@xenotimeyt I do agree with your approach about lifetimes - they are orthoganal to type-types, and could be implemented independently. Re propaganda vs happy users. Yeah, of course ... tools are tools and they all make happy users when used for the correct application. There is still a culture of fanboism though around a lot of new tooling - Rust included. Pretty much none of that comes from actual programmers though - its mostly ... other types of ppl who seem to think they can benefit from a certain tech "winning" or something dumb like that
@@steveoc64 Thanks for the civil reply. I agree with all of that, except the “culture of fanboyism”. That fanboyism definitely exists but from my particular view it isn’t that prevalent. The Rust community has fanboys (of course) but it just seems like you’re trying to insinuate (mostly in your initial comment) that the Rust community is just a bunch of dumb fanboys who shouldn’t be taken seriously. And I really don’t think that’s the case. Also in my mind “propaganda” means some sort of organized effort to spread wrong or misleading information, and I definitely don’t see that at all with Rust. But maybe this comes down to me not being on Twitter lol ;)
![Seungmin "그렇게, 천천히, 우리(As we are)" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/kAzmhLHePqU/mqdefault.jpg)
![BLACK BAG - Official Trailer [HD] - Only in Theaters March 14](http://i.ytimg.com/vi/Du0Xp8WX_7I/mqdefault.jpg)
this problem makes me think of those proofs that 1 = 0, where each individual step is correct but the complexity hides a division by zero
Fr it actually is a lot like those.
They’re totally correct tho, it’s a shame the government keeps suppressing them. /s
_"each individual step is correct"_
No, that would make the proof valid.
@@user255 ok I assumed they meant “looks correct”
@@xenotimeytHmmmm.... maybe.
@@user255 probably
that white house press release at the end mentions rust by name for those who dont know :)
great video! i watched both and you explained what the problem is and why the problem is very well
Can confirm that it does. Glad you enjoyed it!
This is just a sidebar, but I honestly wish they would decouple lifetimes from types in a way that is similar to what ocaml did. The lifetime system should act like its own typing system orthogonal to the normal type system and this would allow them to also expand the rules for lifetimes to allow for explicit cases like this without effecting the type system itself and visa versa.
That’s something I really do agree with. We ended up with kind of a worse version of colored functions- with infinite colors. It’s just like here’s a variable. It has a type which is described what you can do with it. It also has where it is and how long it exists. I think of those as pretty much orthogonal. Just my thoughts tho, I don’t even know much ocaml (yet).
Where can I read about lifetimes in OCaml?
@@tee1532 I learned about them from the "Oxidizing OCaml" posts: blog.janestreet.com/oxidizing-ocaml-locality/
I'm not super familiar with them tho but I do understand the basic idea.
Note that, from what I understand about OCaml, Local variables are the equivalent of only being able to have lifetimes at argument position, and disallowing all lifetimes on return types. This would make a LOT of Rust code impossible to represent using that system.
I think Local variables are a good fit for OCaml because they're trying to make a GC managed language more efficient by using the stack more, and lifetimes on arguments are perfect for that use case.
However, if we tried to retrofit Rust with Local variables, it would have the opposite effect; people would start cloning and using the heap more in order to erase the lifetimes on the return types.
@@jedel1043 Obviously you couldn't just take ocamls implementation and run with it in rust, though I dont think it would be as bad as it seems to have only two types of explicit lifetimes; global and local in rust, with global having a scope beyond the original scope of the variable etc. Instead what I'm suggesting is to make the current system of rust lifetimes detached from the types. Right now a 'static T is a different type from a 'a T which leads to some silly implications. It adds needless complexity to a type system which is already somewhat complicated in strange ways. For example, rust's generics are not really true generics, they are trait arguments and this also ends up creating some weird edge case problems. I've used rust for 6 years in prod and I still run into trouble at times when it comes to the type system and the lifetime system because it's not as intuitive as you'd expect in certain corner cases. This wouldn't be impossible and it would also simplify the way the compiler checks and expands the code. The other nice side effect is that you could rewrite the lifetime system without having to worry about the type system.
This is not programming, this is like writing poetry
Poetry with long lines
What is programming, if not writing poetry?
The amount of enthusiasm about rust... Liked and subscribed 🤝🏾
Glad you enjoyed it! :)
This video is not endorsed by the Rust Foundation.
-Yes? That’s correct?- I’m dumb lol
@@xenotimeytIt's a joke about some dumb trademark thing the rust foundation was trying to do a while ago
@@jameswise9171 oh I totally forgot about that lmao.
Yeah I’m actually in legal trouble for using their logo. Watch the video again so I can get to monetization and pay my legal costs pls.
Damn what a rabbit hole this comment led me into. Seems Rust is ran by a bunch of tyrannical political activist weirdos. Why do ppl bother with this language? Also no wonder the White House loves it so much.
I consider the error to be the conversion of only one 'x to 'static, and leaving the other 'x to be 'x.
The function definition makes use of two lifetimes, 'x and 'y. Buy in step2 we make use of three lifetimes, 'x, 'y and 'static. The compiler lets us change the definite lifetime of 'x into 'x and 'static. Of course it then later gets confused which of the two actually applies to the output of the function.
@connemignonne already mentioned it as well.
There’s absolutely nothing wrong with switching out only one 'x for static. The type `&'static T` is a subtype of `&'x T` just like `Cat` is a subtype of `Animal`.
It’s just like converting a `fn(Animal, Animal)` to a `fn(Cat, Animal)`. We’re not substituting one for the other like when we substitute 'a for 'x and 'b for 'y in the last step. It’s just subtyping.
Hopefully that makes sense.
@@xenotimeyt
To me this feels strange. When I use the same name of lifetime ('x) in multiple places, it should mean that the lifetime behaves in the same way; all 'x have the same relationship to all other lifetimes. When the function has type `for fn(&'y &'x T, &'x T) -> 'y T` it implies that `'x: 'y`. When I than change the lifetime 'x to different things on different places, I break the implicit relations.
In the examples with animals, it is same as having `Type` and `fn(Type, Animal)` here `Animal` is implied to be `Bird` because it is used with `Type`. Changing it to `fn(Type, Animal)` is than wrong because the implied constraint is no longer there.
For this I see two solutions:
- Don't allow conversion from `fn(Type, Animal)` to `fn(Type, Animal)`. When subtyping you must subtype everything consistently in the same way.
- Don't allow `Type`. All constrains must be explicit. In the case of the rust code, that would mean that the constraint `'a: 'b` on the function `weird` would be part of the
type of the function, which would than disallow the conversion of `weird` to type `for fn(&'y &'x T, &'x T) -> 'y T` because it doesn't have the constraint.
@@bonny4 I see. The first case is totally legal and should be because borrows are covariant. Just like any `Vec` is a `Vec`, any `&'y Pidgeon` is an `&'y Animal`. This is a very fundamental rule loads upon loads of existing code depends upon. If any `S` is a `T` any `&S` is an `&T`.
As for the “you must subtype everything consistently in the same way” part this just doesn’t make sense- there’s absolutely nothing wrong with treating a `fn(&Animal, Animal)` as an `fn(&Pigeon, Animal)`. Why would there be? All we’re doing is making the first argument more restrictive- it can’t just be any animal now it has to be a pidgeon.
Your second solution is the actual solution mentioned in the video- kind of. Right now doing that (disallowing the broken conversion from `weird` to `step1`) would break all existing code that uses nested references in functions getting passed around. Want to `map` on an iterator nested reference? Well now you have a `for fn(&’x &'y S) -> T` that you need to create and then pass around. Because you can’t put the constraint `'x: 'y` anywhere it’s not legal to construct this. Loads and loads of code is again broken. :(
But if you implement the solution from the video where you can add constraints to `for …` expressions, now you can make the constraint explicit (basically do what you said in solution #2) and existing code will be fine.
Hopefully that clears it up. :)
@@xenotimeyt I don't think there is a way to fix it without any breaking changes, in the end fixing this would break at least cve-rs :). The main focus of my comment wasn't on how to fix it without breaking changes, but why conversion from `for fn(&'y &'x T, &'x T) -> &'y T` (`step1`) to `for fn(&'y &'static T, &'x T) -> &'y T` (`step2`) may be bad.
I'm still not convinced that the conversion is ok. The problem I see is that in `step1` the two occurrences of 'x are connected - the first 'x constraints what the second 'x can be; it makes the second 'x more restrictive. By converting to `step2`, we are making first 'x more restrictive (that is ok), but we are breaking the connection of the first 'x that was restricting the second 'x. So after conversion to `step2`, the second 'x is less restrictive, and that is the problem I see.
By "you must subtype everything consistently in the same way" i meant that you cannot restrict (subtype) something in one place, when it would cause some other restriction to be lost.
In animal example, `fn(Animal, Animal)` is not representing `step1` very well. It doesn't show the connection between the two animals. I think that more accurate way would be to show it as `fn(A, A) where A: Animal`. It is ok to convert `fn(A1, A2) where A1: Animal, A2: Animal`, to `fn(Cat, A) where A: Animal`, but it is not ok to convert `fn(A, A) where A: Animal` to `fn(Cat, A) where A: Animal`.
Another way to show that the problem may be in the conversion from `step1` to `step2` is that calling `step1` at the end of the `extend` function still gives error but calling `step2` does not. This can mean that some restrictions are lost in this conversion.
I think that there are two equally valid ways to look at where the problem is: in the conversion from `weird` to `step1` or in the conversion from `step1` to `step2`.
This is partially speculation on my side:
I don't think that just disallowing the conversion from `weird` to `step1` would be break as much code because in many cases (probably majority (including `map`)) traits are used to pass functions around instead of types 'pointer to function', so there are no conversions. And if I understand it correctly the problem may occur only when there is conversion.
But I very much agree that the best way to fix the problem is what you suggested in the video.
What does "subtype" mean in the context of lifetimes 'x and 'static? It seems like it's abstract vs concrete. Making it concrete in one place but not others breaks the consistency of 'x.
I think the key point that causes this is, when we call the weird function directly, the compiler checks the lifetimes properly because it knows 'b must outlive 'a in order to "cast" a &'a T to a &'b T. But that constraint is not part of the function's signature (and as such, its type), so when we store it in a variable the lifetime constraints are only the the ones in the functions signature/type. The fact that 'b must outlive 'a is "forgotten"
The way to fix it would be to disable magical inference of a lifetimes constraint from the mere presence of a parameter. If the user is forced to specify the constraint manually it will need to carry over and prevent function casting at some point.
Well yeah that would be a solution but now we have no way to store any function that uses nested lifetimes in a closure or even by just storing a variable in a function.
Sure for a declaration we can make the programmer say `fn …(&'b &'a …) … where 'a: 'b …`. But now we can’t store that in a variable without having to specify a single lifetime. Which would make it impossible to use that (function stored in a variable) with arguments of different lifetimes.
An fn type like for
@@kocsis1david Well remember that these constraints are all over existing code with nested references. Forcing any of it to be explicit would break that existing code. Also adding constraints to HRTBs is definitely not easy, it’s one thing to write down an idea for how something should work but another to go through every possible case in such a complex type system and another still to implement those into the beast that is the Rust compiler.
@al-entropyYou don't need to force people to be explicit about their closure or function variable types. Type like: for
@@niceshotapps1233 Sorry if I misunderstood- yes the compiler would infer `for
0:11 Is it just me who read ‘twas as the name of a lifetime?
lmao you’re in rust mode
finally I can write C in Rust🚀
Hmm, just writing unsafe Rust is still easier than hacking safe Rust. 🧐
@@jongeduard yes but I love Rust so much because it is not only blazingly fast🔥 but also exclusively safe🔒
@@sunofabeach9424Adding seat belts to a car doesn't ensure you won't get killed.
@@HansBezemer while my comment was ironic, your analogy is completely wrong. Rust's type system is designed around being completely safe. No exceptions. And the type system is actually safe, this thing shown in the video is an implementation bug and it must be fixed. It is not something you should be aware of and work around. If there will be a compromise, it won't be Rust anymore
@@sunofabeach9424 Yeah, I heard that about Ada, BASIC, Pascal and Java as well. You should read what Dijkstra wrote about "safe" languages. Quite enlightening.
So, this is mostly an issue in that this can end up in packages that you include in your codebase. Most people using Rust wouldn't want to do this themselves. Maybe, instead of fixing the issue in the language, they can just add some linter rules to the compiler to disallow this particular pattern, or set up some pattern checking in cargo to reject packages that use this.
This isn’t something you can easily detect as a linter, at least not without reimplementing the way the compiler determines implicit lifetime bounds….
The problem is fixable then like you said, by encoding the lifetime restriction as additional data (the where syntax, but the compiler can actually do this implicitly here) in the function symbol at step1. Then, when you attempt to call the function, it will see that the assumption must hold that: ``` 'a : 'static ``` which it can immediately deny the function call for in this case. The original problem by the compiler is allowing the lifetime restriction from the witness to inform the conversion of the other variable, when that information can be deleted.
Yep, although it doesn’t really matter whether you’re doing it implicitly or not. You still have to be able to construct these constrained HRTBs and adding those to the type/lifetime system is the real challenge.
6:59 now *this* is the step i dont get
if the function signatur was just "(&'y&'x()) -> &'y T" then i totally understand why contravariance would allow you to change that to "(&'y&'static()) -> &'y T". likewise, i can understand why the full signature "(&'y&'x(), &'x T) -> &'y T" could be changed to "(&'y&'static(), &'static T) -> &'y T"
but now, how the heck is "(&'y&'static(), &'x T) -> &'y T" also valid??? for any 'x ≠ 'static, this new signature would not be contravariant with the original one, would it?
i feel like my intuition is betraying me here but i dont understand how, what am i missing?
We aren’t substituting 'static for 'x - it’s no different really from converting an `fn(Animal, Animal)` to an `fn(Cat, Animal)`. No need to change both.
So what you’re saying is “call this function, but with the added constraint that that first ‘x reference should actually also be ‘static.”
@@NStripleseven yep!
@@xenotimeyt shouldnt it be more like substituting a fn(T, T) for a fn(i32,T) though? that substitution shouldnt work, as the original wouldnt accept parameters of type (i32,f64) but the second one would
but yet it seems like rust accepts something like an fn
@@paulamarina04 You arent substituting- it’s just subtyping. Any `&'static T` is a `&'a T` just like any cat is an animal.
The difference is that all `i32`s are not `T`s. Could be but if you choose `T` to be `f64` obviously it’s not.
However the reference type `&'static` is special in that *all* references of any lifetime are subtypes of it. That is to say, for any lifetime `'q`, even one where we know nothing whatsoever about it, an `&'static T` is always an `&'q T`.
Hopefully that shows you how it’s like `fn(Animal, Animal)` becoming an `fn(Cat, Animal)` and not like your `fn(T, T)` example.
But to add one more thing imagine that Rust had a type `!`, which is convertible to any type whatsoever. How is that possible? Well you can never construct `!`. It’s a type you can never construct, so it’s convertible to anything. I need an `i32`. Take a `!`. This is totally type-safe because I’ll never actually get there. Functions that never return (like that infinite loop) return `!`. So you can say `let z: i32 = never_returns();` and there’s no problems because it will never return.
The `!` type (which is a real thing but not stable yet) is the type version of 'static. It can freely be used as any other type the way 'static can be used as any other lifetime.
What we’re doing is like converting a `fn(T, T)` to an `fn(!, T)`. No matter what `T` you choose, `!` will be convertible to it, so our first argument will be a valid `T`.
I think you're maybe (?) not right when you say the problem is in the let step1:... line, because the type signature of the witness should still implicitly enforce the correct lifetime constraint: taking a &'y &'x () still tells us everything we need to know about the constraint implicitly without the kind of explicit where-constraint you suggested
It feels like the constraint is lost in the let step2:... line (because the witness was still forcing it for step1 but no longer is on step2) and I don't see why the borrow checker couldn't in principle recognise that and complain at you for contravariantly generalising just one instance of 'x in the type def to 'static where this drops a constraint that previously existed
A solution that might be too heavy-handed but I think should work just fine is if it forced you to demote all instances of 'x to 'static in the definition when performing this generalisation. This fixes it, right? (e.g. we instead only allow step2: for &'y T = step1 and we complain if you try to only generalise some instances of 'x but not all of them). although I'm sure there's a less intrusive way of doing the same thing - just check whether the definition of step1 has an implicit constraint (like the one given by &'y &'x ()) that step2 doesn't when doing the type coercion. what am I missing?
> _"don't ask me why I did that"_
_ok_
lol
6:40 I don't understand why/how "this is completely valid". The signature defines the lifetime 'x to be the same for both parameters. If we change the lifetime 'x of the first parameter for a more specific version, it should require that the lifetime 'x of the second parameter be changed in the same exact way. Is the fix could be that simple?
A ton of people have misunderstood this. We aren’t setting 'x to 'static, what we’re doing is like converting a `fn(Animal, Animal)` to an `fn(Cat, Animal)`. All cats are animals, and in the same vein, all &'static () values are &'a values (for any 'a).
Hope that helps. :)
@@xenotimeytI understand that's why rustc is allowing this but from my understanding this isn't right. So without the need to express a full "where" relationship, the signature already defines that `fn(Animal, Animal)` where both `Animal` are the same, so it should not allow `fn(Cat, Animal)` but only `fn(Cat, Cat)`. I'm trying to understand if there is a way to fix the issue without a new rust edition.
@@OlivierEble It’s not that both lifetimes are the same- Rust lifetimes are sort of upper bounds- a reference having lifetime 'x means it lasts at least as long as 'x, not exactly as long as 'x. So narrowing only one to 'static is fine.
@@xenotimeyt thx for explaining that's clear now.
@@OlivierEble Np, glad it made sense! ;)
wow what are the prospects for adding constraints to the lifetime relationships in an upcoming rust release? how is this understood as a safety problem by the rust dev team?
If you go to the GitHub issue it links to a projects board with all the major unsoundness issues they’re working on ( github.com/orgs/rust-lang/projects/44 ). It seems like they’re working on a new trait solver and are waiting to address this until that gets done.
They’ve clearly acknowledged it (and have since 2015).
Disclaimer tho: I don’t actually work on the compiler or anything so I can’t say anything for sure.
Say we have two function pointer types:
`for fn(&'a &'b (), &'b T) -> &'a T`, which implies that 'b will outlive 'a.
`for fn(&'a, &'static (), &'b T) -> &'a T`, which implies that 'static will outlive 'a, but says nothing about which of 'a and 'b will live longer - so 'a might outlive 'b for this type.
I would assume that in short, the problem is that Rust's type system is currently not capable of checking these implied lifetime bounds, so it permits you to cast the former into the latter despite the former having a lifetime bound that the latter does not satisfy. Right?
I mean yeah checking these implied bounds would be very very difficult. It would be *a* solution but it definitely isn’t a great one.
(I don’t have an example to hand but you could probably make finding these derived bounds almost impossible using GATs also, but that’s just a thought and a whole mess I don’t want to have to explain in yt comments lol.)
The only place that checks whether 'a outlives 'b is the return of borrow in weird because there the compiler knows that borrow's lifetime is 'a and we're supposed to return 'b, where _witness adds the outlives constraint.
So it's sort of a loop of 3 links:
1. _witness says 'a outlives 'b
2. we take 'a and return 'b
3. what we take and what we return is the same thing
And when you assign the function you're losing the point 3
I don’t think “what we take and what we return is the same thing” is an actual constraint anywhere. Really the only requirement of a return ever is that it outlives the function. So in this case the caller chooses 'b, and because we’re returning a value that secretly has a lifetime 'a that means 'a: 'b. The fact that _witness ensures 'a: 'b just sort of implicit and not something that is checked as part of the `for fn(…) -> …` type.
When we assign the function we lose the constraint 'a: 'b, since the way rust is designed the responsibility of checking that is given to callers, but here we have no callers (not until after we mess up the type enough to have that constraint be nowhere to be found). Hopefully that makes sense.
It seems to me the problem is the contravariance conversion is incorrect. The type system is saying that `'static` is a more specific lifetime. But in reality is actually a more generic lifetime
Contravariance of arguments says:
> If any A is a B, then any fn(B) is a fn(A)
Any `&'static T` is an `&'x T`, so any `fn(&'x T)` is an `fn(&’static T)`.
It definitely feels weird because `'static` seems like the most general lifetime, but it’s actually the most specific. The `'static` lifetime is a single special case of any lifetime.
The more specific lifetimes outlive the more general lifetimes. Sounds weird, but 'x *really* means (anything at least as long as 'x) which is much more general than (anything at least as long as the whole program).
Hope that helps. :)
@@xenotimeytActually yes I retract my former statement. I would then suppose one of the problems is the multiple 'a and 'b become uncoupled during the conversion.
Since this would make sense if all 'a and 'b were 'static. But falls apart when only some are and some aren't
Im sure there is a good reason for this but I dont know it
@@thomasjhickson Glad it made some sense.
The thing is that 'a and 'b and 'x and 'y and 'nyancat are all special cases of 'static. No lifetime include more lines than “the entire program”. So `'static: 'a` for *any* 'a whatsoever- and the compiler knows that.
@@xenotimeyt I found that explicitly adding `where 'a: 'b` to the original function stops this conversion. Why is the implicit bound treated differently from an explicit one I wonder. The original function has implicit requirements that are not reflected in the conversion. I would be interested in a video with an example of why this exists! (assuming there is a reason)
In the video you say its an expresiveness problem, but that doesnt explain the difference in the way the bound is treated. Also if it was purely an expresiveness problem then it should just not compile. It compiling is a soundness problem. But I suppose thats why its a bug
@@thomasjhickson Rust can’t currently encode lifetime constraints in HRTBs, so there’s no where for the explicit constraint you added to go. So rust complains.
On the other hand Rust doesn’t notice the `'a: 'b` constraint if you don’t say it explicitly. So it lets you convert just fine.
Sure I think you could say that the fact that it doesn’t notice that constraint is the problem- but if you run with that and try to fix it you’d break totally fine existing code.
Like I think you’re imagining the compiler just throwing `where 'a: 'b` at the end of these kinds of functions, but now you’re not allowed to store *any* function that takes a nested reference with generic lifetimes in a variable.
That seems more specific than it is, any function that takes a nested reference where you don’t bother specifying the lifetimes, i.e. `fn blah(x: &&i32) {…}`, is treated by the compiler as `fn blah(x: &'a &'b i32) {…}`. So you couldn’t have a function pointer to `blah` with only that fix.
Why do you have to explicitly type for
Rust can infer that you mean `for`).
@@xenotimeyt Haskell enforces capitalized types and typeclasses so there's no ambiguity
@@Turalcar Ah right I forgot about that- good catch. :)
ive asked this in the last video, and i still dont get it: why step 2 is valid? how can you say that &'y &'static is a valid &'y &'x in the context of the function definition? clearly the second parameter must have the exact same lifetime as the second reference! i mean, &'y &'x doesnt exist by itself in vacuum, it is only a valid statement as long as the second parameter is &'x. For example, fn(T, T) must check that the first argument and the second have the same type, but here its completely ignored?
Whatever gets passed into weird just has to live at least as long as x, and something that lives forever will always last as long as x.
@@michawhite7613 I don't have a problem with that, I have a problem with coercing the type of step1 into the type of step2, the second is clearly more permissive than the first
@@michawhite7613 by your logic, fn(Animal, Animal) is coercible to fn(Cat, Animal)... Oh wait wtf it's right
Step2 is less permissive than the first. 'x only has to live for a particular period of time. 'static specifically requires that the reference lives forever.
Ooooooh
the pokemon bit was super random, you won another sub
Glad you liked it lol :)
So you lost me at the solution... You say it boils down to not being able to properly express the type of that `extend` function for the purposes of assigning it to a variable. I understand what you mean there, but I'm confused as to why that's the reason? I'm sure it's just something I'm misunderstanding, but since you have &'y &'x () as the type of the first argument in step 1, does that not automatically indicate that 'x must outlive 'y just due to how the borrow of a borrow interacts? I wouldn't expect to need extra syntax to fix this, it seems the compiler is failing to recognize this relationship between the two lifetimes, but maybe that's happening because of the HRTB and the lifetimes aren't known? But still I would think it should still have enough information to recognize this relationship.
There was another similar comment- basically the problem with that is that is now you have to have the compiler somehow search through all types and find these constraints itself.
If I hand the compiler some time that somewhere deep inside it uses &'y &’x T it currently has no idea that 'x: 'y. Having it recognize this would mean every time you wanted to convert something it would have to search through and find all these constraints- and also you would have to make sure there’s no way to trick it by making types that could resolve to &'b &'a T eventually.
Even if I do need an `&'y &’x T` for something there are plenty of circumstances where using an `&'static &’x T` instead is totally fine. Just because you have a reference `&'y &’x T` somewhere doesn’t necessarily mean there’s a function type signature relying on it to assume `‘x: 'y`. So you’d be being a bit too strict and break valid programs (currently accepted and should be accepted) even if you did implement it.
Hopefully that clears things up a bit.
I don't understand this. If 'x: 'y is not proven then *any* instance of &'y &'x T is unsound because the underlying type of the &'y reference doesn't outlive it, and so you would be able to access dangling references. It's the basis of the lifetime model and therefore no programs with instances of these should be accepted.
@@SolarLiner That’s what I’m saying. You should have to have proof `'x: 'y` to construct the type. But unfortunately on the current state of rust there’s nowhere for that to go.
@@xenotimeyt yes, but then any program that is legal under this bug should actually be illegal and unsound, and so we shouldn't care that they'd break? I understand the need to gare this change under a new edition but there shouldn't be considerations for invalid programs that do compile due to a bug
@@SolarLiner Doing that would make it impossible to do basic things like map over an iterator or references- the function type would be `fn(&&Element) -> …` which desugars to `for fn(&’y &’x Element) -> …`. If you can’t store the constraint `'x: 'y` in the HRTB this would be illegal under that rule.
Basically any function values involving nested references would be screwed.
I thought the notion of „ &‘y &‘x „ implicitly states that x must outlive y?, how otherwise is &‘y possible? but I guess thats not true…?
The Rust compiler is thinking that way and treating `&'y &'x ()` like evidence that `'x: 'y`. No need to put another constraint somewhere.
But this isn’t correct- because any `&'y &'static ()` is an `&'y &'x ()`. Any evidence that `'static: 'y` is clearly not evidence that `'x: 'y`.
Hopefully that makes sense. :)
Makes total sense now! :)
Thanks for the video & response
No problem, glad it helped! ;)
I don't understand why the compiler lets you do step 2, there are two generic lifetimes in the signature of weird, and for some reason it lets you pass 'static as the lifetime 'a in the witness but doesn't force the other instance of 'a (which should be the same lifetime) to match that, I think that's the actual issue here, since at this point all arguments passed with lifetime 'a are assumed to have static lifetime without any check.
If there's a reason that this isn't the issue I'd love to know, I'm no rust expert.
You aren't passing in 'static or like substituting 'x for 'static. You're just making one of the parameters more restrictive. An `&'static ()` is an `&'x ()` the same way a `Cat` is an `Animal`. So what we're doing is really like converting an `fn(Animal, Animal)` to an `fn(Cat, Animal)`. No need to change both. :)
Thank you for this explanation. The type checker allows the type incorrect assignment to step1. So it's simply a bug in the type checker that in principle could be fixed. But then: Why has it not been fixed in all these years?
Because the correct type is something that you can’t express in Rust’s current type system. You *could* just disallow conversions like step1 which would plug the hole but also break all code passing around functions that take nested references.
@@xenotimeyt These type unsafe conversions must of course be forbidden in the next rust edition. Programs that use them are potentially unsafe. They will still compile for older editions. (cargo new --edition 2021)
@@holzmaurer1319 Where did you find that?
Or is this something you’re saying? In which case I really *really* doubt that making it impossible to for example `map` over an iterator of references to would be worth fixing this not-that-that-serious issue.
I think the issue is with covariance of &'y: by itself 'static : 'x should not imply that &'y &'static T : &'y &'x T. More generally, R: S should only imply &'y R : &'y S if S: 'y as well (not sure I have the syntax right for the last bit).
The way things currently work the Rust compiler treats `&'y &’x ()` like evidence that `'x: 'y`. With this yes it’s correct to say that borrows being covariant is incorrect.
But the real problem is the expressiveness one I mentioned in the video, if you force `'x: 'y` to be a constraint then you have nothing to worry about. Borrow covariance is an extremely fundamental rule that the type and lifetime systems rely on at several levels.
@@xenotimeyt I agree that the issue with breaking it up into steps where you preserve all the constraints (and possibly the compiler proving/failing to prove that this sequence of steps is sound) is an expressiveness problem. But I think once you've broken up it into the steps in the video the problematic coercion is in step2, where a function taking &'y &'x T is being coerced to one taking &'y &'static T without evidence that 'x: 'y.
In other words, I think the correct (covariant) coercion rule is (&'y &'static T where 'x: 'y) : &'y &'x T, and not &'y &'static T : &'y &'x T, even though the first thing looks somewhat unnatural at first glance.
@@ronnodas Ok sure we can have that be the coercion rule- but how is the compiler supposed to know whether `'x: 'y`, these are generic lifetimes from the HRTB and we don’t even know what they are yet?
So let’s say we’re pessimistic and just say that you don’t have proof 'x: 'y so you aren’t allowed to have nested borrows be covariant. This will break existing code- having HRTBs and nested borrows just won’t work properly. Even if by the time you actually substitute for the HRTB’s lifetimes everything is perfectly above board and the 'x: 'y constraint is obviously satisfied.
The only way to actually distinguish between “you have a nested borrow and it’s fine” and “you have a nested borrow and things are about to go boom” is if we store constraints in the HRTB. Hopefully that helps. :)
Isn't the fix for that to instead of just checking the constraints on type cast, to also reintroduce any constraints that would be lost. That would be your "where x outlives y". And do that by taking the constraints of the original type, check that they hold for the target type, "BIND" the new lifetimes, created by the cast, by copying the constraints from the original types to also constrain the types they are casted to?
Not sure I understand, which cast are you talking about?
@@xenotimeyt Every cast. I believe the cast from a weird function to a function variable declared by a supertype of the type of the weird function is the one where the constraint was lost, if I understood the videos correctly.
During the casting of `fn
@@tomasruzicka9835 Yes that’s right. So let’s say we are converting `weird` to `step1`, we need to make sure we have the constraint `'x: 'y` but we can’t put it anywhere- that’s the whole problem. In this case they don’t hold for the target type and there’s no way to make sure they do.
You don’t want to break code that stores a `fn(&&i32)` in a variable (or passes it around).
0:55 you did this because you felt bad you have nothing to do with the cards
to be fair, you as human can spot it if you look for 'static, right? or could you do anything with this without using 'static ?
Not sure what you mean here. There are loads of perfectly safe uses of 'static.
Could you explain the rest of the cve-rs?
I mean there honestly isn’t much to talk about for UAFs and all the ordinary vulnerabilities, if you’re really curious the code that *uses* `extend` is pretty understandable.
The exception is their reimplementation of `transmute`. I might make a video about how that works tomorrow but we’ll see.
I just released an explanation of transmute (with a little twist at the end)
ruclips.net/video/qqgDuEi7OU0/видео.html :)
What I don't understand (I'm no expert, and lifetimes aren't intuitive to me yet) is why feeding
fn(&'y &`static (), &'x T) -> &'y T
into
fn(&'y &`x (), &'x T) -> &'y T
is allowed.
To me it sounds like changing 'x to 'static in the first argument should imply also changing 'x to 'static in the second argument (and changing the first while not changing the second should be a compilation error) :
only allowing to feed
fn(&'y &`static (), &'static T) -> &'y T
into
fn(&'y &`x (), &'x T) -> &'y
,enforcing that 'x in the first argument is the same as 'x in the second, resolves this unsoundness doesn't it ?
I feel like I'm thinking about lifetimes the wrong way, maybe a bit too much like regular types ?
The way I see it, using actual types this time, no type checker would allow this:
fn example (a: Type_A, b1: Type_B, b2: Type_B) -> Type_A
let not_allowed: fn(a: A , b1: B1, b2: B2) -> A = example
even if both B1 and B2 impl Some_Trait
I see I'm missing something here to understand why this is allowed in the first place with lifetimes...
The two values don't need to live for the exact same amount of time. The reference just needs to live at least as long as x, and 'static lives forever.
@@michawhite7613 Yea, and I was wondering why "as long as 'x" is considered good enough in this case. To me it feels like saying:
fn(&'y &`x (), &'x T) -> &'y T
isn't the same as saying
fn(&'y &`x (), &'z T) -> &'y T where 'x lives at least as long as 'z
and saying the former really should mean (to my intuitive understanding) `x in the first argument lives just as long as `x in the second argument (same lifetime, no more no less).
And if it did there would be no unsoundness here, as feeding this
fn(&'y &`static (), &'x T) -> &'y T
into this
fn(&'y &`x (), &'x T) -> &'y T
wouldn't be allowed (because it would drop the constraint that the first argument's 'x lifetime is the same as the second argument's )
Sorry if this isn't clear ^^'
We aren’t really doing anything with 'x, we’re just making the function more restrictive with its arguments by saying actually the first argument doesn’t just have to live for 'x, it has to live (in a dramatic movie villain voice) FOR-EH-VAAAAA!
(Sorry about that I’m still more than a bit sleep deprived lol)
But `'static` is special in that for any lifetime whatsoever `'q`, an `&'static T` is an `&’q T`. Just like a Cat is an Animal.
So we’re really saying converting a `fn(Animal, Animal)` to a `fn(Cat, Animal)` which is of course safe and fine.
Hopefully that clears it up. :)
@@xenotimeyt To me it feels like it makes it:
- more restrictive: the first 'x cannot be any lifetime anymore, it now has to be 'static specifically
but also
- less restrictive: the second 'x doesn't have to live as long as the first 'x anymore, which I thought it did when written like this: fn(&'y &'x (), &'x T) -> &'y T (obviously I'm wrong since Rust doesn't complain ^^ )
To me, what you're describing is putting fn(&'y &'static (), &'x T) -> &'y T into fn(&'y &'x (), &'z T) -> &'y T where 'x: 'z , which I agree is quite logical to be allowed
But I feel like fn(&'y &'x (), &'x T) -> &'y T should have the same meaning as fn(&'y &'x (), &'z T) -> &'y T where 'x: 'z and 'z: 'x , in which case putting fn(&'y &'static (), &'x T) -> &'y T into it wouldn't work unless 'x is also 'static .
What I understand from your video is that even tho in both parameters the 'a lifetime is present, there is no hard constraint that both 'a lifetime are "the same" , they can be "the same" or any of them can be 'static which is where the unsoundness is.
Thx for the video anyway, this is great food for thought (and thanks for trying to explain it again to me ^^' )
PS: Like I said, lifetimes aren't intuitive to me yet, and I have no real type theory background, so it might just be that I missed something about how lifetimes work in general, and ended up confused :'p
Could something similar to the value restriction in sml and ocaml work.
So assigning the variable step1 to step2 would be illegal.
I'm not super familiar with ocaml, but as far as I can tell HRTBs sort of already behave like weak type variables. The value restriction is about generalization as in introducing a type variable (i.e. `None` is of type `'a option`), the what we're doing when we convert step1 to step2 is actually narrowing the type instead of generalizing it, and it's narrowing it in terms of a subtype relationship rather than introducing a type variable.
Again, I'm somewhat of an ML noob so maybe it isn't what I think it is.
gnome fellow 💚
There are two things that point to step1 -> step2 looking "sus" in here:
1. if the existence of a value of type "witness" implies a nesting constraint in lifetime 'a and 'b that is missing from the type of weird. It could be implicit in the usage of &'a &'b in the signature parameter, but the nesting constraint must exist. In that case it should not be possible to convert step1 which has that constraint on the parameters to step2 which is unbound (because the &'a &'b is removed from the parameter. In other words either step1 has an implicit constraint that 'a is narrower than 'b, that should be replicated explicitly in the type of step2 that has no implicit double reference to assert that for the values to be compatible, or the restriction isn't there, in which case the argument types are not compatible (see point 2)
2. controvariance means that U->T can be converted into W->T if W is a subtype of U (that is if the values that are valid W are also valid U). In this example to allow controvariance between the types of step1 and step2 we have to require that for all possible 'x and 'y the type &'y &'static is narrower than the type &'y &'x. This is simply false if there is no restriction on the types 'x and 'y. If we take for example two scopes 'x and 'y such that 'x is not narrower than 'y the type &'y &'x is empty while &'y &'static has plenty of possible values. This problem with reference covariance is what I was pointing out in the last video, but in term of converting the argument types rather than the function type
This means that either the language has to drop the assumption that the existence of a value statically typed as &'x &'y implies that 'x is narrower than 'y, and thus it should stop compiling the "weird" function until we add the constraint about the two types (or let the compiler add it for us, as it does for most of the lifetime stuff) or it has to drop the double reference covariance that allows values of &'a &'b to be converted to the type &'a &'c if 'c is narrower than 'b (and thus 'b is a subtype of 'c).
The contravariance is not necessarily the problem here, unless this unsoundness is something that can only be shown in contravariant functions. I'll try to create an example with covariant values (e.g. generic traits and return values) to see if it is done properly there
1. You're correct that the types are not compatible. You shouldn't be able to say `let step1: ... = weird;`.
But consider the function `fn blah(thing: &&i32) { ... }`. This is desugared as `fn blah(thing: &'a i32 &'b i32) { ... }`. If you (correctly) disallow these kinds of (unsound) conversions (like the step1 line), now there is absolutely no way to store `blah` in a variable or pass it to a function or anything. So just straight up banning this (unsound) conversion and walking away would break loads of existing Rust code. I know I've passed around functions with nested-reference-arguments a number of times. So you have to add the constraint to the type like I said in the video. Then everything would be nicely inferred and work out.
2. In a world with proper constraints you would have to prove `'x: 'y` to have the type `&'y &'x ...` in the first place. But in current Rust, without the proper fix, yes `&'y &'static ()` is not actually a subtype of `&'y &'x ()`. The problem is that you can get that it *is* a subtype only by applying some very fundemental rules the entire Rust type/lifetime system relies on. So all you need to get a paradox is:
- `'static` outlives all lifetimes
- 'a outliving 'b implies a borrow of lifetime 'a is a subtype of a borrow of lifetime 'b
- Borrows are covariant (`S` being a subtype of `T` implies `&S` is a subtype of `&T`)
- A borrow must have a lifetime that outlives the lifetime of what it holds
So there are three options:
a) Have a paradox and the system is unsound (where we are now)
b) Break at least one of these fundamental rules (not a good idea)
c) Make it so you can only have the type `&'y &'x ...` if you know `'x: 'y` (what the proposed solution would entail)
So the two things you mention in the "This means that either..." paragraph are:
1. The compiler automatically adding the constraint `where 'a: 'b` to functions that take nested references like `weird`: Yes, this would fix it, but as mentioned above would also make it impossible to store, return, or pass around functions that take nested references (unless the proposed solution is implemented then everything would be fine).
2. Breaking the covariance of the borrow type would break loads and loads of existing code. So that's not a great way to go about it either.
Hopefully that clears things up a bit. :)
@@xenotimeyt Maybe there is a compromise wherein you break double-borrow-covariance specifically for types _that contain lifetimes under outer "for" quantifiers?_ Only those types seem to contain the inconsistency, because they contain as-of-yet-undecided lifetimes 'x and 'y. If those were already known by this point, then the compiler could just directly check: "&'y&'x is a malformed type in this context, because 'x is less than 'y".
@@СергейМакеев-ж2н I feel like that could work but it wouldn’t be much of a compromise given the fact that most lifetimes are inferred and so end up in HRTBs. Any function with an plain old nested reference like `fn(&&i32)` is desugared by the compiler and is secretly `for fn(&'a &'b i32)`.
All borrows have lifetimes- the compiler just usually figures them out for you. And if you’re doing this inside a function being stored or passed around, it’s using HRTBs and just not telling you.
So that wouldn’t be a great compromise since loads and loads of code would fall afoul of that- just because you don’t type the `for` doesn’t mean it’s not there.
@@xenotimeyt Sure, but then, the only time one must actually *use* those quantifiers, is when the function is converted to a subtype (by contravariance) *without* substituting known 'a and 'b first. That is the case that, we both agree, should be straight up invalid.
@@СергейМакеев-ж2н The problem is that both HRTBs and converting nested borrows to subtypes both happen all the time. Here's a very simple toy example I made that obviously should compile, but under your compromise wouldn't:
play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=f74e0b996501d2fd49672553cf9e4cc3
Obviously that's a toy example but there's loads of code like this so hopefully that makes it clear why we can't just change the way borrow subtyping works all willy-nilly. :)
Isn't the problem in step2? The line step1 still has `&'y &'x ()` which implicitly says that `'x` outlives `'y` (for the same reason that `'a` outlives `'b` in weird), but step2 has `&'y &'static ()` instead which doesn't imply any relationship between `'x` vs `'y`.
Ideally you would have to write `let step2: for &'y T = step1` which would have the lifetime constraint that was previously implied, but no longer, in it, which would solve the problem.
`for
@residual-entropy I know, I was trying to ask whether the issue was actually on line 2 (like I thought) as you seemed to say it was on line 1 (8:40).
Edit: additionally, if you change `step3(FOREVER, borrow)` to use step2 it compiles, but it doesn't with step1 due to lifetime not living long enough, which indicates to me that the issue is what it does after step1 in step2
@@HamishArb It’s only natural that when using only `step1` it doesn’t compile- that’s where you actually exploit the soundness bug.
The only rules we apply to move from step1 to step2 are:
1. Function arguments are contravariant
2. Any `&'static T` is a `&'x T` for any 'x
Both of these are pretty fundamental assertions.
(Breaking them technically could be a way to solve this issue but would require a huge amount of change to everything since you’ve changed how subtyping works)
Hopefully that makes more sense. :)
rust dependent types & lifetimes when 😳
I mean const generics are kind of close to dependent types so who knows, maybe someday. :)
@@xenotimeyt i'm expecting some agda level of dependent types /j
I'd like to expand on this topic a bit more if I can
Maybe treating a 'static as a `subtype` is indeed what is the root of the problem
I understand that the 'static is a valid type of `any` lifetime 'x, as the 'x exists for all lifetimes we provide, the problem is that we expect a supertype 'x to exist for any lifetime of 'x provided.
As the 'static is a very special lifetime that is the supertype of all lifetimes, this premis is broken, the function that took in any lifetime of 'x and below, now takes any lifetime and 'x and below, where the compiler believes that 'x will encompass 'static
As 'x will never be a supertype of 'static, we could just throw an error to force the 'x to be a 'static,
No additional logic would be needed, just ensuring that in the 'x bounded to 'static, any other 'x is 'static
Or we could require the 'x's lifetime to be proven to contain the 'static lifetime
'static is a special case after all.
I don't think that there's any loophole other than that, but feel free to correct me
TLDR; Why not just throw error when the 'x is bounded to 'static and any other lifetime
When do we say does 'x have to outlive 'static? If that were the case then yeah the compiler would be able to tell no such 'x is possible, but I don’t think we ever say that.
@@xenotimeyt We pass in 'static as 'x in step2, so now the only valid lifetime of any 'x of step1 would be 'static, that's what I meant. It's basically where the bug is manifested, in the idea that 'x could be anything other than 'static when any 'x lifetime is initialized with 'static lifetime
@@francp We don’t pass 'static as 'x. We make one of the argument types more restricted by saying it doesn’t just live for 'x, but forever.
All of this to wind up with a language more complex than C++ that has more bazookas to shoot your legs off with.
Is it more complex than C++? Arguable. More bazookas to shoot your legs off with? Huh? The rust compiler is constantly checking everything you do so you can’t shoot your legs off. That’s like the whole point.
Shooting yourself in the foot in Rust isn’t easy, in C++ you shoot yourself in the foot by looking the wrong direction.
Don’t get me wrong I’m no way saying Rust is always better than C++ or C or anything. But I really don’t think your second clause is true at all.
@@xenotimeyt You are clearly able to manufacture such bazookas in this apparently 'safe language'. The compiler passes these off as safe.
Isn't this much, much worse than having the compiler give you no guarantees so you have to avoid such pitfalls yourself?
In C++, you have been warned. In Rust, you might have a false sense of security. How is this better?
@@JSzitas In Rust this is a) incredibly unlikely to be triggered on accident and b) a bug.
So the only real way in which Rust is worse is that if you somehow manage to trigger this accidentally, you may well have that false sense of security.
But I don’t think “crazy lifetime voodoo can trigger memory unsafety - it’ll get fixed eventually” is worse than “looking the wrong direction can trigger memory unsafety”.
As for the “false sense of security”- that’s a big part of why I made these videos! All you really need is to have in the back of your mind that it’s possible to break memory safety with crazy lifetime voodoo.
Hopefully that all makes sense. :)
@@xenotimeyt I agree with much of what you say, but I disagree that it is so easy to create bad memory bugs in C++.
If you are using modern tooling (an IDE) and follow at least some modern conventions (do not use raw pointers willy nilly without reason), you will not run into many memory errors unless you explicitly try to make them.
If the case is the same for Rust, why bother? That's the gist of my argument.
I write C++ professionally, in a multimillion loc codebase that does some voodoo for sake of performance. I would still say we are rather okay most of the time.
I think many C++ programmers will feel the same. If the tangible benefit is 'your code might be safer maybe if you avoid voodoo', then we have been used to that for a long time. Why switch?
@@JSzitas If it’s so easy to not have memory safety error when why are there so many CVEs and security issues caused by memory safety issues?
Surely the entire android team uses “modern tooling and at least some modern conventions” but they still mess up a ton.
If you’re making a program to generate graphs with very well specified input only you will ever interface with, it’s probably not a huge problem (although debuggability sucks). “This code breaks if you set this config flag to eight characters where the last is a backslash” isn’t a problem. So there are some cases where this isn’t a huge deal.
But for a part of your web service that processes random http requests, hippity hoppity your server is my property.
Of course someday that plotting code might just be integrated into a little web dashboard and….
Now only if somebody can get this solution to nightly rust and battle test it...
It requires a next-generation trait solver for that to even begin to happen, according to a Rust contributor on the related issue
@@Speykious "next-generation trait solver" don't know about it. Is there anywhere I can read about it ?
You can see on the projects tab its listed as "blocked on -Ztrait-solver=next": github.com/orgs/rust-lang/projects/44
I found this: rustc-dev-guide.rust-lang.org/solve/trait-solving.html but that talks about `-Znext-solver` and not `-Ztrait-solver=next` like on the project board so I'm not totally sure this is the same thing.
Regardless it seems like they're waiting for the next trait solver to be stabilized before adding new behavior (such as something like where in HRTBs to it).
Again though I do not actually work on the compiler and for a real answer you would have to ask someone who does. Let me know if you do more interesting details tho. :)
@@xenotimeytThis is helpful , thanks...
I'm surprised there isn't a way to write it as `for &'b T`
I mean that would just be syntactic sugar for / the same as `for fn(&'b &'a (), &'a T) -> &'b T`. Just like with for example trait bounds you can say either `fn blah...` or `fn blah... where T: Trait ...`. They're the same thing. :)
The problem seems to me that in type theory the semantics of a_lifetime : another_lifetimes means a_lifetime
@@ghostx-z8m There's absolutely nothing wrong with a borrow `&'q &'q T` (they come up all the time tbh). Both the thing you're borrowing and the borrow itself last for exactly the same amount of time. No problems there.
I sometimes say "outlives" when I really mean "lives at least as long as"- sorry if that tripped you up. With English had a single word for "lives at least as long as" lol.
But yeah `'a: 'a` should always be true, if it wasn't then converting an `&'a T` to an `&'a T` would be illegal even though it obviously isn't.
Hopefully that makes some sense. :)
@al-entropyMy point is that you should be able to say that a should be strictly less and not less or equal to avoid the replace a with FOREVER and break the model.
Something like this...
Currently
for 'a
@@ghostx-z8m If you can say `for where 'a: 'b …`, with the ordinary version of of `:` meaning “lives at least as long as”, *and* you have the compiler properly infer the type of functions with nested references (like `weird`) as `for where 'a: 'b …` and the like, this is fixed and there are no problems.
Are you saying that adding a “lives strictly longer than” constraint is necessary to fix the soundness issue? It isn’t. An ordinary constraint would live until step3 when you substitute 'a and 'b, and then the compiler would force you to say `'a: 'b` and everyone’s happy.
Sorry but I don’t think I understand what you’re getting at (if thats not it).
nobody's memory is safe around me. (i use c)
Well plaid
would be cool to get syntactic sugar that's like
for
which already expresses the constraint right within the for
Well yeah presumably it would work like trait bounds where you can say 'x: 'y where its declared or in a where clause. But you’d actually have to add that to the type/lifetime system first.
🧠💥💥💥
well... writing safe c++ is easy.
First of all it was about making Safe Rust not writing safe Rust. But also…. no?
If it’s so easy then why do memory safety bugs keep happening lol
@@xenotimeyt Well, for starters the survey is highly biased and incorrect about the prevalence of memory safety bugs.
@pyyrr Depends on what you're doing. If you're only single threaded, then idiomatic C++ is indeed easy and quite safe. It's when you get into multi-threading that you realize every single language is an annoying PITA and it's a lot more difficult to reason about the code.
@@anon_y_mousse The Android developers found that the majority of bugs they had were caused by memory safety, and this issue went away when writing Rust. Is that also biased? What possible reason would the Android developers have to make their lives worse?
@@anon_y_mousse I didn’t even mention a survey so I don’t really know what you’re talking about. Memory safety bugs happen often and when they do they’re really really bad.
First off, I am not deep into rust. I have solved a few AOCs with rust and thats about it.
I get the problem which is described here but I am still confused how this could have happened in terms of language design.
The fundamental problem as I understand is that part of the type of the lambda is bound late at the callsite while the object represeting the lambda needs to be partially instantiated before that.
But why wouldn't you craft this process in the same way normal functions work? Why leave this kind of loop-hole of lifetime checking?
I mean, I wouldn't have noticed this of course but I am not a rust language designer. Shouldn't you ask yourself why you are dropping lifetime checks in the lambda case compared to a normal function immediately as a language designer?
I am not trying to accuse rust language designers of being stupid or something. I just have a hard time understanding how this went under the radar.
Do we know how this happened?
Can’t say for sure but it’s not like you’re dropping lifetime checks- there’s this one feature which is that if you have nested references the compiler automatically infers certain bounds and then this other feature where you can store functions abstracted over their lifetimes with HRTBs. Turns out there’s no way to keep put those constraints in the HRTB version.
I doubt many people even knew that Rust auto-infers nested references => weird implicit lifetime constraints in this way. I certainly didn’t. But then someone who did had to realize wait so what happens to those when we decide to throw them in an HRTB- and remember that Rust has been evolving a lot since 2015 and the fn trait I’m pretty sure didn’t work quite the same back then.
So there’s no weird exception, but just the classic not thinking of a specific way in which two features interact.
And don’t forget that the whole idea of a lifetime system at all was basically completely new to the programming world (save some academic research into linear and session types and whatnot).
Most of the quirks of Rust’s lifetime system are usually well-hidden, but having everything be mostly seamless for simple programs makes it quite the beast. It’s not just implied bounds from nested borrows and HRTBs but splitting borrows, complex elision rules, GATs, PhantomData, unbounded lifetimes, integration with type inference, and a thousand other things. It’s easy to look at two puzzle pieces when you see them in a video and think “why did nobody think of putting these puzzle pieces together”, when in reality it’s a 4,000 piece all-white puzzle.
Not trying to say you’re being a jerk or anything, but I’d say it is always good idea to try and think about ways in which things can be more complex than you realize. ;)
@@xenotimeyt I get what you are saying of course. I wouldn't have expected anyone from catching a problem like this based on type and lifetime inference by the compiler of course. But you can write it down explicitly as you demonstrated and it still fails. Its not something the language prevents you from doing explicitly and only happens because inference does something funky. That is what I was thinking about.
I simply don't know enough about the true complexity behind the scenes of course, and I can absolutely understand such a mistake creeping in in an iterative process where each individual change on its own seems fine across a long time period but they eventually end up adding up to such a complex bug.
If that is kind of what happened then this is just a thing that is going to inevitably happen in any project eventually of course.
Your explanation in the videos just sounded to me like lambda object arguments were flawed in this way since their inception. But I guess that was an unfounded leap.
@@lexer_
> Your explanation in the … unfounded leap.
Sorry if I mislead you there, but yeah you only need to be able to stick where clauses on HRTBs to accommodate these weird inferred nested reference lifetime bounds. If those didn’t exist things would be fine (well you wouldn’t be able to pass around functions that take nested references but in terms of soundness it would be fine).
Who would have thought that having layers upon layers of unnecessary and near opaque complexity ... would lead to hard-to-find bugs and safety issues ?
The blatant propaganda around Rust is getting way out of control.
I’m absolutely on team “Rust is flat out too complex and it’s a problem”.
Unnecessary though? Rust lifetimes are a chaotic mess but they solve a problem. Memory safety without a runtime cost.
And propaganda? There are lots of people who are happy users of Rust anyway.
Just because something has its fair share of problems doesn’t mean there can’t be loads of people who like it anyway. I’m one of them.
@@xenotimeyt I do agree with your approach about lifetimes - they are orthoganal to type-types, and could be implemented independently.
Re propaganda vs happy users. Yeah, of course ... tools are tools and they all make happy users when used for the correct application.
There is still a culture of fanboism though around a lot of new tooling - Rust included. Pretty much none of that comes from actual programmers though - its mostly ... other types of ppl who seem to think they can benefit from a certain tech "winning" or something dumb like that
@@steveoc64 Thanks for the civil reply.
I agree with all of that, except the “culture of fanboyism”. That fanboyism definitely exists but from my particular view it isn’t that prevalent.
The Rust community has fanboys (of course) but it just seems like you’re trying to insinuate (mostly in your initial comment) that the Rust community is just a bunch of dumb fanboys who shouldn’t be taken seriously. And I really don’t think that’s the case.
Also in my mind “propaganda” means some sort of organized effort to spread wrong or misleading information, and I definitely don’t see that at all with Rust.
But maybe this comes down to me not being on Twitter lol ;)
let whitehouse_press_release = extend(&cve_rs);
Make a video about Rc, I promise you will have lot fun.