Improve your Rust APIs with the type state pattern
HTML-код
- Опубликовано: 27 июн 2024
- Today I'm teaching you how to build Rust APIs that are impossible to misuse by taking advantage of generics and zero-sized types!
FREE Rust Cheat Sheet: letsgetrusty.com/cheatsheet
Code: github.com/letsgetrusty/gener...
Chapters:
0:00 Intro
0:28 Example overview
2:30 First solution
4:20 Second solution
8:13 Third solution
14:18 Outro 
Наука
📝Get your *FREE Rust cheat sheet* :
www.letsgetrusty.com/cheatsheet
Corrections:
10:00 - PhantomData isn't needed here. The Locked and Unlocked structs are already zero-sized types.
11:40 - The lock method should return PasswordManager
That was great, thank you! Amazing how it catches so many errors before even running; and, the hints are dynamic as well. Nicely demonstrated.
There is a bug in your final lock() method on lines 27 and 31, since it just returns an unlocked password manager. You should change the state to locked in both lines. Other than that minor issue, great vid.
I like the idea of using "types" as "markers" of sorts (regarding using generics on a struct just for differentiation). Better than using a field as that's runtime data.This was a great little video. Helped me see things a little different. A "rust patterns" mini-series would be good. Definitely some distinctions from other languages.
Lmao, this was done in OOP for past 50 years.
@@shinobuoshino5066What does OOP have to do with the topic of the video? This is about encapsulation verification (access control) at compile time via Rust's type system. How does OOP do the same via types?
You mistyped the lock method it still returns an unlocked password manager
Good catch, it should be locked.
@@letsgetrustyI suggest to always return Self to simplify refactoring and guard against such cases for the future. One can also construct and return a Self {}.
@@mihaigalos279 Isn't Self the wrong type? If you call unlock then Self is PasswordManager but you want to return a PasswordManager
@@mihaigalos279 can't use Self when changing the state since Self represent the type with the current state
Thank you, I thought I was going crazy.
Brilliant thank you! PyO3 (the rust crate for python extensions) uses this method to confirm the user has the GIL (Global interpreter Lock). At the time I gave up trying to understand the phantom data, but thanks to you I understand now! Now if only there was a way to make the macros that crate uses less mysterious...
Soon it might also be possible to define states in an enum using const generics. Currently this requires the unstable "adt_const_params" feature, otherwise you could only be allowed to use integer and char types which isn't really readable.
This would have the benefit that it's easier to see what states are possible currently there is no real connection between the Locked and Unlocked types and the PasswordManager other than the impl blocks.
Amazing video. Thank you! This solution is very elegant. Thank you for exploring the initial, non-optimal solutions first. It makes it easy to see the benefits of the final solution.
This is the first time I understand PhantomData. Thanks!
The type state pattern, love it!
It’s a beautiful use of the type system. Definitely gives me something to think about.
This is awesome! Almost the same thing was implemented in ATS2 Postiats socket library, which is basically a header files for interop between "C". It uses something like the phantom types but with the refinement types. So not only you can describe the exact flow for you api, but also bind this flow with actual values you pass into functions. This is done via dependent type system
Recently I have came across a situation where I had to do something similar and this video immediately came to my mind. Nice work!
Wow man. First 12 hours with Rust and it has already blown my mind so many times.
Fantastic. I love short, to the point tutorials like this.
This video is hot sauce. I had seen Phantom data before, but did not fully comprehend it. Nice work Bogdan.
Beautiful! Thanks for sharing. Looking forward to more intermediate Rust contents like this. :)
That was super straightforward, thanks for sharing!
As an alternative implementation, we could have the unlock method return an owned token that is a required parameter for the list_passwords method. Upon locking, the token is consumed again by the PasswordManager.
This is definetly the proper way to solve this specific problem. Not only you get compile time checks for API usage, but your API is now closer to supporting being unlocked independently in two different functions.
@@GrzesiuG44 however, this requires allocating extra data, which isn't what the demo was trying to demonstrate.
@@tubebrocoli cannot this token be a zero-sized struct too? "struct Token;" If it can, how does passing a zero-sized struct as a function parameter affect the function's call stack?
Your best video yet in my opinion, thanks for your work! :D
Great video! I’m just starting to learn rust but everything was explained very clearly. Coming from typescript I’m very excited about the existence of PhantomData.
didnt know about zero types. really nice video. as always keep the great work up
Zero-size Types is new to me. I have learnt something. Thanks. I have used similar approaches in Scala / Java earlier, even though the effort was more. Moreover, in Akka-Typed, the approach is very similar, even though the implementation is cumbersome.
Thanks again, for uploading this.
I've been doing that instinctively for a long time now. But great example and explanation nontheless! The Rust specifics with the zero-size types was new and pretty informative!
Wow, that was genuinely an awesome vid, thank you!
That's the exact feature I needed the other day and I didn't know yet !
Nice video, really useful, didn’t know about this pattern and state before. Cheers
I think this is maybe your best video
I use this pattern for data sanitisation in my backend. Works great!
I haven't seen this one before. Very cool!
I'm pretty sure you don't have to specify the type of PhantomData since it would be done by the compiler, aside from that very small thing, this is one of your best videos so far. It's providing information on an intermidiate level while being explained very well!
Indeed, usage of `PhantomData` seems redundant since `State` is already a zero-sized type.
(BTW, there is another error: in the third solution, the method `lock()` should return a `PasswordManager`, not `PasswordManager`.)
@@julytikh Yeah, I also noticed both errors. But the video is still great!
I'm new to Rust, but I felt weird, when he said, that just using "State" would take up memory, since Locked amd Unlocked are zero sized types, just like PhantomData. I'm glad to see, that I wasn't mistaken.
What are idiomatic uses of PhantomData then? I know, it can "own" nonzero size types (e.g. PhantomData), but when is it useful compared to using just plain unit structs?
The only thing I can see, is if somebody by mistake created PasswordManager (which wouldn't be prevented by compiler), they'd be wasting some memory in case, where "state: State" ("state: i32": size is equal to size of i32), as opposed to "state: PhantomData" ("state: PhantomData": size is 0).
But there's a way to restrict the State generic to only include the right types, which would prevent you from creating idiotic types like PasswordManager, which would be even better, than using PhantomData to deal with it.
You can create a (possibly private) marker trait yourself, let's call LockState and have Locked and Unlocked implement it, and then restrict generic State: LockState. At that point there's no need to use PhantomData in this case, and it's absolutely impossible to create idiotic types (e.g. PasswordManager).
I guess PhantomData is useful in cases, where your intent is for your type to be able to be marked by any types user of your library wants (e.g. YourType, YourType, YourType). But I don't know of such scenario (because YourType would not actually contain values of those types, it's only marked by them).
@@tri99er_ One case where this comes up occasionally is when you need a type dependency for the working of your type (e.g. for a return type in one of its methods, for example) but the type isn't actually used in the struct itself. In that situation, you can use a PhantomData to satisfy the compiler that you are "using" the type in the struct so that you can use the parameter in your methods.
@@tri99er_ e.g. the implementation of 'dyn Trait'
Always handy Bogdan appreciate the share!
Great! Great! Great! The best explanation of the Type State Pattern on RUclips! You rock man! How can we support your work?
Nice, saw this once before in a "chaining builder" pattern that prevented setting the same property twice.
I personally would make UnlockedManager a newtype for &mut PasswordManager, that way, do don't need ownership and you get the re-lock for free when it drops.
Nicely explained, thanks
i like this kind of video. keep up the good work man
I learned a lot from this. Thank you!
This is so useful!
Thank you for sharing.
Woow, that was very cool. Great teaching. Thanks
this was a really good explanation!! thanks for this
Very cool stuff man
Beautiful. Will be looking for ways to use this, thanks!
you code your password vaults like this? 😱
Interesting pattern! My default is to revert to state variables, usually enums.
This video is one of your best
This been really useful. Thanks!
This was very cool. Thanks !
Wtf. You can actually do this? So much potential! And head wacking because I need to make sense of the structure
This was great!
great pattern. Man, I just love rust.
Very interesting and illuminating ! A while back ago, I was heavily into generating code from UML state diagrams. Generated state machine code either would use a traditional state transition table or state classes. Now duplicated code isn't that much of a problem, if the code is generated anyway, however to avoid duplicatation I could also a use traditional inheritance. So "LockedPasswordManagerr" and "UnlockedPasswordManager" would both inherit from "PasswordManager" which would implement common code (here: "version()" and "encryption"). It never occurred to me, that I could do the same with generics.
There's no inheritance in Rust. And traits don't have access to implementor fields, so you'll end up with the same amount of duplication.
Really cool! thanks Bogdan!
thanks for this, I really wondered how typestate pattern works since it was said that it only works well with rust..
I really like this kind of videos 🤛
Till the end I have been waiting for raii cause I think it fits here well
Enjoy this video so much!
Loved this video! ❤
In some other languages like Haskell they call it the Indexed Monad pattern. Feels pretty similar to me.
The Rust library called Graph uses this in their builder pattern graph constructor, so it can infer at the type level if your graph is directed or undirected, and if the edges or nodes contains values.
Also, thanks for this. The non generic example was a nice touch. Should it be put in the lock impl, as it defaults to lock or is this better as it changes with the default state?
Indexed Monads are an extension of "plain"-er GADTs with DataKinds. Before those were usable, people used open phantom types. GADTs + DataKinds are more common than indexed monads because a lot of operations can be represented with simple non-monadic functions.
This was beautiful
Thank you, this can become handy
Thank you, now i know how to do something like inheritance in rust
Incredible!
Nice stuff, it'll help a lot
Would be interesting to expand on this further by implementing auto lock on drop, raii style.
Just make LockedPasswordManager a newtype for &mut PasswordManager. That way, it makes the locked manager avaliable after it gets dropped automatically thanks to Rust's ownership system.
u can just impl the Drop trait cant u
This was relay helpful. I couldn't come up with such a smart system. However, you should have a State trait that is implemented for the two state types.
Please use Self as a return from your methods, instead of rewriting all the time the struct name. It makes code reformatting wayyyy easier! :)
Why? If you rename the struct shouldn‘t the IDE rename that too?
lock() and unlock() do not return Self, they change the type
Great video!
interesting, looks like a builder pattern, but returns an instance of same struct with just a state 0 size in memory and template reference thing. Great combo
I liked the general concepts of using types to represent the state.
But wouldn't one problem be that after locking the password manager one could still have a reference to the unlocked one. Since lock/unlock returns a new instance
I.e it's not like an FSM with one state but rather a struct where you can have multiple instances all with same content with different state.
That's the whole point: lock and unlock take their self argument not by reference but by ownership transfer (i.e., move). So the caller loses ownership, and there cannot be any references to the old instance or the whole thing does not typecheck.
Awesome this is the idiomatic way of using Rust type system for states. It van be also extended to state machine or state chart.
Amazing !
In the third solution, you didn't update the lock() method to return a locked password manager. But it doesn't matter, you got the point across and I bet many people didn't even notice.
how would using just state: State waste memory? Isn't it also 0-sized?
I'm also confused about this. I can certainly see PhantomData being useful if the struct holds some data, but in this case there really shouldn't be any additional memory used.
I checked it on goldbolt and both structures with :State and :PhantomData takes 72 bytes for me, so I guess it is optimized out in both variants
What he is saying is very confusing, if T is a ZST you don't need PhantomData to make it ZS because it already is, PhantomData exist primarly for lifetimes: for example there are structs that owns a pointer for optimizations, but still need to hold the lifetime of the backing data, so you add somewhere a PhantomData
It wouldn't, I made a mistake. Will point this out in the pinned comment.
I like TypeScript exactly because it provides similar functionality to implement this 👍
Thanks 4 z great videos :)
This perfectly explains Phantom Data types, I never really grokked this but this instantly made me understand and Now I can see how it can be used in other places! It kind of reminds me of how Two structs of the same Type but with different life-times are treated as two completely separated types so you can't return something with the wrong life-time.
Actually, `PhantomData` is redundant in this particular case. The types `Locked` and `Unlocked` are already zero-sized (because they have no fields), and `PhantomData` does not improve upon that.
@@julytikh can you explain how not to use the phantom data in this example
@@LukasCobblerxD just use `State` instead of `std::marker::PhantomData`.
great video. thank you
Does the compiler help make sure you handle all possible states for structs as it does in match statements?
Sweet!
thank you so much 🙏
Wow. It is interesting 😲
Great video
Nice video thanks
This looks amazing. But is only suitable for state that changes infrequently right? Whenever we transform from one state to another, we are creating a copy of the structs fields
It does not need both the default generic type and the last impl block. Just define `new()` in whatever state you want it, e.g. in the `impl PasswordManager` block.
Very nice
Can you also create an impl block that defines methods for multiple states?
There might be methods that shouldn't exist in all states, but in some.
OP Thumbnail
Excellent. I really enjoyed learning this one.
What minimum version of Rust can you use this pattern on?
Looks like the PhantomData struct is since 1.0.0. I don’t see when default generics were added but there’s a spec for it from 2015.
Best vid❤❤❤❤
The problem I ran into is when I try to implement a trait for such state-parameterised struct, I either get "trait not implemented" or "duplicate definitions with the name " errors.
In the last example, if you switch states, does rust have to move all the data to the new struct? So its basically allocating a whole new object when switching states?
Instead of zero sized structs you can use empty enums here. They are not only zero sized but actually not constructible.
This would solve the problem of user building the wrong state by itself
The issue is that you can't implement different functionality on a struct based on an enum variant. Therefore, we only know if the manager is locked/unlocked at runtime, resulting in panics or results like the first solution. As he showcased, generics allows multiple implementations. The only concern I have is restricting the available State variants, since his implementation allows arbitrary types to it. Perhaps having a sealed trait would solve this as a constraint => pub type PasswordManager {}
@@ErikCampobadal You didn't understand me correctly. I'm saying turn `struct Locked;` into `enum Locked {}`, an enum with 0 veriants which is a non-constructible type in Rust.
@@Artentus ahh interesting, indeed. This combined with a sealed trait could be a great solution
Would you say this is the better approach? Is it less common?
I think in the final solution, you can merge the first and 4th impl. PasswordManager and PasswordManager is the same
This is correct, but I think it's clearer, because we are not creating a locked manager, but just a manager(Even though it will be locked anyway)
Would an enum LockedStatus work well here too?
The lock function returns an Unlocked PasswordManager, instead of a locked one.
Question here though, why PasswordManager implementation for the constructor doesn't need a generic like the common methods implementation "encryption" and "version"?
Given that you consume the original password manager, is it necessary to clone the members of the struct (password list and string)
why the lock method returns the state Unlocked? It should return the state Locked, right? Also, in rust should be possible to force the manager cannot be used after calling to lock or unlock? (to force do the switch of the variable?)
Since Locked and Unlocked are Zero-Sized Types (ZST's), why do we need to use PhantomData? What if our state needed to carry a little bit of information? (such as who unlocked it)
How did you get your "todo!()"s highlighted like that? Looks really helpful.
Seems like TODO Highlight vscode extension
The todo-tree VSCode extension + adding this line to settings.json:
"todo-tree.regex.regex": "(//|#|
@@letsgetrusty TY
Bogdan,
Rust neophyte here. I get confused sometimes when my variables get consumed when I don't expect them to be.
How do you know quickly when a variable is going to be consumed, either by using it as a function parameter, calling an implemented method on it, or even iterating over a vector? Do you just have to be aware of the exact syntax of every function call?
Also, in the Type-State pattern is there any way to avoid moving each member from one state of the type to the next? That could get tedious for a non-trivial number of states or members.
How do you include another generic type with bounds? Say I want my password manager to also store another value, and I want that value to implement a list of traits. In each impl block I would have to specify that my generic type needs to implement the traits. How can I do this without repitition?