Agreed, what a fantastic talk. I was just reading some papers about consistency models across many databases. This talk walks through it in a very cool way
This is quite cool. I just built and released a Python package that has 12 CRDTs (with hundreds of unit and e2e tests). Using them to build coordinator-less replicated storage systems is on my list of things to do by the end of the year. I'm thinking that the sha256 of a blob is its content ID, and deletions and restorations can be tracked in an Observed-Removed Set. I'm also thinking that an ORSet for insertions and deletions with each row in a sqlite table being its own Last-Writer-Wins Register (UUID4s for row ID in that case) would work pretty well.
Great Talk! I have one question: At 18:27, when the quorum_read is introduced: It looks like the quorum_read is for a single player only. So if you imagine that every player will play a game every Δt against another player, then the ranking might take more games into consideration for the players that are processed later in the loop in comparison to players earlier in the loop, which might give an advantage, resulting in blood. Or alternatively, the ranking might include the win of one player over another, but not count the loss for the other player from the same game - or the other way around. Are my assumptions correct, or does the system take that into account and the quorum_read is "lifted" to the for loop? Or are there other commands in the language that could solve this?
Hi! Good eye! The code overall is still running in a transaction block---which means it still will "appear" to be atomic, at least at that particular replica. Intersecting the per-replica transaction guarantees with the cross-replica quorum_read guarantees will result in every quorum_read reading from the same system-wide snapshot! So it works.
@@MaePiersonMilano Ah! That makes a lot of sense. Thank you for elaborating! Could have probably figured that from the word "transaction" at the top. 😅 Oh well.
I thought something similar. AFAICT the talk given here is interesting as using an aspect-oriented language artifact (the strong/weak keywords) to help compile-time checking against the necessity for linearization on generic distributed programming. Consensus in blockchains would be one example of a distributed linearization algorithm that is meant to enforce the strong consistency. But general distributed programming might have different requirements per use case on whether or not this strong consistency is required. For example, if a certain player's data always is sharded to always live on one database instance, then when you make calls for that player's personal data you don't care about what the other fresher/staler db instances might contain (weak/eventual consistency). But when you make calls that are global across all instances, then you would probably want to look at all instances (strong consistency). That's how I'm seeing it anyway. Looks like an interesting language feature for the distributed programming universe. With blockchains specifically, maybe it could apply to cross-chain programming models.
Blockchains (to my extremely limited knowledge) invalidate data that clients initially assumed to be accurate during the recovery from a split (or whatever the blockchain people call it). This leaves a lot of work to the clients, who still need to deal with data that might not be properly synchronised yet. E.g. you might invite players under the assumption that your version of the blockchain will remain valid only to later discover that you should have invited different ones when a split gets resolved. Just waiting until enough blocks have been added for a split to be discovered being exceedingly unlikely isn't an option in many applications because it takes too long and simply synchronising all shards would be faster.
This might be the most technically interesting talk I've seen out of Strange Loop this year. Kudos, Mae.
Agreed, my head hurts.
+1
Agreed, what a fantastic talk. I was just reading some papers about consistency models across many databases. This talk walks through it in a very cool way
This is a great talk, with an outstanding delivery. Kudos to Mae Milano.
Nice video professor Milano.
Fantastic talk! But, why is _iso_ being offered to Swift only? Why not to Rust as well? ☺
Great presentation, very interesting topics covered
Amazing work and great talk
Great and interesting talk! Sounds like ”iso” is somewhat the opposite to ”volatile” keyword. Not sure I understood it fully, but sounds useful.
This is quite cool. I just built and released a Python package that has 12 CRDTs (with hundreds of unit and e2e tests). Using them to build coordinator-less replicated storage systems is on my list of things to do by the end of the year. I'm thinking that the sha256 of a blob is its content ID, and deletions and restorations can be tracked in an Observed-Removed Set. I'm also thinking that an ORSet for insertions and deletions with each row in a sqlite table being its own Last-Writer-Wins Register (UUID4s for row ID in that case) would work pretty well.
this is an amazing talk
Great talk!
Great Talk! I have one question: At 18:27, when the quorum_read is introduced: It looks like the quorum_read is for a single player only. So if you imagine that every player will play a game every Δt against another player, then the ranking might take more games into consideration for the players that are processed later in the loop in comparison to players earlier in the loop, which might give an advantage, resulting in blood.
Or alternatively, the ranking might include the win of one player over another, but not count the loss for the other player from the same game - or the other way around.
Are my assumptions correct, or does the system take that into account and the quorum_read is "lifted" to the for loop? Or are there other commands in the language that could solve this?
Hi! Good eye! The code overall is still running in a transaction block---which means it still will "appear" to be atomic, at least at that particular replica. Intersecting the per-replica transaction guarantees with the cross-replica quorum_read guarantees will result in every quorum_read reading from the same system-wide snapshot! So it works.
@@MaePiersonMilano Ah! That makes a lot of sense. Thank you for elaborating! Could have probably figured that from the word "transaction" at the top. 😅 Oh well.
Thank you for the awesome talk! I’m taking lots of ideas for a distributed systems lecture with me (mainly things I should do better)
Very nice talk!
Where can I find links to the three batch of papers?
How do I find that Derecho thing?
Isn’t this what consensus in blockchain look like ?
I thought something similar. AFAICT the talk given here is interesting as using an aspect-oriented language artifact (the strong/weak keywords) to help compile-time checking against the necessity for linearization on generic distributed programming. Consensus in blockchains would be one example of a distributed linearization algorithm that is meant to enforce the strong consistency.
But general distributed programming might have different requirements per use case on whether or not this strong consistency is required. For example, if a certain player's data always is sharded to always live on one database instance, then when you make calls for that player's personal data you don't care about what the other fresher/staler db instances might contain (weak/eventual consistency). But when you make calls that are global across all instances, then you would probably want to look at all instances (strong consistency).
That's how I'm seeing it anyway. Looks like an interesting language feature for the distributed programming universe. With blockchains specifically, maybe it could apply to cross-chain programming models.
Blockchains also need Byzantine fault tolerance. A lot of database tech assumes nodes are not malicious
doesn’t the blockchain provide far too strong guarantees so it adds unnecessary overhead?
Blockchains (to my extremely limited knowledge) invalidate data that clients initially assumed to be accurate during the recovery from a split (or whatever the blockchain people call it). This leaves a lot of work to the clients, who still need to deal with data that might not be properly synchronised yet. E.g. you might invite players under the assumption that your version of the blockchain will remain valid only to later discover that you should have invited different ones when a split gets resolved. Just waiting until enough blocks have been added for a split to be discovered being exceedingly unlikely isn't an option in many applications because it takes too long and simply synchronising all shards would be faster.