"Consistency without consensus in production systems" by Peter Bourgon

When software developers think of distributed systems, they typically think in terms of consistency via consensus. That is, enabling a heterogeneous group of systems to agree on facts, while remaining robust in the face of failure. But, as any distributed systems engineer can attest, it's harder than it sounds. Failure happens in myriad, byzantine ways, and failure modes interact unpredictably. Reliable distributed systems need more than competent engineering: they need a robust theoretical foundation.
This talk describes the state of the art in consensus-free consistency: the CALM theorem, "ACID 2.0" semantics, and CRDTs. Roshi-a production CRDT system built at SoundCloud to serve high-volume time-series data-serves as an example. The goal is to provide engineers in industry with a set of powerful building blocks, which will enable them build and improve their own distributed data systems.
Peter Bourgon
SoundCloud
@peterbourgon
Peter Bourgon is a distributed system engineer who has seen things. He works at SoundCloud, building and improving their data systems and infrastructure.

[@rangelspasov]

Great talk for anyone interested in CAP, ACID, CRDTs, and distributed databases/systems in general.

[@gabrielgiussi4422]

My two cents, CRDTs give you causal consistency which is much more powerful than eventual consistency.

[@pengdu7751]

excellent talk

[@Manish10napstar]

While a read query in the proposed system is returning from the single pool, and listeners doing read repair in the background, won't the system return inconsistent data, as it is returning the value read from the single pool ?

[@KacperPerschke]

I'm just referring to corba.
From what I understand, Plan 9 Filesystem Protocol ( en.wikipedia.org/wiki/9P_(protocol) ) was already, promised less and had working impelmentation. Rob Pike still hopes for a successful follow-up ( ruclips.net/video/ENLWEfi0Tkg/видео.html ).

[@pm71241]

regarding "ACID 2.0". The argument is correct but the slides are flawed.
It's not because 1U1=1 that it's Idempotent.
It's because 1U1U1=1 too.

[@zantrua]

+Peter Mogensen 1U1=1 => (1U1)U1=1

[@ruslanfadeev3113]

@@zantrua 1U2U2={1,2} is also necessary for idempotence, but does not follow from 1U2={1,2}

[@lootic]

@@ruslanfadeev3113 It does follow from the rule stated though:
2U2=2
1U2={1,2}
1U(2U2)={1,2} (replacement 2=2U2)

[@carlosmiguelsoto3853]

An element e is idempotent iff e*e = e (for some operation *). In this case you only need 1 U 1 = 1 for it to be idempotent. You're thinking of the function f : Set -> Set, f(s) = s U 1. To prove that f is idempotent you need to check that f o f = f, which is the same as f(f(s)) = f(s) for all s, which is the same as s U 1 U 1 = s U 1, which is what you're saying.

[@houcebr]

Interesting like any talk given by Peter
A question (that won’t never be answered here) as a user in my stream how do I get the aggregated data for all artists across all partitions ? And how do they deal with pagination and so on ?