Big Data in Real Time

Jay Kreps is a Principal Engineer at Linkedin. He was the original author of Voldemort, a distributed key-value storage system recently recognized by the OSCON Data Innovation Award as one of LinkedIn's major contribution to the open source community to support data analytics.
Jay has also made key contributions to Kafka, a persistent distributed message queue, and Azkaban, a simple batch scheduler for constructing and running Hadoop jobs or other offline processes.
His team builds the core, data-driven features that delight LinkedIn's users, including People You May Know, Who's Viewed My Profile, Skill Pages, and the collaborative filtering applications for LinkedIn's various recommendations.
Jay has a BS and MS in computer science from the University of California, Santa Cruz.

[@AbhishekPathak755]

Great talk.

[@navneet4735]

awesome talk !

[@markrmadsen]

At the ~20 min mark the discussion about ordered streams, being able to roll back in a stream or replay, making it more standard for interfacing, etc reminded me of the arguments streaming SQL projects / vendors make for why basic queues/streams. Too bad FlumeBase is gone. Everything else I've seen is closed source or does it in yeat another proprietary (to the vendor / project) way.
Any reason why things like SQLstream aren't a good approach to this model / use case/

[@MasHaweoye]

Hey! Have you heard about - LinkedIn Buzz Academy (Google it)? Ive heard some decent things about it and my work colleague got excellent money using it.