*timecodes* / *chapters* 0:00 - Intro 1:44 - explanation of the use case 2:39 - installing MySQL and enable Change Data Capture (CDC) 6:59 - adding Apache Kafka, Kafka Connect, and configuring Debezium connector 12:53 - validating that data flows through from MySQL to Kafka 13:44 - materialize data from Kafka using ksqlDB 19:32 - it's a wrap!
Dude, you saved me hours of work with these 2 lines: "transforms": "unwrap", "transforms.unwrap.type":"io.debezium.transforms.ExtractNewRecordState" Thank you very much. CDC has been a pain with all those metadata,... but now I figured out that we can rely on SMTs for some dirty jobs.
Want to clone your brain Viktor. Lol great edit. Miss all the jazz from the livestream. Thanks and keep up the sharing we all benefit from a Db superhero.
This one was quick but great! Maybe one cool think would be to make a KTable from an actor topic coming from debezium too and join it with the movie stream...
Does the connector let you present both before and after image. Ie is it tied into the log of the db or is it just polling? That soundtrack could get you demonetized :-)
> Does the connector let you present both before and after image Here is a CDC event spec. The Debezium MySQL connector generates a data change event for each row-level INSERT, UPDATE, and DELETE operation debezium.io/documentation/reference/connectors/mysql.html#mysql-events > That soundtrack could get you demonetized :-) don't worry, I pay for my music 😎
Viktor: * releases a new vid about stream Processing Followers: how to have the same termal setup 😅 Jokes aside it's just my alias for docker-compose up -d Same for dcdn - docker-compose down. Hope this helps!
*timecodes* / *chapters*
0:00 - Intro
1:44 - explanation of the use case
2:39 - installing MySQL and enable Change Data Capture (CDC)
6:59 - adding Apache Kafka, Kafka Connect, and configuring Debezium connector
12:53 - validating that data flows through from MySQL to Kafka
13:44 - materialize data from Kafka using ksqlDB
19:32 - it's a wrap!
20 минут - идеальный формат. Спасибо!
Dude, you saved me hours of work with these 2 lines:
"transforms": "unwrap",
"transforms.unwrap.type":"io.debezium.transforms.ExtractNewRecordState"
Thank you very much.
CDC has been a pain with all those metadata,... but now I figured out that we can rely on SMTs for some dirty jobs.
👍💪
Good stuff! Would be great to see Postgres, Redshift and Snowflake integrations in future.
very impressive CLI tools!
Want to clone your brain Viktor. Lol great edit. Miss all the jazz from the livestream. Thanks and keep up the sharing we all benefit from a Db superhero.
Good stuff. Excellent presentation.
thanks for sharing ... very educational and easy to understand. 😃👍
The man has come
This one was quick but great!
Maybe one cool think would be to make a KTable from an actor topic coming from debezium too and join it with the movie stream...
@cedric I don't have a follow up idea to enrich a stream of movie ratings with movies info. Stay tuned
Awesome stream 👏
Does the connector let you present both before and after image. Ie is it tied into the log of the db or is it just polling?
That soundtrack could get you demonetized :-)
> Does the connector let you present both before and after image
Here is a CDC event spec.
The Debezium MySQL connector generates a data change event for each row-level INSERT, UPDATE, and DELETE operation
debezium.io/documentation/reference/connectors/mysql.html#mysql-events
> That soundtrack could get you demonetized :-)
don't worry, I pay for my music 😎
How to fetch flat file from Kafka and load into sql server
Where do we create the kafka topic where debezium connector will dump the data?
i have difficulties in converting timestamp and datetime from mysql
need help
the debezium source connector disconnects from postgres DB after 15-16 minutes
Goodh Bhideo
Nice presentation - is it possible to use a connector to hit a kafka rest endpoint?
Yes, you can use this connector www.confluent.io/hub/castorm/kafka-connect-http
Victor, what do I have to install on my Mac in iTerm2 in order to run command in style like you - 'dcupd'?
Viktor: * releases a new vid about stream Processing
Followers: how to have the same termal setup 😅
Jokes aside it's just my alias for docker-compose up -d
Same for dcdn - docker-compose down.
Hope this helps!
Good Russian Man
Can anybody instruct me how to install kafka-avro-console-consumer on mac os please
Install Kafka via brew and console tools will be installed as well
hello i need to show the changed columns and values only as before and after
@ViktorGamov