Redpanda Data
Redpanda Data
  • Видео 79
  • Просмотров 92 916
Enhancing AI Workflows with Real-Time Data Processing
Learn how Redpanda Connect integrates with OpenAI and other AI services to enable real-time AI-powered data processing. Consumes unstructured data, transforming it via embedding models to generating embeddings for fast, scalable AI workflows.
Просмотров: 113

Видео

Introducing Redpanda Migrator
Просмотров 152День назад
In this video, I walk you through the process of migrating data from a source Kafka cluster to a Redpanda cluster using the new Redpanda Migrator. I demonstrate how to configure Redpanda Migrator to seamlessly transfer messages, schemas, and ACLs between clusters. Finally, I show how Redpanda Migrator ensures continuous data flow without disrupting consumer groups during migration. Watch the fu...
Seamless Multi-Cloud Data Convergence
Просмотров 5614 дней назад
Learn how to pull data from an SFTP server hosted in Google Cloud and AWS SQS, normalize it with data transformation, and stream it to your downstream systems in real-time. Dealing with diverse data formats, Redpanda Connect makes Data Convergence simple and efficient.
Low-Latency Data Storage for Long-Term Analysis - Redpanda connect to S3
Просмотров 6214 дней назад
Learn how to efficiently store large volumes of streaming data for long-term analysis. This quick demo shows how Redpanda Connect simplifies streaming real-time data to scalable cloud storage solutions like Amazon S3. You'll also see how easily it deploys on Redpanda Serverless, with no need to manage the underlying infrastructure.
Strongly Typed AI Pipelines - Redpanda Connect
Просмотров 8314 дней назад
Showcase the power of Redpanda Connect's latest features, including support for structured outputs from OpenAI's API and JSON Schema integration within the Redpanda Schema Registry. Learn how to create an efficient data pipeline that categorizes and processes emails using Redpanda Connect's new OpenAI processor, ensuring adherence to predefined schemas. Demonstrate schema-based output handling,...
Building a Dynamic RPG with Redpanda Connect: Multiple AI agent EDA system!
Просмотров 18914 дней назад
In this video, I'll take you on a journey through a fun gaming demo using Redpanda Connect. Explore how Redpanda acts as the backbone of an event-driven architecture, how Redpanda Connect simplifies the integration of various LLM models, making it easy to build complex pipelines without writing code. And learn how to store and retrieve unstructured documents to vector databases with pipelines b...
End-to-End Local Embedding Indexing Pipeline to Quadrant ♡
Просмотров 125Месяц назад
This demo showcase a fully local end-to-end pipeline for embedding indexing using Redpanda Connect and Quadrant. It computes embeddings locally with the Ollama processor and store them in a Quadrant vector database. This setup is perfect for those looking to explore hybrid or managed vector database services. Integrating various data sources with Redpanda Connect has never been easier!
Redacting PII with LLaMA 3 Can't Be This Easy!?
Просмотров 220Месяц назад
Check out this demo of Personal Identifiable Information(PII) redaction using Redpanda's Sovereign AI. We use LLaMA 3.1 70B to redact sensitive data in real-time. We process an outreach email, redact personal information, and provide a JSON mapping of the original data. All processing is done locally without any external network calls. Discover the power of Redpanda Connect for secure data hand...
Gemini Vertex AI Chat Processor with Redpanda Connect
Просмотров 131Месяц назад
How to set up and run a basic Vertex AI chat processor using Redpanda Connect? This simple example takes input from standard input (stdin), processes it with the Gemini Pro 1.5 model via Vertex AI, and outputs the generated text to standard output (stdout). Stay tuned for more updates and additional processors in Redpanda Connect to enhance your AI experience. #gemini #vertexai #redpandaconnect...
Convert Live Data to Embeddings using local LLMs and Stream results into Elasticsearch
Просмотров 271Месяц назад
How to use Redpanda Connect to efficiently transfer data from a Redpanda serverless cluster to an Elasticsearch serverless cluster, while keeping your data safe for AI processing with Sovereign AI. #elasticsearch #ai #redpandadata #kafka #rag #vectordb #vectorsearch #ollama
GenAI simplified! Declarative LLM Integration with Redpanda Connect
Просмотров 405Месяц назад
A groundbreaking feature in Redpanda Connect that revolutionizes how developers integrate with OpenAI or locally running LLMs like Ollama. With this feature, you can seamlessly connect to these powerful language models in a declarative manner using simple YAML files, eliminating the need for extensive frameworks like LangChain. But that's not all! We'll also demonstrate an even cooler example w...
Building a conversational robot with Raspberry Pi and Redpanda
Просмотров 3012 месяца назад
Welcome to my latest demo where I created an interactive smart robot using a Raspberry Pi 4! In this video, I'll guide you through how I turned this resource-limited device into a powerful AI-driven robot capable of answering questions in real-time. 🔧 Project Highlights: Speech-to-Text conversion Dynamic topic assignment for query processing Text-to-Speech playback of responses Utilizing OpenAI...
Running Redpanda Data Transforms in Redpanda Connect
Просмотров 1642 месяца назад
This is a quick preview from latest engineering work to bring Data Transforms to Redpanda Connect. The feature release date is: TBD (We will update the description when it's available)
Redpanda Connect Launch Stream: June 2024
Просмотров 8832 месяца назад
Redpanda Connect Launch Stream: June 2024
First Look at Redpanda Connect
Просмотров 9023 месяца назад
First Look at Redpanda Connect
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Intro
Просмотров 4144 месяца назад
Building Real-Time Data Pipelines: Stream & Batch Processing workshop Intro
User-facing Real-time Analytics with StarTree Cloud and Redpanda Serverless
Просмотров 1805 месяцев назад
User-facing Real-time Analytics with StarTree Cloud and Redpanda Serverless
Simplifying Realtime Frontend App Development with Redpanda Serverless, Vercel and Pusher
Просмотров 4275 месяцев назад
Simplifying Realtime Frontend App Development with Redpanda Serverless, Vercel and Pusher
Zafin CTO on the Advantages of Redpanda
Просмотров 2445 месяцев назад
Zafin CTO on the Advantages of Redpanda
Serverless Stream Processing with Decodable and Redpanda Serverless
Просмотров 1515 месяцев назад
Serverless Stream Processing with Decodable and Redpanda Serverless
Instant Streaming Cluster Setup w/ Redpanda Serverless & Simplified App Access Mgmt via Conduktor
Просмотров 1915 месяцев назад
Instant Streaming Cluster Setup w/ Redpanda Serverless & Simplified App Access Mgmt via Conduktor
Serverless Stream Processing with Timeplus Cloud and Redpanda Serverless
Просмотров 995 месяцев назад
Serverless Stream Processing with Timeplus Cloud and Redpanda Serverless
Scalable, High-performance Real-time Analytics with ClickHouse Cloud and Redpanda Serverless
Просмотров 2135 месяцев назад
Scalable, High-performance Real-time Analytics with ClickHouse Cloud and Redpanda Serverless
Develop an event driven app in a day with Redpanda Serverless & Lambda
Просмотров 2745 месяцев назад
Develop an event driven app in a day with Redpanda Serverless & Lambda
Redpanda Launch Stream -- January 2024
Просмотров 5615 месяцев назад
Redpanda Launch Stream January 2024
Redpanda Serverless Launch Stream
Просмотров 7895 месяцев назад
Redpanda Serverless Launch Stream
Your Friendly Guide to rpk - Redpanda's Powerful Command-Line Tool
Просмотров 5709 месяцев назад
Your Friendly Guide to rpk - Redpanda's Powerful Command-Line Tool
Redpanda Office Hour: Building a Real-Time Leaderboard with Live Coding!
Просмотров 4759 месяцев назад
Redpanda Office Hour: Building a Real-Time Leaderboard with Live Coding!
Redpanda Office Hour: Tiered storage architecture unpacked
Просмотров 16410 месяцев назад
Redpanda Office Hour: Tiered storage architecture unpacked
Redpanda Office Hour: Everything about Redpanda Transform powered by WASM
Просмотров 42711 месяцев назад
Redpanda Office Hour: Everything about Redpanda Transform powered by WASM

Комментарии

  • @marcusjpotter
    @marcusjpotter 7 дней назад

    really appreciate you creating and sharing these videos @Christina :)

  • @ROKKor-hs8tg
    @ROKKor-hs8tg Месяц назад

    what name of ide you are used in video?

  • @raticus79
    @raticus79 2 месяца назад

    Starts 4:36

  • @SebastianSativaLivemore
    @SebastianSativaLivemore 2 месяца назад

    Great demo! The latency was a bit underwhelming, but I suppose this could be mitigaged using Livekit.

    • @weimeilin
      @weimeilin 2 месяца назад

      Thanks :) I actually slow it down a bit by aggregating the voice for 30 sec on purpose, so I have some time to explain what's going on. I'll def take a look at Livekit, maybe with that I don't have to deal with segments anymore

  • @CarlosSantanaDev
    @CarlosSantanaDev 2 месяца назад

    Great demo Christina 🎉

  • @marcusjpotter
    @marcusjpotter 2 месяца назад

    Super awesome presentation strands. We're already starting to utilise Redpanda Connect 'community' processors and our devs teams are loving it. +1 for 'Redpanda managed connectors/integrations' (with the hope that open source remains at the centre of business models considerations moving forward) Thanks @Ash/@Alex for your Alex dedication @Redpanda teams for evolving the streaming processing space, where things surprisingly 'just work' :))

  • @HideBuz
    @HideBuz 3 месяца назад

    Can you do a slow and in-depth tutorial for us slow pandas with aws, example application in lamdba and setting up clusters, price controls etc, pretty please? I am new to event driven architecture and message streams.

  • @paulfunigga
    @paulfunigga 3 месяца назад

    yep, scylla is also faster than cassandra they said, but when i tried it, for my use case case it was 3x slower than cassandra

  • @ManticoreRoko
    @ManticoreRoko 3 месяца назад

    It looks really cool. I'd like to see more comparison videos with Kafka Connect and other bullets 👏

  • @xmanamit
    @xmanamit 3 месяца назад

    Amazing book guys - Fundamentals of Data Engineering! Loved it. I think this is the only book which truly captures this area. There is lots of confusion among people who been in this field for a long time but struggle to find direction in vast variety of toolset. This definitely brings back clarity and helps bridge concepts to tools

  • @danroscigno187
    @danroscigno187 4 месяца назад

    This was really interesting. I worked through the workshop. The Instruqt experience is well done, and the content is very interesting. Thanks!

  • @ginanjarg
    @ginanjarg 5 месяцев назад

    Darran Cooke led me here. I was preparing a new project that requires Consumer Groups with Redpanda cluster.

  • @isaacyork-vr9kl
    @isaacyork-vr9kl 5 месяцев назад

    Very cool

  • @mozinator1
    @mozinator1 5 месяцев назад

    Give this man a high quality microphone please

  • @bhanujuneja58
    @bhanujuneja58 6 месяцев назад

    Nice information. I loved your book “Fundamentals of Data Engineering“ 😊. Thanks to Redpanda for providing it.

  • @atari1040
    @atari1040 8 месяцев назад

    Thank you again for real time question answering during webinar 👍

  • @pubududewagama
    @pubududewagama 9 месяцев назад

    Good stuff!!!!!

  • @BUY_YOUTUB_VIEWS.321
    @BUY_YOUTUB_VIEWS.321 9 месяцев назад

    Thumbs up for another incredible upload!

  • @amenostalgique
    @amenostalgique 11 месяцев назад

    Beautifully explained <3

  • @flyaruu
    @flyaruu Год назад

    For me WASM transformations that can only do stateless is a non starter for me. I tend to need _some_ stateful transformations, and if it means that I’ll need to use Flink anyway, Flink can also do the stateless ones. Is there a roadmap for ‘stateful WASM transformations’ or is there a space where that gets publicly discussed?

    • @rockwotj
      @rockwotj Год назад

      You can always jump in our community slack!

  • @jdrago999
    @jdrago999 Год назад

    Please - who did the voiceover? I'm asking for a friend. 😅

  • @dinoscheidt
    @dinoscheidt Год назад

    40:32 Not touching the first wave of Kubernetes management tools, aka Teraform, with a stick in my life anymore. ArgoCD is part of the modern stack, and if we talk about “using the tools that are popular”, than we would be using Apache Kafka, not considering a rather new and not yet popular tool like Red Panda. So the Moderator here (the Lady), did a great job mirroring public perception and considering where the puck goes community wise.

  • @gabrieldutra4323
    @gabrieldutra4323 Год назад

    Awsome explanation, tks!

  • @Harini1234-h3b
    @Harini1234-h3b Год назад

    ✌️

  • @andydataguy
    @andydataguy Год назад

    These videos are so great! Thank you 💜💜💜

  • @timibolu
    @timibolu Год назад

    interesting video 👍

  • @SanjeevKumar-nq8td
    @SanjeevKumar-nq8td Год назад

    I have configured redpanda with rook-ceph in K8s cluster.

  • @marclamblin6211
    @marclamblin6211 Год назад

    I recall that Zookeper has been removed from Kafka. Am I correct?

  • @tripathi26
    @tripathi26 Год назад

    Informative

  • @mckee_plus_plus
    @mckee_plus_plus Год назад

    What software do you use to draw your presentation?

    • @weimeilin
      @weimeilin Год назад

      Drawing directly on iPad Note :)

  • @pshandy
    @pshandy Год назад

    Best engineers and best team !!!! Go guys

  • @SanjeevKumar-nq8td
    @SanjeevKumar-nq8td Год назад

    Nice tutorial. I am new to Redpanda. Have configured on K8 6 wk node cluster ith the defaults settings. How to install & configure the UI in k8 cluster via helm chart. Any pointers.

    • @vinodprathapa7393
      @vinodprathapa7393 День назад

      which company in india is using redpanda? is thiks skill on demand?

  • @loreleitrageser3904
    @loreleitrageser3904 Год назад

    pքɾօʍօʂʍ

  •  Год назад

    great talk

  • @Mike-cp1tj
    @Mike-cp1tj 2 года назад

    can you invest in a nicer apple pencil, this kind of unbearable drawing is hard to take it seriously..

  • @rk3rn3r
    @rk3rn3r 2 года назад

    Inspiring ♥

  • @padmanabhanvenkatesan3999
    @padmanabhanvenkatesan3999 2 года назад

    Nice to know...

  • @ДинаНосова-х7д
    @ДинаНосова-х7д 2 года назад

    hn8sxu vum.ong

  • @raticus79
    @raticus79 2 года назад

    Ah, thanks, I'm in the middle of trying to explain the simplicity advantage here myself and wasn't sure about the linear growth rate for the log-brokered approach. O(n²) -> O(n). Good point about Data Mesh using the same idea

    • @alexandergallego5899
      @alexandergallego5899 2 года назад

      that's a big flashy one. for the day to day, things like circuit breaker, broker discovery, built in backpressure, etc are just as big of a win that you get 'for free' with the ecosystem Redpanda plays into. there is a huge set of kafka libs out there that do the right thing.

    • @raticus79
      @raticus79 2 года назад

      @@alexandergallego5899 Just made it to the end - I was happy to see the future direction you showed with services as multiple lenses over the log. I agree, that's what I came up with for my own designs, we actually were just talking about key-value with redis and figuring out what to do for graph. Helps to have that confirmation as a bit of a novice. I think your bottom-up image there with log as the base is a helpful way to show its role in this kind of approach (vs the usual ones with log in the middle that everything connects to)