Dagster Tutorial: Dagster Installation and Getting Started with Asset

Поделиться
HTML-код
  • Опубликовано: 5 июл 2024
  • Dagster Tutorial: Dagster Installation and Getting Started with Asset
    #DagsterTutorial #DagsterInstall #DagsterGettingStarted #DagsterAsset
    ========== VIDEO CONTENT 📚 ==========
    In this video, I will walk you through how to install Dagster on both macOS and Windows. I start by verifying Python versions, creating virtual environments, and installing necessary packages for Dagster.
    You'll witness the creation of a Python file to define your very first Dagster asset. I will also explain the core concept of assets in Dagster, illustrating how to enhance asset information with descriptions and logs. By the end of this tutorial, you'll have a solid grasp of Dagster installation and asset creation, empowering you to manage assets effectively in the Dagster web interface.
    GitHub Repo: github.com/coder2j/dagster-tu...
    🔔 Subscribe: Don't forget to subscribe to our channel for more exciting tutorials. / coder2j
    Subscribe, like, and share this video if you find it valuable. Leave a comment to let us know what topics you'd like to see in our next tutorial. Thanks for watching, and stay tuned for more data engineering and data science content! 🚀
    Want to learn more?
    2-hour beginner Airflow Tutorial: • Airflow Tutorial for B...
    1-hour beginner PySpark Tutorial: • PySpark Tutorial for B...
    Video Request: forms.gle/UMp4GA3krcSMMWzy9
    ========== T I M E S T A M P ⏰ ==========
    Throughout the course, you will learn:
    00:00 - Intro
    00:14 - Install Dagster on macOS
    01:14 - What is Dagster Asset?
    02:43 - Install Dagster on Windows
    03:55 - Dagster Asset Materialization
    07:17 - Add Descriptions to Asset
    08:01 - Use logger from Dagster AssetExecutionContext
    ========== L I N K S 🔗 ==========
    Airflow 2-hour FULL COURSE 👉 • Airflow Tutorial for B...
    pyspark Tutorial for Beginners 👉 • PySpark Tutorial for B...
    SQL Tutorial for Beginners 👉 • SQL Tutorial for Begin...
    Airflow Tutorial Tips 👉 • Airflow Tutorial Tips
    Apache Airflow Tutorial for Beginners 👉 • Apache Airflow Tutoria...
    ========== Connect with me 👏 ==========
    Twitter 👉 / coder2j
    Website 👉 coder2j.com
    GitHub 👉 github.com/coder2j

Комментарии • 29

  • @Crow2525
    @Crow2525 7 месяцев назад +5

    Yeah, this is great! Far better than the official tutorials or dagster university...

    • @coder2j
      @coder2j  7 месяцев назад +1

      Thank you!

  • @joshi1q2w3e
    @joshi1q2w3e 8 месяцев назад +3

    Thank You! Please do full tutorials for Dagster!

    • @coder2j
      @coder2j  7 месяцев назад +4

      I hear you. More is coming. What specific do you want me to cover about dagster?

    • @joshi1q2w3e
      @joshi1q2w3e 7 месяцев назад +5

      @@coder2jhonestly as much as possible, cause I hate Airflow lol

  • @0523036
    @0523036 25 дней назад

    Hi, This tutorial is good .. I would love to have more indepth knowledge in Dagster as i am thinking this would be the feature of Orchestration as well as ingestion tool.

  • @totnguyen3308
    @totnguyen3308 8 месяцев назад +1

    Thank you for the tutorial. I have learned a lot from your videos.

    • @coder2j
      @coder2j  7 месяцев назад +1

      You are welcome 🤗

  • @xmagcx1
    @xmagcx1 7 месяцев назад +2

    thanks you, I would like more Dagster videos please

    • @coder2j
      @coder2j  7 месяцев назад +2

      Yes, this is just the start of the series. What specific do you want to learn about Dagster?

  • @ivansimeon7122
    @ivansimeon7122 4 месяца назад

    This is really good for an intro video. Looking forward to more videos about dagster. A comprehensive playlist would be appreciated including deployment for production in AWS for example. Subscribed!

    • @coder2j
      @coder2j  3 месяца назад

      Noted! Thanks for the feedback.

  • @leonvictorrichard3959
    @leonvictorrichard3959 7 месяцев назад +2

    This is a fantastic tutorial! Thank you coder2j. I would appreciate it if you can show us how to add dependencies and add a schedule to, creating a data pipeline.

    • @Crow2525
      @Crow2525 7 месяцев назад +2

      Agree, I am confused on the folder structure and definitions.

    • @coder2j
      @coder2j  7 месяцев назад +2

      Will address that.

    • @coder2j
      @coder2j  7 месяцев назад +1

      Thanks for the feedback.

  • @adityanjsg99
    @adityanjsg99 4 месяца назад

    Wonderful.. Dagster though not as great as Airflow, but installable on Windows. Thats great about it.

  • @yaojialin718
    @yaojialin718 5 месяцев назад +1

    Why am I not getting messages in stdout and stderr? Some log files are missing?

    • @coder2j
      @coder2j  4 месяца назад

      How do you log the messages?

    • @yaojialin718
      @yaojialin718 4 месяца назад

      I'm not sure, how I logged the messages. How can I check it?@@coder2j

    • @sander1426-2
      @sander1426-2 Месяц назад

      I have the same thing

    • @ardaniahjamaluddin360
      @ardaniahjamaluddin360 3 дня назад

      same here. do you get the way to rectify it?

  • @berkkepoglu4248
    @berkkepoglu4248 5 месяцев назад

    I had this error "Failed building wheel for pendulum"

    • @coder2j
      @coder2j  5 месяцев назад

      Please check the Dagster version, it could be some of the packages versions are not compatible.

    • @Tesis-kf4kl
      @Tesis-kf4kl 4 месяца назад

      @@coder2j i have the same error, my code is the same that yours

    • @coder2j
      @coder2j  4 месяца назад

      What is your python version? And dagster version?

  • @VitaliyZlobin
    @VitaliyZlobin 5 месяцев назад

    What to choose in 2024? Dagster or Airflow?

    • @coder2j
      @coder2j  5 месяцев назад +1

      If you want a mature and industrial standard solution, go with airflow. If you want to learn something new and simple to get started, try with Dagster.