Basic Data Engineering Project - End-To-End From Web Scraping to Tableau

Поделиться
HTML-код
  • Опубликовано: 7 ноя 2024

Комментарии • 40

  • @SeattleDataGuy
    @SeattleDataGuy  Год назад +1

    If you like data engineering and you are looking to start you're own data engineering/analytics consulting company, then you should check out my new course here - courses.technicalfreelanceracademy.com/courses/starting-6-7-figure-consulting - and use the coupon code "deconsult" to get 50% off

  • @KrisPatel99
    @KrisPatel99 Год назад +4

    Thanks Ben! I used something like this to fiqure out Opendoor listings across the Atlanta region and ran it against what they bought at and how much decline they are seeing.
    It was fun excersize.
    Kind of cool how much free information is out there as long as your willing to scrap the data and make sense of it.

    • @KrisPatel99
      @KrisPatel99 Год назад +1

      Id also hold on buying a home until 2024... Lot of inventory going to be hitting the market in the next 9-12 months.

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад

      👀

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад

      Oh yea, its interesting to see even from a high level. Houses are staying on the market at least 2x as long as were only a few months ago

  • @shunsuikyoraku3154
    @shunsuikyoraku3154 Год назад +3

    Hello Ben, I just wanted to say that I would love to watch your videos with a better sound quality 😁

  • @splashoui3760
    @splashoui3760 Год назад +3

    I liked the snowflake part of this video.,

  • @dn9416
    @dn9416 Год назад +6

    Hello data man

  • @thetruthsayer8347
    @thetruthsayer8347 Год назад +1

    Thanks for making this video. I’ve been contemplating switching to data engineering. This video just reminded me how boring data is to me😂😂😂😂

  • @MatheusAlmeida-ez4tc
    @MatheusAlmeida-ez4tc 10 месяцев назад +1

    Really good!

  • @TheWizardTower
    @TheWizardTower Год назад +3

    Great content! Just a suggestion, it might be helpful to slow down and enunciate your words a bit more for better understanding. Keep up the good work!

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад +1

      I'll do my best, yeah I think this video in particular I started mumble rapping

  • @yamarh
    @yamarh Год назад +1

    Thanks so much for this video.

  • @dongyan8717
    @dongyan8717 Год назад +1

    Is snowflake free to use to build this project for my portfolio? I only see a trial

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад +1

      You can use it free for like 30days, and you can pretty much keep setting up free instances over and over again.

    • @dongyan8717
      @dongyan8717 Год назад

      @@SeattleDataGuy nice thank you! ill try this project and see if i can get a legit DE role. Right now im Senior analyst and not sure if my company will give me opportunities

  • @macuriels
    @macuriels Год назад +2

    Hi! Recent subscriber here👋 I'm very much a novice on the subject - are the tools you use free? If not, are the free alternatives you can recommend?

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад

      There are always free alternatives, you can code similar things.

  • @vash3331
    @vash3331 5 месяцев назад

    and how do you connect it to tableau?

  • @Ty-ex2kb
    @Ty-ex2kb Год назад +1

    Thank you for the vid!! Not sure if this question is obvious or not as i’m VERY much a novice but when you’re working on snowflake (~17:14) where you using SQL queries or coding in a different language? Just not sure what you used to parse everything

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад

      It's SQL just likely not all ANSI SQL. A lot of solutions like Snowflake and Databricks (actually postgres has some of these as well) have added in additional functionality. For example, when I create the "Stage" thats not regular SQL. Snowflake lets you query from S3 directly (other tools let you do it too). T

  • @parthvisoni7022
    @parthvisoni7022 Год назад

    Do we need to pay for using the api from bright data?

  • @Andrew.Skinner
    @Andrew.Skinner Год назад

    Thanks Ben. Any resources on swapping out a Python selenium script downloading a csv file for the scraping portion in this vid?

  • @arvindsuriakanth5222
    @arvindsuriakanth5222 Год назад

    For the Tableau part, can we use Tableau public or should we use Tableau Desktop

  • @moeheinaung235
    @moeheinaung235 Год назад +1

    SQL parts in snowflake are really confusing and not well explained. Are the scripts available somewhere?

  • @caseypdx503
    @caseypdx503 Год назад +1

    How long did this take you?

    • @SeattleDataGuy
      @SeattleDataGuy  Год назад +5

      In filming time, probably like 4 hours, i went back a few times when I realized I had been mumbling like SBF for like 5 minutes and had to reshoot.
      A little planning too. I had to get used to brightdata, and then there is all the fidgeting around with tableau.

    • @caseypdx503
      @caseypdx503 Год назад

      @@SeattleDataGuy I mean total time. Just trying to get a gauge to compare against a newbie like myself :)

  • @BISimplifier
    @BISimplifier Год назад

    Good content.
    As a sort of native speaker you are speaking too fast and audience would find it difficult to pace with.

  • @Dev_4_hire
    @Dev_4_hire 9 месяцев назад +4

    This vidoe isn't beginner-friendly!

  • @FRUXT
    @FRUXT Год назад +2

    Hard to follow what you saying when you are not english speaker because you don't articulate...