Pyspark Scenarios 13 : how to handle complex json data file in pyspark

Поделиться
HTML-код
  • Опубликовано: 17 окт 2024

Комментарии • 29

  • @anubhavoberoy837
    @anubhavoberoy837 2 года назад +3

    Excellent tutorial, looking forward to many more scenario based questions. Thanks a lot :)

  • @varun8952
    @varun8952 2 года назад +3

    Super bro, looking forward for next json tutorial

  • @higiniofuentes2551
    @higiniofuentes2551 22 дня назад

    Thank you for this very useful video!

  • @shreyaspatil4861
    @shreyaspatil4861 8 месяцев назад

    Thanks very much for the tutorial :) , I have a query regarding reading in json files.
    so i have an array of structs where each struct has a different structure/schema.
    And based on a certain property value of struct I apply filter to get that nested struct , however when I display using printschema it contains fields that do not belong to that object but are somehow being associated with the object from the schema of other structs , how can i possibly fix this issue ?

  • @amanpal5958
    @amanpal5958 5 месяцев назад

    Hi, thanks for the video. In my case my nested struct type column does not contain any nested array type column instead it contains nested struct type. Is there any other way I could use?

  • @samk_jg
    @samk_jg Год назад +1

    Awesome tutorial👏

  • @fchehwan7314
    @fchehwan7314 Год назад

    Very good explanation sir! Kudos

  • @ak1232003
    @ak1232003 2 года назад +1

    Very good tutorial! Can you please show how to 1. extract json schema file from data frame and 2. load complex json files based on json schema file? Thanks!

    • @TRRaveendra
      @TRRaveendra  Год назад

      ruclips.net/video/qfJb45SusMo/видео.html

  • @shubhamaggarwal3676
    @shubhamaggarwal3676 Год назад

    Nice Explanation with good example. Can you please give the video link, that you were talking about, dynamically add columns?

    • @TRRaveendra
      @TRRaveendra  Год назад

      ruclips.net/video/qfJb45SusMo/видео.html

  • @deepakkumar3301
    @deepakkumar3301 2 года назад

    Nice explain sir

  • @4802798649
    @4802798649 Год назад

    How to extract if it is JSON object. This works only for array. Any idea?

  • @JeFFiNat0R
    @JeFFiNat0R 2 года назад +1

    super! thank you!

  • @saiveeramalla5507
    @saiveeramalla5507 2 года назад

    Can you try it dynamically..like if the schema is changing is constantly and without mentioning the column names and all , dynamically parse it

  • @sureshajj
    @sureshajj 10 месяцев назад +1

    Thank you. Good one.

  • @rocku4evr
    @rocku4evr 2 года назад

    great ..did you upload the dynamically exploding one? not able to locate

    • @TRRaveendra
      @TRRaveendra  Год назад

      ruclips.net/video/qfJb45SusMo/видео.html

  • @thebossismael1511
    @thebossismael1511 Год назад

    Muchas gracias amigo, me sirvió de mucho.
    1)existen en un json datos vacios y quisiera que no aparecieran.
    2) Cómo hacer para que en un json me cargue las columnas por defecto y no me agregue el nombre de la columna principal sino el el nombre solo

  • @RD-st4vk
    @RD-st4vk Год назад

    Can u show how to create this multiline json file from table

  • @nakkavenkataramana7445
    @nakkavenkataramana7445 2 года назад +1

    Thankyou Brother

  • @omglucky1063
    @omglucky1063 2 года назад +2

    hi, can you increase video quality: 360p only current video -> increase to 720p, 1080p for better visibility.

    • @TRRaveendra
      @TRRaveendra  2 года назад

      Just now uploaded
      It will take some time to convert full hd

  • @srikanthpatil9469
    @srikanthpatil9469 Год назад

    Can I add a column in nested json array??

    • @TRRaveendra
      @TRRaveendra  Год назад

      Yes, you can add any datatype on dataframe using withcolumn

    • @srikanthpatil9469
      @srikanthpatil9469 Год назад

      @@TRRaveendra In dataframe you cannot modify json nested array

    • @TRRaveendra
      @TRRaveendra  Год назад

      @@srikanthpatil9469 you add a new column dataframe and you can change data in dataframe using cast function. you can try that. using withColumn() for adding a new column and chaging data type