How to Merge Spark DataFrame - Complex type | Spark Scenario Based Question | PySpark | LearntoSpark

[@sravankumar1767]

Superb explanation bro, but little bit difficult to understand 😕

[@harishgandham4902]

Can we consider doing left outer join

[@nareshreddy-l3f]

Thanks for uploading nice content videos ...

[@AzarudeenShahul]

Thanks for your support 😊

[@khushbuagrawal9554]

@Azarudeen - It will be good that If you can put the sample data or file in the description/comment section. So that we can copy paste and do the practical.

[@wasim054]

Thanks for this demo. Can you please help me with any suggestions for one my requirements. It would be great if you have any input on this. I want to do merge like feature in pyspark with target data frame having 2 extra column like isupdated and timestamps. If record is updated in source data frame then isupdated column will be set to yes otherwise no.

[@AzarudeenShahul]

Hi Wasim,
I am not able to understand you requirement.
Can you please help me understanding by, sharing the sample input and required output to my email @ [e-mail]

[@tarunpothala2071]

Hi Azarudeen , I find your videos beneficial. Thanks for sharing this content. I request you to make videos on how to read and write data from\to cloud storages into Databricks.

[@AzarudeenShahul]

Sure, our upcoming item is fully with azure databricks and azure

[@keepsmile564]

Nice session, is thre way Age and other columns should come as a different columns Insted of list education column

[@AzarudeenShahul]

Yes list column can be separated into individual column.. will try to make vdo on how we can do this :)

[@rajasekhar582]

Hi azzar, All videos r u show very usefull. Please can show on spark with scala also in shell scripts

[@saurabhshubham4448]

Very helpful!

[@dhilip77]

Explode function will work on these

[@Tech-Nature-IND]

can you provide us same with scala

[@AzarudeenShahul]

Sure will prepare same scenario using scala

[@TheSumanb]

HI thnks bro for the hard work, could you please share the code repository ?

[@rudrasingh21]

Hi , Its very helpful . Could you please share data and code with github link , so that we can easily practice. Take it as a request, Please

[@AzarudeenShahul]

Now-a-day, I m giving my blog link which has a code snippet and link to data for your practice. For some old videos, i added git link in description.

[@sravankumar1767]

CAN YOU PLEASE SHARE THS CODE IN DESCRIPTION BOX IT WILL BE USEFUL FOR US

[@AzarudeenShahul]

Hey Sravan, I m preparing blog with code snippet for all the scenarios.. will provide the link in description. Hope this helps u

[@sravankumar1767]

@@AzarudeenShahul Thanks Azar

[@Shiva-kz6tn]

Please please post more

[@ashishkumavat2646]

Hi Azarudeen Shahul, I following your videos from the last 1 months and I found it very useful. I would like to make a request, is it possible to upload a video showing how to read data from google sheet using spark scala. I searched for it everywhere but couldn't find it.

[@AzarudeenShahul]

Sure buddy, will try to do one on this use case

[@gayathrilakshmi6087]

Hi,
Can you please attach the data file and code file for us to practice..very useful video :)

[@samisajjad802]

list1=[]
list2=[]
for i in l1:
print(i)
list1.append(str(i).split("`")[1])
i am getting en error message like
----> 5 list1.append(str(i).split("`")[1])
IndexError: list index out of range