114. Databricks | Pyspark| Performance Optimization: Re-order Columns in Delta Table

Azure Databricks Learning: Delta Lake: How to re-order columns of a delta table?
=================================================================================
Re-ordering tables columns is one of the most common requirement in database and data warehousing concepts. It is also improving performance in Databricks delta lake. To know more about it, watch this video
• 114. Databricks | Pysp...
#Deltalake #DataSkipping, #DeltaSkipping, #ReorderDeltaColumns, # RepositionDeltaColumns #SparkDevelopment,#DatabricksDevelopment, #DatabricksPyspark,#PysparkTips, #DatabricksTutorial, #AzureDatabricks, #Databricks, #Databricksforbeginners,#datascientists, #datasciencecommunity,#bigdataengineers,#machinelearningengineers
Delta Lake Internals: • 52. Databricks| Pyspar...
Z-ordering: • 66. Databricks | Pyspa...

[@SriRamTrading]

It's good and new for me as per my experience but here I think how it's make performance optimization

[@rajasdataengineering7585]

Pls watch the complete video and you will understand how it improves performance

[@SriRamTrading]

@@rajasdataengineering7585 in simple it just change the order of column, correct me if m wrong

[@georgesudi]

Hi,
Thanks for sharing those features!
It helped me a lot

[@rajasdataengineering7585]

Glad to hear that! Thanks for your comment

[@passions9730]

Finally ur back..

[@teluguoldmemories8325]

For delta don't specified using delta it will take automatically you can check also using describe extended command.

[@AshokKumar-ji3cs]

Welcome back brother

[@rajasdataengineering7585]

Thanks brother 👍🏻

[@prabhatgupta6415]

Cover real time scenarios in databricks sir.
As well as cicd.

[@rajasdataengineering7585]

Sure, will do

[@vineethreddy.s]

At 13:19 how are you able to append the data with the schema being different. Shouldn't we use mergeSchema?

[@vidyasagar6597]

Hi sir,
Can you please create a vedio on why and when we opt for different scripting pyspark or scala or java or R.. what's the difference?

[@sambitmohanty1758]

Hi can you please make a video on DE project end to end

[@rajasdataengineering7585]

Hi, I have already created couple of videos on this requirement. You can check if that helps you
ruclips.net/video/dxxXWe4gNTo/видео.html
ruclips.net/video/Ia6fDlhlKXQ/видео.html

[@pde.joggiri]

Hi sir
FSCK vs MSCK please explain?
We mostly face manual file delete error/ file not found error and FSCk resolve the issue
When can we use MSCK

[@rajasdataengineering7585]

Hi, sure I will create a video on this requirement

[@bachannigam4332]

Will u recommend any book ?

[@rajasdataengineering7585]

Spark definitive guide is good to understand spark internals

[@abhishekjoshi48]

Hello Sir,
Thank you for video, I have requirements where I need to change datatype of column instead of long it should be string what would be the possible ways?

[@rajasdataengineering7585]

Hi Abhishek, you can use cast method in pyspark