Additionally, there is a good muse for the data wrangler as well.....TDSP (Team Data Science Process) which MS put out about 6 years ago as well. Great profiling tool to emulate
It looks more like a PowerBI Power Query editor for dataframes. So you can implement more of other features available in Power query editor. Data wrangler will definitely help with low code python transformation of dataframes. Thanks for the video.
Yes indeed it may look somehow similar to Power Query | Dataflows Gen2 at first sight, though there are a few options you may not find in Data Wrangler, such as Merging. Merging dataframes can easily be done through Python code, so guess you would need to define it as a custom operation.
6 years ago there was a roadmap put out by Amir for Dataflows, Power Query Online, and almost a comparison to how excel processes data. It looks as though that foundation has been adopted and extended in a similar manner as the Koalas engine in Databricks (low code data profiling) as well as auto-gen'd spark code to facilitate a transation for perhaps less skilled resources to enter the data science and data engineering realm. Or, perhaps I am way off?
Great video, thanks.
Additionally, there is a good muse for the data wrangler as well.....TDSP (Team Data Science Process) which MS put out about 6 years ago as well. Great profiling tool to emulate
It looks more like a PowerBI Power Query editor for dataframes. So you can implement more of other features available in Power query editor. Data wrangler will definitely help with low code python transformation of dataframes. Thanks for the video.
Yes indeed it may look somehow similar to Power Query | Dataflows Gen2 at first sight, though there are a few options you may not find in Data Wrangler, such as Merging. Merging dataframes can easily be done through Python code, so guess you would need to define it as a custom operation.
6 years ago there was a roadmap put out by Amir for Dataflows, Power Query Online, and almost a comparison to how excel processes data. It looks as though that foundation has been adopted and extended in a similar manner as the Koalas engine in Databricks (low code data profiling) as well as auto-gen'd spark code to facilitate a transation for perhaps less skilled resources to enter the data science and data engineering realm.
Or, perhaps I am way off?
Awesome, that Data Wrangler will also support creating Spark code! 🤩
Waiting for Pyspark support for wrangler !