AWS Tutorials - AWS Glue Job Optimization Part-4

AWS Glue Job and Lake Formation Crash Course - • AWS Tutorials Crash Co...
Building AWS Glue Job using PySpark - • Building AWS Glue Job ...
AWS Tutorials - AWS Glue Job Optimization Part-1 - • AWS Tutorials - AWS Gl...
Optimization of AWS Glue Job is an interesting and most-asked topic. There are many ways to optimize AWS Glue Job such as optimizing memory or capacity. In this video, you learn how to use AWS Glue Spark Shuffle Manager to configure S3 as the location for spill and shuffle data for large transformation. It helps in building resilient and reliable Glue Jobs.

[@shanghaitinkar]

What a to-the-point video!

[@AWSTutorialsOnline]

Thanks

[@tranthanhbao9978]

i set all 3 parameters for glue but still facing "no space left on device " . ? Any more step we could do to debug this issue ?

[@AWSTutorialsOnline]

Please enable job insight and check glue job logs for more details. How big is your data?

[@nikhilgupta110]

If I'm using Glue 3.0 , does that mean I can't leverage this optimization strategy?

[@rajeshkumarnimmagadda2547]

You can use.