I think you will need to add a step in the ci/cd process to stop the current job if you can't wait for it to finish on its own. Calling the cli in the ci/cd pipeline is one way or having a Python script using boto3 is another
Thank you for this wonderful video, The content is so useful, I have a question; Is it possible to create an aws glue job with json file instead of python file using the terraform?
Thank you! Great video, is it possible to also stop a job run and start again a job? I mean for a CI/CD process or it is necessary to use the cli
You're welcome! What do you define as job run? A glue job run?
Thanks for the reply, e.g. if I have a streaming job running and I want to deploy a new version of the script
I think you will need to add a step in the ci/cd process to stop the current job if you can't wait for it to finish on its own. Calling the cli in the ci/cd pipeline is one way or having a Python script using boto3 is another
I see, so it is not possible only with terraform. Thank you again
Great video! It would be great if you cover Job bookmarks with Glue
Thanks, What are you looking for the video to cover, just the concept at a high level?
Job bookmarks are a bad ideas , bookmarks can be screwed very easily
Thank you for this wonderful video, The content is so useful, I have a question; Is it possible to create an aws glue job with json file instead of python file using the terraform?
So glue jobs are written in python. If it's a spark job, you have an additional option of scala. I'm not sure what you mean by using a JSON file?