Hi, this setup spike my billing very high, The setup was to build lambda function to read the latest file from the s3 dir and make transformation then finally to s3 target dir, So this all setup with the python script has to run once the s3 notification to lambda function that an file just came to s3. But it went into a loop and made the s3 and lambda billing spike Let me knew what is the issue in my setup that i didn't noticed at first while running this python script in lambda
Were you writing the files back to same directory? You can have event notification on a specific key and write the processed file to a different key on which there is no event notification otherwise it will trigger the infinite loop. Be careful!
very clear instructions and to the point! Thank you for creating this tutorial!
Thanks for watching!
great video, thanks
Thanks you
good keep it up!
Thanks!
can you do a similar video from Sagemaker Code Editor? I'm trying from there but Docker is not installed
Sure, I will give it a try, but any specific issue that you are facing?
Hi, this setup spike my billing very high,
The setup was to build lambda function to read the latest file from the s3 dir and make transformation then finally to s3 target dir,
So this all setup with the python script has to run once the s3 notification to lambda function that an file just came to s3.
But it went into a loop and made the s3 and lambda billing spike
Let me knew what is the issue in my setup that i didn't noticed at first while running this python script in lambda
Were you writing the files back to same directory? You can have event notification on a specific key and write the processed file to a different key on which there is no event notification otherwise it will trigger the infinite loop. Be careful!