ETL | Amazon Kinesis Data Stream | AWS Lambda | S3 | Kinesis to Amazon S3 Bucket using AWS Lambda
HTML-код
- Опубликовано: 17 окт 2024
- ===================================================================
1. SUBSCRIBE FOR MORE LEARNING :
/ @cloudquicklabs
===================================================================
2. CLOUD QUICK LABS - CHANNEL MEMBERSHIP FOR MORE BENEFITS :
/ @cloudquicklabs
===================================================================
3. BUY ME A COFFEE AS A TOKEN OF APPRECIATION :
www.buymeacoff...
===================================================================
Repo link : github.com/Rek...
#etl #kinesisdatastream #awslambda #s3
I see that you have configured to the batch of 100, I just wanted to know for each batch, will there be a separate lambda invocation?
Thank you for watching my videos.
Indeed, it create a separate instance of lambda for the batch.
In production how to grant access for external service. Is it through IAM roles or API gateways?
You can agrant access to external service via multiple options.
1. IAM users.
2. Federated Access (which supports)
3. API gateway as application for tasks
One doubt .. in s3 what files and from where you upload objects in s3 bucket.. can you clarify my doubt
Thank you for watching my videos .
This videos talks about capturing steaming data and storing in it low cost Storage s3 bucket.
I am not able to send the records from kinesis data genrator to s3 bucket
Thank you for watching my videos.
Please let me know what is the error that you are facing now.
Thanks for this wonderful tutorial
Thank you for watching my videos.
Glad that it helped you.
KDG sent 500 messages but why we only see 74 objects in S3?
Thank you for watching my videos.
Please check the concurrency limit on your lambda. May be that could be the reasons
@@cloudquicklabs Can't be, because the invocation limit is 1,000.
Yes, same thoughts here. ^^