You do not lose the ability to query in DynamoDB as you say in the video. What you do in DynamoDB with data modeling is pre-computing the answers to your most accessed queries. Full querying takes time and resources and does not always scale well. Now if you do not know ahead of time the questions you will ask of your data AND you have to do these questions often in your workload, then yes you might need full querying like DocumentDB or a relational DB offers. For good or ill, “it depends” on the workload and the requirements. Take a look at Alex DeBrie and Rick Houlihan's AWS reInvent talks about DynamoDB data modeling for what I am talking about. If your use case needs aggregations and you do not need them completely real time, it is best to do those out of band with a lambda job, not run that query every single time a user needs it. Let's say you have the top 20 articles in a CMS. Which scales better? 1/ Running a query to compute the top 10 articles as voted by users. or have a Lambda job that does this once an hour and puts the content into DynamoDB as one item? The latter. Also, you call it the partition size in the video around 7:25, but I think you mean item size. The item size limit in DynamoDB is 400kb, but for the most part, you should not be putting static large objects into any database. It is not the best use of the most expensive and what should be one of the most performant components of your architecture. For example, let's say your app is delivering a list of recipes. If you can present to the user the title and description in 2 milliseconds, but fetching the JPEG from S3 takes an extra 20 milliseconds, big deal. Storing that image in S3 is so dramatically cheaper than storing it in any database, that is it not funny. You are not paying to store it in more expensive storage, replicate it in a more expensive manner, back it up, etc. etc. Leave the database for the critical data your users are interacting with when you they need it.
Each tool has its own place and there's no good tool. Each of them are analytics service which you can plug it to database or other stacks. Since I haven't worked extensively on it I can't say much about it.
I don't have much knowledge about data migration. Maybe this helps aws.amazon.com/blogs/database/aws-database-migration-service-and-amazon-dynamodb-what-you-need-to-know/
@@RahulAhire Thanks for reply but this is not useful. What I need is data replication not data migration. Like Timescale provides for different sources.
Oh yeah you're right documentDB atleast require $200 upfront cost monthly to get it running being an instance based service. Dynamodb seems flexible and cheap initially but if you have huge predictable workloads then scenario can get quite different Sorry I left to mention that. Thanks for reminding me
![[AUTOML24] AutoML in the age of LLMs](http://i.ytimg.com/vi/xiiVAfjbFQ0/mqdefault.jpg)
You do not lose the ability to query in DynamoDB as you say in the video. What you do in DynamoDB with data modeling is pre-computing the answers to your most accessed queries. Full querying takes time and resources and does not always scale well. Now if you do not know ahead of time the questions you will ask of your data AND you have to do these questions often in your workload, then yes you might need full querying like DocumentDB or a relational DB offers. For good or ill, “it depends” on the workload and the requirements. Take a look at Alex DeBrie and Rick Houlihan's AWS reInvent talks about DynamoDB data modeling for what I am talking about. If your use case needs aggregations and you do not need them completely real time, it is best to do those out of band with a lambda job, not run that query every single time a user needs it. Let's say you have the top 20 articles in a CMS. Which scales better? 1/ Running a query to compute the top 10 articles as voted by users. or have a Lambda job that does this once an hour and puts the content into DynamoDB as one item? The latter.
Also, you call it the partition size in the video around 7:25, but I think you mean item size. The item size limit in DynamoDB is 400kb, but for the most part, you should not be putting static large objects into any database. It is not the best use of the most expensive and what should be one of the most performant components of your architecture. For example, let's say your app is delivering a list of recipes. If you can present to the user the title and description in 2 milliseconds, but fetching the JPEG from S3 takes an extra 20 milliseconds, big deal. Storing that image in S3 is so dramatically cheaper than storing it in any database, that is it not funny. You are not paying to store it in more expensive storage, replicate it in a more expensive manner, back it up, etc. etc. Leave the database for the critical data your users are interacting with when you they need it.
Thanks Kirk for going so deep. I've pinned your comment so that others could get benefit from it
Thank you for the great work Rahul
What is the use of Redshift, Athena, Glue, EMR and more service from AWS? which one is more wanted in the industry?
Each tool has its own place and there's no good tool. Each of them are analytics service which you can plug it to database or other stacks. Since I haven't worked extensively on it I can't say much about it.
Hey Rahul, do you know if we have any connectors from DocumentDB to AWS RDS? The whole purpose being data replication!
I don't have much knowledge about data migration. Maybe this helps aws.amazon.com/blogs/database/aws-database-migration-service-and-amazon-dynamodb-what-you-need-to-know/
@@RahulAhire Thanks for reply but this is not useful. What I need is data replication not data migration. Like Timescale provides for different sources.
@@gurpreetgandhi1907 I really haven't studied it much enough. So, I'm sorry I couldn't tell you anything about it
@@RahulAhire No Worries! I'm novice to AWS as much, just finding out tools. Thanks!
You haven't include ram utilization cost in documentdb which comes out to be over 200 dollars a month which becomes expensive
Oh yeah you're right documentDB atleast require $200 upfront cost monthly to get it running being an instance based service. Dynamodb seems flexible and cheap initially but if you have huge predictable workloads then scenario can get quite different
Sorry I left to mention that. Thanks for reminding me
from where in pune
Wagholi
@@RahulAhire great! I from swarget