Have questions about serverless architectures? Tune into our #AskGoogleCloud premiere on Friday, March 12 10AM PT for answers and a chance to chat live with Google Cloud’s serverless experts → goo.gle/30AdXKY
Awesome video team!!! Appreciate the pacing, not too fast, not too slow! Martin banged out a couple good use cases, and the typo was a perfect real-world error and how to track them down! NICE!!!
I really don't understand why the old video describing this exact coding was removed. Are you also going to bring back the cloudrun video about converting Google sheets to API too ?
Yes, we are re-recording the old videos and adding new videos. We will also update the description of this video when we have the repo ready. Sorry for not getting it ready in time in time for when the video was published.
Thx for this video. After this example we have multiple containers, do we need to pay for these old containers? Do we need to pay for unused service revisions?
Under the hood, your containers are stored in Cloud Storage. So the cost for keeping old containers around is $0.026 per GB per month. Simple containers like the ones we built in this episode are usually less than 0.1 GB in size each. Hope this helps.
@@MayClaude No, the Container Registry doesn't delete container images automatically. But you could potentially automate this yourself by using the "gcloud container images" command: cloud.google.com/sdk/gcloud/reference/container/images
Good idea! We're adding it to our list! In the meantime, here are three lightweight ways to make your Cloud Run service externally accessible without an API gateway: 1. Use the URL that Cloud Run creates for your automatically, like we did in the video. No API gateway is needed if you don't mind a hash in the URL. 2. If you don't want a URL with a hash, you can map your own domain to your Cloud Run service, within Cloud Run. No API gateway is needed. See cloud.google.com/run/docs/mapping-custom-domains. 3. If you want more fine-grained control, for example if mydomain.com should hit one Cloud Run service and mydomain.com/api should hit another Cloud Run service, you can use Firebase Hosting. More details at firebase.google.com/docs/hosting/cloud-run
You can do more things in the Firestore database from the Cloud Console website than you can from the command line. All the tools used in this video have free tiers, which means that you only need to pay if you get significant traffic.
Great video, thank you! A quick question - I'm starting a new project, Python 3 flask web api running under Cloud Run. What are the reasons to choose Cloud Firestore in native mode vs. Cloud Firestore in Datastore mode?
Firestore Native Mode is my default choice. Why? It gives you the possibility to access the database directly from the user's browser if you'd like to do that in the future. Also, I personally think that the API is simpler. The main reason to pick Datastore Mode is if you need more than 10,000 writes per second. You can read a more thorough walkthrough if you search for "firestore native mode vs datastore mode". Best of luck with your project!
@@clickhouseexpert There is no hard cap to writing to a single document. Write throughput to a single document depends on several factors, but the dominant one is the number of indexed fields in a document. This applies to both Firestore Native Mode and Firestore in Datastore Mode. Hope this helps!
Gondai -- agreed, JWT is an important concept. We just recorded an episode about how to use JWT with REST APIs. It is being edited right now and will be published in a few weeks.
Hi @TheMomander I'm wondering if this is still the way to go in 2023? Basically I need to create a REST API to expose data stored in Firestore to the public.
Good question! I would say that the approach in the video is still valid. The alternative would be to use Firestore security rules to make your data readable by everyone. But that would mean all your data consumers would have to learn how to use the Firestore client library. If they are already using Firestore, that may be fine, but otherwise they would probably prefer a regular REST API. Not sure if I answered your question. Happy to answer any follow-up questions you might have!
@@TheMomander Thank you. I was also reading GC docs on the API gateway approach. So having a cloud function that acceses Firestone data, prepares it. Then having API Gateway to access cloud function and pass the results.
@@Mike-er2ih Good point. Putting an API Gateway in front of your Cloud Function is a great approach if you're using specific API Gateway features. But an API Gateway is optional. Some developers reflexively reach for an API Gateway because it is required on some other cloud platforms.
Do you mean data modeling? I don't know of an example of that for social media specifically, but here is a good video about data modeling in general on Firestore: ruclips.net/video/lW7DWV2jST0/видео.html. Hope this helps!
Yes, that's possible and there are many ways of doing it. For example, your web front-end can be your View and Firebase Cloud Functions can be your Controllers, while your database is your Model. Another option is to use a web framework that supports MVC, like Laravel or Django on the server, or React or Vue on the client.
All server less code is available at Momander's github repo . This particular dockerfile is at github.com/momander/serverlesstoolbox/tree/master/rest-api-firestore
Great video. Thanks 1) Is there a way to test locally and access to firestore? I guess firestore won’t run locally, but the troubleshooting feedback will be faster, and 2) could you add Auth to the rest call? Like only for the POST request.
1. Local development -- yes, you can do this. You can either run your code locally and access Firestore in the cloud, or you can run everything locally including an emulation of Firestore. We are working on a video about the former. You can read about the latter at firebase.google.com/docs/emulator-suite. 2. Authentication for REST calls -- you're right, we skipped that part in this video, to keep it short. We are working on another video which will focus on authentication!
Great Video, exactly what I was looking for. However, how would you ensure the firestore cloud rules are being enforced if you are relying on the auth.id or a custom claim token? I don't see a way to access the firestore auth object in the google/firestore nodejs lib.
When you access Firestore server-side (as opposed to from the client), your code has full access to all database contents. Your server-side code would have to enforce any restrictions about who can read what data.
Please can you do a video on cloud Functions. Runtime environments used in Cloud Functions and also the function framework. Thankyou this videos are awesome kudos to helping as learn.
That's a really good question, and really hard to answer. It depends so much on your application. Many applications will cost nothing as they use less resources than the free tier provided by Cloud Run and Cloud Firestore. As your application grows and you start getting thousands of requests per day or you start storing multiple GB of data, you will incur cost. The best way to predict cost is probably by searching for "Google Cloud Pricing Calculator" and plugging in your traffic projections there.
The nice thing about Firebase Hosting is that it includes a CDN (content-delivery network). Your files are cached on SSDs at CDN edges around the world and served as gzip or Brotli. Firebase auto-selects the best compression method for your content. So your files will be served quickly to users regardless of where they are in the world.
Have questions about serverless architectures?
Tune into our #AskGoogleCloud premiere on Friday, March 12 10AM PT for answers and a chance to chat live with Google Cloud’s serverless experts → goo.gle/30AdXKY
Awesome video team!!! Appreciate the pacing, not too fast, not too slow! Martin banged out a couple good use cases, and the typo was a perfect real-world error and how to track them down! NICE!!!
Thank you Mario!
This is a great video. Shoutout to devs that rehearse before film & release.
Thank you!
Great video and good example of troubleshooting with Logging
We didn't plan that at all when we recorded the episode. But sometimes even unplanned errors can be useful :-)
Thanks for this!
I couldn't help noticing the ~2.9 second POST request. Is this because of a cold start of the container or something?
Yes, that's exactly right.
I really don't understand why the old video describing this exact coding was removed. Are you also going to bring back the cloudrun video about converting Google sheets to API too ?
Yes, we are re-recording the old videos and adding new videos. We will also update the description of this video when we have the repo ready. Sorry for not getting it ready in time in time for when the video was published.
The link to the source code repo is now at the top of the video description. Sorry for the delay.
Do you have a video to set up the environment to use visual studio connect to the gcp.
I'm having trouble finding something called "visual studio connect". Could you provide some more details?
Thx for this video. After this example we have multiple containers, do we need to pay for these old containers? Do we need to pay for unused service revisions?
Under the hood, your containers are stored in Cloud Storage. So the cost for keeping old containers around is $0.026 per GB per month. Simple containers like the ones we built in this episode are usually less than 0.1 GB in size each. Hope this helps.
Thx for your answer, can we delete this containers automatic or just keep the last 5 versions or something like this?
@@MayClaude No, the Container Registry doesn't delete container images automatically. But you could potentially automate this yourself by using the "gcloud container images" command: cloud.google.com/sdk/gcloud/reference/container/images
I don't understand how to create the deploy.sh file. May please comment.
You'd create it in your text editor. If you are on Windows, it would be a deploy.bat file.
Thank you! Would you consider a similar video with API Gateway for external access?
Good idea! We're adding it to our list! In the meantime, here are three lightweight ways to make your Cloud Run service externally accessible without an API gateway:
1. Use the URL that Cloud Run creates for your automatically, like we did in the video. No API gateway is needed if you don't mind a hash in the URL.
2. If you don't want a URL with a hash, you can map your own domain to your Cloud Run service, within Cloud Run. No API gateway is needed. See cloud.google.com/run/docs/mapping-custom-domains.
3. If you want more fine-grained control, for example if mydomain.com should hit one Cloud Run service and mydomain.com/api should hit another Cloud Run service, you can use Firebase Hosting. More details at firebase.google.com/docs/hosting/cloud-run
Is there a difference in using firestorm in console or from its own website? And were all the tools in the free tier?
You can do more things in the Firestore database from the Cloud Console website than you can from the command line. All the tools used in this video have free tiers, which means that you only need to pay if you get significant traffic.
love watching them
Great video, thank you! A quick question - I'm starting a new project, Python 3 flask web api running under Cloud Run. What are the reasons to choose Cloud Firestore in native mode vs. Cloud Firestore in Datastore mode?
Firestore Native Mode is my default choice. Why? It gives you the possibility to access the database directly from the user's browser if you'd like to do that in the future. Also, I personally think that the API is simpler. The main reason to pick Datastore Mode is if you need more than 10,000 writes per second.
You can read a more thorough walkthrough if you search for "firestore native mode vs datastore mode".
Best of luck with your project!
@@TheMomander Thank you for your advice :) one last question - what about the 1 per sec write per document limit?
@@clickhouseexpert There is no hard cap to writing to a single document. Write throughput to a single document depends on several factors, but the dominant one is the number of indexed fields in a document. This applies to both Firestore Native Mode and Firestore in Datastore Mode. Hope this helps!
@@TheMomander thank you!
Very informative. Would be interested to see how JWT works in serveless setup
Gondai -- agreed, JWT is an important concept. We just recorded an episode about how to use JWT with REST APIs. It is being edited right now and will be published in a few weeks.
Here is another video where we show how to use JWT with Cloud Run: ruclips.net/video/4HAbnin3nto/видео.html
This videos has amazing content. Pls keep us updated. I'll try right a way to reproduce this. Many thanks for your support
Your GET request is taking 2.59s, how can we speed up our request? it should in millisecond
Hi @TheMomander
I'm wondering if this is still the way to go in 2023? Basically I need to create a REST API to expose data stored in Firestore to the public.
Good question! I would say that the approach in the video is still valid. The alternative would be to use Firestore security rules to make your data readable by everyone. But that would mean all your data consumers would have to learn how to use the Firestore client library. If they are already using Firestore, that may be fine, but otherwise they would probably prefer a regular REST API. Not sure if I answered your question. Happy to answer any follow-up questions you might have!
@@TheMomander Thank you. I was also reading GC docs on the API gateway approach. So having a cloud function that acceses Firestone data, prepares it. Then having API Gateway to access cloud function and pass the results.
@@Mike-er2ih Good point. Putting an API Gateway in front of your Cloud Function is a great approach if you're using specific API Gateway features. But an API Gateway is optional. Some developers reflexively reach for an API Gateway because it is required on some other cloud platforms.
Very informative. Please make a video on serverless service Work flows.
Thank you for the suggestion, Archana!
We recently released a video about Workflows: ruclips.net/video/nJnrkcA7ZdA/видео.html
Thanks a lot Martin fir for the update.
I saw the video and it is very informative.
Waiting for more videos on workflow.
great example. can you please include a link to the associated/example github repo
Yes, we will. Sorry, we didn't get it ready in time for when the video was supposed to go live. Will update the description when the repo is ready.
The link to the source code repo is now at the top of the video description. Sorry for the delay.
Are there any good examples of planning a backend in Firestore for a social network of sorts (Database Planning)
Do you mean data modeling? I don't know of an example of that for social media specifically, but here is a good video about data modeling in general on Firestore: ruclips.net/video/lW7DWV2jST0/видео.html. Hope this helps!
@@MartinOmander Thank you, Yes I meant Data Modeling, I believe I watched those but I'll take another look
Can we use MVC structure in Firebase ?
Yes, that's possible and there are many ways of doing it. For example, your web front-end can be your View and Firebase Cloud Functions can be your Controllers, while your database is your Model. Another option is to use a web framework that supports MVC, like Laravel or Django on the server, or React or Vue on the client.
Where is the dockerfile?
They did not show it but it was in the repo.
@@KevinBoutin Where's the repo? This one on Github is currently empty? github.com/GoogleCloudPlatform/serverless-expeditions
@@FrancisAuYeung same, i cant find the repo?
@@FrancisAuYeung They did not post it in the description unfortunately but they showed the real relevant pieces.
All server less code is available at Momander's github repo . This particular dockerfile is at github.com/momander/serverlesstoolbox/tree/master/rest-api-firestore
Great video. Thanks
1) Is there a way to test locally and access to firestore? I guess firestore won’t run locally, but the troubleshooting feedback will be faster, and
2) could you add Auth to the rest call? Like only for the POST request.
1. Local development -- yes, you can do this. You can either run your code locally and access Firestore in the cloud, or you can run everything locally including an emulation of Firestore. We are working on a video about the former. You can read about the latter at firebase.google.com/docs/emulator-suite.
2. Authentication for REST calls -- you're right, we skipped that part in this video, to keep it short. We are working on another video which will focus on authentication!
Here is the video that focuses on authentication: ruclips.net/video/4HAbnin3nto/видео.html
Thank you very much. I understand this one.
Great Video, exactly what I was looking for. However, how would you ensure the firestore cloud rules are being enforced if you are relying on the auth.id or a custom claim token? I don't see a way to access the firestore auth object in the google/firestore nodejs lib.
When you access Firestore server-side (as opposed to from the client), your code has full access to all database contents. Your server-side code would have to enforce any restrictions about who can read what data.
Please can you do a video on cloud Functions. Runtime environments used in Cloud Functions and also the function framework. Thankyou this videos are awesome kudos to helping as learn.
Thank you for the suggestion, Shadrack! We added it to our list of future episodes.
Fantastic video! It’d be perfect if there’s a server less X cloud SQL X cloud function tutorial !
Wing; thank you for the suggestion! I added it to the list of episodes we should shoot.
Very Good Content! BTW how do Get 1 File at a Time from the Collection ?
If you search for "Get data with Cloud Firestore" you will find an excellent doc that provides example code. Best of luck with your project!
What cost can we expect can you make video with examples
That's a really good question, and really hard to answer. It depends so much on your application. Many applications will cost nothing as they use less resources than the free tier provided by Cloud Run and Cloud Firestore. As your application grows and you start getting thousands of requests per day or you start storing multiple GB of data, you will incur cost. The best way to predict cost is probably by searching for "Google Cloud Pricing Calculator" and plugging in your traffic projections there.
some try catch could help too
Good point!
We can use firebase hosting for this sad, firebase hosting origin only available in USA zone :( they don't even care to mention it in fb documents
The nice thing about Firebase Hosting is that it includes a CDN (content-delivery network). Your files are cached on SSDs at CDN edges around the world and served as gzip or Brotli. Firebase auto-selects the best compression method for your content. So your files will be served quickly to users regardless of where they are in the world.
Is this 100 % free
Building an actual website will be better, example: Small Portfolio or ToDo list app
Thank you for your feedback!
I would've liked the vid, but the like count was 404, which just seemed too perfect for a web service related tutorial.
Hehe! When I read this comment the like count was 479. Please come back and add your like now 🙂
Pay attention to the highlights of the Google Platform not their code implementation. Not a good example of Nodejs RESTful Express APIs.