There is actually no information on how Service discovery actually works or How to validate whether Service discovery is working fine ? .. I was expecting more info on Service Discovery side than on ECS Service creation side
Thanks for the nice tutorial. Would like to ask: 1. Why you need to specify port mapping in python container but didn't for redis, as you were using awsvpc, should each container has it own private IP from subnet? 2. how will service discovery work if there are multi container defined in one task, e.g. if I have 2 containers running restful API endpoint on 5000 with awsvpc, from my understanding the 2 containers will have diff IP address, but since the service discovery were defined in service level, how to distinguish the 2 containers endpoint?
I have a Django application and use celery in the Django application for asynchronous tasks which use Redis but how to configure the Redis in the Django application when Redis is running on a different service like the one you created? When running in my local I just use the Redis docker image but not sure how to connect Redis in celery service.
Very Nice video. When we have more than 2 microservices, do we need to create target group for each microservice and map it to ALB ? In case of having 100+ microservices how service discovery works ?
I have a issue... I have three Ubuntu servers and I have 12 microservices. One server I created 4 services. One server one compose I configured four services.. I run I'm able register eureka all servixes... Zull gateway I can able to access fst servers but remaining I'm not able to access based on serviceId... Can I tell me where I done mistake
I don't think you need to assign Public IP to a task to pull docker images from the internet. As long as the subnet where the task is deployed has a route to the internet, it should be able to pull the docker image.
I followed your tutorial created 3 services inside cluster but only one task in one service which is attached to alb is running other 2 services is not running at all i have waited long but no luck what could be the reason
By default when you assign a new task version to the service, ECS will deploy the new version ALONGSIDE the old version (in parrallel). When the new version passes health checks, ECS will tell ALB to stop routing traffic to the old version and start routing to the new version. When this happens ALB will set the old version to "draining". After 300 seconds(default) ECS will destroy the task with the old version and only the new version will stay running (assuming it passes health check). This is how HA and minimal downtime is maintained. Hope it helps :)
When you create a hosted zone through ECS or CLOUD MAP you can only delete or modify the Route53 through cloud map. So if you delete the cloud map entry then the route 53 will get detroyed aswell.
The same image works fine when not using the ALB, one container is able to communicate with the mysql container using the IP address. When I configure the ALB, above is the error message displayed under 'Targets' tab on Target Group. Also on the task logs, I see the following. 2019-10-07 03:07:172019-10-07T07:07:17.604069Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.17) MySQL Community Server - GPL. 2019-10-07T07:07:17.604069Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.17) MySQL Community Server - GPL. 2019-10-07 03:04:312019-10-07T07:04:31.694473Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' bind-address: '::' port: 33060 2019-10-07 03:04:312019-10-07T07:04:29.381599Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161) 2019-10-07 03:04:312019-10-07T07:04:29.381607Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000) 2019-10-07 03:04:312019-10-07T07:04:30.159602Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release. 2019-10-07 03:04:312019-10-07T07:04:30.159687Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.17) starting as process 1 2019-10-07 03:04:312019-10-07T07:04:31.550768Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed. 2019-10-07 03:04:312019-10-07T07:04:31.554089Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory. 2019-10-07 03:04:312019-10-07T07:04:31.647490Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.17' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL. 2019-10-07 03:04:29MySQL init process done. Ready for start up. 2019-10-07 03:04:292019-10-07T07:04:29.274278Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.17) MySQL Community Server - GPL. 2019-10-07 03:04:27mysql: [Warning] Using a password on the command line interface can be insecure. 2019-10-07 03:04:27Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it. 2019-10-07 03:04:27Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it. 2019-10-07 03:04:20Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it. 2019-10-07 03:04:20Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it. 2019-10-07 03:04:162019-10-07T07:04:16.242533Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' 2019-10-07 03:04:162019-10-07T07:04:16.143583Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed. 2019-10-07 03:04:162019-10-07T07:04:16.146095Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory. 2019-10-07 03:04:162019-10-07T07:04:16.166200Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.17' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL. 2019-10-07 03:04:162019-10-07T07:04:13.843622Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161) 2019-10-07 03:04:162019-10-07T07:04:13.843630Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000) 2019-10-07 03:04:162019-10-07T07:04:14.659431Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release. 2019-10-07 03:04:162019-10-07T07:04:14.659524Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.17) starting as process 80 2019-10-07 03:04:15MySQL init process in progress... 2019-10-07 03:04:14MySQL init process in progress... 2019-10-07 03:04:13MySQL init process in progress... 2019-10-07 03:04:13Database initialized 2019-10-07 03:04:122019-10-07T07:04:12.591016Z 0 [System] [MY-013170] [Server] /usr/sbin/mysqld (mysqld 8.0.17) initializing of server has completed 2019-10-07 03:04:082019-10-07T07:04:08.151071Z 5 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option. 2019-10-07 03:04:082019-10-07T07:04:01.548769Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161) 2019-10-07 03:04:082019-10-07T07:04:01.548776Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000) 2019-10-07 03:04:082019-10-07T07:04:01.549049Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release. 2019-10-07 03:04:082019-10-07T07:04:01.549105Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.17) initializing of server in progress as process 28 2019-10-07 03:04:01Initializing database
@@JavaHomeCloud It is missing about how service discovery work. How to do load balancing if you have more than 1 redis ecs task. How does cloud map, route53 work behind the scene. I think so.
Hello this is a great tutorial ... am wondering how you to connect Fargate Service Container to a Private EC2 instance(I think maybe I create Load Balancer and then call it inside the Container application). Any Ideas will be great 👍 Thank you
The best tutorial i have encountered to unders the working of ECS cluster. Far better than paid ones. Thank you so much 😊
Glad it helped!
There is actually no information on how Service discovery actually works or How to validate whether Service discovery is working fine ? .. I was expecting more info on Service Discovery side than on ECS Service creation side
Very useful. Thanks a lot for sharing.. Great help.
Glad it was helpful!
Thanks for the nice tutorial. Would like to ask: 1. Why you need to specify port mapping in python container but didn't for redis, as you were using awsvpc, should each container has it own private IP from subnet? 2. how will service discovery work if there are multi container defined in one task, e.g. if I have 2 containers running restful API endpoint on 5000 with awsvpc, from my understanding the 2 containers will have diff IP address, but since the service discovery were defined in service level, how to distinguish the 2 containers endpoint?
I have a Django application and use celery in the Django application for asynchronous tasks which use Redis but how to configure the Redis in the Django application when Redis is running on a different service like the one you created? When running in my local I just use the Redis docker image but not sure how to connect Redis in celery service.
Thank you good sir for the video. Packed with information and to the point.
Very Nice video.
When we have more than 2 microservices, do we need to create target group for each microservice and map it to ALB ? In case of having 100+ microservices how service discovery works ?
Good one bro. Need to try this out tomo.
Hope you enjoy
How does a python task connect redis? Is that integration done within docker image?
ruclips.net/video/RQmGQgwp_ns/видео.html
We are setting redis_host="redis" environment variable, python use "redis" as host name to connect.
raddish?
Redis distributed cache
Are these subnet's public or private & how's that environment variable picked up in python, can you please share the code?
Very informative ! Thanks! Subscribed !
Awesome, thank you!
I have a issue... I have three Ubuntu servers and I have 12 microservices. One server I created 4 services. One server one compose I configured four services.. I run I'm able register eureka all servixes... Zull gateway I can able to access fst servers but remaining I'm not able to access based on serviceId... Can I tell me where I done mistake
I don't think you need to assign Public IP to a task to pull docker images from the internet. As long as the subnet where the task is deployed has a route to the internet, it should be able to pull the docker image.
It´s correct
I followed your tutorial created 3 services inside cluster but only one task in one service which is attached to alb is running other 2 services is not running at all i have waited long but no luck what could be the reason
What does the events under the svc say? Please post here.
Would you have the json files for these deployments?
Yes it is possible
When I try to access the DNS I get a 502 bad gateway error.
thanks!
In real time with database tasks. How to do the version update without downtime
By default when you assign a new task version to the service, ECS will deploy the new version ALONGSIDE the old version (in parrallel). When the new version passes health checks, ECS will tell ALB to stop routing traffic to the old version and start routing to the new version. When this happens ALB will set the old version to "draining". After 300 seconds(default) ECS will destroy the task with the old version and only the new version will stay running (assuming it passes health check). This is how HA and minimal downtime is maintained. Hope it helps :)
Can you share the contents of dockerfile you used?
Hi Sir, please let us know how to delete post the session as it created the route53 and I am unable to delete now.
I appreciate you tried the example, hmm okay, go to AWS Cloud Map
and delete entry specific to Route53 and later delete from Route53
When you create a hosted zone through ECS or CLOUD MAP you can only delete or modify the Route53 through cloud map. So if you delete the cloud map entry then the route 53 will get detroyed aswell.
I am getting the following error. Please help. None of these Availability Zones contains a healthy target.
post your error message here
The same image works fine when not using the ALB, one container is able to communicate with the mysql container using the IP address. When I configure the ALB, above is the error message displayed under 'Targets' tab on Target Group. Also on the task logs, I see the following.
2019-10-07 03:07:172019-10-07T07:07:17.604069Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.17) MySQL Community Server - GPL.
2019-10-07T07:07:17.604069Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.17) MySQL Community Server - GPL.
2019-10-07 03:04:312019-10-07T07:04:31.694473Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock' bind-address: '::' port: 33060
2019-10-07 03:04:312019-10-07T07:04:29.381599Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161)
2019-10-07 03:04:312019-10-07T07:04:29.381607Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000)
2019-10-07 03:04:312019-10-07T07:04:30.159602Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2019-10-07 03:04:312019-10-07T07:04:30.159687Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.17) starting as process 1
2019-10-07 03:04:312019-10-07T07:04:31.550768Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2019-10-07 03:04:312019-10-07T07:04:31.554089Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2019-10-07 03:04:312019-10-07T07:04:31.647490Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.17' socket: '/var/run/mysqld/mysqld.sock' port: 3306 MySQL Community Server - GPL.
2019-10-07 03:04:29MySQL init process done. Ready for start up.
2019-10-07 03:04:292019-10-07T07:04:29.274278Z 0 [System] [MY-010910] [Server] /usr/sbin/mysqld: Shutdown complete (mysqld 8.0.17) MySQL Community Server - GPL.
2019-10-07 03:04:27mysql: [Warning] Using a password on the command line interface can be insecure.
2019-10-07 03:04:27Warning: Unable to load '/usr/share/zoneinfo/zone.tab' as time zone. Skipping it.
2019-10-07 03:04:27Warning: Unable to load '/usr/share/zoneinfo/zone1970.tab' as time zone. Skipping it.
2019-10-07 03:04:20Warning: Unable to load '/usr/share/zoneinfo/iso3166.tab' as time zone. Skipping it.
2019-10-07 03:04:20Warning: Unable to load '/usr/share/zoneinfo/leap-seconds.list' as time zone. Skipping it.
2019-10-07 03:04:162019-10-07T07:04:16.242533Z 0 [System] [MY-011323] [Server] X Plugin ready for connections. Socket: '/var/run/mysqld/mysqlx.sock'
2019-10-07 03:04:162019-10-07T07:04:16.143583Z 0 [Warning] [MY-010068] [Server] CA certificate ca.pem is self signed.
2019-10-07 03:04:162019-10-07T07:04:16.146095Z 0 [Warning] [MY-011810] [Server] Insecure configuration for --pid-file: Location '/var/run/mysqld' in the path is accessible to all OS users. Consider choosing a different directory.
2019-10-07 03:04:162019-10-07T07:04:16.166200Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.17' socket: '/var/run/mysqld/mysqld.sock' port: 0 MySQL Community Server - GPL.
2019-10-07 03:04:162019-10-07T07:04:13.843622Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161)
2019-10-07 03:04:162019-10-07T07:04:13.843630Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000)
2019-10-07 03:04:162019-10-07T07:04:14.659431Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2019-10-07 03:04:162019-10-07T07:04:14.659524Z 0 [System] [MY-010116] [Server] /usr/sbin/mysqld (mysqld 8.0.17) starting as process 80
2019-10-07 03:04:15MySQL init process in progress...
2019-10-07 03:04:14MySQL init process in progress...
2019-10-07 03:04:13MySQL init process in progress...
2019-10-07 03:04:13Database initialized
2019-10-07 03:04:122019-10-07T07:04:12.591016Z 0 [System] [MY-013170] [Server] /usr/sbin/mysqld (mysqld 8.0.17) initializing of server has completed
2019-10-07 03:04:082019-10-07T07:04:08.151071Z 5 [Warning] [MY-010453] [Server] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.
2019-10-07 03:04:082019-10-07T07:04:01.548769Z 0 [Warning] [MY-010139] [Server] Changed limits: max_open_files: 1024 (requested 8161)
2019-10-07 03:04:082019-10-07T07:04:01.548776Z 0 [Warning] [MY-010142] [Server] Changed limits: table_open_cache: 431 (requested 4000)
2019-10-07 03:04:082019-10-07T07:04:01.549049Z 0 [Warning] [MY-011070] [Server] 'Disabling symbolic links using --skip-symbolic-links (or equivalent) is the default. Consider not using this option as it' is deprecated and will be removed in a future release.
2019-10-07 03:04:082019-10-07T07:04:01.549105Z 0 [System] [MY-013169] [Server] /usr/sbin/mysqld (mysqld 8.0.17) initializing of server in progress as process 28
2019-10-07 03:04:01Initializing database
incomplete vedio......
May I know What is missing?
@@JavaHomeCloud
It is missing about how service discovery work.
How to do load balancing if you have more than 1 redis ecs task.
How does cloud map, route53 work behind the scene.
I think so.
Never understand the people who uploaded tutorials without the code, why to do that, to show your knowledge?
sorry to say this, what will you get more than this for free??
Hello this is a great tutorial ... am wondering how you to connect Fargate Service Container to a Private EC2 instance(I think maybe I create Load Balancer and then call it inside the Container application).
Any Ideas will be great 👍 Thank you