Thanks for creating this, but unfortunately I am getting an error. User is not authorized to perform: iam: CreateUser on resource: opensearch-poc-user with an explicit deny in a service control policy. Could you kindly suggest what should I use over here? I have followed exactly same steps as mentioned in the video?
Thx for the video. Did you figure out how to limit bulk operation for only movies index? Like you, I needed to give cluster permission but bulk operation can be done in another index. It didn't seem a secure way.
I haven't tried doing it programmatically yet myself. However, since the role mappings is internal to the OpenSearch API and not part of the AWS Control Plane / API, I would use some sort of post-deployment script in Terraform that calls the Security Plugin API, perhaps using a shell_script resource or something like that.
@@YannStoneman Thanks. Make sense. Not sure why but I mistakenly assumed OpenSearch is a aws service (much like s3 or dynamodb) which it clearly is not
@@Amapramaadhy Update: it does seem like the terraform provider has included deeper support for OpenSearch over time: registry.terraform.io/providers/opensearch-project/opensearch/latest/docs
Hey the video was really helpful Thanks, but I have a small query .Is there any way where we can map the backend role programmatically instead of doing it manually?
I had an Error: AuthorizationException message":"The security token included in the request is expired wit this line auth = AWS4Auth(refreshable_credentials = credentials, region = region, service = service), It supose tha the code will be renew the token when expired, can yougive some insight to solve this. Thansk for Sharing men
Thank you, this is very helpful. Do I need an IAM User to assume the role, or is it possible to attach the IAM Role to an EC2 Instance and get my C# Application running on it to assume the Instance Profile Role? I'm having trouble with this currently.
Thanks, Andy! I'm so glad this is helpful. This should be possible using the IAM role of the instance profile. What's the problem you're having with it?
Thanks Yann for taking the time to reply! Glad to hear that it should be possible! I'm still getting an unauthorized 403 error even after mapping the Opensearch Role to the IAM Role. I guess the problem is on the Signing Request portion as the application was built using Elasticsearch's NEST and there isn't a sample based on .NET for the AWS4Auth. I'll try again later!
@@andylbh That makes sense. I haven't tried this with .NET but I was curious just now and found this: stackoverflow.com/a/37218694/9754418 -- let us know how it goes :)
I use Cognito and followed the tutorial from the AWS documentation, but when I log in, I am still being assigned the default_role, altough it correctly assigns me to the correct master_user_role in the backend. Could you make an example on how to use Master User ARN instead of creating a local user?
What credentials did you provide at 9:44? I'm putting in AccessKeyId and SecretAccessKey from the CLI assume role step, but it says invalid username or password on the login page.
Hi Eternal Sunshine - this is separate from the AWS credentials. It’ll be the master username and password you set when you create the domain. Something like un: admin, pw: admin opensearch.org/docs/1.0/dashboards/index/ docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-opensearchservice-domain.html
Thank you for the AWESOME video man ! I really appreciate it.. I am trying to setup this using a cross account user (from another AWS account) the setup should be pretty similar I suppose.
This is a very helpful video. Thanks for that. I am trying to create AWS DMS target endpoint for Opensearch and my endpoint creation fails -- The IAM Role arn:aws:iam::xxxxxxxx:role/custom-opensearch-role1 is not configured properly.AccessDenied
You mention the internal user DB as being not suitable for production… so what should we do? From what I can tell it’s not possible to use our regular IAM users (AWS console login) to log into open search dashboard? Is a cognito managed user pool really “better” than just making opensearch users? As far as I can tell it just changes where you manage the users list since either way a team member needs to have a separate opensearch user/pass from their IAM account
I personally would recommend for a production scenario using SAML authentication so that you can manage your users in one place, “such as Okta, Keycloak, Active Directory Federation Services (ADFS), and Auth0.” See: docs.aws.amazon.com/opensearch-service/latest/developerguide/saml.html
@@YannStoneman I work on a small development team so only 2-3 of us really need access to the Opensearch instance, but we’re owned by a massive corporation that would make integrating with the IT team that runs the active directory a huge headache…. So is it actually unsafe to just have a couple internal users to log into or just not “ideal”? I’d love to have SSO set up too but it doesn’t quite seem in the cards, I just want to make sure I’m still creating a secure solution for accessing Opensearch.
@@zacbackas Nothing specific to OpenSearch in my thoughts here: it's just that with SSO, you have a single source of truth for users (which matters less with only 3 people), and you can have MFA as part of the SSO authentication process, which I guess could matter less if the dashboard is not publicly accessible and only accessible via an MFA-protected VPN or something like that. Nothing about OpenSearch is making me say this -- just the general SSO + MFA thinking.
@@YannStoneman ok thanks for the insight! Sounds like we’re safe enough for now with the internal user DB, but I’ll certainly be starting the conversation about SSO
![opensearch [01] - cert-manager](http://i.ytimg.com/vi/dXfOpp53X58/mqdefault.jpg)
Thanks for creating this. Was very helpful in understanding how we can acquire access via AWS IAM roles. Kudos!
Glad it was helpful! Thanks so much for commenting!!
Yann, thank you for sharing this video, this was very helpful.
Thanks so much, Vimal! Glad it helped.
Thanks for the video. It’s very informative and helpful!
Thanks so much Cecilia!!
This is really awesome!! I followed it and was happy I got the same error messages as you with missing indices bulk write permissions Lol...
Thanks, it helps me save so much time
Thanks so much for watching and commenting Tây!
Thank you.. Was a great video.. Really helped me out.
Thanks Suhani - I’m so glad it helped!
Thanks for creating this, but unfortunately I am getting an error. User is not authorized to perform: iam: CreateUser on resource: opensearch-poc-user with an explicit deny in a service control policy. Could you kindly suggest what should I use over here? I have followed exactly same steps as mentioned in the video?
Thx for the video. Did you figure out how to limit bulk operation for only movies index? Like you, I needed to give cluster permission but bulk operation can be done in another index. It didn't seem a secure way.
Thanks for the video!
Thank you, very helpful!
Glad it was helpful! Thanks for commenting!
Thanks. How would one generate the opensearch role mappings programmatically (a la terraform)?
I haven't tried doing it programmatically yet myself. However, since the role mappings is internal to the OpenSearch API and not part of the AWS Control Plane / API, I would use some sort of post-deployment script in Terraform that calls the Security Plugin API, perhaps using a shell_script resource or something like that.
@@YannStoneman Thanks. Make sense. Not sure why but I mistakenly assumed OpenSearch is a aws service (much like s3 or dynamodb) which it clearly is not
@@Amapramaadhy Update: it does seem like the terraform provider has included deeper support for OpenSearch over time: registry.terraform.io/providers/opensearch-project/opensearch/latest/docs
@@YannStoneman Thanks for following up. Will definitely read up. Hopefully you will cover hardening managed opensearch clusters in future videos 😉
Hey the video was really helpful Thanks, but I have a small query .Is there any way where we can map the backend role programmatically instead of doing it manually?
I had an Error: AuthorizationException message":"The security token included in the request is expired wit this line auth = AWS4Auth(refreshable_credentials = credentials, region = region, service = service), It supose tha the code will be renew the token when expired, can yougive some insight to solve this. Thansk for Sharing men
super helpful
Thank you, this is very helpful. Do I need an IAM User to assume the role, or is it possible to attach the IAM Role to an EC2 Instance and get my C# Application running on it to assume the Instance Profile Role? I'm having trouble with this currently.
Thanks, Andy! I'm so glad this is helpful. This should be possible using the IAM role of the instance profile. What's the problem you're having with it?
Thanks Yann for taking the time to reply! Glad to hear that it should be possible! I'm still getting an unauthorized 403 error even after mapping the Opensearch Role to the IAM Role. I guess the problem is on the Signing Request portion as the application was built using Elasticsearch's NEST and there isn't a sample based on .NET for the AWS4Auth. I'll try again later!
@@andylbh That makes sense. I haven't tried this with .NET but I was curious just now and found this: stackoverflow.com/a/37218694/9754418 -- let us know how it goes :)
I use Cognito and followed the tutorial from the AWS documentation, but when I log in, I am still being assigned the default_role, altough it correctly assigns me to the correct master_user_role in the backend. Could you make an example on how to use Master User ARN instead of creating a local user?
Thanks for watching falc410. Could you please share the link to which part of the documentation you’re following and any additional details?
What credentials did you provide at 9:44? I'm putting in AccessKeyId and SecretAccessKey from the CLI assume role step, but it says invalid username or password on the login page.
Hi Eternal Sunshine - this is separate from the AWS credentials. It’ll be the master username and password you set when you create the domain. Something like un: admin, pw: admin
opensearch.org/docs/1.0/dashboards/index/
docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-opensearchservice-domain.html
Thank you for the AWESOME video man ! I really appreciate it.. I am trying to setup this using a cross account user (from another AWS account) the setup should be pretty similar I suppose.
Yes, this should work the same cross-account. Glad it helped!
This is a very helpful video. Thanks for that. I am trying to create AWS DMS target endpoint for Opensearch and my endpoint creation fails -- The IAM Role arn:aws:iam::xxxxxxxx:role/custom-opensearch-role1 is not configured properly.AccessDenied
Thanks so much for commenting. I’m glad it helped! Please share how to recreate the error (but continue blocking out private details).
what username and password that you used to log in the dashboard?
Quickest way to access OpenSearch dashboard
You mention the internal user DB as being not suitable for production… so what should we do? From what I can tell it’s not possible to use our regular IAM users (AWS console login) to log into open search dashboard? Is a cognito managed user pool really “better” than just making opensearch users? As far as I can tell it just changes where you manage the users list since either way a team member needs to have a separate opensearch user/pass from their IAM account
I personally would recommend for a production scenario using SAML authentication so that you can manage your users in one place, “such as Okta, Keycloak, Active Directory Federation Services (ADFS), and Auth0.” See: docs.aws.amazon.com/opensearch-service/latest/developerguide/saml.html
@@YannStoneman I work on a small development team so only 2-3 of us really need access to the Opensearch instance, but we’re owned by a massive corporation that would make integrating with the IT team that runs the active directory a huge headache…. So is it actually unsafe to just have a couple internal users to log into or just not “ideal”? I’d love to have SSO set up too but it doesn’t quite seem in the cards, I just want to make sure I’m still creating a secure solution for accessing Opensearch.
@@zacbackas Nothing specific to OpenSearch in my thoughts here: it's just that with SSO, you have a single source of truth for users (which matters less with only 3 people), and you can have MFA as part of the SSO authentication process, which I guess could matter less if the dashboard is not publicly accessible and only accessible via an MFA-protected VPN or something like that. Nothing about OpenSearch is making me say this -- just the general SSO + MFA thinking.
@@YannStoneman ok thanks for the insight! Sounds like we’re safe enough for now with the internal user DB, but I’ll certainly be starting the conversation about SSO
Nice video. But now I see the value in elasticsearch cloud.
😂
You are amazing thank you very much buddy
Thanks so much Bakshim!!