AWS Secrets Manager - Rotate Secrets
HTML-код
- Опубликовано: 11 июл 2024
- In this video we will learn how to enable rotation of secrets in AWS Secrets Manager. This is a very useful feature which can provide an extra level of security for your Cloud applications. We will configure rotation for an existing Secret and write our own rotation function using AWS Lambda.
Timestamps:
- Intro: 0:00
- Secrets Manager config: 1:27
- Create rotation function: 5:19
- Configure IAM policies: 17:24
- Finish Secrets Manager config: 18:04
Code example: github.com/endre-synnes/pytho...
Code example from AWS Documentation: github.com/aws-samples/aws-se...
Other function templates for any type of secrets: docs.aws.amazon.com/secretsma...
IAM Policy for Secret Rotation: docs.aws.amazon.com/secretsma...
AWS Documentation - Enable Secret rotation (step by step): docs.aws.amazon.com/secretsma...
Subscribe button animation: touchtechnologyreview.com/sub...
Follow me on Github: github.com/endre-synnes
Thank you so much! You explained it really well. Helped me alot.
Thank you! I'm glad you found It helpful 😄
Simply superb
Endre, This is a really good video. I really enjoyed it. It is very informative. You touched on a very unique topic.
Thank you so much! 😄 Yes, rotating secrets isn't the most eye catching topic but still an important one 😄
this is amazing, thanks so much i was able to finish my work due to this lesson!!!
Thank you! I'm glad to hear that It was helpful :D
Hi,
I am getting below error :
" File \"/var/task/lambda_function.py\", line 11, in lambda_handler
arn = event['SecretId']
"
Let me know how you have passed below variable values, especially "ClientRequestToken":
arn = event['SecretId']
token = event['ClientRequestToken']
step = event['Step']
Thank you so much for the valuable time you spent on this video!
Thank you so much, I'm glad you liked It! 😄
You're a life saver, thanks
Thank you! I'm glad it was helpful 😄
Thanks it's working for me
That's great! :D
This video is a gem ❤
Thank you so much! 😄
nice instruction.
Thank you!😄
Hi Endre Synnes. Very nice explained. Do we have versioning for old/expired secrets? If no, what is the workaround to get the expired secrets? Kindly suggest.
Hi, thank you so much! 😄 Yes it is possible to retrieve the previous version of a secret, as described here: docs.aws.amazon.com/cli/latest/reference/secretsmanager/get-secret-value.html
I would also recommend reading this (docs.aws.amazon.com/secretsmanager/latest/userguide/getting-started.html#term_version) to get a deeper understanding of how Secrets Manager deals with versions of a secret.
I hope this was helpful 😄
@@EndreSynnes Thankyou once again Endre. Yes while using 'AWS PREVIOUS ' argument in Cli , previous password gets retrieved. One more question is it possible to retrieve it by gui also? Kindly suggest.
My pleasure😊 I haven't figured our a way to retrieve previous values using the AWS Console (gui), so my guess is that it's not possible 🤔
@@EndreSynnes I would need your one support that how can we append logic in existing lambda function that our secrets get store in SSM parameter store once rotate. Could you please help me.
Hi Endre, After key rotation the values comes as plain text and not showing as key value pair. I need to use the credential as key value pair at my code end. How can we achieve that??
Hi 😄
I'm sorry for the late response and the confusion this implementation may have caused. As described in the example in the boto3 documentation (boto3.amazonaws.com/v1/documentation/api/latest/reference/services/secretsmanager/client/put_secret_value.html), you can just as well provide a stringified JSON object to the SecretString parameter. This way you will be able to keep the secret as a key value pair.
I have created an updated version of the code (look at line 41): github.com/endre-synnes/python_aws_course/blob/4b51724e6542c0ddfe38a1d8a0609a0cbc637a5f/lambda_intro/04_secrets_and_databases_and_stuff/17_rotate_secret_UPDATED.py#L41
I hope this was helpful, and thank you for pointing it out! 🙌
Hi, what can i do if i want to set the newly rotated key as my new api key? I mean how to reflect the changes in secret manager back to api gateway?
Hi 😄
This depends on how your application is deployed, but preferably your API should call AWS Secrets Manager to verify that the API key is correct. This way Secrets Manager can be your only source of truth, and you will always get the newest/correct version.
If you for example use AWS API gateway then you can create an Authorizer Lambda function which calls Secrets Manager to verify that the API key is correct. docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-use-lambda-authorizer.html
Or you can use the "set_secret" step in the rotate secret function to update the API key wherever it is needed 😄
I hope this was helpful!
Is it possible to set a triger from this secret manager, that when secret rotation occurs triger an event for event bridge?
Hi, I'm sorry for the late reply.
I have not tested this myself, but i'm not sure Secrets Manager rotate events can trigger an Event Bridge event by itself. I think you should be able to do this by creating a CloudTrail tho, or sending the event to Event Bridge from a Lambda function.
Maybe I can look into this in a future video 😄
@@EndreSynnes Thank you 👍
Hi, can we use secret manager rotation to automatically update my key-pair in ec2 instances
Hi 😄
It seems to be possible, but I haven't tried it myself. This article/blog post goes into the details of how to do it: aws.amazon.com/blogs/security/how-to-use-aws-secrets-manager-securely-store-rotate-ssh-key-pairs/
I hope this was helpful! 😄
once secret/password is updated, do we not need to update to the api/db? If not, could you help me how to set_secret.. I am using RDS PostGre SQL DB.
Hi 😊 Depending on how your applications utilise secrets in AWS Secrets Manager, you may not need to do any changes there when secrets are rotated. Of course if the secret values are cached by you applications, then they would need to be flushed/updated. When it comes to your RDS database, then you may need to use the "set_secret" method in your rotation Lambda function. I have not done that myself with PostgreSQL databases, but the examples in the AWS documentation should be a good starting point: docs.aws.amazon.com/secretsmanager/latest/userguide/reference_available-rotation-templates.html
I hope this was helpful 😄
If i encrypted the data with the secret key and stored the encrypted data in database and then key is rotated. Now how can I decrypt the encrypted data present in db ?
Hi,
Sorry for the late reply. If you are thinking about encryption keys, then AWS KMS (aws.amazon.com/kms/) is probably the service you are looking for. Secrets Manager secret rotation is more suited for the password of the database itself. Because as you said, you can't decrypt encrypted data with another key that didn't exist when you encrypted the data.
I hope this was helpful! :)
@EndreSynnes - I am getting below error. Could you please tell if we need to pass any values to arn, token and step? I have defined them as below:
arn = event['SecretId']
token = event['ClientRequestToken']
step = event['Step']
Error Message
----------------------
{
"errorMessage": "'SecretId'",
"errorType": "KeyError",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 11, in lambda_handler
arn = event['SecretId']
"
]
}
Is your issue resolved? Because I am also facing same issue. Could you let me know what to be passed to below variables, especially how to get "ClientRequestToken" details from aws?
arn = event['SecretId']
token = event['ClientRequestToken']
step = event['Step']
Hi,
I can't manage re-produce the issue you have there. I just now tested the lambda function with a new Secret stored in Secrets Manager and it worked just fine. May I ask if the three properties (arn, token and step) are defined inside your lambda handler function, or have you defined them as field variables (outside the lambda handler function)?
If you have written the code as in the example bellow, the three properties should be successfully extracted from the event parameter/object.
def lambda_handler(event, context):
arn = event["SecretId"]
token = event["ClientRequestToken"]
step = event["Step"]
@@EndreSynnes its working.. Now I am getting error as "Module pg cannot be imported".
The same issue with python 3.8 and 3.9.
Could you please confirm which version of python you are using and how to resolve this import module issue.
Hi, I'm using python 3.9 for that specific Lambda function.
I suppose that "pg" is a library you use to communicate with a PostgreSQL database? If so, you probably have to install it into you lambda environment. AWS Lambda does come with some libraries pre-installed, like uuid, urllib3, boto3 etc. but you have to install other 3rd party libraries yourself. For example by using a requirements.txt file or pipenv. I would also recommend looking into Lambda layers (docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html), which may be a topic for another video later on 😄
Does it applicable for multiple secret keys?
Hi 😄
I'm not sure what you mean, do you mean for multiple API keys in the same Secret?
If so I would not recommend it since then can't control access to each specific API key.
@@EndreSynnes Thanks, For me it doesn't work. Failed to rotate the secret, I did exactly what you did but I can't able to rotate the keys. Any Idea about that?
@@mohamedirfan5242 try adding the arn and other details of the secret manager in the trigger block o the labd function. it will rotate
@@anniekoshy1416 Thanks, I don't know how to add, There is any reference to that?
It should not be necessary to explicitly add Secrets Manager as a lambda trigger. Secrets Manager should take care of triggering the Lambda Function after you have enabled rotation in the Secret configuration. But secrets manager needs access to trigger the Lambda function, as shown here (ruclips.net/video/7wkpf0u34Qs/видео.html) in the video. :)
Do you have any logs in CloudWatch that may tell us something about what the issue could be? Or isn't there any logs? :)
how to handle the key error in secrets manager .please help me
Hi 😄
I need more context to understand the issue. Is it your python code that is throwing a Key error exception? If so, It would be helpful if you could provide your code or stack trace.
Hi...when i test the lambda code. I am getting the error.....
{
"errorMessage": "'SecretId'",
"errorType": "KeyError",
"requestId": "xxxxxxxxxxxxxxxx",
"stackTrace": [
" File \"/var/task/lambda_function.py\", line 4, in lambda_handler
arn = event['SecretId']
"
]
}
Where would i have gone wrong?........
Hi 😄
How are you testing your code? Normally this error means that the key "SecretId" is not in the event object.
I guess this would happen if you try to test the lambda function without providing the expected parameters inside the event object. When AWS Secrets Manager calls this function, then Secrets Manager should provide every parameter that is needed automatically.
Let me know if this was helpful 😄
Actually i followed everything a to z in your video. Since i am new to this, i do not know what else extra has to be done. Like passing secretId, client request token or the step....any resource to know about that?
That's strange, maybe you can provide the first lines of your lambda function code?
import boto3
def lambda_handler(event, context):
arn = event['SecretId']
token = event['ClientRequestToken']
step = event['Step']