Storing secrets CORRECTLY in .NET using AWS Secrets Manager

Поделиться
HTML-код
  • Опубликовано: 25 ноя 2024

Комментарии • 119

  • @metaltyphoon
    @metaltyphoon 2 года назад +8

    A tip. If you are setting user secrets on a unix terminal and that value has !, make sure to scape it because it gets interpreted by the shell. That happen a lot when setting connection strings where the password has special characters.

    • @kmorosiuk
      @kmorosiuk Год назад

      Yeah, I got around this issue by using single quotes in bash: dotnet user-secrets set "SecretName" 'SecretValueWith!'

  • @strgaltphil
    @strgaltphil 2 года назад +6

    You must have acces for my browser history. Googled that an hour ago. Thank you for your video!

  • @AlFasGD
    @AlFasGD 2 года назад +3

    The !! feature was stripped away from C# 11, you could touch that topic and give a few bits on decisions like this one

  • @MrAyuub22
    @MrAyuub22 2 года назад +3

    Hey, how would you use one secret key with many key value pairs containing the applications secrets? This would help with keeping costs down

  • @177sams
    @177sams Год назад +2

    Hey Nick, thank you for the explanations. How the code knows which AWS environments to hit? All I was passing a AWS Region Endpoint. In my example it is East-1 for QA and West-2 for Prod.
    I am not able to understand how nuget package able to download the secret by using a region endpoint?

  • @renatogolia211
    @renatogolia211 2 года назад +4

    I'm the author of AWSSecretsManagerConfigurationExtensions. Thank you for featuring my lib. Do you mind if I add a link to your video in the README of the repo?

    • @nickchapsas
      @nickchapsas  2 года назад

      Absolutely not, please go ahead and thank you for the library. You've done a great job.

    • @renatogolia
      @renatogolia 2 года назад +2

      @@nickchapsas It seems I need to add a tutorial on how to pronounce Kralizek tho :P

  • @vamvdotnet
    @vamvdotnet 2 года назад

    I learn so much from you in every video you bring to us!

  • @orening
    @orening Год назад +2

    Hi Nick, I am trying to solve a problem when loading secrets from AWS that change.
    In the DI i use objects that use values from those secrets, and sometimes those are added as singletons.
    How do i get those objects updated with the new values?
    For example, i have an sqlsettings object that is loaded with values from AWS:
    builder.Services.AddOptions().BindConfiguration(nameof(SqlSettings)).ValidateDataAnnotations().ValidateOnStart();
    builder.Services.AddScoped(resolver => resolver.GetRequiredService().Value);
    And later add the SqlClient
    builder.Services.AddSingleton();
    Where SqlClient constructor uses the SqlSettings to initiate a connection (or something like that)
    After a change the secret value, the SqlSettings object changes, but the SqlClient does not.
    Is there a solution to this, or is this singletons working as intended?

  • @Kneebreaker
    @Kneebreaker 2 года назад

    Thank you! Looking forward to any/all aws & dotnet content!

  • @superpcstation
    @superpcstation 2 года назад +4

    One of the very first things I built in my very first proper dev job was a program that had to keep the Dropbox API key a secret. Since a .NET program can be reverse engineered, that's no good. If for example you keep the secret in an encrypted sqlite database, now you have to keep the database password safe. If you hardcode it in your code, it can be discovered.
    These secret managers i feel suffer from the same problem. You put your secrets in a secret manager, but then where do you put you secret manager credentials? Where previously you had to keep your secret safe, now you have to keep the credentials for you secrets manager safe. What am I missing?

    • @nickchapsas
      @nickchapsas  2 года назад +2

      You don't need credentials. In AWS you authenticate using IAM Permissions and security policies. You define that X microservice has access to Y AWS service and AWS knows who to allow in and who to deny access to based on that.

    • @superpcstation
      @superpcstation 2 года назад +3

      @@nickchapsas Thanks for the reply, Nick. Honestly I really appreciate the fact that you read the comments and take time to reply. RUclips's comments are not exactly the best thing for a person's mental health so i hope you do take care of yourself.
      Anyway, i should have clarified that i'm talking about a plane old desktop app. For example if a microservice is running on the cloud and you need to talk to it, you probably need some sort of API key or credentials. Now how are you going to keep those credentials safe? I ultimately ended up believing that managed languages simply can't keep secrets though i would love to be proven wrong.

    • @nickchapsas
      @nickchapsas  2 года назад +2

      @@superpcstation You can't securely authorize a desktop app directly to the secrets manager. You'd have to add some form of user authentication and then build an API that the app communicates to and that API is the authorized one. There are always ways but they are relative to your usecase.

    • @tuxino
      @tuxino 2 года назад +3

      @@superpcstation
      This is not unique to managed languages. Any software can be reverse engineered, and if that software has credentials within, they can be extracted.

  • @PatrickFortunatoJr
    @PatrickFortunatoJr 2 года назад

    I think you did a great job here, but I have 2 issues. First, using pattern matching for names can be error prone. A simple naming error can cause problems. Second, from a security standpoint, you should not be grabbing all the secrets and discarding them. It is more secure to grab only the ones you need by name.

    • @nickchapsas
      @nickchapsas  2 года назад +1

      The filter will only grab the ones you need. It’s a server side filter. Not having a pattern causes conflicts. It’s a very standard practice. If you had a type the secret wouldn’t be loaded in the first place and the app wouldn’t start

    • @keithcarrillo8238
      @keithcarrillo8238 Год назад

      Hi Patrick, do you know how he sets up "dev" and "prod" and pairs it to "ConnectionString" in appsetting.json?. In the video it doesn't show where he gets those from but he's able to use it first-thing in the video.

  • @mersy4405
    @mersy4405 2 года назад +1

    Hi Nick, I have a question that hope you can reply me,
    I'm following your video and its works on local but when I deploy it to production mode it got an error "Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service.". So how can I get a credential for my production mode? Tks for your time

    • @nickchapsas
      @nickchapsas  2 года назад

      Hey there. You would need to create a policy for your role that allows access from your EC2 to the Secrets Manager

  • @IwillFindyousoonorlater
    @IwillFindyousoonorlater Год назад

    Thank you for this great video, but I have a question that's been on my mind. If a hacker were to gain access to my EC2 Windows server, implementing the solution of not allowing the connection string to be read from the appsetting.json file would prevent them from accessing it. However, the hacker could potentially use a tool like dnSpy to reverse engineer the code and extract the connection string. Using an obfuscator would also prevent the hacker from being able to read the connection string. So why would I need this solution?

  • @antonmartyniuk
    @antonmartyniuk 2 года назад

    I will definetely like to see an AWS course

  • @tea_otomo
    @tea_otomo 2 года назад +1

    Would be nice if you would explain, why you did not need to provide credentials at 12:19. Especially, when the app would be in production...where are the credentials for the cloud? This is usually always a loophole, where you need to store credentials somewhere, just to access the secrets from the cloud.

    • @nickchapsas
      @nickchapsas  2 года назад +1

      There would be no credentials. Your pod (k8s), task (ECS) or VM (EC2) will be authed against the service via an IAM role that defines what the service can and cannot access.

    • @raygan3
      @raygan3 2 года назад

      @@nickchapsas I also would like to know how can i authenticate my production VPS (ubuntu) to access aws secrets

    • @nickchapsas
      @nickchapsas  2 года назад +4

      @@Qrzychu92 No they don't need to. You just set up access control on EKS. Here is the process: docs.aws.amazon.com/secretsmanager/latest/userguide/integrating_csi_driver.html

  • @michelchaghoury9629
    @michelchaghoury9629 2 года назад +3

    Is there always free services that we as developers can use, like services that do not have a trial an always free? If so can you please make a vid about those services and if they are usefull and how they can help us pleasee

    • @nickchapsas
      @nickchapsas  2 года назад +6

      I only make videos on things I have used in production, for a long time at scale. There is probably some free version to manage secrets too but I am not aware of it and I won't be making videos on things I am not proficient at.

    • @metaltyphoon
      @metaltyphoon 2 года назад

      For secret management you can use Hashicorp’s Vault.

  • @Gonzo345
    @Gonzo345 Год назад

    Loving it. Thank you so much!

  • @wh33lers
    @wh33lers 2 года назад

    Thank you for sharing. Using SecretsManager for a while now and really love the concept of using a separate service for my secrets. Keep the AWS content coming.

    • @keithcarrillo8238
      @keithcarrillo8238 Год назад

      Hi Wheeler, do you know how he sets up "dev" and "prod" and pairs it to "ConnectionString" in appsetting.json?. In the video it doesn't show where he gets those from but he's able to use them first-thing in the video.

  • @leonardorf95
    @leonardorf95 2 года назад

    Hello Nick, I hope you are well, you see I have the following error when starting the app, Unable to get IAM security credentials from EC2 Instance Metadata Service., I know I need to configure the iam role but I can't find exactly how to do it, do you have any resources that I can follow
    Greetings from Mexico

  • @nbktube1171
    @nbktube1171 2 года назад +1

    Hey Nick, gr8 vid... quick question, how would this work in elastic beanstalk?

    • @ronsijm
      @ronsijm 2 года назад

      Oh I just commented that in another comment:
      If you're using AWS Elastic Beanstalk, you can (kinda) set these LaunchSettings in the Environment properties: docs.aws.amazon.com/elasticbeanstalk/latest/dg/environments-cfg-softwaresettings.html
      Reading it back in dotnet doesn't seem to work with the example they're giving (using ConfigurationManager.AppSettings) - but you can use this helper to merge it with your existing config: pastebin.com/wwzcbsty

    • @nbktube1171
      @nbktube1171 2 года назад

      @@ronsijm thx. Lying in bed with covid ATM, but will defo give that a shot

    • @AbdullahKhan-hz7yr
      @AbdullahKhan-hz7yr 2 года назад +1

      Works exactly the same way as demonstrated by Nick in the video. All you need to do is modify the default aws-elasticbeanstalk-ec2-role and ensure it has access to the secrets manager and hey presto, magic!!

  • @disturp
    @disturp 2 года назад +3

    This video is great, however, it would nice to know how to load that secret when you register your servces.AddDbContext(), instead of using it in the ApiController.

    • @nickchapsas
      @nickchapsas  2 года назад +1

      It’s the same since it is loaded in the configuration provider. You just point to the config name and it will be loaded assuming your app has access to secrets manager

    • @mersy4405
      @mersy4405 2 года назад

      Hi, have you find the solution for that?

  • @hlosanidube-mpofu7328
    @hlosanidube-mpofu7328 2 года назад

    Great vid Nick..thanks

  • @marcelfernandes5000
    @marcelfernandes5000 2 года назад

    Hello guys?
    Can someone help me with this error here?
    "is not authorized to perform: secretsmanager:listsecrets because no identity-based policy allows the secretsmanager:listsecrets action"
    It's working when I'm running my API local and getting the secret from the AWS, but it's not working when I publish the API to AWS

  • @verdurakh
    @verdurakh 2 года назад

    How does this compare to aws parameters and app config?
    Seems to be able to do almost the same thing but is more expensive?

    • @nickchapsas
      @nickchapsas  2 года назад +1

      Parameters store and app config is for general parameters and app configuration. Secrets manager is specifically targeted towards secrets and it has features that you’d only need for secrets

  • @mutazadil5119
    @mutazadil5119 2 года назад

    I am very beginner building c# windows form application with firebase database || how to secure my connection please help !!

  • @tinypanther27
    @tinypanther27 2 года назад

    Very well paced video! Do you have a video about authenticating developer machines for AWS / IAM ?

    • @pilotboba
      @pilotboba 2 года назад +1

      Is basically a credentials file in your ~/.aws folder.
      Google AWS Configuration and credential file settings and it will walk you through it.

  • @tupitech4038
    @tupitech4038 2 года назад

    Hi Nick, thank you very much for this video. I was wondering what would be the case if the Secrets Manager is unavailable or returns some sort of throttling error. How does the package deal with it? Is it possible to configure any resilience or retry pattern? For ECS or EC2 it would be fine, however for lambdas it would be important.

  • @CloudzardTechnologies
    @CloudzardTechnologies 2 года назад

    Hello Nick, first of all great video love your content! I have a question, everything works as expected when running locally, however when I publish to my EC2 I'm still getting the default appSettings value in this case "prod". Any idea why this can be happening? Also I double checked and I'm authenticated against AWS through IAM creds.

  • @ghostslinger3148
    @ghostslinger3148 2 года назад

    Hi Nick, awesome vid,
    Just having a bit of trouble making it work by deploying a docker container to AWS ECS.
    Getting the error: Unhandled exception. Amazon.Runtime.AmazonServiceException: Unable to get IAM security credentials from EC2 Instance Metadata Service.
    Do you have a video that goes through getting a db connection string from either this secret manager or ssm parameter store while running a container on aws ECR,
    Would like to know the best practices for AWS, cause at the moment what I can think of is only giving the credential of an IAM user with only getparameter access, but still it would mean showing the secret key in code.

    • @nickchapsas
      @nickchapsas  2 года назад

      I don’t but you don’t need a connection string. You need to create a policy which allows your ECS task definition to see the secrets

  • @tinypanther27
    @tinypanther27 2 года назад

    Hey this was a great video!
    A slightly off topic question, for the nuget package you used (or in general), how do we know the code on github is the same as the package published through nuget?

    • @nickchapsas
      @nickchapsas  2 года назад +1

      You don’t. You have to manually check it, which is what I’ve done for packages like this. In fact for this one I am using a private fork because I needed to change some logic for my use cases

  • @ansariamin3865
    @ansariamin3865 Год назад

    Guyz I am trying to integrate the secret Manager on on premise web server…. We have jboss eap which is connecting to cloud database now I want to mask the id and password using secret manager on the on prem server… have tried multiple method online but nothing seems to work ……does anyone have document for this will by much help

  • @ArgeKumadan
    @ArgeKumadan 2 года назад

    If u can have a different implementation of that SecretManager packager, u can basically store ur secrets in your database(or seperate db) in a seperate table. So the question here, why do we need this AWS Secret Manager? Obviously your doesn't know where ur secrets are.
    Why not just use something like table-storage or RDS or something else? What is the key benefit of SecretManager?

    • @nickchapsas
      @nickchapsas  2 года назад +7

      Because Secrets Manager is built to store and load secrets in a secure way and it also gives you features such as the secret rotation out of the box. If there is a breach in your database and you have the keys in there you are basically done. Also, how do you get access to a database when the keys for the database are in there? There are too many factors you need to take into account to make such an implementation yourself. It also gives you super fine-grained polices for managing the access to the keys and things like secure audit.

    • @PatrickFortunatoJr
      @PatrickFortunatoJr 2 года назад

      @@nickchapsas If you are all in within AWS, you can use IAM roles to access your resources instead of secrets. I did a .NET POC of this years ago where SQL RDS was using IAM roles as credentials instead of the standard username/password configuration.

  • @neilsg2001
    @neilsg2001 2 года назад

    Actually, is there ways to safe store stuff in mem?

    • @nickchapsas
      @nickchapsas  2 года назад

      Not really. As soon as they are in memory they will need to be accessed and used. Even if they are encrypted and decrypted in memory you can't guarantee that they key won't be there too

    • @neilsg2001
      @neilsg2001 2 года назад

      @@nickchapsas Thanks Nick, I was thinking of things like securestring which is now deprecated. I wonder is there anything in win API...

  • @moranmono
    @moranmono 2 года назад

    Great video. Thanks for the information

  • @lizzaelvillarcruz2282
    @lizzaelvillarcruz2282 2 года назад

    Hey Nick Chapsas, can you do a similar video for Azure?

    • @nickchapsas
      @nickchapsas  2 года назад +1

      I have one on Azure KeyVault already

  • @deepakdagar_change_is_in
    @deepakdagar_change_is_in 2 года назад

    Can these be accessed in ci cd Pipelines

    • @nickchapsas
      @nickchapsas  2 года назад

      Sure if you auth the pipeline to your environment

  •  2 года назад +1

    Nicely explained. The only thing I'm skeptical about is the 3rd party nuget package for configuration. I could see a potential supply-chain attack on a library maintained by a solo developer.

    • @oliverrc_ww
      @oliverrc_ww 2 года назад

      If you are genuinely worried about that then there are two options I can see you taking.
      1) Fork the code or clone it to you machine, read it thoroughly and reference your copy where you fully understand and can ensure the code is not doing anything dodgy.
      2) Pin the Nuget version to a specific version that you have code reviewed and are happy taking. If you target that specific version then any further updates to the codebase should not affect you.

    • @shovitk1
      @shovitk1 Год назад

      @@oliverrc_ww Thank you for this response. @Dalibor's concern was the same as mine.

    • @keithcarrillo8238
      @keithcarrillo8238 Год назад

      Hi Dalibor do you know how he set up "dev" and "prod"? In the video it doesn't show where he gets those from but he's able to use it first-thing in the video.

  • @victorgarcia3526
    @victorgarcia3526 2 года назад

    Why don't you use Azure Key Vault?

    • @nickchapsas
      @nickchapsas  2 года назад +1

      Because I am not on Azure. If I was on Azure I would use Key Vault

    • @victorgarcia3526
      @victorgarcia3526 2 года назад

      @@nickchapsas Thanks for your answer! I think my question was more related to "why you chose Aws instead of Azure", a poor selection of words by me, I'm sorry 😅

    • @nickchapsas
      @nickchapsas  2 года назад +2

      @@victorgarcia3526 It's ultimately not a developer decision but a business one. Both offer similar services with small differences in functionality. Judging factions will be some cloud exclusive features (which are usually rare), business relations and pricing. Developers don't make those calls.

  • @ColinM9991
    @ColinM9991 2 года назад

    14:58 - MY approach?!

    • @nickchapsas
      @nickchapsas  2 года назад +1

      This video was dedicated to you Colin

    • @notmeprobably_
      @notmeprobably_ 2 года назад

      @@nickchapsas I will remember this

  • @scott98390
    @scott98390 2 года назад +1

    ...and when AWS goes down, so does your app, no matter where you hosted. How do you mitigate this risk - use Azure KeyVault in parallel?

    • @nickchapsas
      @nickchapsas  2 года назад +2

      AWS isn’t a single entity. You have georeplication in place for high availability

  • @lincolntx98
    @lincolntx98 2 года назад

    i could understand the improvements on using the secret manager in local development, but in production where i use containers and k8s and my env is stored in an ,env file, why should i change to the aws secret manager?

    • @nickchapsas
      @nickchapsas  2 года назад +1

      Because you shouldn't store those in an env file, that's dangerous. The env files should point either to the secrets manager ARN or use K8s' secrets managers directly

    • @lincolntx98
      @lincolntx98 2 года назад

      @@nickchapsas got it, thanks for the explanation

  • @EvekoShadow
    @EvekoShadow 2 года назад

    key vault master race

  • @notmeprobably_
    @notmeprobably_ 2 года назад

    Press 5. It's essential that you know what the name of this package is.

    • @nickchapsas
      @nickchapsas  2 года назад

      No it's not. Context is way more important and understanding why we will do what we will do is way more important. The package is irrelevant

    • @notmeprobably_
      @notmeprobably_ 2 года назад

      @@nickchapsas Sounds like someone didn't press 5

    • @nickchapsas
      @nickchapsas  2 года назад +1

      @@notmeprobably_ Wait was this a joke because I couldn't pronounce the name? 😂

  • @diligencehumility6971
    @diligencehumility6971 2 года назад +13

    The new thing is "cloud provider independence".
    If you have libraries to support AWS features in the stomach of your codebase, you are dependent in a way your don't wish for your codebase.
    What we've seen with Parler and other services hosted at AWS, they were illegally shut down, censored, for no other reason than political.
    Supporting AWS after knowing this is one thing, but what if you get shut down? Can you take your codebase and host it another place? -If you wrote your code in an appropriate way, yes you can. Any service in fact. But using AWS specific features? No. You will have to re-write large parts of your codebase, and wait weeks if not months, before you are back online.
    This is even worse for Firebase developers. I don't understand them.

    • @nickchapsas
      @nickchapsas  2 года назад +8

      You are missing the point. This isn't about supporting AWS, Azure, GCP or whatnot. This is about people who are getting into AWS or are already in it and they want to know how to do this. People can talk about "cloud provider independence" all they want but good luck finding a cloud agnostic version of DynamoDB or CosmosDB. And no, MongoDB or Couchbase are both a nightmare to manage and scale and I'm happy I don't work with them anymore. You think that Microsoft can't stop supporting aspect of .NET at any point? Where is .NET Framework? Where is WinForms? Where will Blazor be if it keeps being niche? This "tech independence" on any level is a lie people are believing.

    • @Omego2K
      @Omego2K 2 года назад +1

      @@nickchapsas I think his idea is creating libraries that are provider agnostic and easier to replace providers using the abstractions you create during development. Provider locks is also something I am tense over.

    • @mateusnc
      @mateusnc 2 года назад

      ​@@nickchapsas Agree, the concept here is "do not store secrets in your appsettings", you have showed one of many ways to do it and this can be easily rewritten if needed. Anyway, having a package and a few lines in your startup project ins't a vendor lock-in.

    • @nickchapsas
      @nickchapsas  2 года назад

      @@mateusnc It's probably the least invasive way to implement this. You can look at the Azure KeyVault equivelent video that I have out and the setup is basically identical since they're both built on top of a configuration provider

    • @AndrewJonkers
      @AndrewJonkers 2 года назад +1

      Xactly. While I totally agree with Nick that this was not at all the point of the video, you do make a valid point about the conflict between commercial profit and a cloud agnostic consumer friendly standard for a type of service. In the last 40 years in the business, every time I have seen a proprietary solution, within 2 years it has turned out to be an expensive option from which it can be hard to escape. It is the cost plus mentality.

  • @javedhshaik6069
    @javedhshaik6069 2 года назад

    Too lengthy lectures, please try to keep it short and straight

    • @nickchapsas
      @nickchapsas  2 года назад +1

      The length is relative to the information that is needed for people to understand the full context. If it was shorter it would be a bad video

  • @TheAzerue
    @TheAzerue 2 года назад

    Hi
    One question, i saw you previous video ruclips.net/video/J0EVd5HbtUY/видео.html title "Managing your .NET app configuration like a pro". Both are solving same problem which is loading secrets securely in application. Can you please share yours though which one should be used over other in which scenarios.

    • @nickchapsas
      @nickchapsas  2 года назад

      They don't solve the same problem. AWS Parameter Store is, as the name suggests, a general parameter store and all you app's configuration can go in there. Secrets manager is built for secrets management and it involved encryption and native secret rotation. They solve different problems.

    • @TheAzerue
      @TheAzerue 2 года назад

      @@nickchapsas When creating new parameter In types we have option "SecureString" which will be encrypted using KMS. We could have plain text as well as encrypted string. Secret Manager is also encrypting using KMS. Please bear with me if my question sounds silly. Thank you.

  • @hunterwebapps5091
    @hunterwebapps5091 2 года назад

    Azure > AWS

  • @jeffrdrama7984
    @jeffrdrama7984 2 года назад

    Can you do a similar video but about azure key vault?

  • @frmn4678
    @frmn4678 2 года назад

    You don't need to replace __ to :. Dotnet eats double uderscore well as I know, at least it does it for my k8s secrets injected as a env variables.

    • @nickchapsas
      @nickchapsas  2 года назад

      .NET does indeed accept double underscores but for some reason it wasn’t working in one of the demos I did so I went with the approach that was working for all my examples