5 JavaScript API Key Mistakes (and how to fix them)
HTML-код
- Опубликовано: 21 сен 2024
- Don't let other developers find your API keys in your JavaScript applications. In this video, we'll cover 5 mistakes I commonly see JavaScript developers make when it comes to API keys. We'll talk about environment variables, .env files, ignoring files with the .gitignore file, proxy backend API endpoints, and more!
Video from Ania Kubow - • How to hide your API k...
cors NPM Package - www.npmjs.com/...
dotenv NPM Package - www.npmjs.com/...
STAY IN TOUCH 👋
Check out the Podcast - compress.fm
Newsletter 🗞 - www.jamesqquic...
Live streams on Twitch 🖥️ - / jamesqquick
Follow me on Twitter 🐦 - / jamesqquick
QUESTIONS...?
Join the Discord Server 💬 - / discord
Want to know what hardware and software I use? www.jamesqquic...
Learn Web Development 📖 - www.jamesqquic...
Please PLEASE make a follow up. This is such an important concept and most tutorials and sources don’t give it enough attention. I’m currently struggling with these concepts myself.
I can build a backend and secure a server side application. But I have no idea how to go around protecting a request from the front end for example. I had been stuck on how to handle API requests that require a token called from a front end application. This video gave great input on it
I’ll get to work on that :)
@@JamesQQuick YAY!! Thank you!
I used a server side rendered http only secure cookie over SSL (HTTPS) connections. With each request from a browser, the cookie is sent along with encoded information that is bound to user bound session data that automatically invalidates itself without user interaction.
Here's the full story, it's REALLY long:
In this cookie I store information that is encoded/encrypted by the server so its hard for normal people to read it (security by obscurity step).
I bind user information to the cookie, so that the value cannot be used to impersonate another user (security by isolation).
A session is instantiated by the server with an guaranteed unused session id. The session contains one encryption key, and one validation key. The validation key is part of the encrypted payload.
The session id is also stored in the cookie, but not encrypted and serves as a way to bind a user and session together using a cookie.
Each time the cookie is sent to the server, it can quickly lookup the associated session, decrypt the payload using the random generated encryption key, and validate the cookie stored validation key with the session stored validation key.
If I'm sure I'm only going to use short living sessions, I also encode IP information into the session and compare it when the cookie is send. If any of the data does not match up, the cookie is tampered with and the session and cookie van be invalidated. This is a simple way to force The Union Router (TOR) users to login again when their IP changed without them realizing, and thus preventing compromising their session. It also prevents networking traffic inside a user network from being rerouted to a different internet exit point, as the user device will access the server with a new IP by then.
The session automatically invalidates after some time of no activity.
If the cookie is using a session that no longer exists, the server will try to recover it from archiving, or create a new session and update the cookie. It can be a heavy feature to browse through a database full of archived session data.This process is intentionally made relatively slow but steady with a limited resource pool, to prevent DDOS from bringing this feature down. Its very possible the server denies creating a new session unless the user supplies login credentials again or enough resources become available for processing. The bound information also includes a date of expiration, so that if a cookie's expiration date is tampered with, we can invalidate it and block user access.
PS: I often also add a random generated string to a long-term stored user account, and use that code to encode random values. This makes it harder to further analyze random generated patterns unless you know about the encoding happening And even then you have to break into the server-side storage to get the user-stored random string.
This is how I do it, may sound convoluted but every step is crafted in a way to prevent any form of corruption on server side (for example, due the server being DDOS'ed) with minimal performance hit, but with maximum security I could envision. The server uses atomic operations to operate on all the values, which allows the greatest guarantee to prevent corruption by computer instructions going bad.
I thought of different methods such as using no sessions, using simpler data, not using as much random strings but I found ways to corrupt the system without them. Besides, using sessions allows me to store dynamic session data and check permissions on the server side easily, which is useful for having stateful applications such as frontend editors with caching and stream support. I wanted to be sure that people could relatively stream data safely and that the server could process it fully in parallel and asynchronously, sacrificing a fraction of speed for stability and security.
This approach is something I concocted that can be made to work in a distributed computation network such as server parks using only simple synchronization techniques (like using TCP and just send over formatted data). I use sessions to store information about how the user data is distributed over the system.
To pull off stealing someone's session with this approach, you'll have to do a man in the middle attack or hijack the internet exit point, hack into the browser of a user to get session bound data. Fabricating a cookie using random data is out of the question, because if any of the stored cookie data does not match up with the session, it is not deemed valid. At this point, the attack vector is people who can directly access your computer or people who have compromised browsers. There's nothing you can do against that.
It may not be the answer you're looking for but I hope it was insightful nonetheless.
Feel free to criticize my approach. I'm not a security specialist. Definitely tell me if you have more secure methods if you are a specialist!
@@msc8382 WOW, That was like reading a medium article. Thank you for taking the time to write this out. The IP address encoding you do along with the sent cookie is genius, i hadnt thought of that. especially for users on vpns or TOR. Up to now i had just be authing with credentials and a jwt cookie but without any of the extra data. This definitely gave me some ideas. will be saving this in my notes.
Most viewers would be interested on a proxy backend API endpoint demo with load limits. We don't need a production stable app, just an app that is safe regarding exposing our API key (mostly to avoid high bills).
Hi James, thank you very much for this video. I would very much like to see an explanation about JWTs and common mistakes while using them - especially around the storage of these tokens and securly sending and receving it from the back-end.
That’s a great idea. Thanks for the feedback!
Up
Great video as always, I've made plenty of these mistakes!
Here's a little tip, I like to ignore ".env*" instead of typing out each different environment file permutation.
Came to see if someone had mentioned exactly that tip - possibly also *.env as well, as I have seen local.env, for example (not sure if it was with dotenv package, or something else)
Yep I do the same except when I have a .env.example file that I want to leave in for other users to get started
Haha Brad nailed it!
Then do :
.env*
!.env.local
It's a common thing to leave a .env.example file with the description of the vars on it. If you do that you won't be able to have that file.
PLEASE PLEASE do the follow up! This is for sure another pain point for folks just learning about how to handle API's and most like you said thought if you used the .env file that it might be ok but we see here that is NOT enough! Also, can you add a short part about how to possibly remove it completely if it did accidentally get pushed up to Github? Your stuff rocks THANK YOU!!!!
Glad you enjoyed it. For the “accidentally pushed to GitHub” there’s not really a way to completely remove. You need to regenerate a new key. I’ll work on the extra video too :)
@@JamesQQuick Oh wow ok gotcha!! Thank you!
I really don't understand why, in 2022, there isn't an encrypted & protected enclave within the browser to load env params, which are never visible to an end user. Perhaps it is not possible, but it seems like it should be (for people much smarter than I am).
The example in the video is a perfect case as to the necessity to add further layers of abstraction, which may be to protect just 1 api key..
Great video & hope this saves someone the pain of leaking env's!
Interesting take!
@@JamesQQuick I’m an optimist, if nothing else 😂
@@everyhandletaken Me too. You were absolutely right about encrypted and protected layer for this kind of authentication via API's. There must be a way to do this without exposing the secrets to the whole internet. I mean we are trying to reach Mars. Have a good time and thanks for your answer. I back this 100%.
Excellent video on a relevant topic. I solve this problem by issuing a unique CSRF token (Cross Site Request Forgery) from my API to the FE page that is consuming a public resource from my API. I persist the token in a DynamoDb table.
Then each subsequent request from the FE needs to pin the token to the request header e.g X-CSRF-Token. Then server-side I validate the token by looking it up in the DynamoDB table. If the token exists I know that the request is valid. Tokens can be designed to be one time or multiple use. They can be time base and have an expiry (which should be short).
This, combined with rate limiting and an API key will offer a good level of protection from someone abusing your public end-points.
CSRF Tokens can't defend against this. You can have a workflow using Postman/Insomnia/HttpClient to call the csrf token endpoint, extract the token from the response, inject to the next request.
Security is a real challenge for frontend development.
CSRF token should only be provided to authenticated and authorized users who have a valid session.
@@Subs-rj3zk
Should use other API as an example because there are two different types of API keys, Public and private API keys.
Public API keys, like Google Map API key, for example, will be always exposed to the internet because you need the key to access the map. This kind of keys should be restricted by limiting IP addresses, referer or domains to secure the usage.
Private API keys are for internal use, for example: APPs to APPs or servers to servers communication, so they shouldn't be exposed to public. Instead, they should be secured by ways mentioned in this video.
Great video James - Objective, honest, and on point, without unnecessary things!
Actually you can use env variables in Astro if you're not using your API keys in client side code. Since Astro runs in build time, the requests that need the API keys will be sent during build time and no js code with the API keys would be shipped to the user.
I'm already making popcorn for the follow up :)
Would like to see a demo of this explanation
Please suggest in detail how to manage multiple dev/test/production .env files shared among multiple developers and testers.
Also, how would ex-developers (potential saboteurs) be denied access? Generate new keys and rebuild/redistribute a new generation of .env files?
Also, please address Pavel Pirogov's comment that many api keys are host-limited, apparently mitigating key secrecy as a serious issue.
Thanks James. Properly put up in an order to understand
Cors is good if backend consuming from website but if you have mobile app?
Sure I need more help and explanations about how to solve the problem number four 😂. Nice video!
Thanks for this video James, it was actually Ania's video that led me here and it's an issue I've been wrestling with for longer than I'd like to admit!
Ah so cool! Glad it helped :)
Thanks for this video, james. Yes, a demo would be nice.
I also like to see a follow up video with all the security stuff :-)
Thats where i am stuck i created proxy node backend but i think its useless now because you can call it directly and get the data😂😂😓
Hi James. Thanks a lot for this post! I'm still learning about securely using my API keys. It seems that this will be a long, long way to go. I almost can't believe that there are no more robust and secure methods implemented in modern browsers. Seems like one has to go to university to be able to securely use widespread API's. But hey, let's figure it out. I like your content, and thanks again.
Actually this would be so great to cover more specifically incl. demo of best practices ! :D
Thank you so much this helped a ton James!!
I've been burned by this before. I'm happy for Nextjs 14's server-side implementation to allow for secure calls. However, if you don't have Nextjs, these tips are still good. Thanks.
Good video, can you do a follow up with actual examples? Cheers
Looking into it :)
Hey James, I recently being diving in deeper in this topic and I been learning a new tool API managmenebt and Key Vault. Was wondering if you have knowledge on or could do a video about key vaults?
Absolutely need a demo with some visuals
Ok, this obviously works fine when using node.js, but what if you are using client side Javascript?
Afaik the only way to hide the keys here is by doing a custom request to your own server instead of sending it directly, am I right about this?
Thanks for this question, Daniel. This would be interesting to know for me too.
i wait more detailed videos. This issue more important. Thanks
Please make a demo. Even if it's a multipart video to avoid all mistakes.
Backend protection, use Firebase auth with custom claim with JWT
Hello DasDune Technologies. Thanks for the tip. Appreciate it. I will check that out for sure. Have a good time.
Deeper please!
Yesss. :)
That was very helpful thank you.
Do I have to give the web developer the secret key to implement the payment method?
Hi, James. Thank you so much for this valuable video. Actually I have stored all my env variables in vercel, but when I see in browser/inspect/source tab, I could see all of them even though my github username, developer key etc which is very sensitive info. How to hide them? 🙏🙏🙏🙏
Not so much info about Chrome extensions and JWT. Would be nice to see some tips/tricks :)
So would this also be true for using an API to say gather just simple blog posts from a CMS; I mean do you need to get special tokens for that type of application?
Hey bro. How much longer before the everything svelte course is dropped?
shouldn't we use a vault service to securing your API key and get api key dynamically to request backend.
I have to use Google Maps in a project. I was wondering if anything that you said would apply for the Google API key, but I guess it doesn't. Google forbids using Maps with a proxy as far as I know. Restricting the key to a host (via Cors in the Google account settings) is the only possible protection I guess.
I guess what I don’t get is, yes I want to minimize exposing my api key or auth token via github or dev console, but I also want people to use the app I’ve made without having to make an account and login. Don’t some of these approaches go a little too far in “protecting” the keys? What’s the trade off if people can’t even use the thing. And what’s to stop multiple accounts from signing up and spamming your key anyway?
It certainly does seem like a lot. The simple fact is if you expose your keys, they can be used by anyone to spam your content so you certainly don't want that espceially for paid products. To the question of, couldn't users spam your content anyways...yes, they could, but at least that way, they don't have direct access to the key AND you can throttle or even remove a user. You can also put general throttling logic in your application to help prevent it to start.
Very impotant video! Thank you! Please, make video about JWT!
Is it okay to expose an API Key that is limited and retrieves information for demonstration purposes (i.e. demo projects hosted on GitHub Pages)?
Thank you ❤
As long as it’s being used on the client, the key can still be seen on the browser if you dig deep into it, no? Unless you mask it with a serverless function then yes it’ll be completely hidden.
but then that function can be invoked by anyone. :D how do you protect that. Anything that has to be client side can simply not be hidden.
@@sayamqazi I meant the key will be hidden if you use serverless function, it does not mean you can prevent people from using it. What exactly do you mean by protection? What are you trying to keep hidden? A serverless function already hides that for you.
@@Notoriousjunior374 Oh got it. But what's the point of hiding key. if the attacker can just move his goal post now.
@@sayamqazi These answers can easily be found on the internet. It’s your choice whether or not you want to expose to the client. You can implement any features on your end. For example you may have an api key that has access to your crypto wallet and that you can perform trades with it. You’re planning to ask your fans to buy any shit coins for you but of course you’ll want to have some kind of limit that your fans will not be able to do. Hide that key from them and do write some custom functions to do whatever you want. It’s really that simple.
these are not effective solutions, there are ways to read and expose keys.
this only covers accidental key leaking but does NOT cover actual security.
such as tricking an api end point to return its environment variables.
So there no solution to prevent a stranger to trigger our backend from a server ? (Apart login auth)
A simple like counter which is available for guests (no login) that be stored in a db can’t be secured to a stranger call?
Firebase App Check might be a possible solution.
what if don't want to make localhost:3000/weather data available to all users, just to the ones that are authenticated?
Suppose in react before push it github or hosting craete .ENV file and write REACT_SECRET_API_KEY=abcd...........etc etc
And my index.js file code {process.env.REACT_SECRET_API_KEY} then add .ENV file to . gitignore after that if i push it github then okay or probelm ? I mean can hide my api key that's way ??
Your key would not be exposed since you did not upload it to the internet, but accordingly your app also could not read the key as it cannot access .env.
I have an api-key error case, my api-key is case sensitive, in javascript my execution does not progress, due to the error it shows in api-key, please does anyone know how to solve it?
"x-api-key": "swGwRN7X65XLuBqFFsthpwxMjhXjxL9CrUmvtW70"
error displayed: message: 'The request message is not properly formatted.'
Aww this time Cap guy did not make a cameo in the video intro
Haha I’ll add him back!
Why not have it on a database on the web, then construct a class with private values if ur shipping an app with API keys.
Cannot not use npm dotenv package in vanilla JavaScript.
Well you also wouldn't include private credentials in a frontend vanilla JavaScript site
Great video as always!
Thank you!
Thanks for you video, what about JavaScript UI applications (in React namely) where they run on Docker ? We run into a problem with this Dockerized apps because in our DevOps "Philosophy" we build one image independently from the environment (remote DEV, UAT, PREPROD, PROD) and we want to add environment related variables later, in Java it works well, because the java applications can read from a property file shipped out of the container, but in JavaScript I didn't hear about an application reading properties from an external file, did you encounter a situation like this ?
You can pass env: ./env and pass in .env file for environment variables
Not entirely sure I understand, but you can pass an env file to Docker container using arguments when you run the container, or in a docker-compose file & the env file is picked up from outside of the container. Not sure if this could be a remote path, but can be anywhere on the filesystem of the Docker host. I may be misunderstanding though, sorry.
Can you please make an Tutorial on SSR Streaming with Next.js
CORS is not enough. A simple HTTP Request (not XHR) can bypass CORS. CSRF.
People just do not understand security at all. All that .env bullshit about api keys ruins here: anyone can open devtools, network tab and see all exact headers browser sends to 3rd party api including your 'secret' key
The whole point is that you don't include the API key on the frontend because of that...
This is what I wanted
Vercel environment vars work great for prod. Thought you maybe are highlighting this as not the best solution.
NextJs newer version of 12 has resolved the ability to get env vars into the front-end. If the var is prefixed with NEXT_PUBLIC_..., then the value is return and not return of a blank. Although, it only works for NextJs. And, I do believe Vercel's NextJs would offer this only if it were secure approach.
It sucks though cause how do you use a api key that is stored on vercel client side with out it being exposed to the client
So people won’t see the key
Please make follow up video
That is so many files and folders for the project you are showing. How do you even learn to create all those? It's overwhelming.
Yes on the DEMO!
Haha totally understandable. The main thing is that those files get created one bit at a time. That’s whT comes with experience is being able to break down something big into smaller components
DevTools > Network > see api key in plain text in the request
=/
encrypted over https...
@@JamesQQuick it's in the url I can see. Any other user can see it too, duh =)
A problem with public-facing web pages.
gitignore and put it in CI, what else is there
So I with 0 dollars in my pocket basically can't use any API because I don't have a server to make it secret... veb dev is a giant headache.
Please make the follow up video
Yes, demo please :D
This is stupid! Most services that require api keys provide keys for exact host and if request goes from another host key will not work. So there are no reason to hide your api key because noone else would be able to use it any way. And if service allow one key to be used from any host that would mean that there is nothing bad can happen if someone else will use your key because there are no functionality to make any damage to you. Usually no host check would mean that it's purely made and it's better not to use this service.
Hi Pavel. Thank you for your answer. I'm relatively new in this field. I started learning to code earlier this year, but I care a lot about security and I want to learn how to do things the right way. If I understand you right, then a good API specification is using API keys that are bound to a certain host and can only be used on this host but not from any other than this. This seems to make perfectly sense for me. You advise that we should better not use any API's that allow the use of API keys from a random host. Did I get this right? At this point I'm not dealing with securing one of my own applications, because I have not coded one. All I want to do now, is learning to consume API's with JavaScript's fetch or with curl. After seeing this video I almost think that there is no way to do that in a very secure way. To be honest, I will look for API's to learn with that do exactly what you described here. I want to find some good examples that use host bound API's. Thanks again and have a good time.
Do you not sing a song anymore?
Which theme u are using ??
Midnight synth :)
Yes, but after building a pure react app the environment variables are injected into the code, not called by process but actually hard coded into the build files
Yep that’s right
Firebase does all of this so it's all secure with Firebase
** I think Supabase might do this also...?
All I know about Supabase is it does not have everything Firebase has yet.
Great
So you didn't explain how to solve it. Nice. What a waste of time.
whats the point of this video?
And how does reading from env variable make it any secure ?
Any one who can look at your network tab have access to the curl and hence the key.
Please stop making these videos which gives people a false sense of security which is more dangerous in practice .
client secret in a react/single page application !!!
There is only one way to secure your JavaScript client and using oAuth.
Security is such a sensitive topic and videos like this only makes it worse.