You're typing process.env wrong
HTML-код
- Опубликовано: 17 апр 2023
- Typing your environment variables with Zod is SUPER ergonomic and, if I say so myself, rather beautiful.
TS Playground: tsplay.dev/WYe7EW
Become a TypeScript Wizard with my free beginners TypeScript Course:
www.totaltypescript.com/tutor...
Follow Matt on Twitter
/ mattpocockuk
Join the Discord:
mattpocock.com/discord
Honestly these short videos are replacing dumb memes and stuff for my daily quick watch or youtube shorts feed. So I really appreciate your work in these tid bits, not all will be useful to me but the seed is planted for future edge cases.
I was using the NodeJS namespace with the z.infer trick, however I found it better to do `export env = envVariables.parse(process.env)`, Then in my envVariables schema, I use transform or coerce
- so my env becomes typed (eg, numbers, booleans etc from a .env file)
I like this - it gives control and one place where settings change and are managed, still in a global/static way.
Well, there's actually a limitation to this approach, if you try to intialize something globally (for instance, database connection) when the server starts-up, you will notice that the `env` will always/sometimes return undefined, as it's not ready.
@@chungweileong Sounds like you have some other issue. Import loop? Or async parsing perhaps?
@@chungweileong I think mutating the env or lazy loading it is a problematic practice, you could store lazy loaded or deferred stuff in another globally available variable.
The approach of exporting env globally works fantastically when you rely on env variables that should be present on the program's initial start, whether it be a node server or even a frontend framework
@@chungweileong don't mutate your env from inside your application. Env is for passing data to your application.
I would rather import the parsed env from a separate file than mess with the global interface that expects strings. It’s not really much effort considering the intellisense just auto imports from the correct place anyway.
and you can't forget to actually validate the process env. For example if you're code has multiple entry points you need to make sure that everyone of them actually calls envVariables.parse
Matts approach seems to be quite elegant and is impressive. But what @zheil says, is the way! The usage of zod to simplify validation is really convenient though 🎉
@@davidadamson1588 To be honest, to me this jumps out as a great opportunity for Typescript to make some improvements to interaction of env vars with the rest of your code.
@@Gigusx I am not sure if I am a little old school…
But I prefer to have such things as configuration in its own place, like inside a class. This helps to comply with the SOLID principle which again ensures the correct usage of eg a strategy pattern. Since when done properly, it will allow me to simply exchange the source of configuration by swapping out the configuration strategy. Strategies could be: loading configuration from a file or env vars or a combimation of both or what ever suits your needs.
Hardcoding against process.env is not a thing you should do (in my opinion). Code against abstractions not concretions.
People might argue that this is a „java thing“, but it has the same advantages when used with typescript, especially when working on large scale projects.
@@davidadamson1588 It has the same downsides too. You added complexity that you carry for the life of the project which you may never take advantage of. Put them in a class sure, but strategy pattern? abstractions? If you love acronyms then don't forget KISS.
These shorts are quick, condensed and really useful. I love them! 👌
Really appreciate the shorter format, yet clear explanations
Incredible! And yes we love the short videos. Thanks
Thanks Matt! keep em coming, learning a ton from these vids!
Keep putting out more of these video. you're making great content
I really like these short videos!
Good to hear!
I love these interesting short videos.
THIS IS SO HELPFUL - Matt, I can't tell you how much I've learned from your channel in a short amount of time. Thank you - keep up the great work!
I absolutely love this
awesome find! thanks
God damn thanks a lot dude! I searched now the past few hours for it, but I wasn't able to figure it out by my own. And thank god I found your video. Subscribed your channel, so that I can hopefully see more such hepful content. :thumbsup:
Awesome stuff!
wow! I recently gave up trying to type these env variables. Thanks a lot!!
oh cmon man, id love to use that on my engineer degree project if only this video was created 2 months ago. I was so struggling with process env on typescript.
This was so helpful either way, thank you
In our team we use JOI. Works perfectly fine. We create functions and group similar env in its return obj and import that function and use it at the place we need.
I was using this approach with NodeJS (no TS), but I started using TS with this approach and really got into problems with other types like numbers (PORT for example).
So now I'm using an interface that is applied to a getter function, in the getter I can covert the var into real number, and also I added a default value functionality.
So now I have a centralized function working as a getter for the env vars, that can cast types and use default values if you var is not set.
This is gold!
It's quite nice. I really like it
amazing piece of code
Our team has been using the resulting parsed Zod object, but extending it further with coercions. Definitely interesting to override the global ProcessEnv interface.
I personally use cleanEnv from a package called envalid for my env variables. Keep up with your impressive videos.
I was always trying to understand why my process.env.VARIABLE could be string or undefined, this helped me a lot
So nice trick!
You can also transform the env variables inside the zod schema, from strings to eg. Date.
You won't be able to do the `interface extends ...` trick, because that still only accepts strings.. But you can parse the envs and export them, just like you did for a second there.
This is great! What do you think of using `z.input` instead? Then you could even do transforms, for example to convert CUSTOM _STUFF to a number, while still retaining autocomplete and correct types on process.env.
z.infer is an alias of z.output, but sometimes you want the input types instead, such as on process.env.
Zod is such a great and incredibly useful library, I feel like I'm not exaggerating when I say it's currently the best package on npm by far.
One thing to note about zod though, is that it sucks with cross field validation
@@parlor3115 wouldn't the refine method serve that need?
@deepkit/type is far superior to that garbage
nice, didn't think about it 👌
you could easily export transform env vars from such a file. For example for a positive integer you'd do: z.string().regex(/\d+/).transform(val => +val)
the refine method of zod could also be used for more complex values instead of regex. Now all I need is some presets for frameworks such as astro and Next.js :)
the t3 stack also does this, but with an own "env" variable that is being exported. Also the declare 2 schemas: client schema and server schema.
i find it pretty useful (yes I know the client schema is a little bit more expense, variable has to start with "NEXT_PUBLIC_..." and so on )
This is really interesting, I think I will try this, I have been doing the second version which is exporting Zod parsed process.env.
But what you did here is made it type safe and globally available without the need to export. Which is cool, but the downside is as you said they are all strings by default, and I prefer, when working with booleans or numbers, to really have them as numbers.
(UPDATE: WHOOPS, this is incorrect)
Many js libraries that provide a build process parse things like `process.env.MY_ENV_VAR`, and replace them with the actual env var found at build time.
If you export the parsed object that won't happen while @mattpocockuk 's solution will work with that.
Without such feature, I'd agree with your solution and the lack of ability to transform data.
@@Erandros no. Matts solution will not work as well when using such build tools! This is due to the fact, that process.env cannot be validated by zod (since env vars are getting replaced individually). Typing process.env globally without having the validation will result in pure nonsense.
@@davidadamson1588 sorry
@@Erandros no reason to be sorry. I hope my comment did not offend you 😨
@@davidadamson1588 not at all, I just felt ashamed that I gave inaccurate information, but that's on me, dont worry
Amazing!
I love these small videos, please keep sharing these superb ideas with us!
I already did something really similar to that and the only problem I had is if you add to this env object the isDev property based on the "NODE_ENV === development" when you build to production your bundle might not strip out the code you have that is only related to development
So then to make this work I just used the process.env.NODE_ENV directly for these cases, and other places I access the env properties from this custom object.
@@emkisn Yeah I wouldn't use this for env variables I don't control. NODE_ENV being an example of this, or even PATH in POSIX because it shouldn't matter that they don't exist in those cases, there should be a reasonable fallback for those cases.
@@dealloc Makes sense, the only thing that borders me is that then we will have two different ways to access env variables, so it's not really consistent. Maybe an ESLint rule to only allow certain kinds of it usages like NODE_ENV could help with that.
I prefer exporting the parsed variable so that I can also do z.coerce.number() or something similar which avoid repeatedly doing that
The parse would also fail and prevent the app from starting; instead of running into a runtime error or bug down the line, if the environment variables are not defined, which is also another benefit!
So cool!
Have you heard of the znv package ? It does all of this and then some. Uses zod, parses into whatever type you're expecting your variable to be, and lets you provide defaults for each environment that NODE_ENV might be set to.
I never thought of this approach and I really like it, this video timing was pretty good actually, I had an issue today in production because of an environment variable that I forgot to add in production, this would've helped me find the issue faster.
I'm going to do this from now on, great stuff!
You could do z.coerce.number() or a transform, or refine
nice one
Mind blown!
RUclips should look into adding a love react button, this video requires so much of it!
Thanks Matt, this is awesome!
I migrating to Fastfiy, and I'm also considering Zod now. It seems very superior to alternatives also for documenting API end points / schemas.
ps. Hopefully this Zod + process env also works together with dot env eg. @fastofy/dot-ev?
Nice!!
This is great although it lacks the support of transforms, for example of boolean coercions, using default values etc.
Due to the lack of transforms, a more correct way to infer the schema here is to use the _input type of the validator so that no transforms are affectign the types of the process.env. But this is a nevertheless a great way to get a simple validation that you're not missing any envs on build!
Cool comment
nice approach actually i did follow an export approach
I've used `env-var` in the past to validate, parse and type my environment variables
I love TypeScript and I wish I could be there to understand like all much about it definitely it is wizardry 😅😅😅😅
I use the envalid package for this
I've seen a lot of developers who are fine duplicating static information like that, this is so much nicer :)
this gives me ideas....
I like envalid for this solution
I'm looking at creating environment variables that can be parsed by JSON. Seems there's a standard way to do that, and I wish the example you provided made it easy to do that, as well.
With zod you can infer types so you could do:
CUSTOM_STUFF: z.infer.number()
And it would do the equivalent of String(process.env.CUSTOM_STUFF) , so that's not really an issue.
I still prefer exporting a dedicated object than messing with the node types but both options are acceptable to be honest
This seems helpful if you’re using envars for anything other than the main of your program, but that has turned out to be a bad idea every time I have done it so I just don’t do that anymore.
so many things to learn lol
Helpful video. Wondering how I could use that with a .env file? Would really appreciate support of this. Thanks
This is designed to work with a .env file!
I don't think you can ever get rid of process.env, and this is the best way to handle it, but for env vars in general, I like to have a solution with one more trick up its sleeve:
Wrapping the var in a getter that is refetched/decrypted from a config/secret store at the time of access (with reasonable amounts of in-memory caching where necessary) and then garbage collected. That way, if the node is compromised, the secrets are not on disk, and only a temporary access token can be stolen.
The problem with this is that mutation don't work. If you export the parsed object and use it in place of environment variables, however...
Could you rephrase your question? It works great.
Wym? You want to mutate the env variables?
@mattpocockuk look at libraries like envalid and envsafe. Instead of parsing the result and than doing nothing with the actual result object, we can parse the environment and than export it (I like to than add a path to it in the tsconfig, I.e. @/env) and use the resulting object instead of process.env. That way, any mutation you make to the environment variable (for example, using JSON.parse to turn a stringified object into a string proper) can be used
We can do something like this:
import { z } from 'zod'
const envSchema = z.object({
STR: z.string(),
NUM: z.coerce.number(),
})
declare global {
namespace NodeJS {
interface ProcessEnv extends Record {}
}
}
const envVars = envSchema.parse(process.env)
It results in for process.env.WHATEVER and full power of zod for the envVars
Great video. I would personally add the Readonly wrapper to make sure env vars are constant like this `interface ProcessEnv extends Readonly {}`.
Good idea!
Saw this first in create T3 app, using it since then.
Do they do the declare global thing in T3? I thought they used the export const envVariables setup.
@@mattpocockuk Correct, it uses export const env setup with a variation that allows to throw error if env vars are accessed client side. But extending global namespace interfaces is certainly a neat idea.
nice :)
Is there a way to do the same with ImportMeta?
Wouldn't we want to use the implement keyword in this case?
sorry im new to this but what's the purpose with the zod libraries, can't I just write it with typescript?
❤
I do it without using any other plugin using only typescript
1:51 what is "types node"? Do you mean @node/types or something? I want to use this in our proj for urls and other global settings. Very cool. Thanks.
what font did you use in the thumbnail ? Is that fira code ? it just looks so nice but fira code to me is not really that nice so i am curious though. Thanks
Do you have a way to make the key of process.env non-accessible if not defined? Ex: typescript will throw an error like if you define a constant object `as const` and you try to access the `key` that doesn't exist.
I like this method, but unfortunately, `process.env` still allows access to the key that is not yet defined with zod, which makes accessing `process.env` directly riskier.
Why not use a proper configuration management library like convict or dotenvsafe ?
Is there any reason why this is better than "only adding it to the ProcessEnv"? Why use zod at all for this?
If we forget an environment variable zod will just throw an error for us instantly and we won't have to debug and find out what's missing.
I have that same question, what is the benefit of zod here...
What Nicolai said. Free Validation and typing is the key here
when i import .env file using doteng.config() in app.ts file
i can access process.env.someting in most of the files.
but few files for example helper.ts file or some file which are not directly calling from express router where i am not able to access process.env.something. env are not loading in these files
why ? how to fix this issue
Call dotenv.config() before you do your imports.
I find it better to import the parsed object
I actually like this approach, but the only problem I have with this is that it doesn't stop people from using a variable without define in the zod schema. Instead, I personally prefer to have a `env('....')` function, to use & validate variable on-demand, and allow you to do some cool stuff like throwing error when someone try to use non-NEXT_PUBLIC_ variable in NextJS frontend.
The problem with that is Next is not able to transform your `env(...)` calls during build time, which makes it unusable in client-side code.
The same problem happens when during process.env[someArbitraryKey]. Next cannot inline those environment variables. Same if you were to destructure, or assign process.env to a variable.
Can't you use Zod to enforce all env variables to be strings and never anything else?
What is `interface ProcessEnv` and how is it related to `process.env`?
@Clemens Horn thx, i was wandering from where `ProcessEnv` came, so apparently it is a part of NodeJS typings in typescript. good to know
This looks really familiar 🤔
I employed this and it's great, but then I found `znv` module, so using that instead to keep it even simple🤓
can i do without zod or yup?. just create an interface and pass.
Why do this instead of exporting and using the envVariables object to retrieve the environment variables?
I address this in the video itself. I find it just a bit more truthful. process.env.WHATEVER _is_ available, so why not make it type safe?
@@mattpocockuk only issue is that if you add something like "DEBUG_LEVEL: z.number()" on the object, TS will assume that process.env.DEBUG_LEVEL is a number, which would be wrong
@@scezar8880 That's not correct. It'll conflict with the built-in ProcessEnv type, so it'll give you the proper error.
Actually you better parse the env at one place and export a decent schema because calling process.env involves calling C functions which is a negativ Performance impact when done too often
Thank you for the video, but I think there is an issue.
Imagine you have PORT: z.number() in your schema.
While you extend ProcessEnv, typescript considers it as actual number, but in your process.env.PORT is still string, which can cause the bugs typescript unable to catch.
Try it
Wouldn't it be better to just have a class model and use that as the source of truth throughout your app isolating it from using process.env ?
Cool! I might be confused though - how does this give us "type safety"? Like you said, process.env.X is a string by default. It seems to me that the only benefit here is autocomplete. Am I wrong?
process.env.X is string | undefined by default!
It gives you runtime safety by ensuring that the env variable you are accessing actually exists
@@mattpocockuk ahh, true. Thanks!
Nope, use a module where you parse and export, then use that. no-process-env is an es-lint rule for a reason.
Nice. Or one can use envalid; works great and throws relevant errors.
NO, I will use the global type define :s
only legends use the npm package called 'envalid'
we can do this if you not using zod ==> declare global {
namespace NodeJS {
interface ProcessEnv {
DB_HOST: string;
DB_PORT: string;
}
}
}
Yes, though this won't give you any runtime validation.
someone should just create a library or generator to do this automatically I think.
zod not needed for this? can just do the same thing with the declare global thing
Yep, true! Just wanted to show the awesome z.infer thing
not needed, assuming you're gonna typecheck process.env manually, otherwise you're assigning a type to it that isn't accurate
@@mattpocockuk oh yeah! that way u don’t need to add types manually, honestly could save a lot of confusion!
I personally just create a singleton class with the values in the form i want:
export class AppEnvironment {
public static readonly DATABASE_URL: string = process.env['DATABASE_URL'];
public static readonly CORS_WHITELIST: string[] = !process.env['CORS_WHITELIST'] ? [] : process.env['CORS_WHITELIST'];
public static readonly PORT: number = process.env['PORT'] ? parseInt(process.env['PORT']) : 3000;
public static readonly API_SECRET: string = (() => {
if (!process.env['API_SECRET']) throw new Error(`API_SECRET not set in process.env`);
return process.env['API_SECRET'];
})();
public static readonly AWS = {
S3_ARN: process.env['S3_ARN'],
SES_ARN: process.env['SES_ARN'],
};
}
I think it's cool for autocompletion but it's not a single source of truth, since you have to keep the .env file in sync with the types file, and actually that can cause issues in runtime (zod string trying to parse an undefined env variable). Correct me if I'm wrong
The point of this is that zod ENSURES that your .env file is correct.
But I pass VITE_BASE_API=, it's not throwing error
I prefer validate with min()
const envSchema = z.object({
VITE_BASE_API: z.string().min(1),
});
Why should I bother doing this in Zod instead of just creating a regular interface? (Honest question)
Because you'll be sure that at runtime, the correct env variables are being passed in
@@mattpocockuk Ah yes. That makes sense. Thanks for your answer!
that's nice, but it doesn't work with Vite ImportMetaEnv.