npm is unsafe*
HTML-код
- Опубликовано: 24 авг 2024
- People seem scared about NPM's safety. I think that's kind of silly. Decided to explain why I feel that way. And yes, thumbnail is a @LowLevelLearning shoutout
My video about the polyfill thing: • One Script Tag Just Pw...
Check out my Twitch, Twitter, Discord more at t3.gg
S/O Ph4se0n3 for the awesome edit 🙏
my life of developer is unsafe
Same bro 😊
My favourite case of loading js from other websites is still barclays UK loading a js file from an older waybackmachine copy of their own website.
no f-ing way lmao
That’s awesome 😎
Gaaaahhhhh!!
wtf...do they not know how downloads work?
Abusing waybackmachine as a CDN, well that's next level!
15:30 I think you kinda misunderstood, what many people are worried about. It's not, that known good packages (like react) get compromised, but that you don't really know what many dependencies do. Maybe that small utility package used to add some spaces to the left of your string also has code to send every keystroke to some server and grab the passwords of your user. Nobody is looking at the code of the hundreds of packages many projects include.
Agreed. However, he should know better. This video is unsafe... lol.
You can interrogate what packages you have installed, and audit them with tooling dependabot for example.
It's even more obscure with python for example as very difficult to track which thing installed the sub dependencies.
Yep
Like, nice-try module exists and has >10M weekly downloads
@@_modiX React/JS fan boys are the worst.
@@_modiXTrue, but a lot of that has to do with unsafe versions, which yarn is a lot better at handling
It is not about NPM as such, it is about having too many unreliable dependencies. In C/C++ dependency management used to be hard so historically people tried hard to keep their dependencies few and small.
that's the thing I love about C/C++ dependency management.
Just the fact that npm audit has to exist proves you wrong. We cannot keep track of the thousands of left-pad, is-number and similar packages, so we have to build a workaround for it.
"Back in the day" you had a database wrapper like pdo, and maybe some Apache modules like Headers, Session etc. and that was it. Easy to keep track of and less points of failure.
I think this is an issue with js in general, because so little needed functionality is built in and the trivial code is usually very flawed because of some edgecase.
In other modern languages like C# you usually only need 2-10 nuget packages for a large project and even then most are provided by Microsoft.
That’s my issue with JavaScript and its ecosystem.
don't try checking the dependencies of react-native on npmgraph 💀
Is IsEven one of them?
Please tell me it is
@@no_name4796 sorry to disappoint :( ...but hey, is-number is there
@@no_name4796 hot take: using iseven is better than writing it yourself. Are you prepared to write and test all of the edge cases for iseven? What if iseven is passed null, undefined, "0", " asdf", NaN, [0], etc.
Its better to use the dependency and pin it to a specific version and dont touch it unless you absolutely have to.
All modules (1 top level, 524 dependencies) lmao
Jezus christ.
Slight disagreement on Theo's take on dev dependencies not being as much of a risk (as regular dependencies). Having to hijack the build step vs being conveniently bundled is a trivial difference for a malicious author.
How can code that doesn't ship to user be *equally* as dangerous as code that ships to the user? You say you disagree it's less of a risk, right?
@@codyrap95 You're correct that code that doesn't ship to the user can't hurt them. You're mistake is assuming that malicious devdependencies can't ship code to the user without you knowing. The attack vector in Thompson's "Reflections" is not exactly unfeasible in JS. Heck, the expectation now is that any JS served to the user is obfuscated and minified anyway, so a great hiding place for nastiness.
@@himagainstill As someone browsing websites, I feel unsafe knowing that everyone thinks it's fine to serve me a page that depends on obfuscated and minified JS and in many cases doesn't show me *any* content without JS.
Not to mention that malicious packages will often push malware on developer's computer rather than code meant to compromise the project. In that case it makes no difference whatsoever.
@@codyrap95 If the build tools are compromised, they can inject malicious code during the build step.
The secret:
It's unsafe all the way down.
Hell the packages aren’t even hash checked on installation. FISA court could order NPM to load a secret sauced variation of the repo just for you. You’d have to fork every package to private hosted GIT then use the SHA of the commit. If I was an externally hired dev in Asia working on datacenter dashboards or goober things none bats an eye to reading outputs from highly classified systems I’d be so scared.
>static typing enjoyers hate him
Your screen at 3:56 shows the thing that really stresses me out about npm; everything you download throws up twelve thousand warnings about how there's something wrong with it. And I'm a noob, what do I know, it usually works anyway and tutorials all act like that's normal, so I learned to just ignore it, but it feels like learning to ignore an overreactive smoke alarm.
In any other system, this would prompt people to fix their package and remove the warning. In NPM land, people just complacently ignore and accept mediocrity.
Literally no one thinks npm can somehow pwn your website at runtime, the point is that you're downloading and executing dozens of megabytes of arbitrary code on your machine.
You're not safe just because there's no active NPM scandal, literally open any year old project and NPM will automatically warn you about a dozen new security vulnerabilities that were discovered, it's safe to assume there are dozens more that have not been reported yet and are actively being exploited by malicious actors.
And why assume devdeps are safe? If a package like eslint got compromised it could trivially inject malicious code into your bundle. Just because it's supposed to only run on your machine doesn't mean it can't still access files in your dist folder.
But one person on twitter said it so that is what everyone is thinking!
I'm surprised that no one has mentioned that his dependency versions are not "locked", you know that the ^ means that you can automatically download ANY new minor or patch version...
so in this example, you can download react 18.4, 18.3.2, 18.5...etc
if you actually want to lock the version, you have to remove the ^
its all in the documentation on semver
in practice, its also better to set up your own private npm which proxies the public one, solutions like nexus can do this automatically for you, so you can host your packages and still get all the packages from the same URL
to be more clear, when React 18.3.2 is released, and someone runs npm install, they will get 18.3.2 using this package.json file (not 18.3.1).
I believe that the karet symbol means "Include everything that does not increment the first non-zero portion of semver", in the react example, that is 18...so anything after the 18 can be upgraded
Yeah... Theo willfully misunderstood the issue and then offered a solution that wasn't actually a solution. Incredible stuff, really.
the package-lock.json would fix this, but I was just pointing out that the semver in the package.json is not locking you to a specific version, but rather to a specific major version.
@@MaximoJoshua Doesn't `npm i` regenerate the package-lock.json on every call, anyway? So using `^version` in package.json should update dependencies on `npm i` regardless of what `package-lock.json` says
@@MaximoJoshuayeah, you should being using SAST to identify your vulnerabilities in your package-lock.json file and then deploying exactly that set of dependencies via npm ci
Honestly the biggest risk with npm is the near heart attack when you have a typo in a package you npm install and it downloads a package with 19 total downloads
It feels unsafe when I go npm install on a project where my colleagues and I are working on and it installs a multitude of dependencies because we don't just use plain react and react-dom. We use other packages which do have a bonkers dependency graph. That then installs 1500 packages with multiple vulnerabilities which you can't fix because they are multiple dependency levels deep and those packages that depend on the vulnerable packages sometimes depend on specific versions so you can't just go all willy nilly and update all the vulnerable packages because you might break your entire appllication. That is something that feels unsafe
Exactly. I knew most of what he talked about and still feel uneasy about aspects of NPM (even if I'll continue to use it) because even if React doesn't have a ton of dependencies, projects end up having these huge trees filled with tons of warnings and errors that I can't do much to fix and Theo brushed those aside as not being relevant.
So true 🥶🥶
@@viccie211 that's not an npm related problem man that's just your project being terrible... you would have these exact issues whether or not you used npm for the package stack you chose.
It doesn't matter if the npm server itself is "safe." What matters is the number of maintainers of those packages you depend on. At ANY moment, ANY number of them can go full evil mode and start pushing malware into the package they control.
good point. 5 modules from the same dev (getting about the same love) is no more or less secure than that same code being in single module.
it's the number (and "quality", for lack of a better word) of maintainers that counts.
That's what package-lock is for. If something malicious is shipped the moment AFTER you already installed the packages, you're not installing the later versions (the affected versions) unless you specify to use latest always. The maintainer can't force users who are already on an older version to install the compromised one if the package-lock is already set.
@@ThisIsNotVincent the "after" is doing a lot of work there, good thing you spelled it in all caps to give it more strength. :)
malicious release is one thing but if you think about a broader category of unsafe/buggy code, and consider that in any codebase there's a non-zero chance there is a bug / possible security hole which could have been exploited, then locking on a package version is sort of locking in the bug/exploit.
you may lock out the bandit but you're locked in with a petty thief or an incompetent janitor, or cynical guard which which still might get corrupt and let the bandits in anyway.
thing is, if you don't have a consistent, informed and good level of trust towards the source of your SW, you're kinda screwed.
whether you get there via using conservative source, conservative approach to adding deps, or raw-dogging your stuff is a different question. but for npm nowadays it does not seem to be feasible.
I don't think you even understood their worries here. You still remember the xz backdoor, right? I've seen projects relying on random hobbiests' repo. Not that those hobbiests are doing a bad job. The problem is that you have no idea what level those devs' securtiy awareness are at. Offensive operators will focus on the weakest link, and it's a disater when even experienced devs have no idea where the weakest links are at.
You missed the point by a mile here theo
Yeah, if you install everything as most do you did not understood the main point
Lately all of Theo's takes have been: "actually it's not that bad". About things that are pretty bad.
Here, he's not recognizing the fact that the average npm-based application has around 700 dependencies. And nobody who writes an application like that is even skimming through them to have an idea of what they do and why your application needs them. Much less an in-depth code review to check that it's not bundling some keylogger into your application.
He's also not recognizing the fact that dev-dependencies have a say on what gets bundled in the final artifact, so they are as much of an attack vector as regular dependncies.
At the very least some solid advice would be to avoid taking additional dependencies when possible, unless (1) you don't really care about security (which is fair in some cases) or (2) you're willing to go through the dependecies that you're taking on, at least enough to reasonably assume that they're safe.
Every dependency is a dice roll for your project's stability and security. Any one of those dependencies, at any time, has a nonzero chance of either improperly making breaking changes, failing to patch a security flaw, failing to bump versions of its sub-dependencies, or actively turning malicious.
The average node package makes you roll that dice a _lot_ more than even python, which is also famously terrible about dependencies.
Few people know about "npm ci"... That's what this misunderstanding boils down to.
Everyone is using "npm install", but "npm install" ignores the lock file, while "npm ci" honors it.
This is honestly just bad -design- naming. npm i should just install like composer install. Not update at the same time. On top of that, there is no option to just install from lock file. You can do cleaninstall with ci, but it deletes node_modules and then installs based on lock file. Having something like npm l(ock)i would be great.
Yeah, was surprised that Theo didn't mention that behavior. It's important and probably what was referred to when asked what package-lock.json is even good for.
really if someone is having this problem, they shouldn't be programming with node. its been common knowledge for years how the cli works. npm install is convienent when you need to update things like patch versions in your package-lock.
is it npm's fault that some idiot "engineers" that use npm daily do not bother to ever learn how their package manager works?
if people suck this bad at programming and can't understand the difference between install and ci, then they shouldn't be programmers. so tired of these HTML/CSS people complaining about how they can't understand something so unbelievably simple.
How does that help? When your project depends on 2k deps, no one is looking through all of these to check for security flaws, and neither are you. Sure, you lock down the versions, but you’re assuming those versions are perfectly safe, and I doubt you’d bet money they are.
@@davidhruza1134 it's more of naming issue than a "design" issue, assuming by naming we are talking about structural stuff and not labels.
Hate to be harsh about this one, but maybe someone could give Theo a give lesson on how npm actually works before his next video.
Cause this video includes alot of misinformation, and does not at all address the valid concerns regarding npm.
- npm install - ignores you package-lock file. So yeah, you would get the newest version within your semver range in you ran the command. Hence why we got npm ci.
- devDependencies could absolutely be included in your bundle. As far as I know the bundlers just include everything you reference in your code. So unless your team are very strict with devDeps vs deps when installing packages you might very well have included something just listed in devDeps.
- no mention of "postinstall" scripts, which is basically a remote code execution whenever you install a package.
the problem is not that you host directly from npm... I dont know a single person that got that confused, why did you spend so much time on it?
The problem is that even a simple project can easily have hundreds of dependencies and there is no way you can verify that the source code doesnt contain malware or a backdoor,
all it takes is one developer screwing up or getting screwed and push a malicious package that gets included by other projects and remains unnoticed, because seriously, a lot of code is unchecked on npm,
and you end up with a malicious app either for your users or compromising your own machine.
all it takes is a node js file downloading and executing a binary on your system...
Dont pretend this is not an issue, there is a very good reason deno the successor to node went with a very strict permission model, where you have to opt in to things like file system access, etc.
You don't need to verify everything, the community does it for you.
It’s not only about what’s unsafe for your users. You are basically opening up your file system to unknown entities, so it’s unsafe for you. Whenever you run a script locally, there’s so many scripts that you don’t know about that now have access to your files.
Everyone thinks that hacking something is very difficult and technically challenging but in reality all you have to do is offer a free cdns for javascript and even developers will fall for it....
It cost money and hosting knowledge
@@pencilcheck That's part of business plan. You calculate your costs and expected revenue and then do the business. I would take some fixed price we hosting like $10 per month and then choose to host only few most common javascripts. And then as soon as I exceed million hits per day I add a little extra to these scripts.
lock file and hash does not guarantee security; at most, what it guarantees is "reproducible builds" and integrity.
You can have a lock file and even a Merkle tree of all the hashes of all packages in a project, and your project can still contain malicious packages.
Yeah, but the point I think Theo is making, if you know your build is clean (at some point in time), and you create a lockfile, then it's of course secure, for the exact reason you just gave, "reproducible builds". Of course the trick here is knowing if your build is clean.. :)
@@keithjohnson6510 "Of course the trick here is knowing if your build is clean.. :)" yeah, a little teeny tiny sneaky nuance.
And if you update one of the packages, you need to review its source code and all of its dependencies. EVERY. SINGLE. TIME.
"Getting always the same thing" is not about security, it's integrity.
Unsafe package can pass integrity check and still to be unsafe.
Dependencies are unsafe. If you care about security you pick a stack without a deep dependency hierarchy.
Like coding C all the way down?
@@KangJangkrik that's too high level
Jquery ftw
The bundler doesn't distinguish between dependencies and devDependencies. Using devDependencies does not exclude those deps from being shipped to the user. The bundler ships everything to the user that is referenced from your source files.
Rather, devDependencies controls what is being removed on "npm prune --production" from your node_modules directory. That is important for node apps without bundling since the require function in commonjs will load js files from your node_modules folder.
But if you bundle your apps, you might as well just have everything be dev dependencies, since your dependencies' code will be *bundled* into your build artifact anyway.
I don’t think anyone thinks NPM is downloading assets in realtime. The problem is that you’re often blindly downloading possibly malicious code during the development process, and you have no idea if that’s contaminated your published site, because you didn’t audit the code.
thumbnail looks like low level learning
Look at the description lolz
One thing that wasn't quite clear in the video, the 'Initial npm install' the latest version that resolves within the semver range specified in package.json gets installed, i.e. if you have "react" : "^18.2.0" in package.json it would actually install 18.3.1 (Latest version within that semver range as of this date), so the package-lock.json would have 18.3.1 listed, even though package.json uses the semver range "^18.2.0" specified. Subsequent calls to npm install would read the exact version 18.3.1 from the package-lock.json file.
The package lock is nice but when you upgrade you can still have this dependency tree that is overloaded and devs just aren't going to review. This happens in most packaging systems of course but maybe has a bigger risk exposure in JS.
Rare hard disagree with the interpretation of the problem.
The problem is not the big react package, but the smaller packages that get added over time, with tons dependencies of dependencies.
And most projects I see have too many of these small helpers with a ton of baggage.
This problem exists in Linux and other software too. And the awareness there increases too.
The lock file helps keeping the software running without breaking changes due to small package changes. But nearly no one checks for security after an update to the lock file.
Web development is looking more and more like deploying a native application including shared libraries.
The way he describes npm makes it sound more like a statically compiled version of the code though. There probably are bundlers that include the deps like shared libs but the example in the video is not like that.
@@sakurajin_noa But there has to be a html page which bootstraps the inclusion of the bundle?
@@krumbergify the bundle also contains your code. The HTML Index Page is more like a bash script launching the program. All of the functionality is in the statically compiled js while some style data gets loaded from a css file.
@@sakurajin_noa Thanks for educating me. I’m just a curious C/C++/Go developer without any web experience.
@@sakurajin_noa Thanks for the clarification
The python ecosystem is Dependency hell. Matplotlib or Scipy have so many dependencies it's insane. If you look at all the ml stuff that is built on top of this stiff it gets even more crazy.
doesn't caret automatically use the latest version with the same first number? like if someone just pushed something bad as react 18.3.2 only people with exact version specified will be safe
btw I totally thought it's written like carrot lol
Perspective of a new dev who who just did a basic training course and a few personal projects, so pretty much a baby with not much knowledge yet.
The concept of using cdn has something that have felt crazy to me from the first time i heard about it and i never want to use any forein cdn in any of my projects.
I do consider package managers due to lock versions which will prevent automatic updating to newer versions with possibly security issue and that is certainly a major step in securing the depencies overall.
But the constant fear lurking in the back of my head is that this just protect from the introductions of future security issues, in all cases we are still adding dependencies in an effort to not waste time reinventing fire for every use case. That means we add a lot of foreign code and among that sea of dependencies there may be some code that has been cleverly hidden for malicious use and not found yet. It gives the feeling that a decent chunk of the ecosystem is built on trust, and trust is a very expensive commodity in our era.
Go has a decent standard library which is often big enough for basic tasks and services.
The issue is that even if don't add these files directly to our website, but the bundler still use that code from hundreds of unknown packages. Also even if we are using well reputed package, it might have some dependencies that have some vulnerability or it was never updated to cope up with latest security standards, so it is indeed unsafe.
There is a risk but that's for any package management solution. A project or library could be compromised without people knowing - e.g the recently caught SSH compromise thay was luckily caught by an MS dev. This was some malicious code in xz utils - it was almost purely a fluke thay it was discovered so quick.
By relying on any third-party library you're creating a potential supply-chain risk, period. NPM would be a risk based on the sheer volume of third-party dependencies. Just because a dependency could have a limited window of compromise, it doesn't make it any less likely or risky you'll end up distributing it.
It is a big issue but it's not unique to NPM. It's not even unique to package Managers.
thats why i only use vanilla javascript and css, just rawdogging ajax
theo with the low level learning thumbnail style got me a bit confused at first lol
honestly a huge fan of this style its so clean imo
Our solution to this NPM is just to require any code that goes to our CD/CI pipe to have any deps added to our own private repo. Nothing builds that requires external resources, all NPM packages are mirrored (and usually audited) on our own server. This way we can remove problematic packages when needed and prevent them from ending up in our apps. Also it prevents using bleeding edge things and including things like leftpad as there is tad more work to get a new NPM package than implementing it ourself with 3 lines of code. Also if package is removed from NPM for any reason, we can keep it on our server forever and keep updating it ourselves. Also we can patch/modify any NPM package without needing to interact with upstream.
The solution for the company i work for is often that we load a version of stuff like polyfill and put them in our own cdn.
We have some sites where we was forced to use plain html, css and JavaScript.
So this was our workaround to avoid external cdns
i think you completely misunderstood the concern people have. you rely on the npm service and source not getting compromised or having any zero day exploit in it. and then you rely on that being the case for every single packet and dependencies you use. not to forget that the bundler itself is also something that can be compromised.
and i think the point is not that this is by default bad or dangerous, but to even notice that it CAN be a problem or risk to blindly trust in stuff like this.
and btw the "^" is not there for fun, it has a function and drives your point totally absurd. also the take on dev dependencies never being included in the output is simply wrong.
I used to think everyone clowns on Theo for trolling purposes, until today 💀
Yes you provide the code from your server but how do you guarantee that deps from npm you've provided are safe? What are you on about?
My experience with node is when I tried a git visualization utility and node downloaded ~4000 dependencies and tried to build them all and broke at dependency 2736 and I decided npm is an infuriating mess and stayed as far as I could from the whole joke of an ecosystem.
When people complain of dependency bloat, these are the experiences they're drawing from. Not consummate developers using package locks and appropriately vendoring their dependencies and all kinds of other voluntary best practices that half the community ignores.
A lot of times, the requirements of a project compels you to work with a dependency that doesn't have its dependency sprawl nearly as well-checked as react, and just like with reading EULAs, nobody has time to do the due diligence to do a depth-first tree traversal of the dependency graph of everything to ensure they're including is trustworthy.
I know that left_pad is ancient history by this point, but it doesn't seem like the fundamental dependency recklessness that precipitated that disaster has been corrected.
I think what people dislike about npm is that prepare scripts can, by default run automatically on installation, so you can't open up your node_modules and "review the code on your machine" because the malware would be a supply-chain attack and it would be located in the prepare script which gets run when the module is installed (examples of this in legitimate cases like how lerna initializes, like how git commit hooks work, etc), so they could cover their tracks theoretically before you knew what hit you.
People who use arch linux run the same risk installing any package, it's all abt risk tolerance really when it comes to supply-chain attacks. would you rather things be on the bleeding edge or delayed by a year or two so that things can be safe and approved... (npm equivalent would be like a corporate npm mirror that scans for malware)
Excellent lesson, really well done.
Its easy to never learn what package-lock does unless you've experienced problems with not having it or updating it by mistake.
I remember pre-lockfile era zipping node_modules and having that be the only way to get the right packages and build. Otherwise one or more semver compatible updates would break something. Nightmare.
i love you theo, thanks for teaching really og stuff. keep doing this type of content
People are using tools now knowing anything about them.
The biggest security issue is not NPM, but those people.
The problem with NPM really is the granularity and no code quality checks. These two things go together: I you have a dependency graph with 100s of nodes you're never going to check every dependency. You're also not updating individual packages at that point, and just "update all", without any review of source code, or even change logs. The larger the dependency graph the less time you can realistically spend per dependency. I'd love a better NPM, with a staff that actually reviews committed packages for basic style and safety, but that would mean that every idiot suddenly couldn't publish on NPM, which would mean you lost 99% of the JS dev community.
This is of course not just a problem with NPM, all the open repositories for code have that same problem, e.g. pip etc., but JS devs seem to depend on a great number of very small packages of unknown quality a lot. Maybe it's the education/documentation we provide with JS: It's always easier to use a library or external dependency, and so a lot of tutorials etc. always use lots of libraries(And for the time-spent in the tutorial that's a good thing; JS devs just don't realize you don't need jquery to get an element by ID).
Honestly I'm surprised that this video didn't mention manifest confusion - since that's one of the big risks of NPM, as far as I understand. Darcy Clarke has a great blog post about it. (and his new company is a new registry)
The bigger point in security issue of npm is the "XZ-effect".... look at how tricky and complicated malicious actors hide stuff around and now look at npm.... do you really have any real ability to check your dependencies? I prefer minimal amount of very clean dependencies and no "package management". It really goes the wrong direction...
This video ignores the fact that even many non-malicious packages can have bugs leading to security vulnerabilities, and that many packages have deep dependency graphs that are hard to understand and impossible to audit. The lockfile helps a bit, but doesn't solve the underlying problem. If we had a fully featured, safety-checked standard library, we would need a lot less external packages, and most of these packages would only depend on the standard library anyway.
Beautiful!!! I'm so grateful for this video for reasons I can't go into. 😅
Thank you for posting. 🙂
The ^ caret before the package version number in both your package and package-lock files meant "Compatible with version", meaning that npm install will update to the latest compatible version (using semver). Does that not apply to the lock-file? As I understand, the only way to properly pin all of your your dependencies to a version would be to remove every single one of those carets.
Because you're using the caret (`^`) in your `package.json`, there's still a risk that an exploit could be introduced through minor or patch updates, even with a lock file in place.
I'm a little confused but think the versioning info Theo mentioned might not be 100% correct. The version "^18.1.2" will use newer version provided the '18.' doesn't change. If one really wants and lock into 18.1.2 the version should be "18.1.2" putting the caret '^' will lock to 18.x.x. And to bind the range one could use ">=18.1.2 < 18.3.9" so 18.3.1 or lower are fine... Sorry if i'm being a bit pedantic i'm thinking if some hacker really wanted to be evil, they could just as easily put 18.1.99 and slip in a rouge version. I'm more a tester at heart and not a developer. I'm only looking at if i'm a hacker how could i slip in a bad version... I might be conservative and assert on the production build build the versions are locked down to one exact/specific version... i.e. I tested and certified the app with these exact version, anything different... It is a completely different product...
You misunderstood this. With npm, you install a package it grep a lot of packages that grep packages, and you end up with 100 or more, no one read all of the code them, if 1 of them is bad we are done, it is so easy to get bad stuff in, just make left pad like code, and wait for a lot lazy dev to download and you upload a update and boom
Having experience with apt package manager I kinda felt relieved when I first opened a package-lock file, there's no reason at all for that tweet
Package locks don't fix anything, big dependency trees are just inherently unsafe. There could be a piece of malware hiding in a small dependency many layers up the chain which persists across many versions of the package, pinning a single version does not fix this problem.
What needs changing is the JS ecosystem's mindset on modular dependencies. Big monolithic dependencies of trusted code is always going to be safer option.
“Freezing” a lockfile does also pin the sub-dependency versions
@@toooes And who guarantees that they don't contain a flaw or malicious code?
@@piff57paff that’s not the point I am addressing
@@toooes exactly ;) But I think it's the point made by @delibirdempire4792
Great video. Only one thing, it's not because a package is in devDependecies that it won't be in the final bundle. It won't be in the final bundle because it's not imported and used in the frontend code (tree checking by bundlers).
How many dependencies does it take to create a button?
Html, chromium, V8, 7 layer osi and much more abstractions, its just dependencies all the way down
You need a package to check if the button size is a number, so yeah...
About dev dependencies at 11:30, as far as I understand "typescript" is there to translate your .ts code to shippable JavaScript, so if it get's hijacked it basically can append to your code whatever it wants. May be there are other similar options, so I wouldn't be so sure if it's a good point to care less about devDependencies security. Otherwise I would agree, if you need to include some third party code, package management is the best option, no doubt.
CDN has always always seemed so stupid to me on anything beyond a MySpace page or whatever. HILARIOUS that a site the size of Hulu would blindly load JS to their users from somewhere they do not control.
The difference with NPM (which I am sure Theo will say) is that it isn't blind. You can scan you dependencies, you can version lock the code at a place where you know it is safe, and while the code comes from someone else's place, you then put it on your actual server's filesystem so you know it won't change into something malicious. There are definitely issues and concerns with going library-crazy in Javascript-land, but my god people who think that NPM has the same problem as the Polyfill exploit are pretty dumb, tbh.
The reason that NPM gets a bad rap is that it has had a large number of packages detected that were *intrinsically* malicious (by running code at install on dev machine or including it in for deployment later along with legit functionality). Thus the pinning of versions, etc. does not help any, since they started out that way. The other reason why, secondarily and less importantly, pinning is a problem if it provokes the idea that it is hard to upgrade when one *should*. (This is where the tools can accidentally encourage laziness, etc.) Both are by way of comparison to other sources; it is unclear to me whether or not this comparison is useful. (E.g., to nuget, PyPi, etc.)
this moment section appears scared of things they don't understand (such as how the security risks of having a large number of dependencies work)
No, it wasn't about being scared about hosting all those files yourself, that wasn't it at all, and if you think that, you've already failed.
The idea of hosting the JS files on a CDN was for many reasons, 1 was for speed, hosting them on a CDN meant that the JS files would be closer to your users, so if your server is in Sydney, Australia, and you have users in say London, they wouldn't have to wait for the files to be pulled from your Sydney server, they would come from the closest CDN node to them.
The next reason was for caching purposes, it's possible if you are hosting the file on your own server, it's possible the browser won't use a pre-cached version that it pulled when you visited a completely different site. If it is on the CDN, it's precached based on the CDN, so if you visit 50 different sites, and they all use the same JS from the same CDN, then they only had to download that JS file once, every other site would be using the locally cached version.
Dependence is unsafe
Something I like about Deno is that the standard library is very comprehensive -- it has a built in test runner, linter, formatter, collections package (ie: lodash), etc. The amount of tooling you need to reach for outside of the Deno ecosystem is much smaller, and the reduced surface area means less opportunities for supply-chain attacks.
F*king GOOD CONTENT!!! Your awesome at explaining!
thank you for making this video.
You completely misunderstand this person's point. The problem is that we're importing massive amounts of code that we do not know what it does and there are almost definitely security holes throughout its inevitable. So we end up importing tons of security holes just so that we can get that one little function we need. The library bloat is just obnoxious and one of the things I hate the most about node.
Yes batteries are not included for example you can seed your random generator (i wanted to shuffle a list randomly in a deterministic way for a given user to pagination to work). So I had to get a random generator from npm. But also on the other hand there are packeges like is odd and is even for those I'll happily wear dependency shaming hat.
man, this video is a mixed bag, this is my personal take, but I think the opinion that using thrid party dependencies is unsafe isn't about packaging and what ships and what doesn't.
Yeah, you're hosting the code yourself, but this means nothing to make sure that the packages are safe unless you go and actually read the code on your dependencies, I know it sounds costly, but just blindly trusting someone else did this for you is how you can get in bad waters. And self hosting/forking dependencies is also a good practice to create resilience on your product instead of just relying on 3rd party.
One key point in here is pinning to versions vs latest. After the solar winds supply chain issue people were STILL downloading the malware’s version AFTER the brouhaha
To be even more sure of a package, NPMs new trusted build feature allows packages to be distributed with a build log of how we got to the package you’re installing & other build specific info :)
"I still don't know what is used for" imagine admitting that you don't know this very basic thing about your development tooling
There are some really good points in this video around how to improve your security posture by pinning npm packages. I do think however that since we are frequently upgrading packages and, in many cases, automatically using tools such as renovatebot/renovate that it isn't much better than including a CDN link in your html. Yes, there is more of a chance to that a hacked package gets identified but as we have already seen, it is not always the case, particularly where the maintainer(s) are they ones who introduced the malicious code.
There are also malicious packages in npm so we should be cautious about which packages we are using and the reputation of it's maintainer(s).
People running `npm install` instead of `npm ci` are the problem here.. And possibly npm for having a bad UX.
good explanation but you didn't seem to understand the argument
It's not anymore unsafe than your Linux distro... Which is proven in academia to be easy to sneak malware into. Heck, not even the distro, the fudging kernel.
Good video!
I learned that lockfiles have integrity checks!
Clone and mantain your own dependencies per project. You can then vet every package used for a project and sleep peacefully at night.
A lot of people pick on JS because JS. Yes, it's got some BS...but other languages have that as well and you rarely if ever have to deal with JS being wierd (...okay stateful regex was dumbest thing ever created).
thank you for explaining that package json stuff, was useful for me
I usually always vet my packages before I install them, which even then is rarely. We typically have such specific functionality that it makes more sense to just implement it yourself rather than relying on packages.
I'm a SecOps engineer, the main concern I have with npm (as someone who uses it for dev work on an in-house security automation app) is more about the dependencies themselves. I try to avoid using dependencies when I can avoid it, even if it means extra work. I also try to only use modules with frequent updates and weekly downloads in the millions. Still, if you're using modules like this, you should always be using a SAST tool like Snyk.
Think you missed the point, it's not about bundling what you downloaded. It's about what you downloaded from NPM. Not too long ago there was a popular library that was compromised and once installed it would delete your entire project.
Dev dependencies can still compromise your bundle. It's not that far-fetched, really.
Are you checking all the diffs for your thousands of dependencies? You're being very naive. See xz 🙂
I was on vacation for 3 weeks so now i'm catching up 3 weeks of Theo videos.
Elm packages are probably the safest you can get. Since all functions must be pure, there are no hidden side-effects at all.
If a package wants to (for example) do a http request, the developer will surely notice it, since it’s going to be in the function return type and you must explicitly pass it back to the elm runtime to execute it.
Pure languages for the win.
The Roc language also have a really interesting approach for this problem, but it’s kinda too much to explain in a RUclips comment ;)
Conceptually, I love Elm. It would have been sick if Elm had taken off more. Or made it out of beta for that matter.
@@nikomancer69 I really like elm. But the biggest impact could be to show the world a safer path is possible. Better error messages, simpler design, better linters (like elm review) and safer package ecosystem are doable. To name a few.
Anyway, there are some exciting news coming. Like open source Lamdera (soon), elm land, elm pages, and Evan is still working on some backend stuff (elm studio).
I don’t think it will become mainstream. But still my favorite place to be in the frontend world.
I never heard of anyone being "scared" to host their own JS. The reason folks embedded external JS or did the whole CDN thing was performance. Your website would load just a bit faster if you're user already had the common jQuery, for example, in their cache.
Excellent! Thank you.
Major L take. It doesn't matter if you know the exact version of a dependency you're shipping if you don't know that that version is safe in the first place.
10:40 I think lots of people think react has tons of dependencies due to their first experience with React being create-react-app, which visibly installs a ton of dependencies.
It's kind of like calling your computer unsafe because it gets updates and you don't know every dev working on your os, it could be but what's the solution write everything from scratch and hope you never make a mistake. Don't get me wrong I love writing things from scratch but I'm also not very productive and no one is waiting for my code.
I just download jQuery to the websites root, and yes I still use jQuery, and an outdated version of PHP for my personal site, there is no private user/personal data accessible on it though.
This guy is cracked in web dev!
Ljharb stiked again?