Say Goodbye to Containers - Ephemeral Environments with Nix Shell
HTML-код
- Опубликовано: 8 июн 2024
- With Nix Shell, you can instantly spin up the exact environment you need, tailored to specific projects, and discard it with ease once you're done. No containers!
We'll walk you through the installation of Nix and various ways to craft custom environments with Nix Shell.
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
Sponsor: Cast AI
🔗 cast.ai/devopstoolkit 🔗
▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬
#NixOS #NixShell #EphemeralEnvironments
Consider joining the channel: / devopstoolkit
▬▬▬▬▬▬ 🔗 Additional Info 🔗 ▬▬▬▬▬▬
➡ Gist with the commands: gist.github.com/vfarcic/8ebbf...
🔗 Nix: nixos.org
🎬 Dagger: The Missing Ingredient for Your Disastrous CI/CD Pipeline: • Dagger: The Missing In...
▬▬▬▬▬▬ 💰 Sponsorships 💰 ▬▬▬▬▬▬
If you are interested in sponsoring this channel, please use calendar.app.google/Q9eaDUHN8... to book a timeslot that suits you, and we'll go over the details. Or feel free to contact me over Twitter or LinkedIn (see below).
▬▬▬▬▬▬ 👋 Contact me 👋 ▬▬▬▬▬▬
➡ Twitter: / vfarcic
➡ LinkedIn: / viktorfarcic
▬▬▬▬▬▬ 🚀 Other Channels 🚀 ▬▬▬▬▬▬
🎤 Podcast: www.devopsparadox.com/
💬 Live streams: / devopsparadox
▬▬▬▬▬▬ ⏱ Timecodes ⏱ ▬▬▬▬▬▬
00:00 Intro to Ephemeral Environments
04:14 CastAI (Sponsor)
04:54 Ephemeral Shell Environments with Nix
14:31 Nix Pros and Cons 
Наука
What do you think of Nix? Can it replace containers on your laptop (for tools, not apps)?
Nix does not do distributed deployment (multi-host) [of applications/containers], I use Kubernetes for that. So I use Nix for the host (servers, workstation), development environment and containers. But not for deployment beyond a single host.
I run NixOS on my desktop and laptop and I like it a lot, but I'm not comfortable enough with the Nix language to really use it effectively for building my own packages and environments yet, it has quite a steep learning curve.
@SuperHeroINTJ I do not use Nix to run apps. That's where kubernetes excels. I use it as ephemeral environments for tools i need.
@sligit oh yeah. It has a very steep learning curve if you want to go beyond simple use cases. For me it works well since I'm not building packages or doing anything "fancy". I use it almost exclusively as ephemeral ebbs for tooling (not apps).
@@DevOpsToolkit Yeah it's great for that.
I'm loving the fact that someone is finally speaking about this! ❤
Viktor - your use case is the exact same use case I have had teaching students and nix has become a foundation in the platform I am building. Love hearing your experience!
Nix is awesome, switched to NixOS on everything, doing local dev with Nix and custom modules, doing CI with Nix and Gitlab through a custom module, etc.
I want Nix everywhere, it's extremely nice as a developer to be able to configure and "program" everything, like writing custom modules etc.
Thank you for the amazing video. Your ability to showcase the tool while teaching how to use it is truly a superpower!
❤ Wow, very nicely done and what a great find! I can tell you times when my world changed, VMWare, docker, lambda, python virtual environments, and now NIX shell!
This is indeed, a game changer, thank you for making this video and sharing it with all of us!
I am now subscribed to your channel with notifications, enabled, and of course, thumbs up!
Thank you,
Ernie in Pennsylvania
A video which addresses more Nix features would be awesome.
Finally, an explanation of Nix package manager that I can understand. Thank you so much for this video.
Nix is amazing good. I have been using to synchronize all my 24 hosts, host K3s, and to generate containers. Yes! Nix generates containers that are equal bit by bit no matter in what environment. And in a declarative manner!
Nix unifies the software stack. The dependency chain is same be it on a host, server and containers. Nix builds once. Then, you can re-use the artifact/cache everywhere. Also supports Musl. It can generate container images smaller than Alpine when using NixNG.
This great video rekindled my interest in Nix. Thank you so much!
The nix hash bang thing is the killer feature here... Love that!
Very interesting. I really like that they are solving this problem. Thank you for sharing!
Nix has it's own CI called Hydra for continuously building Nix packages. Hydra can upload to a S3 storage, then you consume Nix packages from binary cache (S3, etc). Hydra is a bit more clever when building packages because it sends individual packages to cache instead of doing full builds and only then sending to cache. This is helpful when having lots of builders building simultaneously. It is possible to re-use cache early. I'm unsure if Nix without Hydra does that too.
Amazing content! Thank you for sharing! I recently started exploring nix and I think this is pretty exciting for the reason you mentioned in the beginning - keep the mac virgin and lightweight. I am curious about mixing nix with tmux and nvim.
I was talking with my team about Nix right before this video went up. It's an incredible piece of tech. I'm exploring how to separate runtime dependencies and build artifacts using nix flakes; this might also make a good video. Well done.
Also kinda nice to use the nix dockertools to build your container images.
Wow this is insane! I do some contracting on the side and often have to download various software to work on different projects (I cant imagine how much random clutter I have rn), this can greatly simplify my workflow for different clients. Then once the project is done clean up is super simple!
That's my primary usage of Nix. Spin up what I need when I need it and destroy it once I'm done. Since everything I work on is in a Git repo, I just keep `shell.nix` in each and start my working session with `nix-shell`.
At work we use nix flakes in each repo defining tools (and specific versions) required for deploying and working on what's in the repo, e.g. terragrunt, terraform, kubectl, etc. Overall it's been handy, We've considered moving to containers instead for that use case, but since it works it's not a high priority, and doubtless there are pain points we'd run into.
fantastic video and explanation, many thanks.
nice video. I've been using nixOS for a few months now and as a developer it simplifies my workflow significantly and for all of the languages and frameworks I use. no more worry about version conflicts. one thing I was going to mention about nix and build environments in containers, I would absolutely reduce the size of all ontainers by volume mounting the nix store.
Please pin this
First time viewer, recommended by the algorithm. I subscribed :)
Odlican video i kanal, ne znam kako tek sad nailazim na njega! Pozdrav :)
I would love to see this catch on.
I've been running nixos on one of my machines for a month/so and I love the implications on sscm this ecosystem has
thank you for the very informative video!
Great video. Thanks
Thanks for this!
Nix looks like a great tool for ex2 based automations.
That intro goes way harder than expected
it's like virtual environment but for the shell .. very nice. this seems to be a great tool for managed/hosted development environments.
Very nice! Would you suggest using nix when building pipeline containers vs doing apt-get installs and lots of other makeshift things?
My recommendation would be to do neither. Build images with what you need and use them in pipelines. They are well equiped to use containers. Switch to nix if that is not an option (which is rarely the case).
Thanks for another great video! Generally speaking I find this approach good whenever you want to have your tools with you to do something ephemeral, the CrossPlane course example is nicely fitting here. In that case the container approach may be overkilling.
I even think that there are similarities here with the container based approach, like you have to install nix on the host, you still have a local cache of what you install, and so on.
My preference still goes to containers whenever you're able to define a set of tools that you always use for some reason, for example when preparing a develoment environment.
Containers in my opinion are great to enable remote development with tecnologies I use, like devcontainers and DevPod.
Certainly I woudn't use them to run pipelines locally!
I agree. When working remotely, containers are a better choice. But, when local work is in wurstion, i switched fully to nix, except for the apps themselves.
@@DevOpsToolkit sorry I didn't notice that you already wrote "except for the app themselves" before! I agree this is a better approach
Nix is very good, and Nix Shell is a killer feature. Wish more OSS supported it to make contributing easier to get started with.
Nice tool! really eager to start using it and see use cases. It would work with packages that requires sudo priviledges?
Yeah. It's essentially a (more powerful) Shell.
Wondered when you would cover this :)
I've been using this for a while now for work stuff. The other important thing can be to get everyone using the same versions of things. You can do this with flakes. The file is flake.nix, inside you define one or more devenvs, and the command is nix develop. The first time you do this you get a lock file that specifies the versions.
Nix is great, Nixos is even better
I know Nix and I've always wondered if we should use a Nix-based container image, or a scratch-based container image with a static nix-shell (if that's even a thing), for example for pipelines. I still have no solid conclusion
Very keen to know your thoughts on comparing this approach with immutable infrastructure. Great video by the way!
I would not compare it with immutable infrastructure. I use Nix to generate local environments, mostly focused on tools I need. My apps are still running in containers which are running in Kubernetes in one of the hyperscalers.
Nix makes it totally easy to make portable OS agnostic development environments everywhere as well. I have a folder where I'm teaching myself GoLang by building a web CMS app/framework and I can use my Mac or Windows (running WSL) or even in a NixOS VM. So whenever I have some free time, Just pop into one of the environments, do a git sync, tinker and code, git sync and done!
That's my favorite use case. Put nix shell file into a repo and everything i need to work on it is there.
This was the best Nix tutorial I have seen so far, and I have given up on a couple of them.
I was really excited until I saw Windows requires WSL2. As a minority linux user I often want to share my TTPs with my majority Windows colleagues but if they have to install WSL I might as well stick to containers.
I think that WSL is a must, no matter whether one uses Nix or not. Most of what we do is on Linux and being able to be in a similar environments as production is important. On top of that, trainings, tutorials, scripts, etc. (almost) always assume that one is using Linux or Unix . MacOS is essentially a UI on top of Unix. There are many other reasons but it all boils down to Linux/Unix being a standard. WSL is, in my opinion, the most important addition to Windows.
P.S. I rarely use Windows myself so I might be completely wrong on that one.
@@DevOpsToolkit WSL + the 'Windows Terminal' store app. But people are so lazy or so hard to make make a change of what they are used to... I'm tired talking colleagues into start using WSL, tired of seeing someone going VS Code + PS for terminal - it's beyond me why it's so big of a hesitation for most people ..
@@DevOpsToolkit WSL + the "Windows Terminal" app from the store! Mentioning it here as I know some people have never heard of it, and IMO this is the closest you can get to a real Linux multitab terminal. WSL also pairs nicely with VSC with the WSL extension which is another "a must" .
I've been hearing about Nix for a while now and love the appeal. However, I see a lot of similarities to cloud- init. Not necessarily the ephemeral stuff, but more the reproducibility via a central config. It seems you can create ephemeralility via a cow file system with snapshotting capabilities but of course would have to roll up the solution yourself to leverage cloud-init. Curious what your thoughts are. Have been really appreciating your channel and videos. Also heard you as a guest on the Dev-Ops Podcast. Thanks for your content!
Very interesting, haven't dug into Nix yet, thanks for the intro.
On a separate subject, I'm hoping you'll be talking about Apple's new Pkl language and how it compares with Cue for managing Kubernetes manifests. I've been meaning to dig into Cue, based on your past recommendations, but now I'm curious whether it makes more sense to look at Pkl. It sounds quite promising.
I'm still in the process of experimenting with pkl. I need a bit more time before i start working on the material.
The problem I have with Pkl is that it is built on an inheritance based model whereas CUE is explicitly immutable.
That is indeed one of the big difference. Personally, i prefer CUE immutability but I also recognize that is one of the reasons for CUEs steeper learning curve.
Here it goes: ruclips.net/video/Nm1ioWPRRVQ/видео.html
Hi Viktor. I'm curious about something in your demo. You were using "kind" in your nix-shell, but I didn't see you also add Docker to the package list for the nix-shell environment. Does nix-shell install Docker as a dependency to "kind"?
It does not work with Docker. I do not use Nix Shell for any non-cli app like VSCode, Slack, or Docker, but only for CLIs.
@@DevOpsToolkit I see. So, to use Kind while in a nix-shell, you also need to install Docker separately (outside of nix-shell) on the host, right?
@easmithdev i think so. I haven't tried to run docker through nix so i might be wrong.
Quality content
What about kernel and os arch type bindings? More on shift left / true devops point of view
Wow this changes everything! BUT....there is 1 major flaw. I started setting up my configs and immediately noticed that there is no real way to specify the package version. For example, I need an older version of the Velero cli to match the version of Velero running on the cluster, however nix does not seem to have anything like velero:1.9.4. And no, I don't want to go to lazamar and specify the tar hash in my configs for each package.
so if you take a look at the nix package repository in a lot of cases you'll see the mainline package having a bare name, "python" for example which is currently at version 3.11 and different versions having different package names, like "python312" for the release candidate version. when you're building your shell.nix obviously you can use those packages, but if your needed version doesn't exist, then you may need to create your own nix repository. then it becomes as simple as copying one of the other build expressions, updating the source location and hash (which nix-build does for you) and giving it a package name that matches the version. then all you have to do is specify that channel in your nix shell script and you're good to go. it's really not as bad as it sounds
How does Nix handle pinning versions of packages? And how up-to-date are the packages?
You can specify a version so instead of `normalPackages.nodejs` you can set it to `normalPackages.nodejs-12_x`.
As for how up-to-date packages are... It depends on maintainers of a specific package (just as anything else in open source depends on maintainers). My personal experience is that packages are up-to-date most of the time, sometimes one version behind, and rarely old.
@@DevOpsToolkit thank you!
Nix is first and foremost a language of the functional kind, which just so happens to be able to build and install packages. Now cover flakes, you'll love them, honestly. Reusable Nix.
LOL
No, it's not.
I like to think of nix as being primarily a software deployment model. As per Eeclo’s thesis. The nix language is just a (partial) means of realising this model. You could create and build derivations without using nix language at all. tweag’s blog post: “NIX - TAMING UNIX WITH FUNCTIONAL PROGRAMMING” is a good, and relatively short exposition of the core ideas
Nix is awesome and there is a newer way of doing this (different, not specifically better) using just the `nix` command.
However, it might be an issue for your purposes because by default the commands are not enabled.
How do you compare Nix versus DevContainers? Can Nix help with the problem of enabling repo maintainers to declaratively set the development environment for anyone who cloned and attempts to build the repo source?
I prefer using it for local development. For remote stuff containers are a better choice.
Nix excels at this. By using a shared nix config all developers are ensured the EXACT same environment, fully provisioned, with a simple command. Docker containers are good for deployment/distribution and can be used to share dev environments but that's what nix is best at. You used the word "declare" which hits the nail on the head: nix is declarative while containers are imperative.
Hmm. Very interesting. But how much reliable is this? I will try it anyway...
There are a few tools in this space including, asdf, mise/rtx moonrepo proto, devbox.
there's a fundamental aspect completely missing in your video: versioning...
does nix-shell support package versioning? How can i use a fixed nodejs or java or anything else version?
at work we use various script with asdf to allow this kind of setup, with specific versions based on the directory you go into, thanks to direnv... and we had to take care of fixing the asdf azure-cli plugin because it didn't work anymore after python update to 3.11, for example...
Yes it does support versioning.
One of nix's strengths is the granularity of it's pinning... not only version level but commit level. Dockerfiles are convenient but don't ensure true reproducibility while building a Docker image with nix does (no apt install/update stuff...)
so I'm assuming that Nix would only be able to handle stuff that is command line driven in nature, it won't be able to do stuff with like GUI IDEs, correct? Say for example if I was using PyCharm and I wanted to use Nix to install a whole bunch of tools, there wouldn't be anyway to get PyCharm to use those tools installed by Nix? Like a specific version of Python say or c compilers etc.?
Not sure... I use it almost exclusively to setup CLIs since i tend to work trom a terminal most of the time (excluding editing and observability).
In my very limited experience with Nix, I solved this by installing my IDE in the nix shell and then running it from the nix shell itself. I don't know if a better option exists.
I don't think that this has been stated explicitly...nix provides truly reproducible build environments (which containers typically don't/can't). Nix is not a deployment tool. It can produce a Docker image without needing a Dockerfile. Then you deploy with k8s or podman (or docker itself if you must ;-)
Docker and nix are different tools for different purposes. Nix is for consistent builds and dev environment.
Struggling to understand something - I see how nix-shell can standardize development environments across developers (awesome) but then when it comes time to replicate the same environment on production machines, how would one do this without also running prod in nix (likely not great / isolation etc?)
Very interested in this tool - thank you.
I do not use Nix to create environments where apps are running. I use Kubernetes, preferably remote, for that. My use case for Nix is to standardize tooling I need for developement. For example, the project I'm working on right now, needs `gum`, `git`, `gh`, `helm`, `kubectl`, `kind`, `yq`, `jq`, `bat`, `buildpack`, and `aws`. Instead of having all those permanently installed on my laptop, I just enter the directory with the repo and execute `nix-shell` which picks up those tools from the `shell.nix` file in that same directory. On top of those, it is, at least for me, extremely useful when experimenting. Instead of constantly accumulating new CLIs instelled with `brew` and, probably, left on my laptop forever not matter whether I use them or not, I just pull them into `nix-shell` session and `exit` when finished.
In other words, `nix-shell`, in my case, is not a replacement for app environments but, rather, a temporary env. that brings me all the tools I need. On top of that, anyone who wants to work with me on the same project can just start a `nix-shell` session without having to worry whether he or she has the tools I use.
@@DevOpsToolkit thanks for keeping me on the rails. We've struggled for long enough with unification of development environments and runtime environments that my brain didn't recognize the contrast between the tooling bits and the runtime bits.
I can see how this would be extremely valuable for that.
Presently we use docker to handle rt environment but do nothing for tooling. I'll be considering that now that I'm presently and consciously aware of that divide.
Thanks again
I really like nix containers for some uses, but I'm still new to them - can nix environments block access to the computer, I'm more and more using containers and have zero dev tools on my host because I can't trust the package maintainers to not be crooked or get hacked.
Especially with the npm/node.js ecosystem, my new dev box has zero node.js /npm/yarn etc on the host.
You need to trust someone or something. If you're using containers as a way to install tools, you still have packages either through base images or by adding `apk install` or whatever you might be using. Unless you are using Linux on your laptop, those containers are running in VMs which is isolated but you are probably mounting volumes and processes so it's not bullet proof either. At the end of the day, you need to trust something unless you're willing to inspect the code of whatever you're using.
@@DevOpsToolkit I've been a long time linux user, for 30 years or so. So I know the foundations of trusting trust. I'm careful about what to trust and NPM/Node.js have proven to me, that they are not trustworthy at a rate that is high enough to just give them access to my machine.
I expect that docker and the container ecosystem that is powered by the linux kernel to be robust enough to prevent the random arbitrary code from encrypting the entirety of my harddrive and returning it for the cool sum of a single bitcoin.
I know that it will not stop an exfiltration bot or a crypto miner from running
I never allow the docker socket to be mounted, so they cant use that to elevate privileges and I'm aware there could be bugs that get exploited overall
I don't mind random code running on my machine so much as I just don't want it to screw over my day.
I'm just worried about the liability of lost productivity due to something malicious happening to the data on my computer.
The policy of 'default to everything open' was design of the past, and should be considered deprecated at some cost but not all costs.
The idea is cool, cgroups and namespaces should be enough to have an isolated and reproducible ci. Also LXD containers.
But they're still not useful for a mature CI environment where you need to integrate with an IDE for building and debugging code, export artifacts like reports, security scanning and signing immutable layers.
Are they going in this direction with Nixos?
Assuming that you're talking about local IDE, Nix Shell works like any other Shell and everything you're normally using in an IDE should still work.
P.S. That video is all about Nix Shell, not NixOS.
@@DevOpsToolkit sry I meant nix shell for the CI and nix Flakes for the CD
I'm not sure I understood. Why would you use an IDE in pipelines (CI)? Independently of that... If you do use Nix in Pipelines, it is effectivelly a replacement for containers (for tools, not apps).
@@DevOpsToolkit I mean we need the same tool with same settings to be available as in the ci pipeline as in the development process ( integrated with the Ide possibly) as we do today with docker.
I don't know if today this is already possible to replace with nix ecosystem.
@alessandro-affinito you're right. If everything needs to be exactly the same, you need to choose one of those (Nix or containers) and use them everywhere. That will also mean that you will not be able to use IDE for any tasks. If everything needs to be exactly the same and given that IDEs are silly to use in pipelines, they are out as well.
Personalky, I think those are extremes that are not beneficial.
Actually starts at 5:00.
In the context of local development, sure you may not need containers. But I think you'll agree that containers are still very much desirable for other things.
Absolutely. All my apps run as containers in production and since i ephemeral environments to be as close to production as possible, apps are always in containers. I do not use Nix to run my apps. I use it to get the tooling required to work on a project.
@@DevOpsToolkit it's just that the title made it seem like one doesn't need containers at all :)
@ekolteenarp true. I should change it.
My fear of using this shiny new ways of installing packages/environments is that I never know if I'm stalling a compromised version of the package. I still prefer to install something the way the producer recommends rather than a more practical, well intentionned packager. But maybe I'm just paranoide. Containers at least are a little bit sepparate from your OS, nix shell looks like python virtual environments, a workaround with PATH. Anyway, thanks for the video.
You can also make a video on devbox which uses nix for dev environments.
Adding it to my TODO list... :)
you've beaten me to it 😅
Done: ruclips.net/video/WiFLtcBvGMU/видео.html
Looks very interesting. You say it will allow windows, macOS and Linux devs to work on the same repo in the same env, but what about a repo that uses symlinks?
I haven't tried it with symlinks, but i guess it should work.
@@DevOpsToolkit i just asked Bing chat and it said that would still be an issue 😢
A case (for me) of using symlinks would be one repo with one directory for front-end and one for backend and then having shared types files between them
I haven't tried that but it should be easy to verify.
What's the point of this wheel reinventing? Docker can pretty much do the same. Even if Nix does provide less configuration fuss and more concise command usage, it still lacks many packages which Docker has
How does Nix work with secrets and authentication. If I start a Nix shell do I need to reconfigure ssh for example?
Nix Shell is a shell just as sh or zsh or bash so there's no need to SSH into it.
@@DevOpsToolkit Sorry, I meant what is the nix way of setting up ssh or secrets to access other resources from a nix shell. For example does my ssh config used for github or a bastion host persist across nix shells?
@hambroman yes it does persist. I think you're mixing OS with Shell. If i would create a new VM with NixOS, if would need new keys and what so not. However, I was just switching from one Shell, from zsh, to another, to nix-shell.
I tried nixos 4 years ago and was basically just confused. I got kubernetes to work on it but due to it not using standard paths longhorn didn't work.
So I put it away. But now there's a lot more tutorials available for it and I'll try again to daily drive it.
Longhorn works fine in NixOS. The solution for that problem is simple: Just extend PATH environment variable this way: "PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/run/wrappers/bin:/nix/var/nix/profiles/default/bin:/run/current-system/sw/bin" I usually use Kyverno for that.
NixOS K3s has been well maintained. @euank did good work there.
@@SuperHeroINTJ that's cool, this workaround wasn't available then - but I switched to Talos for K8S deployment now already, which just recently got longhorn support :)
what about security/isolation like firejail?
I haven't used firejail so I can't comment on it.
What about networking? nix or guix solves development environmenk in terms of libraries and dependencies, but one of containers appeal that you can create complicated stacks with private networking for dev, test, production environments....
For me, Nix can never replace app environments simply because they need to mimic production as much as possible and that means either kubernetes or cloud services like Google Cloud Run. The part of nix that i am using is related to tooling using for development but does not go to production.
Going in, I'm hoping you will compare nix to what I would naturally turn to: devcontainer and devpod. (Is nix the emacs of k8s?) After watching, it will be interesting, for me at least, to find out how wrong I was going in. Nix runs as non-root with seccomp restrictions?
Nix is many things but, in my case, it is first and foremost a way to start a shell with all the tools i need without installing any of them on my machine. Devpod is, essentially, a VM.
After watching, I wonder which (PSS) admission policies I need to disable on my namespace before I can run nix.
@joebowbeer i never tried running it in kubernetes (assuming that's what you meant by namespaces). I don't see it being of any use there.
@@DevOpsToolkit oh! I was thinking the title referred to k8s ephemeral containers. sorry for not tracking.
@joebowbeer that's my bad. I should have explained it better.
Which most lightweight, nixshell or container docker?
I prefer Nix for tools and containers for apps.
direnv could be even simpler as you can select cource for separate gcp azure aws
I'm using Charm Gum scripts so that people can choose and provide info interactively.
how network work inside a nix shell ? can i access a service running on the host ?
I would not recommend nix as a destination for your apps. Use kubernetes for that. I use Nix as ephemeral environments for the tools i need, not for the apps.
@@DevOpsToolkit I think you should explore Nix for building containers. That is a very valid use case. And common too. And dependencies can be shared with shell. So it is less work than having to double package it for Dockerfile & Alpine.
yes, its for a tool that connect to database to exec some sql script
@@DevOpsToolkit
I started Nix 2 months ago. My biggest regret is that I didn't start earlier.
I love the concept of Nix. But for some reason I personally struggle with getting everything working as I want. Probably because I’m so used to Arch, and recall much from Debian and Red Hat, and Nix is a total departure from those.
Maybe one day I’ll get good with Nix. It does in fact appear to be superior.
And if you like LISP, you might find Guix amusing. Based on (idea of) Nix, kinda 🙂
I tried Guix after using Nix for about a year, and it turns out that at least for me that guix was a gateway drug to more lisp. 7 years on guix now, and I'm addicted.
How is this different from something like Fedora CoreOS or Tumbleweed
To begin with, it works on every OS. I, for example, am a Mac user who i not planning to ditch macOS for CoreOS.
I think everyone who knows a certain amount of Nix would love to use it for deployments... AND YET...
The way to do this seems forever non-obvious
As far as the pipelines goes, why not just have your pipeline cache/restore that nix cache? Then you have a really clean setup that replicates your local dev without having to potentially manage both solutions depending on the context.
I think I have a better solution which I'll explore in a video in a couple of weeks 🙂
What i was trying to say is that I'm not sure that rewriting pipelines to nix provides enough return of investment. For new ones it is a good option.
You said don't use nix for CI/CD pipelines, use containers. Agreed. You should rebuild those. But couldn't you just use nix in the container build? That way your dev and CI/CD environments use the same setup?
I started doing that, but with Devbox instead of Nix directly.
@@DevOpsToolkit lol. After writing that comment, I saw your devbox video and have spent the last 24 hours rebuilding all my workspaces with devbox. I think I'm in love. I haven't tried the devcontainer/codespaces stuff yet but that will be huge.
Maybe you can also do a follow up video on nix flakes
Adding it to my TODO list... :)
Combining what you demonstrated with direnv which makes it even more convenient.
It's also a rather minimal use case for what Nix provides.
Does nix run on windows yet? No wsl2 doesn't count
It doesn't (without WSL).
cube-cuttle or cube-control?
I tend to say cube-cuttle but i heard others saying at least 5 variations of it.
I'm confused ... Isn't nix a stand-alone OS? How'd you install it? You create a virtual machine with nixOS installed? obviously, not. Cause, I'd be easier to install docker and be given a container. Got to be missing something.
Nix is quite a few things. I explored in that video only Nox Shell, not the OS.
One of the confusing things about nix is what the hell it is 😊. Nix is both a package management system and a scripting language which can be installed on any OS. NixOS is an operating system built with nix but you don't need to run NixOS to use nix.
Nix is indeed very confusing. It has a very steep learning curve and, as you said, it is too many things at once. Personally, I use it only as a package manager where those packages can be installed anywhere and are ephemeral in natuve. I don't use it directly but through Devbox (video is coming up tomorrow).
seems to have issues with selinux.
Isn't it just a fancy flatpak?
To begin with, Flatpack is a package manager for Linux while Nix packages work everywhere...
This doesn’t address the problem of differing or missing library versions, which is why containers are a thing. If anything the ephemeral-ness of containers is a problem.
Assuming that you're referring to tools we're using while developing and not libraries in app code (as what I shown in the video), you can pin them to any version you need.
And now onto nix flakes? :D
Correct
A solution in search of a problem
Not clear to me how this is better than a writing a "Dockerfile" with the packages you need installed inside.
Assuming that you are using windows or macos, containers are running in a VM and that complicates things like, for example access to binaries you already have installed, filesystem, local network, etc. Nix Shell is just a shell like bash or ish with the additional abilities to temporarily install additional packages.
Ok but file systems and local networks can be mounted/exposed in containers. Access to other binaries on the system kind of goes against the notion of "isolated environments" as well.@@DevOpsToolkit
@shazmailz you're right, but i think we're talking about different types of environments. I guess you're referring to preview environments like those you would spin up when creating a PR while I'm talking about a way to create an ephemeral environment that will bring me all the tools i need while working in my laptop. In that case, i am not concerned about isolation and i still run my apps as containers one way or another. What i was talking about is s way to bring in all the tooling i need (e.g. kubectl, jq, yq, g loud, etc.) in a way that is not permanent. I got sick of having thousands of random binaries/CLIs floating around my laptop, many of them to be used sporadically, some even never again. When i execute brew list, I do not even recognize more than half of the output.
I'm still confused as to the difference/advantages. This is exactly how I use a Dockefile that can be built according to local context, filesystem mounted, run interactively or truly ephemerally (one command at a time), using just what is needed from the host environment, and deployed to pipeline. Requires the same number of prerequisites on every host system (including WSL on Windows). And you can deoloy the same image to CI/CD, no duplication. I see no effective difference but I know i must be missing something.
@BDnevernind I'm was not talking about the pipelines but the tools (CLIs) you need while working (e.g. kubectl, yq, jq, helm, nodes, etc.). Are you spinning containers for those while working on your laptop?
I use VS Code with Dev Containers.
Why gh and not git?
gh Has some GitHub specific commands like, for example, to fork a repo or to create it.
Going to the website to create repos always annoyed me. With the CLI I can just `"git repo create" and I don't have to open my browser. Plus other conveniences.
Looks like conda
I like the prospect of nix. But the installation and removal of nix itself makes it a hard sell for my team members
10 mins in u were just downloading packages...
I guess you never heard about packer and vagrant.
I know that one can create a VM as a work environment but that is cumbersome. That's why we have containers.
@@DevOpsToolkit Docker isn't commercially available for free anymore though...
Docker isn't, but Docker is only one of many ways to run containers. Have you tried Rancher Desktop?
@@DevOpsToolkit You're right. Still. Ecosystem needs to equalize for a solution that the big players want. Right now that's VMware (which is rapidly falling apart ofc), and Hyper-V. These engineers come for cheap and they do good work, these platforms are GOOD. Arguably Hyper-V has an advantage today because of inconclusive licensing.
In my opinion - containers are proper for certain endpoints you want to access, but for your major infrastructure - I would still go for local AD, exchange at azure, and devops on either cloud providers k8s, and then production on the cheapest provider for your service, if you're a data provider that'll be different that if you're a retailer for example. Naam sayin'? :D
tldr: I don't think the ecosystem of containers is completely ready to take over full production.
Containers are not ready to take over complete production. Nothing is. Some will say that VMs are not ready for complete production and will run some workloads on bare metal. Nothing ever replaces everything. Containers are ready for significant amount of workloads and the ecosystem around containers is unlike anything we saw in the past.
NIx is a multi killer technology, can kill any template rendered, like TF HCL, kustomize, go templating.... etc, and any package manager, deb, yum ,apt..
from source to deploy this is nix. real reproducible environments I use it for all my personal configs and start introducing it at work as well.
20 years old technology, why is not leader market yet?
- highly complex language
- hard to debug or errors that does not describe the real problem
- unorganized documentation or actualy deprecatted.
regarding deployment yes there are tools to deploy remotely via nix, basically deploy-rs, colmena, nixops.
also even have their own CI system hydra.
btw @DevOpsToolkit maybe you want to mention about devbox jetpack, is an "starter" options for those who don't want to learn nix lang but still want dev environments :)
I think Kubernetes handles deployment better than those Nix tools for deployment. Particularly in a dynamic environment. I'm not saying there aren't tools trying to cover that space. It is just that Kubernetes does that well. As Nix can generate containers using Nix packages. It is okay to use Kubernetes from there on for deployment. At least, that is what I have been doing succesfuly.
Oh boy, so true. I already thought I was the only one ...
I am really having problems with that language (and I've been working with Linux and various scripting+programming languages way longer than Nix exists). Then comes the whole "are Flakes the only way to go - or experimental, that's why we do not document them properly" question.
I installed NixOS and really wanted to love it. And ditched it after several weeks.
So, I love the basic idea. And the way @DevOpsToolkit uses it (Nix shell and package manager with minimal configuration) is simple enough - that works fine. But when you get into the details of that programming language ... run as fast as you can ... 😛
@MichaelRedinger i feel that what i showed can be used by anyone and that anything else in Nix requires dedication and uphill battle with the docs.
@@MichaelRedinger You can't learn Nix in a too short period. It is something to be introduced in small doses. Like, start introducing only the shell or only home-manager in another distro or some other host that is not critical, maybe a VM. And then, incrementally build your skills. Nix scope is too large. You haven't got to know Nix yet. Flakes is an abstraction for composing Nix w/ using git repositories. So, for each repository, you have a flake for that. It's an interface. Listing inputs and outputs. It's simple. But you have to learn the abstraction. There is no way you can be productive without understanding the basics of it. And Nix language is a simple language. If you are thinking it is complicated. You haven't understood it. It's so simple that your mind repels. For example, import is `copy and paste`. Nix is bash. Etc.
"Say Goodbye to Containers - Ephemeral Environments with Nix Shell" Na, Containers are not going anywhere
I prefer gnu guix over nixos and nix because of funky nix DSL. nix language becomes burden for nixos... gnu guix uses guile which is a dialect scheme, a powerful language which is famaous tool to write compilers. It is more concise to write packages with guix. Only downside for me of guix is that it does not use systemd. Gnu herd which is systemd equivalet is not as mature as systemd and is very tightly integrated with guix package manager...
voice audio needs to be clearer, too much bass
I'm trying a different audio. It will be in videos starting a month from now.