You're a great speaker and you have an extreme passion for all things nix. Your breadth of knowledge and technical proficiency is amazing. I love the way you don't waste any time giving the audience useless fluff, you're all business. I have shared your videos with all my colleagues and friends.
I really liked this talk. I also like how you can make really minimal images with Nix. Normally the minimal way was to just use the Alpine image, but with Nix you essentially build from scratch pretty easily.
Great talk, thanks for doing it! Format is a little too speedy but that's not your problem :). A good area for future public education here would be deployment. I'm evaluating this for work / life - one benefit of docker is that it plugs in quite easy to GCP Cloud Run. Going to have to research if flakes allow a similar easy integration.
Indeed, the industry of Flake native providers and hosting companies has to be created, and I'm working on that too. You still need to create legacy formats like OCI if you want to use GCP Cloud Run.
I did not know about AArch64 emulation this way. That could definitely make creating software stacks for my Pine64 devices easier. Reproducibility across AArch64 and x86-64 is what attracted me to NixOS initially.
Definitely a bumpy presentation to watch, lots of thoughts and not enough time for them all. Cool idea to use nix for container creation. Next time i would love to see bigger fonts on a light background to help with contrast. As nice as dark mode is for work, projectors do not do it justice. Love to see what you do in the future! :D
Hehe, that's definitely how it felt. And I plan on giving this talk a lot more in order to get better at it. Was definitely good to get up and do it, regardless.
you can pin a container in docker via sha ... admittedly this isn't very common I'd be surprised if we couldn't do the same using something like apt I'd also be pretty stoked to get the C build process down. I don't like muddying my path for headers 3/3 talk
Not only is it uncommon, but Nix wouldn't allow you to make this mistake. Nix demands that you specify a sha256 when performing impure computation, as a language should. Dockerfiles are not a language, so they do not prevent mistakes like this.
Dockerfile can have specified versioning applied, even if you want you'd could use dpkg to retrieve specific dependency versions such as; sudo apt-get install apache2=2.4.29-1 in the case of Debian based distr. such as ubuntu example used here There are definitely ways to make docker builds specific that reproducibility is near guaranteed
The point is that Dockerfiles allow you to make this mistake, whereas the Nix expression language doesn't permit this mistake, and forces you to specify a version, which is clearly better than allowing people to be lazy and make silly mistakes that will lead to unfun and boring debugging sessions in the future. Your debian example is apples to oranges, because those are binaries and aren't compiled from source like Nix. They are definitely not reproducible, because when you `apt install` something you're simply grabbing a binary, not reproducing anything. And even if you wanted to, it would not be trivial to build it from source again, therefore making yourself reliant on third party infrastructure to host those binaries, with no easy way out.
In the past I've encountered problems with Debian based images that are no longer supported (Wheezy for example), because the official repositories where moved to archived so apt stopped working or the gpg signatures are no longer valid, etc. This is a nice idea, but it don't feel that it is worth the effort for me (and likely to must people). The syntax is (imho) ugly compared to the simplicity of a docker file. Also If some one has to fix something I did that person is more likely to know dockerfiles than flake.nix. I guess that what I'm trying to say is that most of the time docker files are good enough.
I mean, really cool but what if I don't care since 99.9% of the times it just works. I like the NIX approach and I think it makes more sense to lock the version but still he is making the approach of using lates look very bad when often is just fine. On top of that you can use version numbers instead and if you use a stable base system you should not worry about breaking changes by design. NIX has it's beauty and place but I don't see this as a game changer that will replace docker.
plus if you want you can version your OS dependencies on an LTS release to get a pretty fine grained control over what is installed on top of such a base image using again dependency versioning; e.g. apt install apache2=2.3.4
All of this sounds more like a problem with common Docker practices than the actual functionality of Docker. 3:08 You can specify a specific base image using an SHA instead ubuntu:latest if you need/want that level of specificity. Or use a specific Ubuntu release tag as a middle way. 3:53 You can create a base image with all your apt installation already done. You can specify specific versions for apt packages if you need/want package installation to be part of the dockerfile. Nix also relies on the Internet to download stuff so I don't see how that is different. Ex., pulling the hello-world tar from GNU servers. 8:13 "Nix guarantees that we're going to get the input from the Internet every time".... How on earth can you do that? You might be able to check if you got a different input if a hash has changed but that's not the same thing as getting the sams input every time. Docker builds in a sandbox too. That's the entire idea behind a container.
Yes, and using a language like Nix prevents you from making those mistakes with common practices, and also eliminates the need for a container runtime as a plus.
@@matthewcroughan your statement would have merit if you could nix container in kubernetes, as far as im aware it's for Docker exclusively. Secondly, it's not always a requirement to have that level of reproducibility and as said Docker has ways to make it more specific to mitigate such cases pretty much.
@@Cenot4ph OCI images are a standard, and run everywhere, whether it's Podman, Docker or in Kubernetes. I also don't believe there's any scenario where reproducibility should be valued less. What's the point of software if it doesn't run everywhere? If you build it to be reproducible at the bottom, then it will remain reproducible when you're done. If you give up the principle early on, then you'll have to salvage it later, which is a waste of time in my opinion.
The best hope here is that nix can easily produce a docker file, that integration would let people switch without adding the overheard of managing two similar sets of tools. Is there any chance of that?
Not necessarily. The point of this presentation, for me at least, was to show why Docker is not very reproducible and why Nix is a better choice. This naturally leads to questioning whether you need Docker at all in the first place.
@@matthewcroughan because of the enormous amount of preexisting functionality around it. Adoption is easier with a bridge, don't make people swim if you can help it.
@@DrewIsFail I think you might be using Docker wrong then, if you think this collides at all. Docker is for running software, not building it. That's the point I'm trying to make.
Because it doesn't produce the same result twice when you run 'docker build', it produces two different rootfs contents. You can build it and redistribute the tarball, and mark it as golden with a tag. Of course distributing something you built one-time is reproducible. But, if you took the same Dockerfile and ran 'docker build' on it twice, it would not produce the same rootfs.
Thanks for the reply! Checking out nixos.. just trying to wrap my head around the use case I guess. In my head if I write a docker file and am using tags its seems to me to be repeatable, I can control what is pulling into the container, what deps are used, etc.. Thanks man! Going to give nixos a go
@@danepane527 You can't control what apt does when you apt install hello. You're at the whim of it, because it isn't reproducible. Whereas nix uses the nix expression language to give you full control over the inputs. Apt, by comparison, at least by default, will go and do some random stuff and populate a package database differently every time it is ran. Note how you first have to do `apt update` which populates this database differently and non-deterministically every time it is ran.
All the same flaws exist there. Podman and Docker are just container runtimes. How the OCI (Tarball) image is built is what this talk is about, and Podman's 'buildah' is just as unreproducible as a Dockerfile. Podman would be no better than Docker at building software reproducibly. It still allows unconditional access to the internet in the build environment and doesn't provide you with a domain specific language to make builds happen reproducibly.
@@lonterel4704 I can agree with that. But I don't think it takes anything away from what Nix is, or is going to be. It will only get better. You could probably say the same about the Python2 and Python3 transition. I don't think that detracts from Python at all, just because the transition was messy.
Наука
based and nixpilled
Love the distinction between repeatable and reproducible :)
You're a great speaker and you have an extreme passion for all things nix. Your breadth of knowledge and technical proficiency is amazing. I love the way you don't waste any time giving the audience useless fluff, you're all business. I have shared your videos with all my colleagues and friends.
Thank you, Matthew, for presenting this presentation! It was awesome! Please do more! :-)
I really liked this talk. I also like how you can make really minimal images with Nix. Normally the minimal way was to just use the Alpine image, but with Nix you essentially build from scratch pretty easily.
Goddam! The best high-level explanation in the whole internet! Now, I comprehend what Nix is!
Love the nixOS shill at the end. Great presentation.
Great talk, thanks for doing it! Format is a little too speedy but that's not your problem :).
A good area for future public education here would be deployment. I'm evaluating this for work / life - one benefit of docker is that it plugs in quite easy to GCP Cloud Run. Going to have to research if flakes allow a similar easy integration.
Indeed, the industry of Flake native providers and hosting companies has to be created, and I'm working on that too. You still need to create legacy formats like OCI if you want to use GCP Cloud Run.
I did not know about AArch64 emulation this way. That could definitely make creating software stacks for my Pine64 devices easier. Reproducibility across AArch64 and x86-64 is what attracted me to NixOS initially.
Definitely a bumpy presentation to watch, lots of thoughts and not enough time for them all. Cool idea to use nix for container creation. Next time i would love to see bigger fonts on a light background to help with contrast. As nice as dark mode is for work, projectors do not do it justice.
Love to see what you do in the future! :D
Hehe, that's definitely how it felt. And I plan on giving this talk a lot more in order to get better at it. Was definitely good to get up and do it, regardless.
@@matthewcroughan a bumpy presentation is better than no presentation at all! Keep it up! 😎
fantastic presentation, was about to ask for a longer one but just seen there is one.
Great! Finally something new. I just what i needed
Awesome, I love it, great presentation. I'm going to start learning this right away!
the best presentation about nix, Show me how powerful nix is, and it's so enjoyable to learn nix.
nice comparison of nix vs docker
Thanks Matthew!
so good
Good job!
Nice talk, subbed
Quite interesting! What was the software you used for the slides? I really like the look of them
github.com/maaslalani/slides
the Paddy the Baddy of software development :)
Impressive, self-apparent competence. Fuck ya man, much respect.
you can pin a container in docker via sha ... admittedly this isn't very common
I'd be surprised if we couldn't do the same using something like apt I'd also be pretty stoked to get the C build process down. I don't like muddying my path for headers
3/3 talk
Not only is it uncommon, but Nix wouldn't allow you to make this mistake. Nix demands that you specify a sha256 when performing impure computation, as a language should. Dockerfiles are not a language, so they do not prevent mistakes like this.
🔥🔥🔥🔥🔥 😎
Is it possible to read the thesis?
I'll get back
no crows in your cornfield, thats for sure!
Dockerfile can have specified versioning applied, even if you want you'd could use dpkg to retrieve specific dependency versions such as; sudo apt-get install apache2=2.4.29-1 in the case of Debian based distr. such as ubuntu example used here
There are definitely ways to make docker builds specific that reproducibility is near guaranteed
The point is that Dockerfiles allow you to make this mistake, whereas the Nix expression language doesn't permit this mistake, and forces you to specify a version, which is clearly better than allowing people to be lazy and make silly mistakes that will lead to unfun and boring debugging sessions in the future.
Your debian example is apples to oranges, because those are binaries and aren't compiled from source like Nix. They are definitely not reproducible, because when you `apt install` something you're simply grabbing a binary, not reproducing anything. And even if you wanted to, it would not be trivial to build it from source again, therefore making yourself reliant on third party infrastructure to host those binaries, with no easy way out.
great cap, where did you get it from?
Made it :)
In the past I've encountered problems with Debian based images that are no longer supported (Wheezy for example), because the official repositories where moved to archived so apt stopped working or the gpg signatures are no longer valid, etc. This is a nice idea, but it don't feel that it is worth the effort for me (and likely to must people). The syntax is (imho) ugly compared to the simplicity of a docker file. Also If some one has to fix something I did that person is more likely to know dockerfiles than flake.nix. I guess that what I'm trying to say is that most of the time docker files are good enough.
So Docker isn't idempotent? Dang.
i need to up my game. this guy is a f*ing chad
Gigachad
I mean, really cool but what if I don't care since 99.9% of the times it just works. I like the NIX approach and I think it makes more sense to lock the version but still he is making the approach of using lates look very bad when often is just fine. On top of that you can use version numbers instead and if you use a stable base system you should not worry about breaking changes by design. NIX has it's beauty and place but I don't see this as a game changer that will replace docker.
plus if you want you can version your OS dependencies on an LTS release to get a pretty fine grained control over what is installed on top of such a base image using again dependency versioning; e.g. apt install apache2=2.3.4
All of this sounds more like a problem with common Docker practices than the actual functionality of Docker.
3:08 You can specify a specific base image using an SHA instead ubuntu:latest if you need/want that level of specificity. Or use a specific Ubuntu release tag as a middle way.
3:53 You can create a base image with all your apt installation already done. You can specify specific versions for apt packages if you need/want package installation to be part of the dockerfile.
Nix also relies on the Internet to download stuff so I don't see how that is different. Ex., pulling the hello-world tar from GNU servers.
8:13 "Nix guarantees that we're going to get the input from the Internet every time".... How on earth can you do that? You might be able to check if you got a different input if a hash has changed but that's not the same thing as getting the sams input every time.
Docker builds in a sandbox too. That's the entire idea behind a container.
Yes, and using a language like Nix prevents you from making those mistakes with common practices, and also eliminates the need for a container runtime as a plus.
@@matthewcroughan your statement would have merit if you could nix container in kubernetes, as far as im aware it's for Docker exclusively. Secondly, it's not always a requirement to have that level of reproducibility and as said Docker has ways to make it more specific to mitigate such cases pretty much.
@@Cenot4ph OCI images are a standard, and run everywhere, whether it's Podman, Docker or in Kubernetes. I also don't believe there's any scenario where reproducibility should be valued less. What's the point of software if it doesn't run everywhere? If you build it to be reproducible at the bottom, then it will remain reproducible when you're done. If you give up the principle early on, then you'll have to salvage it later, which is a waste of time in my opinion.
The best hope here is that nix can easily produce a docker file, that integration would let people switch without adding the overheard of managing two similar sets of tools. Is there any chance of that?
Not necessarily. The point of this presentation, for me at least, was to show why Docker is not very reproducible and why Nix is a better choice. This naturally leads to questioning whether you need Docker at all in the first place.
@@matthewcroughan because of the enormous amount of preexisting functionality around it.
Adoption is easier with a bridge, don't make people swim if you can help it.
@@DrewIsFail I think you might be using Docker wrong then, if you think this collides at all. Docker is for running software, not building it. That's the point I'm trying to make.
Why not both?
Because only one actively prevents you from making mistakes.
Docker just doesn't work for me
I don't get it.. you could use docker + tags, create you own base image, etc... How is a docker file not reproduceable?
Because it doesn't produce the same result twice when you run 'docker build', it produces two different rootfs contents. You can build it and redistribute the tarball, and mark it as golden with a tag. Of course distributing something you built one-time is reproducible. But, if you took the same Dockerfile and ran 'docker build' on it twice, it would not produce the same rootfs.
Thanks for the reply! Checking out nixos.. just trying to wrap my head around the use case I guess. In my head if I write a docker file and am using tags its seems to me to be repeatable, I can control what is pulling into the container, what deps are used, etc.. Thanks man! Going to give nixos a go
@@danepane527 You can't control what apt does when you apt install hello. You're at the whim of it, because it isn't reproducible. Whereas nix uses the nix expression language to give you full control over the inputs. Apt, by comparison, at least by default, will go and do some random stuff and populate a package database differently every time it is ran. Note how you first have to do `apt update` which populates this database differently and non-deterministically every time it is ran.
Use Podman ;)
All the same flaws exist there. Podman and Docker are just container runtimes. How the OCI (Tarball) image is built is what this talk is about, and Podman's 'buildah' is just as unreproducible as a Dockerfile. Podman would be no better than Docker at building software reproducibly. It still allows unconditional access to the internet in the build environment and doesn't provide you with a domain specific language to make builds happen reproducibly.
@@matthewcroughanhmm, gonna have to look into Nix then.
@@uziboozy4540 you would not. There are messy docs about old nix, now its more messy with flakes
@@lonterel4704 I can agree with that. But I don't think it takes anything away from what Nix is, or is going to be. It will only get better. You could probably say the same about the Python2 and Python3 transition. I don't think that detracts from Python at all, just because the transition was messy.
@@matthewcroughan my point is lack of docs exists already several years. Nixos community should delegate someone to update docs.
No.
Maybe.
Yes.
Definitely.
Go on then
Thanks, I hate it. I'd much rather use dockerfile.
Lol.