Making Simple Linux Distro from Scratch
HTML-код
- Опубликовано: 16 окт 2023
- In this video I will demonstrate how you can create a small and simple Linux distro from scratch, together with the kernel I will use BusyBox for the user mode utils and Syslinux for the bootloader.
Installations: apt get install bzip2 git vim make gcc libncurses-dev flex bison bc cpio libelf-dev libssl-dev syslinux dosfstools
Linux Git URL: github.com/torvalds/linux.git
BusyBox Git URL: git.busybox.net/busybox 
Наука
This ~12 minute video explained it better than the entire "Linux From Scratch" book.
yeh
The difference is that as the video said, this setup isn't usable or stable for production. Linux From Scratch, if done properly, can be
@@takodanotdakota you can theoretically just use this busybox and init script as a bootstrapper for a fuller distro, like alpine
true ngl I made a simple distro with gnome desktop environment with this video
My guy really using Ubuntu on Windows Terminal with DWM to build a distro from scratch
i legit had to go to the bathroom bc of that
Your "Guy" mabye use WSL (Windows Subsystem For Linux)
@@skybloxstars'your "guy"' most definitely did not use WSL lmao, he literally spun up a docker container step 1
@@huhulili9021 Sorry
@@huhulili9021 docker desktop (windows) depends on wsl but it exposes the docker command to use in cmd/psh
I have run Linux since kernel 0.99 (Slackware on floppies) and never really knew/thought about the individual bits and pieces that make up a Linux system. This is super educational. I would love to see an expanded version of this that allows for a basic Linux distro (that includes everything that you mention it would need for production) but without any of the bloat that normally comes with distros. A tiny server Linux if you will. Also, just in general, thank you for making these videos - they're awesome! I appreciate the time and effort you put into them.
Thanks! That's a cool idea :) maybe also in the direction of making a minimalist distro for raspberry pi
@@nirlichtman - Yes - as is said "you stole words from my mouth", raspberry pi. now i use diet pi but I still want to see the development from start for educational means. and 1 more thing now my eyes will be looking for the files shown in your video as Linux boots up on my laptop 😆
@@nirlichtman Yes, that would be awesome!
@@nirlichtman, thank you for the videos, that you make. A minimalist distro for raspberry pi sounds nice. Please do it, when you've got time. Waiting for it.
agreed, it would be very helpful!
Wow, so this is the way to a truly unbloated distro bliss!
Good stuff!
real unbloated implies kernel only has necessary options to run on hardware, busybox without the extra cruft...
WinXP Desktop Pure genius How long ago?
@@nonenothingnull Bro! How deep goes the rabbit hole?!?
@@alexviralata1356 very, tcc instead of gcc, musl, etc..
@@nonenothingnull who cares about busybox, it's only needed for a second at the startup. The real pain is choosing only the needed kernel options, while also maximizing the performance.
I think this is the best educated Linux tutorial I've ever seen
Are you lying?
No@@bones.24
@@bones.24 why would he?
@@yesh420 prophets lied, why wouldn’t he
@@bones.24 wdym?
I have been studying how to do this for thirty years and this is the most coherent explanation/demonstration of how to make a bootable system that I have seen. This is a magnificent work. Thank you.
Even though I have no interest in doing something like this myself, I found the whole process rather interesting. 🙂
Really cool to see this, thank you for the simple and easy to follow tutorial. I look forward to diving deeper into how this works.
Thank you this video! I have been using Linux for years, but I am interviewing with a company that builds their own kernel for their product and I wanted to know something about the process. This helped me so much!
my dream of creating my very own linux distro one day, is a step closer. thanks!
i tried LFS a long time ago, but gave up half way through did something wrong didn't know what.
but yeah a full distro from scratch how cool is that. for educational purposes too.
I mean, lfs is more for developers. People who already know their way around on debugging the worst and shittiest situation possible
@@Mempler feels like this is for a certain kind of user too. Using docker and not volume mounting /boot-files, so that you have to docker cp later...
omitting the configuration of syslinux
using vi, instead of cat or echo to create files.
It's nonetheless great to have many people with different ways of achieving what they need.
@@LewisCowles agreed, doing lfs helped me to learn linux in depth. funnily enough, i never read the book.
I literally just googled a shit ton and reverse engineered arch linux lol
I see LFS as something that everyone serious about Linux should install once, and then never actually use it. (If you really _want_ to use it, you can, but it probably won't be very enjoyable.)
Very nice video! I would enjoy more like these!
You did this in 12 minutes?! That's crazy! 🤯
That's linux, once you go black you don;t come back.
Amazing and very useful for studying Linux!
That's so cool :O
I did LFS like three times and i always wondered how could i simplify the process to the maximum
Very cool and well communicated. Thanks!
Thanks, Nir. It was my dream to build my own distro. I printed my name on the distro. Thanks again!
BRAVO, BRAVO! That was great!
this make me recall the time we were trying with LFS distro - just doing it without understand anything :)
super clear and interesting! Subscribed.
This video is gold, thanks man. Ever thought about making a video on your very custom distro using the knowledge you showed us in this video?
This is the absolute best tutorial on this topic. Please do a tutorial on compiling more packages for the custom operating system. Packages like GCC, sudo, nano and whatnot. And please also do videos on setting up ttys that work hand in hand with key configurations, I've set up my ttys but they don't work with key configurations. A lot of demands here but please also do a video on setting up the root user, sudo (root access) users and normal users, as well as, how to compile and set up wayland, xorg, sddm and possibly even KDE Plasma.
May God bless you bro😂 I needed this tutorial. I tried this at first but since i knew nothing about where to place the kernel, or about how to create and use a bootloader, the best i could do was only to make a chroot folder (and i became really good at it lol i even forged my own "terminal" and got it to work in chroot) but now im gonna be practicing this new knowledge everytime i get a chance.
Thanks a Lot. For now all i can do is add you a new sub. 🎉
Thanks! :)
@@nirlichtman pleasure bro. One day you'll get as many views as you deserve
@@nirlichtman also, have you tried to make a syslinux boot disk with a drive size of say 8mb/8 counts? It doesn't work. Do you possibly know any reason why?
You mean you are trying to write syslinux on a physical disk of 8MB?
@@nirlichtman I mean a disk file (dd if=/dev/zero of=eight bs=1M count=8) kind of thing.
Another issue I have seen is that it loads the kernel and the initramfs but it kinda gets stuck there , even after I followed the along with your tutorial.
Its hilarious how BusyBox is basically a legitimate add-on to docker in that both are containers but BusyBox works as a "from scratch" image for Docker, more so than the docker official scratch image itself
Oh wow, my genius brain clicked on this video thinking you were about to build a basic operating system written in Scratch. And now I want to see someone actually try to do that.
I mean, there is linux on scratch
yeah but that's just using the linux kernel making a disto, not a new OS itself@@commander3494
@@commander3494 Is that a word play on Linux from scratch?
@@ghosthunter0950 im not sure if ifs intentional or not
@@ghosthunter0950no but it's just running a compact RISC-V emulator to run a VERY RUDIMENTARY Linux implementation that does not include most of the basic system utilities, and even text editing is limited to "ed" only, not even "vi", and forget about "nano"
simply amazing...
Happy to see this!!😃
good and precise to the point
This is super cool!!
Funnily enough this is also exactly how Linus Torvalds created the first version of Linux back in the days. He just fired up docker and booted some Ubuntu image
I laughed so hard, just opened the video and scrolled to the comments and saw that
Subscribed right away
cool! combined with something like sinit and sbase and voila - extremely small desktop linux :)
Instead of
dd if=/dev/zero of=boot bs=1M count=50
you can also use
truncate -s 50M boot
truncate changes the size of a file. If the file does not yet exist, truncate creates it. If the size is shorter than it used to be, the file is in fact just "truncated", hence the name. Yet if the size is bigger, zeros are added at the end until the file reaches the desired new size.
Not only is the command shorter, it also has another advantage: truncate creates a sparse file on systems that support that. That may not matter in this example but it would matter if the file was 100 GB in size. In a sparse file, only blocks of that file that contain actual data also use real disk space. All blocks that are still zero filled are virtual, they don't use any disk space and they are also not actually created when you created the file, so file creation is also ultra fast, as all that really happens is that a directory entry is created for that file. Yet when the file is accessed, it will always behave to any file operation as if the file really was 100 GB in size and all filled with zeros. If you jump to the middle of the file and write some data there, only blocks covered by that data will materialize and actually consume disk space, the rest of the file stays virtual.
Nice, didn't know about truncate, will be useful for my upcoming graphical Linux from scratch video, thanks!
@@nirlichtman just seen this one, can’t wait :)
First time knowing about sparse files, I love it!
Thanks for the great comment ⚘️
Very instructive thanks!
@@nirlichtman or use something like "fallocate -l 50M image" to do it in yet another way
Yes u made my day. 🎉😊
Hello. If I understood correclty the ramfs is indeed for syslinux. The kernel will mount the cpio archive pointed out by syslinux to the kernel through the command linux. I remember It is also possible to embbed the cpio inside the bzImage itself (option in menuconfig).
Great tutorial Nir! I have a few followup questions.
1) You used the default kernel config file that came with the repo, instead of replacing it with your Ubuntu machine's config file. And in the default config file, the only change you made was enabling 64 bit. Would that cause any issues if you were to use this kernel in production, since it wasn't "handcrafted" for your machine?
2) Why don't you have to run 'make modules' after you make the kernel? Does the default config file just set all modules to be compiled into the kernel?
3) If you set some modules in the config file to be compiled external from the kernel, where would we copy those .ko module binaries so the kernel could use them? Would they need to be copied into the initramfs cpio archive? Or would they need to be placed on a real disk in a root filesystem the kernel would switch to after it goes through initramfs?
4) Since you copied BusyBox directly into the initramfs archive, the kernel didn't need to mount any real root file system on a real disk. However, suppose we did want to use a real root fs on a real disk, and install BusyBox there instead. How would we need to change the init program and initramfs directory structure and content to make the kernel mount the real disk and change the root directory to the one on the disk?
Looks like someone else also watched Rob Landley's video on a minimalist Linux. Although this is even tighter than his, even if it doesn't exit properly.
Excellent
Following this tutorial was easier than installing arch Linux
Except this is not arch so it doesn't help you in the slightest
splendid
thank you
Why are you using Windows though?
Gotta base yourself
what a mad lad
Hello! I might be doing something wrong, but I tried to boot this on an older laptop without EFI support.
Seemingly everything seems to be fine (booting it with GRUB, actually), but after I get a few messages from the kernel printed to the screen,
everything goes black, my pendrive (the boot device) blinks a few times and I get stuck there.
What could possibly be the issue?
technically with EFI stub compiled into the kernel, you can directly boot it from uefi, no bootloader needed.
Hello NIr,
thank you very much for all your "in-depth" and short, very well explained videos! Great!
Do you like to do a "real" minimalistic linux system with a graphical interface with that approach? What do you think? Or is it too much to think of and better take arch linux and do a minimalistic linux with dwm, vim and so on from there?
Thanks, shalom and best regards
Daniel
Shalom Daniel, Thanks :) I haven't tried running Xorg on this distro so I am not sure if it would work, I do skip a few things that may be necessary to make the graphical interface work (for example running a proper init and setting up the tty devices), but this is a great idea for an additional video about actually setting up a minimalist graphical linux distro from scratch
@@nirlichtman any update on this?
I have few things to request if you don't mind creating a video.
1. Can you create a distro using a base debian or ubuntu image?
2. Illustration of how to use systemd as a init system.
3. And use Grub2 as the bootloader.
such amazing video. Can U publish similar but to create minimal system to boot on old raspberry 1/2 ? Regards!
Nice.
I wanted to customize the kernel for development boards (embedded systems)
and this video being what i realy expects by building the ninux kernel from scratch making this video as a series that show the other options for production standards
Nice video!
I wanted also to implement the switching from the initramfs to a normal ext4 partition. It took me the last 6 hours because I had to use a different bootloader (systemd boot on aarch64) and because the fucking UEFI implementation for QEMU (asahi linux) did not had a SATA driver built in, I was not able to boot from any SATA disk, which means I have to use a virtio disk to boot and the Linux kernel did not want to detect a virtio disk but an SATA disk which means, I am currently having two disks with the same img file just to be able to boot from it and mount it in Linux. Arm is nice but the next time I am doing it on my Desktop with x86 xD
I too managed to do this but after a few days of reading and reverse engineering the archlinux iso
mine is the same thing as in the video up until the init file, which instead of calling /bin/sh, mounts the hdd, chroots into it and asks for login without the kernel dying if you press Ctrl+D
its very simple and not very operational so far but I think I can easily add systemd, git and gcc, and eventually make an actual distro out of that you can add your own packages
what do you recommend i look into/do if i want to go further in making it useable?
I had to do somth9ing very similar before to be able to boot systemd system when using a btrfs file system. I however did not use busybox as I did not need a full enviroment to work in. The system basically just needed to load up enough system to be able to read the file system and start systemd, then pivot root into the real FS. Funny enough what you did is pretty much what dracut and other programs are actually doing when they create the initramfs image.
Based
Slackware installer is my ideal distro 😂
I love your video, can you make one on how to make http request for windows os?
Yes, I plan on covering some winsock related stuff as well :)
Thanks for guide.
I wonder , how extendable this system in your opinion ? I have been through Lfs book and I can't get rid of intrusive thoughts to build a (mostly) gnuless system from a linux installation you shown. I completely understand that it doesn't make any sense, pure enthusiasm and curiosity.
It is as extendable as any other distro except that you need to manually bring over the package binaries for every tool you use (or manage to get some package manager running). To make it actually usable for day to day you would probably also want to setup proper tty devices, init (and not just start the shell right away like i do in the vid - busybox also includes init built in but I didn't use it in this video) and a persistent file system to start after the initramfs
@@nirlichtman nah just write every program you need in vi like a real hacker
@@nirlichtmando you have some documentation or recommendations on the persistent storage thing?
@@itsoctotvYou could create a 1GB zeroes file, format it with EXT4 and mount it at boot. The first part is the same as the FAT one here, then you have to pass it to QEMU and also write an fstab that mounts it, etc.
@@nirlichtmantheoretically it maybe supports a subset of Debian packages since a minimal dpkg is present on it by default
Great video. Thank You .... Only I would change one thing :) ...as a total beginner, it would be nice to see the result / what to expect on the beginning of the video, and then the explanation how to do it. For me the experience was as : command after command, and I had no idea where we are going :) ...on the end it was totally worth it :) Thank You
Thanks for the feedback! 👍
Can you make a follow up video like configuring syslinux to auto boot the os and adding persistence or burning it to a iso/img
Yes, that is a good idea! BTW the image in the end that I boot with qemu is a disk image you can burn also to a physical disk using dd for example
I know a lot of people have asked the same question, but how would I write this to a USB drive to boot it on a real machine? I've tried to use tools like dd or genisoimage, but even after a few hours, I still can't get it to work. Usually, it just says there's no bootable device, other times, it says the device doesn't support UEFI booting.
windows -> wsl -> ubuntu -> dwm -> distro from scratch
sweet
How to create an iso from the img, maybe with grub to automatically load kernel +initramfs
How did this come to be? I mean... I already knew what a bootloader, kernel, busybox and initramfs did, but I never knew (or even got close to thinking for that matter) that you could put them together just like this. Did you first come up with the idea to do the bare minimum to get a shell working as init process, then read how the initramfs worked in detail just to get the system running and finally looked for a minimalist bootloader? Just how did this pop in your mind?
hmm... simple panic fix would be while [ 1 ]; do /bin/sh; done? xD
At any rate fun stuff, I used to do LFS for fun when I was a teen, makes me miss those days.
Great job, thanks!
Hey you installed busybox differently but *Build Minimal Linux in 1 Hour* guy did make a static binary and linked it inside /bin/busybox
Can I ask what is the difference which is better approach?
Hi i keep getting an error when mounting boot as m (mount: m: failed to setup loop device for /boot-files/boot.)
Try it with sudo
me too, idk why and in docker im root
Could you theoretically install a desktop envirement and make an iso out of it. thx, btw. good video
Have you done one about buildroot? To make custom os for customs apps light weight
Make a video showing how to install gcc on that distro
How long did it take to compile??
mine is going for about 30mins
It would be nice to start with Docker installation. Not everybody knows what it is.
Also, do you think you could please make a tutorial video on how to compile the custom distro into an ISO file?
No need to create an ISO to get this running on a real computer, since all you have to do is write the boot image to a physical disk using something like dd for example (I should have added .img to the boot image name in the video to avoid confusion)
why was this recommended to me at 12:53 AM and why am I watching this
I am gonna try once i get some time to spare. Really interesting. Can we make a distro with AI integration ?
Great question
how would i go about implementing apt or opkg?
Is this minimalistic environment sufficient to bootstrap Linux From Scratch?
how to add a desktop environment into it? and store it to a usb?
Instead of running the shell with the shell, you could just use a symlink I think?
Yes, that is a good point :)
How?
@@abrahimzaman360 ln -s
Good point if you're not going to extend script further. Which I doubt if you would like to play with your distro and extend it.
Can you explain how you got DWM on what I assume to be windows?
Explanation in the welcome link on my channel description
@@nirlichtman OK thank you
- Artimis, bird of war.
6:15
Ahh, it confuses me, much like the chicken and egg situation 😅
Are you talking about the fact that I used the shell to start the shell? I am just running the shell and the first command the shell is going to run is the shell :)
I have a custom built distro without boot configuration. So based on this tutorial, i did convert my whole root directory into init.cpio (init script also added). And did the same boot configuration(based on distro size, i have created the boot file with count=4000 (4GB)). But when i want to run the kernel via qemu, "init.cpio...failed, no such file or directory" error occures. Does anyone have a idea? The cpio file exist i had double check.
I can’t remember the name but in the pre Linux kernel 1.00 days there used to be a single boot floppy that would ask a bunch of questions, download and compile from sources. In those days I’d boot it and answer everything in the morning then let it down load, configure, compile, and install till the next morning. Lol. Ah the good ole days
The step I'm not following fully is at 9:25 when you mount boot into m. What happens when you mount a file to a directory then copy those files into that mounted directory?
The file I am mounting on the directory is a disk image I initialized as a FAT file system, after I copy those files into the mounted directory it adds those files to the FAT file system on the disk image so the Syslinux bootloader that I installed on the disk image can find the kernel and initramfs
What version of ubuntu are you using to make this build?
In this video I used the default latest docker tag of ubuntu
anyone else failing with "mount: m: failed to setup loop device for /boot-files/boot." on the first mount command?
Make sure you started the docker with the privileged flag
is there any way to install a package manager onto this?
Yes, you can bring for example the package manager that comes with Alpine that is called apk-tools, I may make a video about making a distro around that package manager.
If I want to use the default init from the busybox instead of the simple bash you provide, what should I prepare?
LFS will guide you :)
Just compile busybox with it and "ln -s sbin/init init" so the kernel can use it
So busy box is a user space? What are alternatives, primarily interested into using apt, if applicable a basic GUI (I have my mind of what Icon set I want to use). Those options are and or conditions. Just want to know some user space alternatives and its names.
ofc I meant user space interface
You could also use gnu coreutils or uutils (rust rewrite of coreutils)
These are the minimal set of Unix tools to have a functional cli
You'll also need a proper init system like runit,initrc,systemd if you want to build a modern functional os
@@Person1873 Thank you very much. I just started to stick my head into this topic. This is very useful information, as before I wasn't able to find much about it without knowing what to look for.
hey i wanna make my own ubuntu based distro with some custom tools amd softwars which i can then install on real computer too, with custom boot logo and all, can you guide in some main points like what to learn and from where to start, waiting for reply please must guide me
Hey man nice job on the tutorial! Only thing im stuck on is mounting the boot file. It says that it keeps on saying that vfat is unknown. Im not sure what I did wrong.
Perhaps you forgot to install the dosfstools ?
@@nirlichtman weird thing is that I have it installed. I'll reinstall it again and check it
@@nirlichtman yeah it still gives me that error for some reason
Hey I have a project as a part of my OS Course called "Linux Kernel Development". Can i make this as my project. Or kernel development is far more beyond what you did in the video?
Depends on the project criterions, on this video I did not make any modifications in the kernel code but mostly worked with it as a whole component, I have a video about adding a simple system call to the kernel, that may be more relevant. Another option is my video about making a simple kernel module.
So this Linux system wasn't mounted to the root filesystem and instead stuck in the initramfs?
I notice the root filesystem is not created. Wow looks interesting, didn't know that it would work.
who needs arch or gentoo when we have this?
Next video: Making a lunar module from scratch
MFW your x64 kernel compiles into the arch/x86 folder ._.
Oh, yes, my dream is now one step closer to being realized. Soon enough you all shall tremble before the power of my own linux distro, liNUTS.
So I've tried doing this on my M1 mac and spent an hour swearing and cussing ;-) So the problem is the arm architecture (even the menuconfig is different). It creates a slightly differnt image (Image.gz) etc. And the worst problem is the bootloader. Syslinux doesn't exist there (but there is some syslinux-common package). If you could suggest some workarounds, it'd be great. Tried playing with u-boot, but since I have very little experience with bootloaders, I didn't make much progress.
In the end I emulated the right architecture (docker run --platform linux/amd64) and I'm doing it from scratch again. However it's visibly slower. On original architecture, the kernel compiled in 8 minutes. Now I'm way past it and it's still compiling
A possible solution to decrease the build time is to cross compile, you can try for example buildroot which should help easily cross compiling for different archs
me too, i can feel you
"mount: m: failed to setup loop device for /boot-files/boot." ???
problem solved... did "apt upgrade" in addition to "apt update"...
How can we use GNU instead of Busybox?