1. Unused RAM is wasted RAM. 2. Sometimes high CPU usage is a good thing. 3. Inodes need to be tracked, as well as available space. df -i command is importatnt. 4. Large number of distros is good thing, a superpower. It allows you to change up the user interface. 5. Use LVM (logical volume manager) to get more control over your storage, treating multiple things as one. You can resize file systems online. 6. You don't have to memorize Linux commands. Memorize everyday ones; it's okay to look up lesser-used ones. 7. Always have a backup distro. 8. GIT version control is not just for software developers but also for system administration. 9. LVM has a snapshot feature, useful for trying new things. 10. Especially for cloud servers, take all individual volumes when backing up data.
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it.
For years i was obsessing about have "free memory". And getting mad, that my operating system, "ate" all my memory. Untill someone, asked me why i wanted a slow inefficient computer.
The only real thing that needs to have a free excess is hard disk space. Everything else only needs to have a buffer of free. Its not about how much you have free, its about how fast that ram can do its job. That's all.
Around 2002, when I was fed up with the repeated crashes of my system, I went to ask three questions on the usenet forum: - "As a non-IT person, do you think I can install and configure a GNU/Linux system? - Am I obliged to know by heart a myriad of magic formulas? - Will these orders still be valid in a few years? The old wolves reassured me, and told me that if I got into the habit of reading the manuals and getting information before doing any manipulations, things would fall into place quite naturally. Moreover, the commands that I will have to use will be the same in ten years. They were right, and I just jotted down some "cooking recipes". 8-)
In my experience, most GNU utilities tend to keep the options the same over the course of time, just adding new ones. But, if you jump over to BSD the utilities may have the same name, but different options, and sometimes the GNU has things BSD doesn't. I encountered such issues with the "whois" commands, so I had to tweak how I automated checking for expiration dates and name servers when jumping between the two.
Fast forward ten years and you have systemd with totally new commands and concepts, plus a lot of other replacements (ip address show instead of ifconfig for example).
@@illegalsmirf very true, I didn't say that we don't get new software. Like getting a list of open sockets, we now have ss -tpln to see what daemons are listening on which TCP ports. I have also moved from ipchains, to iptables years ago, and I think that we have moved on from there. But I don't manage several firewalls for ISPs anymore, so a bit out of the loop there.
Thanks for being transparent about not having to memorize commands. You just lifted a huge load off of me brother. I have definitely built my own cheat sheet. Which is very helpful. As always, I love your content. Keep up the good work.
The real skill you need is to be able to refresh your memory quickly. So you can look up what you need and understand what you are reading. Only the real world tests this - it applies to every aspect of computing and most other things as well.
I now remember cut because I've used it a few times in BASH scripts, but I don't really remember awk and sed, so I just consult some scripts I've used them in. I throw those scripts up on github and consult them when I need to remember what I did before. Also tab is a real blessing trying to remember or discover commands.
After you’ve used certain commands and options a few dozen or hundred times, you do automatically memorize them. For the rest, there’s man (among other sources). This is why I’ve started writing man pages for some of my own programs. Nitpick: grammatically, it should be “I wish I had known”, not “I wish I knew”.
I agree with you, Jay, regarding the certification process. It seems that the primary focus of these certifications is more about generating revenue rather than genuinely ensuring quality or skill. Many organizations appear to leverage these certifications as a way to monetize professional development, rather than offering a truly valuable and rigorous credential. This commercial approach can undermine the credibility of the certification and the true expertise it is supposed to represent. It's important for us to be discerning about which certifications we pursue, considering not just the cost but also the tangible benefits and recognition they provide in our field.
The biggest handicap I experienced with Linux, which is what I have used for years as my primary OS, was a an absence of well documented applications. That has greatly eased over the last 20 years or so, and generally anything you can do using a Windows (or Mac?) system can be done on a Linux system.
More or less, yes. There's always the learning curve, though depending on the person. A lot of people can pickup Linux pretty quick and others will struggle a lot.
Most things, yes. The difference is more often “how easily” or “how quickly”, rather than “at all”. Frex, I just found out that there’s something very much like Time Machine for Linux that’s been around for a while, but it’s nowhere near as easy to set up or use. And sometimes the deficit in UX is a dealbreaker-for some software, the UX is as much or more important than the functionality. For me, the main software I’m not willing to give up in order to use Linux is Scrivener. Several open-source and commercial imitators now exist for macOS, MSWindows, and Linux , and some people find some of them just as good as Scrivener. I don’t. There’s other software that I’d have trouble replacing (BBEdit, InDesign, 1Password, Things, frex), but I either already know of Linux alternatives that are “good enough”, or they’re more “what I’m used to” than “what I love”, so I’m sure I could adapt to new software and new workflows.
not really true. audio production does not have a workflow on linux that isn't a huge pain in the ass. and most plugins are not supported. its unfortunately the reason why i can never use it as daily driver. also, no photoshop isnt quite a deal breaker but its close. there are plenty of examples of this where you can technically do the same thing but you are forced to using a greatly inferior (much of the time) app to do the job. the fact is that even here in 2024, no one uses linux as their daily driver because its good but because they just hate windows and apple too much to stomach using them anymore.
@@vivsavagex There is an immense difference between can't be done, and "I don't like the work," which is your position. Many computer users confuse having to learn an interface with "difficulty." There are many entries these days on audio production on Linux. Many even recommend Linux as a platform. That said, the first time I ever met someone who did his own processing, he was a black jazz musician who used Linux. I still have a CD of his.
To explain the multitude of distros, I like to use a car analogy. There is one brand of engine, Linux, and a bunch of related models (aka versions). Then there are a couple of popular brands of chassis (Debian and Redhat) and a few other less common ones (Arch, Suse). On these chassis, companies then add the controls (gear stick, steering wheel, pedals) and interface, generally using off-the-shelf families of controls (KDE, Gnome, XFCE), and then add the body and styling and such. So all the cars have the same engine, and there is a small number of choices of chassis, and then each distro adds its own customisations on top of that engine and chassis. Ultimately, like driving a car, once you are comfortable with changing gear and steering, you can easily move from one car to another and only have to re-learn stuff like how to use the radio. Some cars will go fast; some will let you carry more cargo; there are vans and go-carts, and such. By comparison, Apple is like a Rolls Royce that makes three brands of cars, using entirely in-house engines, chassis, and so on, and with a policy that if the electric windows stop working, the solution is to buy a whole new car.
I've heard OSs compared to cars and the makers compared to dealerships before... the analogy I heard involves a busy street intersection. At the corners, there are two dealerships, and also a large paved lot full of various groups of people in tents and makeshift sheds working on tanks. One of the dealerships sells sleek, modern, sports cars. They're difficult to maintain and almost impossible to fix yourself, and overpriced... but they look fancy and slick, and owning one is a status symbol. If it breaks you can bring it to the dealership for servicing, but most likely they'll tell you to buy a new one, or a repair will cost more than a replacement. The other dealership sells family sedans. They're cheaper, but also less slick and stylish. They're technically easier to service and somewhat less tightly integrated and hermetically sealed up, but nobody who buys one tries to service it, beyond maybe a new paint job or an aftermarket horn. They break down, you take it to the dealership and pay the dealership to fix it. The guys across the street, are teams of volunteers. They make tanks. The tanks are sometimes uglier than a standard car, and can be far more confusing if you pop the hood, and the controls aren't always as simple as the dealership sedans or slick sports cars. But they're insanely safe, extremely smooth and fast on the road, and if you know what you're doing, far more accessible to tinker with. They come with extensive manuals and documentation. And the volunteers will literally give them away to you for free. If something breaks - they will give you the manual and tools to fix it if you broke it, or come and repair it for you if they broke it. These people do this because they wanted the tanks, and believe that something of this nature should be made available to anyone who wants to use it, for free and with the blueprints available openly for anyone to build their own or modify the blueprints. Multiple times a day, someone will come to this street corner with a broken or out of date sports car or sedan, or looking to buy their first car. The volunteers will tell them all about the tanks, and try to get them to take one. They'll protest they don't know how to maintain a tank. When told they don't know how to maintain a sedan or a sports car either, they'll say "But I can bring it to the dealership and pay them to fix it when it breaks." The tank people explain that they will fix it when it's their fault, will never charge you again for the newest model, and if you break it they will give you the tools and documentation to fix it, for free. The car shopper will still go into one of the dealerships and buy their overpriced product, because the tanks sound overwhelming. (On the other side of the spectrum, nowadays, a parent will pull up to the tank people in their well maintained tank, their young child in tow, seeking to pick out a good tank for their child to learn to drive in.) Once or twice every so often, someone who didn't come looking for the tanks will actually take a tank. Most of them return it in frustration and buy an overpriced new sedan.
For anyone wondering where the previous analogy came from, it's from "In the Beginning... Was the Command Line", an essay by the fiction author Neal Stephenson.
And when you go from a car that just works and us uniform to a tank and you want to set the chair and inside mirrors, suddenly something breaks and when you bring that up the volunteers say "well mine never did that, it must be a rare issue" and you have to learn everything to find out what the problem is and then the seat is too small but you are caugh by choice paralysis and just go back to the sedan bc it's easy, it works and it doesn't expect you to learn a lot about a topic that doesn't interest you BC you don't care about cars or tanks, you use them as a tool, not a character trait so having em just work is something you just want
1:47 The key thing to identify this situation (in the “top” display, or output from the “free” command) is to look at “available” RAM, not “free” RAM. The former includes cache usage, the latter doesn’t. So don’t worry if the latter is small, so long as the former still shows some useful amount.
This video made me feel much better in regard to memorizing all the commands. I took the Red Hat course through my school and was overwhelmed with all the commands. Thanks.
Yeah, that is something that took me a while to get comfortable with as well. I learned very early on to be very specific in my comments in my code, etc. because, next week, next month, or next year, I might need to do that same exact thing, but I will not remember it. If I wanted to memorize stuff, I guess, I could have been a Lawyer. LOL!
As I love it, to add the costs of the items in my cart, having the exact amount in my hand, when approaching cashier, see his smile cause it went so quickly, so I like to write cmd because stuff is quickly done. To memorize ~30cmds should be no problem for average pple.
@@DrKellieOwczarczak This is exactly me as software engineer. I always add comments and detailed docs to my projects (especially APIs). I do other stuff like flowcharts, system schemas etc. I always write test for my code and have backups for all of these. I even created myself a tool (as a project) that keeps details with tags, topics and description, which I can pull if needed. Also in most cases I prefer code readability than length or syntax, and that helps a lot. In that way I am not stressed and worried about my work.
I think it's such an important point that even the most experienced people do not memorise everything, and thank you for saying it because I think it's often a scary thing for us newer and less experienced people often worry about. It's something I learnt with Photography that even the best photogs do not turn up take one shot and it's perfect. they take thousands of pictures to find their best shot to show the world. I've been watching lots of the "into the terminal" series form Red Hat recently and it's so great seeing one of their most experience people there even makes mistakes and forgets things. It's just down to what you use on a daily basis is different from the next person. I'd love to learn more about your career, how you transitioned and what you do now. I think it would also be really great to learn what you use for your note taking. Obsidian is such a widely discussed tool these days. I've taken a look at Logseq which looks really interesting but I cannot wrap my head around how to use it effectively and always find myself comparing it to Obsidian which I probably shouldn't. Do you host things like your notes or do you keep them local on your computer?
Yeah, gonna be directing my students to you. Your calm explanations are sound. Heck, I learn things and I've been around the block a few times. Thanks for putting these out.
I ran into the inode issue a very long time ago, also on a mail server, however, it wasn't the messages themselves, it was a file system based grey listing solution I was using to help with spam. The answer then was to dedicate another drive to that solution, but to make the clusters very small, so very many inodes. As to LLVM, I avoided it on physical servers, because of the physical backup and recovery of a failing drive was much harder with LLVM in the mix. I see it as a great plus in a virtual server system.
4:56 It would be easier just to say that inodes limit the maximum _number_ of files you can create on the volume, regardless of the space they occupy. So if you run out of inodes, you cannot create any new files (or directories), you can only extend existing ones.
@@CTimmerman Depends on the config, you can specify a number of inodes per X MB's .... but using XFS they are created dynamically, so you never run out.
8:28 The point about using a different distro if you don't like the user interface - I would take this one step further and point out that for the most part you can install pretty much any desktop environment or window manager on any distribution. You don't have to even switch distros. Ubuntu is famous for it's unique spin on Gnome, but you can easily rip that out and replace it with anything you want. Or, you can install several desktop environments and switch between them as your mood dictates without any real issues.
I was just thinking about this the other night, while trying to sum up why I like Linux to a friend. The extensibility is actually insane when you think about it too much. You can do everything up to and including compiling a custom kernel. Literally every aspect of the way your machine behaves, looks, acts, and reacts is up for modification. But even before that you have desktop environment/window manager choice and extensibility, and before that you have the, by Windows standards, already insane levels of customisation that the vanilla install of something like plasma or gnome offer. People in Linuxland get mad about sysD, people in windowsLand get served ads on their desktop and have to just like it.
Hell yes. This was a great video. I wish you'd cover a lot of things you didn't know at the first. Maybe, organize them according to complexity (or career-period) versus just random topics.
When it comes to using Linux commands, I think it’s worthwhile to try to remember the general syntax of commands you use frequently, but don’t worry about remembering all the options / flags / arguments/ etc. So long as you have the name of the command, most commands are a simple “command -h” or “command -help” away from getting a list of available options and ways to use it. Yes you can do “man command”, but manual pages are confusing and terrible! 😀 They’re either way to brief with no additional info or WAY too long and convoluted, so at least see if the command has a -h or -help flag first.
@@brettsjoholm yep. Sometimes -h or -help doesn’t do much better, but often it will give a nicely condensed blurb of the command syntax and what each of the possible flags / arguments do.
2:27 Yes, it most certainly is true. Remember, userland apps like GNOME are not responsible for managing filesystem cache--that is done by the kernel. So any RAM the apps are using is actual regular application RAM, and if they are using a lot more than other comparable apps, then yes, that counts as “bloat”.
Thanks so much for your Linux Instructional Videos: When I first jumped into Linux 15 years ago, as an alternative to Windows, I just wanted to be a user ... not a script writer, analyst, or administrator. I soon found out that to be a Linux User, is to wear all of those hats. The ability to self-educate was difficult for me as the availability of quality instructional material was difficult to find ... at least at first. I do have a friend who is an experienced Linux Administrator and that has helped over the years ... especially with the catastrophes that can happen. I am glad I found your channel ... it's been a good source of quality information. I still have a long journey ahead. Thanks once again.
Yes, you are not the only one. I'm moving away from WIndows, because I think LInux is more the future. So, I'm trying to educate myself too. And, it feels like a long road too.
I use flash cards for Linux Commands & specialty terminology. I wanted to remember ... to memorize certain things because they were new to me when I first learned them, and I planned on using that info regularly. For me, repetition was the most effective memory tool for Linux.
I had what I called the big 2 when I started in 2004 - Ubuntu, Fedora and Opensuse. These were what I used either as a primary driver or my backup. Lately I now use Linux Mint. I've learned to use a Home partition to make this easier.
The main disadvantage of having 100s of distros is that it makes extremely difficult for a 3rd party vendor to supply support for their products. Imagine having to teach your support team Ubuntu, Fedora, OpenSuse, Arch and then all it's dozens of variants. It's insane. The one thing google has done with linux is take it out of open source and streamline it into ONE OS period.
Yes, that's info that I can definitely use. Now I will use git and LVM in ways that I didn't realize. I see them all of the time and thought that only admins and devs needed them. I had forgotten about inodes because I haven't had problems with those for years. But it's a good idea to know how to fix the problem in case it ever comes about. I think that it happened to me a long time ago but I thought that the distribution was buggy. 😏
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it
I think it depends on the intended use case. If you're running a database, it's a good idea since as the index grows it just occupies more RAM. If it's something that has more sporadic usage, maybe just let it use some of the swap space when it spikes.
I've been in Linux System Operations for about a dozen years but have been using Linux since 2.2. This is a well done list. I think the only thing I'd add to really introductory material like this is this: du and df. They might (probably will) report different usages. This can be frustrating when tracking down a "disk full" issue. This, really, is far more common than the exhausted inodes issue. If a file is deleted (but still actually open by a process), df will report that space as still used, while du will not. If this deleted file is very large, the difference could be huge. lsof (list open files) will help you track down the culprit in this case. Often this is caused by a log rotation that does not restart the application doing the logging.
LVM is not a backup solution but many think so because it can include a “cloud” service. Data exists only where it located. In other words if the data in any one place, like an LVM cloud space there is no backup. Unless there a considered plan to make data replicated in more than one place it can be lost. Usually data should exist in three places, the original, a live backup, and an offline backup (one offsite and dead to changes, think of a place that is kept in a safety deposit box.
Whew!! I’m so glad to know I’m not the only Linux noob using man pages. I learned a few things reading the man ls pages the other day. So much more than I realised. Great video. I’m going get that book too.
I've been using S.u.S.E. Linux since 1994. One thing I never understood is why you would want to install a new distribution just because you don't like the GUI. Not a fan of gnome? just install KDE. Problem solved. Well, if your distribution doesn't make it easy to install another GUI, then yeah, change the distro.
Great video, Jay. I'm always learning things from you. It seems that you have an endless supply of Linux knowledge to share. That's OK, though; 'cause I have a pretty healthy appetite for learning those kinds of things. Thanks for all of your videos and books. They've been a great help to me. From the Linux Mint Essentials book that got me started with Linux (back in the day); to the Mastering Ubuntu Server books of recent times... What a long "road" we've traveled since the CP/M days. ...and you've helped to make the trip a great experience for me. Thanks bunches, Jay.
Number 7: It seems, due to the path Ubuntu is going, Vanilla OS and Pop OS will derivate themselves directly from Debian, no more from Ubuntu. Things are changing.
Dont forget that you dont have to change distros if you want a specific desktop environment. More often than not you have plenty of tutorials on how to swap DE without remaking your whole system or learning the ins and outs of another distro. Distro is more of an idea of an amalgamation of what a gnu/linux system should look and work like as the foundation. I might be wrong but the distro selection usually revolves around what package manager it uses... While newcommers are usually just looking into the looks and popularity (which isnt wrong at all). I use arch (btw) because i didnt mind the rolling releases (tbh you can kind of opt out of this ) and the fact that it comes out of the box with basically just the kernel, bootloader and package manager, from there i just pick my "goey" and some apps. You like apt-get more but you wanna use a specific distro that uses something else? Noone is stopping you and its usually not that painful to swap package managers
I also want to add. Notion has been super useful for creating my own cheat sheets. I basically extract all the useful stuff from the man page for the command, and than embellish that with stuff I’ve learned elsewhere.
@@PoeLemic it’s an app for taking notes. It basically uses an abstracted form of markdown language. You just type backslash followed by what ever your looking to add (ie table, bullet, heading, url, page).
For point 7: I don't really care about the distribution, I care about the DE. I'm a desktop user, not a server admin, so the look is more important to me than the underlying stuff. But the point still stands: if KDE goes away, I'll have to have a hard look at which DEs provide the most Windows-like environment.
I personally like GNOME because I was using Windows since 2001 and i'm just bored of seeing a bar with a menu so the way of Gnome to show apps and all always makes me to use it. But I don't like some ideas of GNOME and lack of customization even there's extensions that can break the system in every update... I'm using Debian rn
The TL;DR on inodes is that a directory entry (when you see a file in your file manager, or when running ls, what you're really seeing is a directory entry) points to an inode. An inode then tells Linux where the file's data is on the disk. The number of inodes is (I think) set when you create the filesystem. Each file or directory requires an inode. So no available inodes means you can't create a new file, only allocate more free space to existing files. A nice thing about inodes, and actually something you get with NTFS (and I think it only got put there because MS wanted to have a minimal Posix subsystem in Windows NT so that they could get government contracts) is that you can have multiple directory entries pointing to the same file (hard links). This means you can efficiently create a clone of a directory hierarchy, which I tend to do before doing any destructive power renaming (e.g. applying a naming scheme to a bunch of media files) in case my Python script has an error which messes things up. With NTFS, hard links are hard to make unless you're running cygwin or git-bash.
High CPU usage might not be an issue on a server, but if your desktop is idling at, say 80%, you don't really have much overhead for other applications
Thanks very much Jay, this was very useful advice. I'm particularly interested in the LVM implementation - will definitely be checking out your content for that. 👍
You spent a bit of time talking about LVM and it's snapshot and backup capabilities, but in my experience, LVM has basically been supplanted by zfs in most instances. for pretty much any system that requires redundancy in file systems (multiple disks / some level of RAID) zfs is significantly superior when it comes to filesystem integrity over stuff like ext4 and btrfs, as well as using hardware raid controllers (which often do the wrong thing in a collision). I would suggest folks learning about linux learn about the zfs tools rather than using the old lvm ways of doing things
This was another great video! Interesting things you'd learned, practical uses for familiar tools, and an honest look at how much you need to "know" to run a server. Since 1994, the `apropos` command has been a stalwart friend!
i agree with Shawn Lewis below after hearing what you said about commands. I've always felt exactly as you do, but didn't think about this idea of certification tests demanding knowledge of commands mostly likely soon to be forgotten post exam. I totally agree that we have documentation at hand, and quickly on the Net, for all Linux commands. And the one's that we use often are the ones we naturally memorize.
Couple of handy analogies for gamer types: think of Linux like modding a game, example Skyrim. Linux is your Skyrim and the various bits you add are the mods. Some are content like a new merchant or gear (apps, package manager, games, etc) Others are basic frameworks or dependencies, like XP32 for skeletons and ragdolls. As for the I-nodes that's like inventory systems, everything has weight (kb, mb, gb) while the nodes are your inventory slots, doesn't matter if your carry capacity is 10,000kg if you're out of inventory slots. Yes I know Skyrim doesn't have limited inventory slots but it is quite crash happy if you collect too many items in a single map location, so the concept is close enough.
Thank you for the effort put into this video. I would try to minimize the redundant sentences however. For instance the section anout inodes really could have been explained in a much more direct fashion.
Cool thing about lvm, sounds like you were suggesting you can just span your volume across drives. You csn, but people often consider that unwise. What you can do that's cool though is move your data while the server is online. Add new drive to the old one as 'free blocks", then online move the old drive's blocks to the new drive. Then you remove the old drive (easy since it's empty, block-wise) and then expand the partition to use the rest of the new drive's free blocks. Doesn't play nice with btrfs because what does?
Memory from the cache can be provided to any application which needs it. So basically we should have free memory more, instead of Cache memory as cache memory is not that important. If we have cache memory more, we can either reboot the system to increase the memory and eliminate it from cache, or we can use a command -> [echo 3 > proc/sys/vm/drop_caches] Correct me, if I'm wrong 👍🏻👍🏻
I agree about not trying to remember commands you don't frequently use. As long as you can remember a search term or part of the actual command your trying to utilize then you can look it up online or in the man pages, etc.
Great video Jay, as always! I have never really understood the fixation some Linux distro reviewers have with looking at how much RAM the system is using after first boot, or what it's using during normal operations. I'm like, what is this, 1998?
I had a terrible weekend trying to fix an error related to disk. And yes it was inodes. You learn about that really fast if the main website is down...
Server CPU can be low if it's waiting for you - like my nextcloud on my raspberry pi ;) But you are right and this was a major driver in virtualization and later containerization - use the resources to the max with minimal overhead and bottlenecks. Yes, backups are only really in existence if you tested them and retest on a schedule!
I think that the CPU usage is widely variable. I manage web servers and email servers for customers these days, so unless there's a lot of activity, CPU usage is very low. But I used to deal with monitoring servers for ISPs, and high loads were common. When pulling data from thousands of devices on a regular basis, the servers were kept very busy.
I’m getting there with the Linux commands. I think my biggest hurdle right now is learning how to effectively use pipes. Also I feel like I’ve barely scratched the surface of things like grep and awk.
I really don't think there's a Linux channel on RUclips that's like this. There's some really great content on this channel. I love the way Jay explains things. He makes it very easy to understand.
I've seen in practice that using LVM in a cloud environment is not the best solution. LVM carries the inherent risk of data inconsistency, especially during restoration. For example, we had a case involving 14 hard drives. The restoration process was complicated due to issues in correctly mapping the drives. Consider using cloud provider’s native snapshot and backup tools instead. These tools are designed to handle such complexities and provide seamless volume management and restore capabilities.
There should be an additional point: Linux plays well along with the other „kids in the yard“ like Windows and MacOS, even when they are not vise-versa that friendly against Linux. You can mount and interact with the others quite well, even that they pretend they are the only one’s that exist. That even goes to the users of those systems! I can‘t count, how many discussions i had in it support like: „Thank god we are using windows and not Linux!“ - „Dude, you know that you wrote that mail on a Linux-terminal, which remote controls a windows system, which is virtualized on a Linux system …. right? And by the way, each system has his advantages and disadvantages just saying“
4:24 you could argue that the purpose of a server is not doing work but providing the possibility that work could be done if it is demanded. So even an idling service provides the service of the possibility and so is not without purpose.
nr 7. unfortunately there is not always easy to have a backup distro at hand. in many environments there are other constraints than the distro itself and the app running on it: availability of official/verified images in public clouds, your security agent may only support a limited number of distros, etc. some distros, even if supported by your apps get support last of all (debian :) in some cases you just have to go wherever the big players are going.
Man I’m feeling this right now. Bought a Dell Micro 7000 to run my plex/arr set up and it’s sitting at 2% even under “heavy” use. Could have just used the old i5 I had lying around.
And LVM can do raid , check the doc on how and what kinds of raid . A great way to learn Linux is to read the man pages , as many as you can stand . LVM has great man pages . Also name LVM name conflicts with LLVM which is Low Level Virtual Machine , so some distros are naming LVM , LVM2 now which stands for version 2 but also gives it more of it's own name , since LVM conflicts with Level Virtual Machine.
If I can remember a few letters of a command I once used, sometimes I'll drag .bash_history file into a text editor so I can use the search function. If you use a shell other than bash, the file name will be different. Although the Up-arrow will eventually get you to that past command, sometimes I used it two or three years ago. That can mean a lot of clicks.. Olden days .bash_history would get truncated to save resources, but on desktop systems the file can stay alive for years.
I have a shell script that displays a text file in the terminal. It's my personal help system. Plus I also keep a lot of notes whenever I perform tasks in the OS. Like say when I compile some software I create a text file with notes about how I built it. I can always refer to those notes when I want to do that task again.
I use the ZFS filesystem as root filesystem with native encryption on /home and direct boot from UEFI (no grub) with the Gentoo distribution. ZFS has some cool features like deduplication, which is very hardware intensive, and snapshots like described in this video on LVM. Gentoo allows me to not use systemd (which I hate), compiles the code specifically for my CPU model, is a cutting-edge rolling release, and has very good tools for compiling/upgrading the kernel. I am giving this as possible alternatives, but not as advice. Use whatever works best for you.
The number one thing I would tell people who are new to Linux: don't let *anyone*, no matter how experienced they are, tell you that there is one "right way" to do something. Whether you choose floating windows or a tiling window manager, desktop or terminal window, "beginner" distro like Mint or compile your system from the ground up with Gentoo - doesn't matter. What works for you and what you like is what you should go with. Period. Yes, Linux people will have (usually well meaning) opinions about tools and OSes you should check out. Take their advice if you want to, but don't feel obligated to do *anything* because it's your computer. You're the one who will be daily driving it, so do with it what *you* want; not what the Linux Community thinks is right.
Yes please. I would like to see more of '10 Things I wish I Knew about Linux'! For example on a sideway, different level of technologies through your career. What you met what you knew and what new technologies surprized you? Thank you for your educational videos.
Yeah, I said same thing -- rather than random stuff; he should organize it by career-period or grouping-ideas to topics (e.g., disk management or server installation), etc.
I’m a Windows user who used Linux infrequently and is very rusty. I was researching curl and I saw this complex example in Stack Overflow using Lynx, SED and Awk and instead of installing Windows versions decided to install Mint on an old laptop to apply this example I found (which had typos). I was gratified to connect to the internet and then change to the command line with ct-alt-F1 still connected to the internet to play with curl. I do remember running Redhat 2.4 which ran beautifully with sound and I then ran a distro upgrade and my anguish when the upgrade broke the sound driver. I was using a weird-ass motherboard sound chip and I ended up buying a common sound card just to run with Linux. SoundBlaster from memory. The mobo sound chip was Aureal and the company went out of business so the support had stopped. Follow-up: Creative Labs (maker of SoundBlaster) had sued Aureal over patent infringements (multiple lawsuits) and Aureal won but the legal costs bankrupted the company and Creative Labs acquired the IP at the bankruptcy hearings and Redhat withdrawing the drivers would have been the nail in the coffin for Aureal products. Sucky yes, it really is a dog eat dog world.
Thank you for the content. But I want to ask you about that stained glass light in the background - I dabble in stained glass and wonder where you sourced this lamp. I like the glass/wood mix and the overall design. I found your video because I'm thinking of moving to Linux. Best wishes from Ireland. Yesterday we had a whole day without rain......
Jay, you can't imagine how it was a relieve to me to know now, that I don't have to memorize so much in Linux. I thought I just getting too fast too old LOL. That was really a main bother to me when working in the shell. Now, I just continue to use my cheat-sheet §8-) Thanks man for being so transparent. 🤗👍
I use Manjaro and Linux Mint on my main PC, Fedora on my laptop for work, Ubuntu on my x86-tablet, Tuxedo OS on my private laptop and Proxmox, OpenSUSE, Debian and Arch on my Servers.
i was trained that on a virtual memory machine CPU utilixation should be 30% ... for responsiveness ... over 50% and the system is thrashing .. moving pages and not working and not responding to requests
Git is great for everyone - whether youre writing documents, anything from memos, a PhD thesis or novels to hardcore software projects like the Kernel, less formal code projects like data analysis/data science projects or sysadmin scripts. If you write anything in your job, it's worth learning git. Just learn to write mostly text based (like .txt, Markdown, LaTeX, code files) rather than flooding your repo with large binaries like Word docs
So true about certifications. I mean, exams in general in education. They expect you to memorise things as if you couldn't look it up with some research, even before the internet, you could just go to the library.
Could'nt agree with you more Jay about Cert exams. They should focus more on teaching you to be more "Resourceful". A much more important skill to have then a great memory. Your channel is fantasic!
12:58 writing down commands goes double for anything tricky, obscure, or troubleshooting. I’ve got a notebook in evernote of these sorts of things, ranging from my favorite parameters for rsync [actually, I should go add that one] through standard settings I want to change after a new install (or check after a major version upgrade) and aliases I want to add to my terminal config, to the magic incantation to patch around Solus Budgie’s inability to detect my screen resolution or maintain it once set. I started this notebook after a particularly frustrating time configuring openvpn on my server because I (1) hadn’t changed anything, but it stopped working and (2) hadn’t touched it in a couple years and couldn’t remember what I had done. So I wasn’t even sure what the settings _should_ be or where some of the config files needed to be or what responses indicated the vpn was running and funneling all the traffic through it. And when I went back to the resources I had originally consulted, the webpages were either gone or had been updated to give instructions for different software than I was using.
1. Unused RAM is wasted RAM.
2. Sometimes high CPU usage is a good thing.
3. Inodes need to be tracked, as well as available space. df -i command is importatnt.
4. Large number of distros is good thing, a superpower. It allows you to change up the user interface.
5. Use LVM (logical volume manager) to get more control over your storage, treating multiple things as one. You can resize file systems online.
6. You don't have to memorize Linux commands. Memorize everyday ones; it's okay to look up lesser-used ones.
7. Always have a backup distro.
8. GIT version control is not just for software developers but also for system administration.
9. LVM has a snapshot feature, useful for trying new things.
10. Especially for cloud servers, take all individual volumes when backing up data.
Good notes :)
Video sounds like an AI hallucination
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it.
tab completion and bash history were the things that I wish I'd known about from day 1.
Thanks for the notes
For years i was obsessing about have "free memory". And getting mad, that my operating system, "ate" all my memory. Untill someone, asked me why i wanted a slow inefficient computer.
Why wouldn't you use what you paid for? Only worry when there is little left or it's frequently close to full.
"Unused memory is wasted memory."
This is the most concise explanation I've heard of this concept.
The only real thing that needs to have a free excess is hard disk space. Everything else only needs to have a buffer of free. Its not about how much you have free, its about how fast that ram can do its job. That's all.
@@scheimong Is your swap file the size of your free space yet? Virtual memory matters!
@@lliamthrumble Chrome gets really slow when the OS starts to swap, so i built an OOM killer to save my SSD and other programs' data.
Around 2002, when I was fed up with the repeated crashes of my system, I went to ask three questions on the usenet forum:
- "As a non-IT person, do you think I can install and configure a GNU/Linux system?
- Am I obliged to know by heart a myriad of magic formulas?
- Will these orders still be valid in a few years?
The old wolves reassured me, and told me that if I got into the habit of reading the manuals and getting information before doing any manipulations, things would fall into place quite naturally.
Moreover, the commands that I will have to use will be the same in ten years.
They were right, and I just jotted down some "cooking recipes". 8-)
In my experience, most GNU utilities tend to keep the options the same over the course of time, just adding new ones. But, if you jump over to BSD the utilities may have the same name, but different options, and sometimes the GNU has things BSD doesn't. I encountered such issues with the "whois" commands, so I had to tweak how I automated checking for expiration dates and name servers when jumping between the two.
@@javabeanz8549
Rigor seems to be in order at GNU.
Thank you for this information about BSD that I personally don't know well.
Fast forward ten years and you have systemd with totally new commands and concepts, plus a lot of other replacements (ip address show instead of ifconfig for example).
@@illegalsmirf That's right.
Luckily we still have distros without this ugly thing.
@@illegalsmirf very true, I didn't say that we don't get new software. Like getting a list of open sockets, we now have ss -tpln to see what daemons are listening on which TCP ports. I have also moved from ipchains, to iptables years ago, and I think that we have moved on from there. But I don't manage several firewalls for ISPs anymore, so a bit out of the loop there.
Thanks for being transparent about not having to memorize commands. You just lifted a huge load off of me brother. I have definitely built my own cheat sheet. Which is very helpful. As always, I love your content. Keep up the good work.
The real skill you need is to be able to refresh your memory quickly. So you can look up what you need and understand what you are reading. Only the real world tests this - it applies to every aspect of computing and most other things as well.
And me!
I have no reason to think you should have to memorize a bunch of commands. I'm interested in why you might feel it necessary?
I now remember cut because I've used it a few times in BASH scripts, but I don't really remember awk and sed, so I just consult some scripts I've used them in. I throw those scripts up on github and consult them when I need to remember what I did before. Also tab is a real blessing trying to remember or discover commands.
After you’ve used certain commands and options a few dozen or hundred times, you do automatically memorize them. For the rest, there’s man (among other sources). This is why I’ve started writing man pages for some of my own programs.
Nitpick: grammatically, it should be “I wish I had known”, not “I wish I knew”.
I agree with you, Jay, regarding the certification process. It seems that the primary focus of these certifications is more about generating revenue rather than genuinely ensuring quality or skill. Many organizations appear to leverage these certifications as a way to monetize professional development, rather than offering a truly valuable and rigorous credential. This commercial approach can undermine the credibility of the certification and the true expertise it is supposed to represent. It's important for us to be discerning about which certifications we pursue, considering not just the cost but also the tangible benefits and recognition they provide in our field.
The biggest handicap I experienced with Linux, which is what I have used for years as my primary OS, was a an absence of well documented applications. That has greatly eased over the last 20 years or so, and generally anything you can do using a Windows (or Mac?) system can be done on a Linux system.
More or less, yes. There's always the learning curve, though depending on the person. A lot of people can pickup Linux pretty quick and others will struggle a lot.
Most things, yes. The difference is more often “how easily” or “how quickly”, rather than “at all”.
Frex, I just found out that there’s something very much like Time Machine for Linux that’s been around for a while, but it’s nowhere near as easy to set up or use.
And sometimes the deficit in UX is a dealbreaker-for some software, the UX is as much or more important than the functionality.
For me, the main software I’m not willing to give up in order to use Linux is Scrivener. Several open-source and commercial imitators now exist for macOS, MSWindows, and Linux , and some people find some of them just as good as Scrivener. I don’t. There’s other software that I’d have trouble replacing (BBEdit, InDesign, 1Password, Things, frex), but I either already know of Linux alternatives that are “good enough”, or they’re more “what I’m used to” than “what I love”, so I’m sure I could adapt to new software and new workflows.
not really true. audio production does not have a workflow on linux that isn't a huge pain in the ass. and most plugins are not supported. its unfortunately the reason why i can never use it as daily driver. also, no photoshop isnt quite a deal breaker but its close. there are plenty of examples of this where you can technically do the same thing but you are forced to using a greatly inferior (much of the time) app to do the job. the fact is that even here in 2024, no one uses linux as their daily driver because its good but because they just hate windows and apple too much to stomach using them anymore.
@@vivsavagex There is an immense difference between can't be done, and "I don't like the work," which is your position. Many computer users confuse having to learn an interface with "difficulty." There are many entries these days on audio production on Linux. Many even recommend Linux as a platform. That said, the first time I ever met someone who did his own processing, he was a black jazz musician who used Linux. I still have a CD of his.
@@vivsavagex same with graphic design tools or even cad programs
To explain the multitude of distros, I like to use a car analogy. There is one brand of engine, Linux, and a bunch of related models (aka versions). Then there are a couple of popular brands of chassis (Debian and Redhat) and a few other less common ones (Arch, Suse). On these chassis, companies then add the controls (gear stick, steering wheel, pedals) and interface, generally using off-the-shelf families of controls (KDE, Gnome, XFCE), and then add the body and styling and such. So all the cars have the same engine, and there is a small number of choices of chassis, and then each distro adds its own customisations on top of that engine and chassis. Ultimately, like driving a car, once you are comfortable with changing gear and steering, you can easily move from one car to another and only have to re-learn stuff like how to use the radio. Some cars will go fast; some will let you carry more cargo; there are vans and go-carts, and such. By comparison, Apple is like a Rolls Royce that makes three brands of cars, using entirely in-house engines, chassis, and so on, and with a policy that if the electric windows stop working, the solution is to buy a whole new car.
I've heard OSs compared to cars and the makers compared to dealerships before... the analogy I heard involves a busy street intersection. At the corners, there are two dealerships, and also a large paved lot full of various groups of people in tents and makeshift sheds working on tanks. One of the dealerships sells sleek, modern, sports cars. They're difficult to maintain and almost impossible to fix yourself, and overpriced... but they look fancy and slick, and owning one is a status symbol. If it breaks you can bring it to the dealership for servicing, but most likely they'll tell you to buy a new one, or a repair will cost more than a replacement. The other dealership sells family sedans. They're cheaper, but also less slick and stylish. They're technically easier to service and somewhat less tightly integrated and hermetically sealed up, but nobody who buys one tries to service it, beyond maybe a new paint job or an aftermarket horn. They break down, you take it to the dealership and pay the dealership to fix it.
The guys across the street, are teams of volunteers. They make tanks. The tanks are sometimes uglier than a standard car, and can be far more confusing if you pop the hood, and the controls aren't always as simple as the dealership sedans or slick sports cars. But they're insanely safe, extremely smooth and fast on the road, and if you know what you're doing, far more accessible to tinker with. They come with extensive manuals and documentation. And the volunteers will literally give them away to you for free. If something breaks - they will give you the manual and tools to fix it if you broke it, or come and repair it for you if they broke it. These people do this because they wanted the tanks, and believe that something of this nature should be made available to anyone who wants to use it, for free and with the blueprints available openly for anyone to build their own or modify the blueprints.
Multiple times a day, someone will come to this street corner with a broken or out of date sports car or sedan, or looking to buy their first car. The volunteers will tell them all about the tanks, and try to get them to take one. They'll protest they don't know how to maintain a tank. When told they don't know how to maintain a sedan or a sports car either, they'll say "But I can bring it to the dealership and pay them to fix it when it breaks." The tank people explain that they will fix it when it's their fault, will never charge you again for the newest model, and if you break it they will give you the tools and documentation to fix it, for free. The car shopper will still go into one of the dealerships and buy their overpriced product, because the tanks sound overwhelming. (On the other side of the spectrum, nowadays, a parent will pull up to the tank people in their well maintained tank, their young child in tow, seeking to pick out a good tank for their child to learn to drive in.) Once or twice every so often, someone who didn't come looking for the tanks will actually take a tank. Most of them return it in frustration and buy an overpriced new sedan.
For anyone wondering where the previous analogy came from, it's from "In the Beginning... Was the Command Line", an essay by the fiction author Neal Stephenson.
And when you go from a car that just works and us uniform to a tank and you want to set the chair and inside mirrors, suddenly something breaks and when you bring that up the volunteers say "well mine never did that, it must be a rare issue" and you have to learn everything to find out what the problem is and then the seat is too small but you are caugh by choice paralysis and just go back to the sedan bc it's easy, it works and it doesn't expect you to learn a lot about a topic that doesn't interest you BC you don't care about cars or tanks, you use them as a tool, not a character trait so having em just work is something you just want
1:47 The key thing to identify this situation (in the “top” display, or output from the “free” command) is to look at “available” RAM, not “free” RAM. The former includes cache usage, the latter doesn’t. So don’t worry if the latter is small, so long as the former still shows some useful amount.
This video made me feel much better in regard to memorizing all the commands. I took the Red Hat course through my school and was overwhelmed with all the commands. Thanks.
Yeah, that is something that took me a while to get comfortable with as well. I learned very early on to be very specific in my comments in my code, etc. because, next week, next month, or next year, I might need to do that same exact thing, but I will not remember it. If I wanted to memorize stuff, I guess, I could have been a Lawyer. LOL!
As I love it, to add the costs of the items in my cart, having the exact amount in my hand, when approaching cashier, see his smile cause it went so quickly, so I like to write cmd because stuff is quickly done. To memorize ~30cmds should be no problem for average pple.
@@DrKellieOwczarczak This is exactly me as software engineer. I always add comments and detailed docs to my projects (especially APIs). I do other stuff like flowcharts, system schemas etc. I always write test for my code and have backups for all of these.
I even created myself a tool (as a project) that keeps details with tags, topics and description, which I can pull if needed.
Also in most cases I prefer code readability than length or syntax, and that helps a lot.
In that way I am not stressed and worried about my work.
Jay, I think you’re a great teacher and I learned so much from you. Thank you for being on the channel.
This is one of the worst channels ever. He rambles too much without saying anything of substance. There are far better channels out there
I think it's such an important point that even the most experienced people do not memorise everything, and thank you for saying it because I think it's often a scary thing for us newer and less experienced people often worry about. It's something I learnt with Photography that even the best photogs do not turn up take one shot and it's perfect. they take thousands of pictures to find their best shot to show the world. I've been watching lots of the "into the terminal" series form Red Hat recently and it's so great seeing one of their most experience people there even makes mistakes and forgets things. It's just down to what you use on a daily basis is different from the next person.
I'd love to learn more about your career, how you transitioned and what you do now. I think it would also be really great to learn what you use for your note taking. Obsidian is such a widely discussed tool these days. I've taken a look at Logseq which looks really interesting but I cannot wrap my head around how to use it effectively and always find myself comparing it to Obsidian which I probably shouldn't.
Do you host things like your notes or do you keep them local on your computer?
Yeah, gonna be directing my students to you. Your calm explanations are sound. Heck, I learn things and I've been around the block a few times. Thanks for putting these out.
Would have been good to know this was about your career before I clicked on it. I am looking at personal tip for home use.
I ran into the inode issue a very long time ago, also on a mail server, however, it wasn't the messages themselves, it was a file system based grey listing solution I was using to help with spam. The answer then was to dedicate another drive to that solution, but to make the clusters very small, so very many inodes.
As to LLVM, I avoided it on physical servers, because of the physical backup and recovery of a failing drive was much harder with LLVM in the mix. I see it as a great plus in a virtual server system.
4:56 It would be easier just to say that inodes limit the maximum _number_ of files you can create on the volume, regardless of the space they occupy. So if you run out of inodes, you cannot create any new files (or directories), you can only extend existing ones.
How soon would a 1TB drive with Etx4 run out of inodes?
@@CTimmerman Depends on the config, you can specify a number of inodes per X MB's .... but using XFS they are created dynamically, so you never run out.
@@openevents Is XFS better than ZFS? A redundant storage pool of random old drives sounds useful.
I would love a series of videos like these. Tips for current Sys Admins that are still learning.
8:28 The point about using a different distro if you don't like the user interface - I would take this one step further and point out that for the most part you can install pretty much any desktop environment or window manager on any distribution. You don't have to even switch distros. Ubuntu is famous for it's unique spin on Gnome, but you can easily rip that out and replace it with anything you want. Or, you can install several desktop environments and switch between them as your mood dictates without any real issues.
I was just thinking about this the other night, while trying to sum up why I like Linux to a friend. The extensibility is actually insane when you think about it too much. You can do everything up to and including compiling a custom kernel. Literally every aspect of the way your machine behaves, looks, acts, and reacts is up for modification. But even before that you have desktop environment/window manager choice and extensibility, and before that you have the, by Windows standards, already insane levels of customisation that the vanilla install of something like plasma or gnome offer.
People in Linuxland get mad about sysD, people in windowsLand get served ads on their desktop and have to just like it.
Hell yes. This was a great video. I wish you'd cover a lot of things you didn't know at the first. Maybe, organize them according to complexity (or career-period) versus just random topics.
When it comes to using Linux commands, I think it’s worthwhile to try to remember the general syntax of commands you use frequently, but don’t worry about remembering all the options / flags / arguments/ etc. So long as you have the name of the command, most commands are a simple “command -h” or “command -help” away from getting a list of available options and ways to use it. Yes you can do “man command”, but manual pages are confusing and terrible! 😀 They’re either way to brief with no additional info or WAY too long and convoluted, so at least see if the command has a -h or -help flag first.
Ohhh. That's awesome. I never knew that... Because yes, man pages are useless for me most of the time.
@@brettsjoholm yep. Sometimes -h or -help doesn’t do much better, but often it will give a nicely condensed blurb of the command syntax and what each of the possible flags / arguments do.
Try to use "tldr" command - it is VERY helpful for me, I actually use it every day to see stuff about commands in very quickly way lol
Make notes of your most used commands. As you said, navigating the man pages is just terrible.
tealdeer is also handy for common programs. It's basically a mini man page with the commands and options you're most likely to want to use.
2:27 Yes, it most certainly is true. Remember, userland apps like GNOME are not responsible for managing filesystem cache--that is done by the kernel. So any RAM the apps are using is actual regular application RAM, and if they are using a lot more than other comparable apps, then yes, that counts as “bloat”.
regarding commands, installing 'tldr' is a massive help
Thanks so much for your Linux Instructional Videos: When I first jumped into Linux 15 years ago, as an alternative to Windows, I just wanted to be a user ... not a script writer, analyst, or administrator. I soon found out that to be a Linux User, is to wear all of those hats. The ability to self-educate was difficult for me as the availability of quality instructional material was difficult to find ... at least at first. I do have a friend who is an experienced Linux Administrator and that has helped over the years ... especially with the catastrophes that can happen. I am glad I found your channel ... it's been a good source of quality information. I still have a long journey ahead. Thanks once again.
Yes, you are not the only one. I'm moving away from WIndows, because I think LInux is more the future. So, I'm trying to educate myself too. And, it feels like a long road too.
I use flash cards for Linux Commands & specialty terminology. I wanted to remember ... to memorize certain things because they were new to me when I first learned them, and I planned on using that info regularly. For me, repetition was the most effective memory tool for Linux.
Truth in its purest sense
I had what I called the big 2 when I started in 2004 - Ubuntu, Fedora and Opensuse. These were what I used either as a primary driver or my backup. Lately I now use Linux Mint. I've learned to use a Home partition to make this easier.
Did you mean to say "Debian" when you said "Ubuntu" ? I ask because the latter didn't exist yet.
@@szr8 TRUE...✌👍
Ubuntu came out in October 2004.
@@szr8Ubuntu came out in October 2004 and I started to use it in early 2005
6:51 the large amount of Linux distros is what made me shift to FreeBSD as my main server OS, the insane number of choices just paralysed me
The main disadvantage of having 100s of distros is that it makes extremely difficult for a 3rd party vendor to supply support for their products. Imagine having to teach your support team Ubuntu, Fedora, OpenSuse, Arch and then all it's dozens of variants. It's insane. The one thing google has done with linux is take it out of open source and streamline it into ONE OS period.
Yes, that's info that I can definitely use. Now I will use git and LVM in ways that I didn't realize. I see them all of the time and thought that only admins and devs needed them. I had forgotten about inodes because I haven't had problems with those for years. But it's a good idea to know how to fix the problem in case it ever comes about. I think that it happened to me a long time ago but I thought that the distribution was buggy. 😏
Point 1 applies only if you have high enough ram to begin with. You'd need a decently high amount of ram if your workflow involves sometimes using lightweight apps and occasionally running multiple heavy programs at the same time. No unused memory is really wasted. You're just being prepared for when you do need it
I think it depends on the intended use case. If you're running a database, it's a good idea since as the index grows it just occupies more RAM. If it's something that has more sporadic usage, maybe just let it use some of the swap space when it spikes.
@@TheMet4lGod thats not enough. also swap space is for unused apps. not good for multitasking
@@RenderingUser the swap is the equivalent of the Windows page file
@@TheMet4lGod it saves to disk. Which is slow
@@RenderingUser yeah. I am aware. I feel like you missed the point of what I'm saying.
That book plug was so smooth :D
I've been in Linux System Operations for about a dozen years but have been using Linux since 2.2. This is a well done list. I think the only thing I'd add to really introductory material like this is this: du and df. They might (probably will) report different usages. This can be frustrating when tracking down a "disk full" issue. This, really, is far more common than the exhausted inodes issue.
If a file is deleted (but still actually open by a process), df will report that space as still used, while du will not. If this deleted file is very large, the difference could be huge.
lsof (list open files) will help you track down the culprit in this case. Often this is caused by a log rotation that does not restart the application doing the logging.
LVM is not a backup solution but many think so because it can include a “cloud” service. Data exists only where it located. In other words if the data in any one place, like an LVM cloud space there is no backup. Unless there a considered plan to make data replicated in more than one place it can be lost. Usually data should exist in three places, the original, a live backup, and an offline backup (one offsite and dead to changes, think of a place that is kept in a safety deposit box.
Whew!! I’m so glad to know I’m not the only Linux noob using man pages. I learned a few things reading the man ls pages the other day. So much more than I realised. Great video. I’m going get that book too.
I've been using S.u.S.E. Linux since 1994. One thing I never understood is why you would want to install a new distribution just because you don't like the GUI. Not a fan of gnome? just install KDE. Problem solved. Well, if your distribution doesn't make it easy to install another GUI, then yeah, change the distro.
Great video, Jay. I'm always learning things from you. It seems that you have an endless supply of Linux knowledge to share. That's OK, though; 'cause I have a pretty healthy appetite for learning those kinds of things.
Thanks for all of your videos and books. They've been a great help to me. From the Linux Mint Essentials book that got me started with Linux (back in the day); to the Mastering Ubuntu Server books of recent times...
What a long "road" we've traveled since the CP/M days.
...and you've helped to make the trip a great experience for me.
Thanks bunches, Jay.
Number 7: It seems, due to the path Ubuntu is going, Vanilla OS and Pop OS will derivate themselves directly from Debian, no more from Ubuntu. Things are changing.
Far more informative than I thought this would be. Great tips.
Dont forget that you dont have to change distros if you want a specific desktop environment. More often than not you have plenty of tutorials on how to swap DE without remaking your whole system or learning the ins and outs of another distro.
Distro is more of an idea of an amalgamation of what a gnu/linux system should look and work like as the foundation. I might be wrong but the distro selection usually revolves around what package manager it uses... While newcommers are usually just looking into the looks and popularity (which isnt wrong at all).
I use arch (btw) because i didnt mind the rolling releases (tbh you can kind of opt out of this ) and the fact that it comes out of the box with basically just the kernel, bootloader and package manager, from there i just pick my "goey" and some apps. You like apt-get more but you wanna use a specific distro that uses something else? Noone is stopping you and its usually not that painful to swap package managers
I also want to add. Notion has been super useful for creating my own cheat sheets. I basically extract all the useful stuff from the man page for the command, and than embellish that with stuff I’ve learned elsewhere.
What is Notion?
@@PoeLemic it’s an app for taking notes. It basically uses an abstracted form of markdown language. You just type backslash followed by what ever your looking to add (ie table, bullet, heading, url, page).
That is an amazing tip! Thank you, I use Notion a lot but never imagined this
For point 7: I don't really care about the distribution, I care about the DE. I'm a desktop user, not a server admin, so the look is more important to me than the underlying stuff.
But the point still stands: if KDE goes away, I'll have to have a hard look at which DEs provide the most Windows-like environment.
yeah, options are good, the facts built my own personal Desktop experience built around sway says a lot
LXQT :) its very dull & boring. modern, and boring. classic desktop paradigm. its fantastic.
I personally like GNOME because I was using Windows since 2001 and i'm just bored of seeing a bar with a menu so the way of Gnome to show apps and all always makes me to use it.
But I don't like some ideas of GNOME and lack of customization even there's extensions that can break the system in every update... I'm using Debian rn
Very,very useful for beginners! Respect for video! Thanks 🙏👍
My pleasure 😊
The TL;DR on inodes is that a directory entry (when you see a file in your file manager, or when running ls, what you're really seeing is a directory entry) points to an inode. An inode then tells Linux where the file's data is on the disk. The number of inodes is (I think) set when you create the filesystem. Each file or directory requires an inode. So no available inodes means you can't create a new file, only allocate more free space to existing files.
A nice thing about inodes, and actually something you get with NTFS (and I think it only got put there because MS wanted to have a minimal Posix subsystem in Windows NT so that they could get government contracts) is that you can have multiple directory entries pointing to the same file (hard links). This means you can efficiently create a clone of a directory hierarchy, which I tend to do before doing any destructive power renaming (e.g. applying a naming scheme to a bunch of media files) in case my Python script has an error which messes things up. With NTFS, hard links are hard to make unless you're running cygwin or git-bash.
High CPU usage might not be an issue on a server, but if your desktop is idling at, say 80%, you don't really have much overhead for other applications
Thanks very much Jay, this was very useful advice. I'm particularly interested in the LVM implementation - will definitely be checking out your content for that. 👍
You spent a bit of time talking about LVM and it's snapshot and backup capabilities, but in my experience, LVM has basically been supplanted by zfs in most instances. for pretty much any system that requires redundancy in file systems (multiple disks / some level of RAID) zfs is significantly superior when it comes to filesystem integrity over stuff like ext4 and btrfs, as well as using hardware raid controllers (which often do the wrong thing in a collision). I would suggest folks learning about linux learn about the zfs tools rather than using the old lvm ways of doing things
This was another great video! Interesting things you'd learned, practical uses for familiar tools, and an honest look at how much you need to "know" to run a server. Since 1994, the `apropos` command has been a stalwart friend!
I just do “man -k”.
@@lawrencedoliveiro9104 That's pretty apropos.
Not apt?
i agree with Shawn Lewis below after hearing what you said about commands. I've always felt exactly as you do, but didn't think about this idea of certification tests demanding knowledge of commands mostly likely soon to be forgotten post exam. I totally agree that we have documentation at hand, and quickly on the Net, for all Linux commands. And the one's that we use often are the ones we naturally memorize.
Couple of handy analogies for gamer types: think of Linux like modding a game, example Skyrim.
Linux is your Skyrim and the various bits you add are the mods.
Some are content like a new merchant or gear (apps, package manager, games, etc)
Others are basic frameworks or dependencies, like XP32 for skeletons and ragdolls.
As for the I-nodes that's like inventory systems, everything has weight (kb, mb, gb) while the nodes are your inventory slots, doesn't matter if your carry capacity is 10,000kg if you're out of inventory slots.
Yes I know Skyrim doesn't have limited inventory slots but it is quite crash happy if you collect too many items in a single map location, so the concept is close enough.
Thank you for the effort put into this video. I would try to minimize the redundant sentences however. For instance the section anout inodes really could have been explained in a much more direct fashion.
Cool thing about lvm, sounds like you were suggesting you can just span your volume across drives. You csn, but people often consider that unwise. What you can do that's cool though is move your data while the server is online. Add new drive to the old one as 'free blocks", then online move the old drive's blocks to the new drive. Then you remove the old drive (easy since it's empty, block-wise) and then expand the partition to use the rest of the new drive's free blocks. Doesn't play nice with btrfs because what does?
I skipped a lot of those issues by jumping straight in to a Debian net install 😅
I love have every Linux video has cmatrix running in the background
Memory from the cache can be provided to any application which needs it.
So basically we should have free memory more, instead of Cache memory as cache memory is not that important. If we have cache memory more, we can either reboot the system to increase the memory and eliminate it from cache, or we can use a command -> [echo 3 > proc/sys/vm/drop_caches]
Correct me, if I'm wrong 👍🏻👍🏻
I agree about not trying to remember commands you don't frequently use. As long as you can remember a search term or part of the actual command your trying to utilize then you can look it up online or in the man pages, etc.
Great video Jay, as always! I have never really understood the fixation some Linux distro reviewers have with looking at how much RAM the system is using after first boot, or what it's using during normal operations. I'm like, what is this, 1998?
I had a terrible weekend trying to fix an error related to disk. And yes it was inodes. You learn about that really fast if the main website is down...
Awesome content! Thanks so much for always delivering!
Server CPU can be low if it's waiting for you - like my nextcloud on my raspberry pi ;)
But you are right and this was a major driver in virtualization and later containerization - use the resources to the max with minimal overhead and bottlenecks.
Yes, backups are only really in existence if you tested them and retest on a schedule!
I think that the CPU usage is widely variable. I manage web servers and email servers for customers these days, so unless there's a lot of activity, CPU usage is very low.
But I used to deal with monitoring servers for ISPs, and high loads were common. When pulling data from thousands of devices on a regular basis, the servers were kept very busy.
Love the videos thanks!
Thank you. I have picked up some more sys admin tips and for me today was the 'tippet' on disk space. I appreciate your work muchly.
Hi, just found your channel. You got one more subscriber and got a like in this video.
I’m getting there with the Linux commands. I think my biggest hurdle right now is learning how to effectively use pipes. Also I feel like I’ve barely scratched the surface of things like grep and awk.
I had a block years ago with pipes and grep. The only way I got past that block was through working with them and other commands to redirect data.
grep is useful, but I never bothered with awk. I learned Perl first, and that does everything that awk can do, just as concisely, and more.
The Linux Command line by William Shotts is a great free online resource.
Thanks, that was helpful, I'd be interested in part II.
So helpful! Thanks so much, and I’d like to mention your light and color, it’s awesome 👏
I really don't think there's a Linux channel on RUclips that's like this. There's some really great content on this channel. I love the way Jay explains things. He makes it very easy to understand.
I've seen in practice that using LVM in a cloud environment is not the best solution. LVM carries the inherent risk of data inconsistency, especially during restoration. For example, we had a case involving 14 hard drives. The restoration process was complicated due to issues in correctly mapping the drives.
Consider using cloud provider’s native snapshot and backup tools instead. These tools are designed to handle such complexities and provide seamless volume management and restore capabilities.
Clever plug for the book. :-)
We are lucky to have such an amazing teacher. Great channel, really
Your channel is brilliant! Thank you for sharing your knowledges around the world! Hello from Russia!
Never mentioned exactly what inodes are, nor why they dictate if I can save something or not....
There should be an additional point: Linux plays well along with the other „kids in the yard“ like Windows and MacOS, even when they are not vise-versa that friendly against Linux. You can mount and interact with the others quite well, even that they pretend they are the only one’s that exist.
That even goes to the users of those systems! I can‘t count, how many discussions i had in it support like: „Thank god we are using windows and not Linux!“ - „Dude, you know that you wrote that mail on a Linux-terminal, which remote controls a windows system, which is virtualized on a Linux system …. right? And by the way, each system has his advantages and disadvantages just saying“
4:24 you could argue that the purpose of a server is not doing work but providing the possibility that work could be done if it is demanded. So even an idling service provides the service of the possibility and so is not without purpose.
the Git idea for version controlling config files is genius, so obvious now you mention it.
nr 7. unfortunately there is not always easy to have a backup distro at hand. in many environments there are other constraints than the distro itself and the app running on it: availability of official/verified images in public clouds, your security agent may only support a limited number of distros, etc. some distros, even if supported by your apps get support last of all (debian :)
in some cases you just have to go wherever the big players are going.
As a software engineer, I agree that any resources not in use are being wasted...
In general a very informative list, thank you :)
When your Server's CPU is almost always idle, it could also mean that it is way oversized and you are paying way too much
Man I’m feeling this right now. Bought a Dell Micro 7000 to run my plex/arr set up and it’s sitting at 2% even under “heavy” use. Could have just used the old i5 I had lying around.
If your CPU is running at 70% consistently that can be good or bad. If no sods using it, you’re in trouble.
And LVM can do raid , check the doc on how and what kinds of raid . A great way to learn Linux is to read the man pages , as many as you can stand . LVM has great man pages . Also name LVM name conflicts with LLVM which is Low Level Virtual Machine , so some distros are naming LVM , LVM2 now which stands for version 2 but also gives it more of it's own name , since LVM conflicts with Level Virtual Machine.
There is no need for lvm on virtual servers. 😉 just enlarge existing virtual disk, use "growpart" to expand partition and then grow file system.
If I can remember a few letters of a command I once used, sometimes I'll drag .bash_history file into a text editor so I can use the search function. If you use a shell other than bash, the file name will be different. Although the Up-arrow will eventually get you to that past command, sometimes I used it two or three years ago. That can mean a lot of clicks.. Olden days .bash_history would get truncated to save resources, but on desktop systems the file can stay alive for years.
you can search a file on the command line. Just use grep on it.
Great talk!
Glad I listened.
Thank you for sharing
your learning curve.
I have a shell script that displays a text file in the terminal. It's my personal help system. Plus I also keep a lot of notes whenever I perform tasks in the OS. Like say when I compile some software I create a text file with notes about how I built it. I can always refer to those notes when I want to do that task again.
I Remember back in the amiga era there was a dynamic ramdisk in workbench
Thank you, learning so much , can't wait for part 2.
I use the ZFS filesystem as root filesystem with native encryption on /home and direct boot from UEFI (no grub) with the Gentoo distribution. ZFS has some cool features like deduplication, which is very hardware intensive, and snapshots like described in this video on LVM. Gentoo allows me to not use systemd (which I hate), compiles the code specifically for my CPU model, is a cutting-edge rolling release, and has very good tools for compiling/upgrading the kernel. I am giving this as possible alternatives, but not as advice. Use whatever works best for you.
The number one thing I would tell people who are new to Linux: don't let *anyone*, no matter how experienced they are, tell you that there is one "right way" to do something. Whether you choose floating windows or a tiling window manager, desktop or terminal window, "beginner" distro like Mint or compile your system from the ground up with Gentoo - doesn't matter. What works for you and what you like is what you should go with. Period.
Yes, Linux people will have (usually well meaning) opinions about tools and OSes you should check out. Take their advice if you want to, but don't feel obligated to do *anything* because it's your computer. You're the one who will be daily driving it, so do with it what *you* want; not what the Linux Community thinks is right.
Great content as usual Jay. I do miss the old intro though...
Yes please. I would like to see more of '10 Things I wish I Knew about Linux'! For example on a sideway, different level of technologies through your career. What you met what you knew and what new technologies surprized you? Thank you for your educational videos.
Yeah, I said same thing -- rather than random stuff; he should organize it by career-period or grouping-ideas to topics (e.g., disk management or server installation), etc.
Every video from you worth tons thank you for sharing your knowledge and experience, I have learn so much from you!
I’m a Windows user who used Linux infrequently and is very rusty. I was researching curl and I saw this complex example in Stack Overflow using Lynx, SED and Awk and instead of installing Windows versions decided to install Mint on an old laptop to apply this example I found (which had typos).
I was gratified to connect to the internet and then change to the command line with ct-alt-F1 still connected to the internet to play with curl.
I do remember running Redhat 2.4 which ran beautifully with sound and I then ran a distro upgrade and my anguish when the upgrade broke the sound driver. I was using a weird-ass motherboard sound chip and I ended up buying a common sound card just to run with Linux. SoundBlaster from memory.
The mobo sound chip was Aureal and the company went out of business so the support had stopped.
Follow-up: Creative Labs (maker of SoundBlaster) had sued Aureal over patent infringements (multiple lawsuits) and Aureal won but the legal costs bankrupted the company and Creative Labs acquired the IP at the bankruptcy hearings and Redhat withdrawing the drivers would have been the nail in the coffin for Aureal products.
Sucky yes, it really is a dog eat dog world.
Thank you for the content. But I want to ask you about that stained glass light in the background - I dabble in stained glass and wonder where you sourced this lamp. I like the glass/wood mix and the overall design.
I found your video because I'm thinking of moving to Linux.
Best wishes from Ireland. Yesterday we had a whole day without rain......
I'm pretty sure that the inode issue only affects ext volumes ( ext4.....3? ). Apparently its not an issue on other file systems.
Jay, you can't imagine how it was a relieve to me to know now, that I don't have to memorize so much in Linux.
I thought I just getting too fast too old LOL.
That was really a main bother to me when working in the shell. Now, I just continue to use my cheat-sheet §8-)
Thanks man for being so transparent. 🤗👍
Learn one thing how to navigate and search for specific commands or use cases by using man.
I use Manjaro and Linux Mint on my main PC, Fedora on my laptop for work, Ubuntu on my x86-tablet, Tuxedo OS on my private laptop and Proxmox, OpenSUSE, Debian and Arch on my Servers.
i was trained that on a virtual memory machine CPU utilixation should be 30% ... for responsiveness ... over 50% and the system is thrashing .. moving pages and not working and not responding to requests
Git is great for everyone - whether youre writing documents, anything from memos, a PhD thesis or novels to hardcore software projects like the Kernel, less formal code projects like data analysis/data science projects or sysadmin scripts.
If you write anything in your job, it's worth learning git. Just learn to write mostly text based (like .txt, Markdown, LaTeX, code files) rather than flooding your repo with large binaries like Word docs
So true about certifications. I mean, exams in general in education. They expect you to memorise things as if you couldn't look it up with some research, even before the internet, you could just go to the library.
Thanks, Jay for the advice!! the inodes was relatable
Could'nt agree with you more Jay about Cert exams. They should focus more on teaching you to be more "Resourceful". A much more important skill to have then a great memory. Your channel is fantasic!
I am interested in Linux and I just subscribed to your channel. Now my question. Where do I get that absolutely sick screen saver?
it is a terminal-based application called 'cmatrix'
12:58 writing down commands goes double for anything tricky, obscure, or troubleshooting.
I’ve got a notebook in evernote of these sorts of things, ranging from my favorite parameters for rsync [actually, I should go add that one] through standard settings I want to change after a new install (or check after a major version upgrade) and aliases I want to add to my terminal config, to the magic incantation to patch around Solus Budgie’s inability to detect my screen resolution or maintain it once set.
I started this notebook after a particularly frustrating time configuring openvpn on my server because I (1) hadn’t changed anything, but it stopped working and (2) hadn’t touched it in a couple years and couldn’t remember what I had done. So I wasn’t even sure what the settings _should_ be or where some of the config files needed to be or what responses indicated the vpn was running and funneling all the traffic through it. And when I went back to the resources I had originally consulted, the webpages were either gone or had been updated to give instructions for different software than I was using.
Thanks a lot Jay, learnt some newer things from this tutorial.
You got a little bit of a crackle @14:04. Just thought you might wanna know that lol.
i did not understand first argument. how is it in favor of gnome? if it constantly eats 1gb of ram - it is bloated