The Untold Unix Story
HTML-код
- Опубликовано: 6 июн 2024
- This is an "untold" story in UNIX which had a major effect on the current environment of UNIX-like operating systems such as the BSD's, Minux and Linux. I wonder where we would be today without the seminal work of people like John Lions who took the time to go through the UNIX System 6 source code and document what each piece of the puzzle did, so it could be taught to those of us interested in learning more about the workings of the system. The paper John wrote went viral and even was published as a book. Force to remove it from publication in 1979 it returned finally in 1996 and is still available today. "You are not expected to understand this" became the hallmark quote of this early work.
Amazon Link to John Lions "Commentary on UNIX" Book: amzn.to/3NksYIa
00:00 - Intro
00:48 - Quote
01:40 - OS Approach 1
02:40 - OS Approach 2
03:28 - OS Approach 3
04:29 - UNIX is released
05:52 - 1975 Unix System 6 Release
07:21 - Commentary on UNIX
08:24 - John Lions Quote
09:34 - The Book
10:21 - System 7
12:13 - Minix
12:57 - Linux
13:29 - Re-Release of John Lions Book
15:11 - NextGen OS
16:03 - Tagline
20:21 - Outro
Support me on Patreon: / djware
Follow me:
Twitter @djware55
Facebook: / don.ware.7758
Gitlab: gitlab.com/djware27
#unix #johnlions #commentary 
Наука
John Lions was a professor at the University of New South Wales, in Sydney Australia - not South Wales in the UK. I completed his Operating Systems course in 1981 as part of my Computer Science degree and both the Unix V6 source code as well as John Lions' commentary were required textbooks. I still have my copies, with the source code signed by Dennis Ritchie and the commentary signed by John. Attending his funeral a few years later was a sad day for everyone who knew him as well as the community at large.
The source code is a great teaching resource, including a privilege escalation vulnerability in a system call, discovered (and exploited) by someone in my honours class at uni.
Interesting.
I remember the day - Munro coming back from the lecture excited after JL asked "why isn't this unsigned?". Then proceeding to devise a brilliant hack - and needing to find an account on the "target" machine with an even UID...
I have that book and it’s is amazing, required course book for unsw, mine is yellowish now.
Thank you for this video. I have been working on Unix flavoured operating systems for 30+ years, starting with SunOS/Solaris, then getting into Minix and finally Linux, as well as dipping my toes into a number of flavours of Unix along the way. I never had the money, or the grades (due to dyslexia & dyscalculia) to get into University, but I bought Tannenbaum's "Operating Systems: Design and Implementation" when it came out and started fiddling with Minix. This is where most of my initial knowledge of Unix and Linux came from. When Linus came out with his kernel, as soon as there was a boot and root disk available, I downloaded them and would go around at work when people were not using their 386 systems and boot up Linux to poke around with it. Back then, access to Usenet was one of my primary sources of information, and it's importance in helping to build our current computer world, I think, has been forgotten.
Greay story and yeah Usenet was the place to get all kind of information, do you remember that guy who kept posting in the Unix forums on Usenet to blow up the moon? LOL, those were the days...
@@CyberGizmo Don't remember any threats of a moon shatterin kaboom. LOL indeed! I just went over to the Google Usenet archive, and checked when my first post on comp.os.linux was...May 26, 1992!
In non-unix land I also read the newsgroup devoted to one of my favourite musians, Tori Amos. Through it I came across information that allowed me to meet her during her tour!
Wow!. At 73, I lived through all this at Western Electric and Bell Labs. What a trip down memory lane! Guys like Ritchie and others involved in the development of Unix were legends in the company. I personally learned a tremendous amount by studying the Unix OS code. The most valuable lesson: How not to write comments in the code - as the very sparse and cryptic comments taught so well! Thank you for this most excellent presentation.
Then you'll love these interviews with Brian Kernighan ruclips.net/video/QFK6RG47bww/видео.html
LARRY! On May 18th, 1974 you sat on my lunch at Bell Labs and as a result my cheese and bologna sandwich was _irreversibly_ damaged (you were wearing those heavy slacks and they really did a number on the bread). I have been waiting $((2022-1974)) years to track you down for reimbursement!
... okay, might not have happened.. at all.. but imagine if. it. were. true.
Mister, you are one of the best content creator I’ve seen. You are able to retain audience attention with just story telling, presented in a gracious and coloquial manner, no matter how arid the subject is. Kudos, sir.
Thank you, Corvo
Yes, I agree with you totally, but I highly doubt that DJ Ware finds any of these subjects to be arid. I get the strong feeling that he lives and breathes computers and everything about them. Anyway, I also found this talk to be fascinating.
Hear, hear
DJ ware is the best content creator when it comes to UNIX and unix like systems he know a lot and teaches very well. Keep up the good work.
A detail about the "you are not expected to understand" code is that originally C used "x =+ 8" as the shortcut for "x = x + 8", but that proved to be too ambiguous as "x = +8" meant assign positive 8 to x. So they switched to "x += 8" to eliminate any possible confusion. The code shown is "rp->p_flag =& ~SSWAP;" would be written as "rp->p_flag &= ~SSWAP;" in more modern C, meaning: clear the SSWAP flag pointed to by rp.
Thanks Jecel
Yes, but you are not expected to understand that. 🥸
@@flexairz I for one, still don't understand it even with clarification on the syntax change. I understand what is going on but do not understand why.
@@AkoyTamad Take it from right to left one step at at time.
What is SSWAP? Looking left we see it is applied to a variable with 'flag' in the name, so it is presumably a constant (a #defined value) representing a single bit.. it will look like an integer constant that is a power of two, but its real intent is to control a bit in a particular position.
What is ~SSWAP? Take every bit in SSWAP and reverse it, the result is ~SSWAP. So if SSWAP is an 8-bit integer constant with the value of 4, it would be binary '00000100' and ~SSWAP would be '11111011'.
What is =& (or &= in modern C)? It means take each bit from the values to each side of the operator, 'AND' them together, and store that result in the lefthand expression.
What is rp->p_flag? Specifically, a member 'p_flag' in a struct that is pointed to be 'rp' ; for our purposes we can treat 'rp->p_flag' as an integer variable the same size as SSWAP, the bits of which represent what flags are currently active.
So we bitwise-AND it with '11111011' (our hypothetical value for ~SSWAP) then we will preserve every value except the bit indicated by SSWAP which will always be set to zero, i.e. 'cleared' as a bit flag.
For example, if our current rp->p_flag was '11001100' then ANDed with '11111011' you would get '11001000'. Just the bit that we intend to target is forced to zero. The values of these variables when treated as numbers changes, but that's not important to this kind of operation, we're only interested in how the bit pattern changes.
The point of all this, is that for example, in our own code we can do tests like 'if (rp->p_flag & SSWAP) { ... }' that will be true if and only if the bit indicated by SSWAP is a 1, or we might be writing the entire value of rp->p_flag to a hardware register. It's even possible that 'rp' stands for 'register pointer' and we're actually writing to such a register with this exact operation, but we'd have to see where 'rp' is declared to know for sure.
(Oh, hey, look at that, there's actually exactly such a test shown in the code sample. I probably should've watched the video before commenting, heh.)
Oh, hmm, or you might have meant your question at a much higher level. We'd have to see a lot more of the code to know all the gory details, but the comments would suggest that we're about to turn control over to a process that may or may not have been swapped out, but isn't anymore. So if the flag indicating that it is swapped out is set, we need to clear that flag, and apparently do another thing (that isn't explained clearly) to make it work right.
@@chrisamshey207 Thanks! I did understand the bit operations and the intricacies of the C++ language. I just don't understand the significance of rp, flag and SSWAP. Then again, this has more to do with my lack of knowledge of the UNIX operating system and the PDP 11 architecture. The only OS I did learn from the university was MINIX and that was decades ago. Nonetheless, thanks for your explanation.
I can't thank my college enough (and AT&T) for the opportunity to work with and make changes and improvements to the UNIX system in the late 1970s, that was used for student computing. Having the free source code to examine, edit, and build was a great basis for my career.
Very nice, thank you Eric for that story
12:53: Minix is also hard to escape in current year, because it's in almost every major Intel CPU/chipset.
Per Wikipedia: "Intel chipsets post-2015 are running MINIX 3 internally as the software component of the Intel Management Engine."
True, had forgotten that, thanks ropersonline
Thank you for making this video, it's such a treasure to me.
As a long time Linux and open source enthusiast, I have had such a hard time getting people to understand and appreciate this origin story. The fact that Bell Labs retroactively revoked the ability to use the source is why the GPL is the way it is (and why GNU means "GNU is Not Unix"). That along with Xerox distributing only precompiled binaries (previous to that, it was typical for companies to distribute software + source code and Stallman had custom modifications).
From wikipedia:
> In 1980, Stallman and some other hackers at the AI Lab were refused access to the source code for the software of a newly installed laser printer, the Xerox 9700. Stallman had modified the software for the Lab's previous laser printer (the XGP, Xerographic Printer), so it electronically messaged a user when the person's job was printed, and would message all logged-in users waiting for print jobs if the printer was jammed. Not being able to add these features to the new printer was a major inconvenience, as the printer was on a different floor from most of the users. This experience convinced Stallman of people's need to be able to freely modify the software they use.
These things compelled a lot of smart and passionate people to find a solution, and many of them were doing things in isolation (Minix, GPL, Linux, BSD) as disparate bespoke systems until the world began ultimately to see them as part of a larger single movement.
This is really important to understand, because people have a really hard time understanding why once something is open source you can't close it again - that's the very behavior they set out to prevent with GPL. Finally, I'll add that this story highlights the clear and apparent friction that took place as a result.
Thanks Kamel, yes that one act of revocation showed how important the open source approach is
@@CyberGizmo I suppose I rambled a bit, sorry for that. I guess what I was getting at is thank you for making this so that I have a video to reference when someone is having trouble understanding.
It was also the time period where Sun stopped including a C compiler in the system making it an extra cost option.
@@Kamel419 Well I didn't know this and found your "rambling" (if it was meant to be implied, I don't agree with a negative connotation to that) nicely readworthy. So thanks. ;-)
@@CyberGizmo revocation still keeps happening nowadays, but to "user data", aka, music, games, videos, anything you think you own but its on the "cloud".
which is why I'm antiquated and have a big square box left to my table (it doesn't fit under it). that box is mine and my things are mine, they can't "delete" them if its running on my machine.
Absolutely love video essays like this. What a wonderful story that had a wonderful impact all these years later. Glad it's not lost to obscurity in time!
Thanks Open-Source Anarchist, it helped so many
as a youngster in the field, I really appreciate you sharing this info with us in a really well formatted and paced way. understanding the behind the scenes feels as important because we can learn to avoid mistakes that happened with licencing and distribution and long term development of such a big project as an OS. Great video thank you for sharing!!!
I came through my undergrad CS years just after UNIX had been locked down. So we had no coursework level access to it. However, I had a few friends that were doing their Masters at the time, and they had managed to load a bootleg copy of UNIX onto a PDP-11 they had in the graduate lab. Interesting stuff at the time.
Thanks for sharing the story, Eyes
Explanations: First, I said the Academic version of UNIX was free, this was true later, but they had to pay $150 for an administrative fee.
That was a copy and media charge. a blank tape was anywhere from 10 to 50 dollars just for media depending on the supply contract - which varied from year to year. the rest was inventory and handling charge to put the distribution on the tape. (which, if I remember right was a dd dump of the current system disk to tape, and you had to have the same controller in your system to load it back to disk or you wasted several days writing your own dump utility to copy it to disk.
What a fascinating story! Every time I watch one of your videos, I learn something new and exciting. I often wish you were on the moving television screen doing documentaries about the history of software development. Love it! Best wishes.
I am thankful for the history. I'm just a little younger than you but didn't become a developer. I stayed on the systems consultanting and implementation side. I remember playing with BeOS on my PII 266 MHz and was amazed at the speed. It got me looking into other OSs besides Windows. I fell I love with NetWare. But some Unix history is still beyond my years so I love to understand the roots. Cheers.
Nicely done. That really brought back memories. I was hired at Bell Labs in 1980 and learned Unix and C. I've used mostly Unix and Unix-like operating systems ever since. Currently I'm running Debian Linux on all my machines.
As computer engineer and Linux/Unix enthusiast, I am happy to find such channel with a wonderful creator
Dankeschön
And I am happy you like the channel
My OS class, in the early 90s, the practical side focused on Minix and re-writing the scheduler. It was a fun class. I wrote a test simulator around my code using Turbo C++ under DOS which enabled me write and test my code with a debugger, which enabled me to complete the project much more quickly than my classmates who most where "making changes, recompiling the kernel, and seeing what happens." - which took something like an hour for each iteration.
On my 14 years old main rig, I just compiled two versions of my custom Linux kernel, derived from Linux 6.0.6, slightly patched, one optimized for speed, one for size.
Both custom kernels were compiled in just over 2 minutes each. Both kernels boot my trusty old rig in unter a second (
Cheers! dipi
No bloatware on your 14-year-old rig, lol.
@@jimatperfromix2759 Not if I can help it, right on. At least concerning kernel space: no IPv6, no WiFi, no Bluetooth, no lvm/md/dm/virtualization/hyperthreading/btrfs/initrd or localization bs. None of those things are enabled when I build my kernels. It has been a compact, initrd-less kernel for years now, kernels able to mount their own boot partitions, able to process IPv4 packets and able to talk to the USB keyboard.
My userland, however, happens to be a bit more comfy: KDE/Plasma (but without messenger, file indexing or other bloat) with Okular, feh, SeaMonkey, Firefox, xterm, OpenOffice, gcc, ruby, some vim extensions and Stellarium.
So yeah, parsimonious kernel space, comfy userland. Cheers!
@@dipi71 what's so bad about hyperthreading? genuinely curious.
my experience is that it helps more than it hurts.
@@KenjiUmino It adds unnecessary complexity, making CPUs even more vulnerable to exploits like Spectre or Meltdown. It's also abused by marketing parading around synthetic benchmarks to sell CPUs. It makes caches less effective because of all the MESI-syncing and flushing. It makes optimal code generation even more problematic (those poor gcc and clang devs!). I could go on.
Have been thinking for a while about how to wrap my head around systems for a while so I'm glad to be pointed to a resource like Lions' paper and book. Thanks, DJ!
I'm happy I found your channel, even though it has happened three years later than it should. Thank you. Will enjoy scrolling the feed back.
Welcome aboard!
Please keep putting out videos. You're content is invaluable.
To anyone who enjoyed this video, I can highly recommend Brian Kernighan's (yes, the Brian Kernighan of K&R C fame) 2019 book, "UNIX: A History and a Memoir". Working at Bell Labs during that era would have been awesome.
Thanks for the reminder - it's on my Must Have list.
Agree 100% This is a fascinating, enetrtaining and hugely informatative book by Kernighan, a *must-read* for anyone interested in operating systems or history of computing; you don't have to be a Unix-head to love this book!
Peter H. Salus's "A Quarter Century of UNIX" (1994) also covers the topic.
Thanks for doing these videos! I'm not so young but you make me feel like hearing grandpa's great stories about how we get here as society, full of interesting details! Thanks again!
As a lover of computer/video games history, your videos are pure gold.
Thank you very much.
I learned similar story in my Uni.
I like your channel because it is academic like but easy for anyone to listen to. Right amount of time, keeps attention and on point.
Your a legend! Happy to learn from all of the systems which have come before and build future OSes!
Another awesome video DJ👍Thanks!
I love watching your videos. I've been using Linux since 2019 (Junior year of high school), when I learned about it in my CompTIA A+ course. Anyway, I've recently started just diving in deeper into the OS and learning about the history of Linux, Unix, and other Unix like operating systems. Your videos are great help and I appriciate your deep knowledge of Linux and computers in general.
Hope I can be a very wise IT guy like you.
Wow, makes me feel old. I remember taking a similar course in high school but we weren't actually taught anything real, just BS to pass a certification exam for Windows NT, which I never ended up taking because I found Linux and went in that direction.
@@anon_y_mousse kinda how it was for me. The biggest thing I can note about the course with us though was the fact that the teacher knew he was just trying to please the CompTIA A+ people so he made sure to help us learn that stuff then if we had any questions about anything, come to him. He worked in a hospital for about 20 years in IT so he had quite the experience. He help me grasp networking concepts (advanced), WIndows commandline, Windows Server, virtualization (CompTIA goes over all this but not in the depth I wanted.)
Very interresting and very informative video on the Unix origins. I'm a Linux user since 1996 and although I'm not a developper, I really enjoy using Linux on my personal rigs and also earning a life working in a big company whose infrastructure is running Linux VMs
Smart Captain! Think of all the pain you saved yourself over those years! :-)
In 1975 I was lucky to be a sophomore at East Brunswick (NJ) High School, where Dick Haight lived. Dick is the notorious author of the UNIX find command. He was then a supervisor at Bell Labs Murray Hill doing UNIX development. He got EBHS to buy a PDP-11/40 in 1975, on which he installed Research Version 5. That's curiously not from the branch that became System V, which was the branch Haight actually worked on. I learned C on an all upper case Model 33 teletype with a paper tape punch and reader and a big yellow roll of paper. It was glorious.
I think I first got the Lions books in the red and orange covers (one was the actual code, the other the commentary) as a junior in High School; we devoured them. Dick Haight got me and a couple of my high school friends jobs at Bell Labs starting summer 1978, which presumably was his plan all along. I think it was that first summer I shared an office with John Lions for a few weeks before I left for college. He was so warm and so gently funny. I remember he was working on making the chroot (change root) system call useful.
The story I heard was that the lawyers got wind of the the Lions books when someone ordered a hundred of them for Bell Labs. They started asking why they were paying for copies of software which they already owned. But instead of Big Bad AT&T suing Dr. Lions, they offered him a sabbatical, which is how we met.
I didn't think of "you are not expected to understand this" as a hack, but of course back then I had nothing to compare it to. It was a clever way to switch processes. To switch to a process, all you had to do was restore its stack pointer to the stack pointer register and execute a return, which restores all the other registers that were pushed on the stack. The entire process state was determined by the stack pointer.
Thank you for sharing the story as I can relate to all of it. I started working in Silicon Valley around 1977. By 1984 I was working on UNIX systems that my company built, but later I ended up working for Semiconductor Co learning on a PDP 11/780 BSD of course for a few years and later moved over to super Micro. Moving from sys3 to sys5. I learned a lot of form those day on that PDP 11 until it was sold for scrap. I had good mentors during that time. I enjoyed the journey, over 30 years working on most version of UNIX HPUX SUN Dec Linux etc as I continued working in the valley until 2016. Again thanks for sharing.
Wow, just wow! Thank you so much for sharing. One of the best histories of computing I've heard. I agree with all the other commenters this is a treasure and should be more widely seen.
Great story. Now I am really curious to read that book. Thanks for your channel and content, sir!
Thanks Peter, it is a very informative about how things worked
Excellent informative and entertaining as usual,thank you very much for videos you create DJ.
Welcome Higher Self
The breakup of the Bell System was indeed a two-edged sword. The Labs were a unique resource, non-governmental with all of that baggage and non-commercial with the shortsightedness of commerce. One of the worst results of the use of Antitrust.
Great episode, plenty of interesting things as usual. Thanks so much!
Thanks Alberto and glad you liked it.
Wow, great story and context into why things are the way they are today! ⭐
Great video amd love the HAL9000 screen running in the background! Thanks for sharing
Greate video, it brings back memories as I was there for John's first classes using Unix as his case study. We would be given an area of the system to study and each we week would have a short exam where we were given 20 lines of code and asked to explain what it did without using the commentary. It taught me a lot about operating systems and driver writing and along with the compiler classes (Ken Robinson) I got a good understanding of runtime systems. These are all skill I still use today, even though I'm a hardware engineer. I still have my copies of the original commentary and source code and I still can open it to any page and within a few minutes explain what is going on, and I do understand what that code did :-). As someone else pointed out it was "The University of New South Wales". "South Wales" is halfway around the world, not far from where I am now as after UNSW I attended Cambridge University where I studied even more on operating systems and networking, the area I still work in. Thanks again for the memories.
Thanks for the correction and also sharig your memories of the use of the commentary always a pleasure to meet people like you
David, greetings from the past 🙂.
(Fun times at UNSW)
Nice walk down memory lane. Those were the days. Thank you.
I was using Linux in 1992, and it was effectively open source.
I don't know if it used some specific licensing scheme, maybe that came later, but the way we set up Linux was to download the dozen or so images off Usenet, write them to floppies with rawwrite, install them to a hard drive, and then edit and compile the kernel's source ourselves. It was open source in every practical sense.
I think he meant Minix wasn't open source (couldn't be redistributed) when Linus used it to bootstrap Linux. I actually remember Linus' now famous post to comp.os.minix announcing his new work. I remember everyone was itching for a 32-bit version of Minix but Tanenbaum wanted it to remain 16-bit so it could be used for teaching on older pre-386 PCs. So there were all these patches being distributed to make Minix (sort of) 32-bit. I was reluctant to buy a "pc clone" until I could get something that actually ran some kind of Unix that was reasonably obtainable by an individual (and I really hated MS-DOS). So I was over the moon when Linux was released! Before that I bought a used AT&T Unix PC (3b1) and used it for several years.
In 1980 I attended John Lions operating system course and bought the unix book from the UNSW. As part of my master degree involving Unix level 7.
Interesting. I worked for at&t long lines in the 70's and 80's as a computer technician. Most of our systems were unix in system 5 with Bourne shell. Unix has always been my favorite OS. I would have loved to see the C source code along with assembler. We had several pdp 11/70's along with vax780 and a vaxcluste system of various vaxunits with vax/vms OS. We also had a hp1000 system with hp-ux. I'm 71 and I still wish I could work with these again. Anyways, thank you so much for this video.
Thanks @dj, really informative! 🤙
Welcome Ernest
love this history stuff from you.. any linux/unix history things i love
The SerenityOS project should fill you with joy then.
Will have to go take a look at it.
Thanks for the history lesson and explanation. Most fascinating.
Glad you enjoyed it
I always wanted something to explain the Linux Kernel (or UNIX) at the level Lions did. Thank you for turning me onto this document. Most books go over userspace tools as "How Linux Works".
Try "Understanding The Linux Kernel" (3e, IIRC... or is it 5th ed.?)...
It only took me ~5 years to find the time to read through it and understand. YMMV ;-)
That was fun! Thanks for the history.
Sixth edition Unix was not the first version available outside of Bell Labs. There were earlier versions that Bell Labs made available to universities. For example, at The Johns Hopkins University we definitely had Unix Fifth Edition running on the EE Department's PDP 11/45 in 1975. I believe I even recall seeing Unix Fourth Edition for a short time before that. Also, as far as I recall, the 6th edition from Bell Labs was not called "Unix System VI" but rather "Unix Sixth Edition". The research versions of Unix were named after the edition of the manual that went with them. "Unix System V" was a non-research version that was used more generally within AT&T outside of Bell Labs.
I recall them also being called version 6 and 7. I would have to check, but I think some of the published papers also use version rather than edition. But I was getting annoyed about him saying "System" as well. By edition 8, it seemed that edition was used most of the time.
Some of the early ports were done by Uni's, I think one was to a 32bit mainframe done in Australia.
I used to have a really in depth UNIX history books that covered all the early relases.
When I was at Uni, Version 6 with PWB and 7 the more commercial variants, but Uni's used a v6 license with BSD4.2 on VAXes which provided networking and was tolerated.
The Bell Labs guys were cooperative, it was the commercialisation with System V which made things tough.
The workstation market built on BSD which had sockets and networking
@@petergathercole4565 me too, it said UNIX(tm) version 6
I wonder if "edition" came in because the early commercial applications was for documentation & type setting.
👍
Great discourse on the topic. I enjoy all of your content, per Corvo Attano's comments below. My immediate contemporaries and I are a unique gathering of those genuinely raised by the "Computer Industry." Born in the mid 70s and raised on the first PCs of the late 70s and early 80s, we were fated to study EE, CS or a combination of the two, which was my destiny!LOL We basically had no say in it!LOL Interestingly, I've found that those almost immediately after me (say, my ex-wife and her cohorts who are only 6 years my junior) were taught so differently because the architecture had such quantum leaps, and the availability of "cheap" processing power was so prevalent. Even PnP relegated my ex to knowing even less about how this stuff worked, never having to configure a .sys or .bat file on a PC. My point is deep dives like this are so rare in today's academic space. This coming from a former adjunct. I appreciate your knowledge and your willingness to share. Though I'm deeply versed in what you've discussed, I'm shocked at how much this is not common knowledge for those getting out of school now. Thanks so much.
Thank you so much for sharing these anecdotes. I found it wonderfully inspiring. I feel far less guilty about cheap hacks now! 😁
You should write a book. I would definitely buy it. The history of OS development, especially UNIX, fascinates me.
I might just do that, and thanks for the very kind words, Robert
@@CyberGizmo If you do, please make sure you include all the 'friction'... e.g. SCO / Caldera, the Berkeley variants and 386BSD / Bill Jolitz thing, and the various scumbaggery involved at every step.
Another interesting video! Enjoyed it! Love the NeXT shirt too!
Over the years BSTJ had many articles and several entire issues devoted to UNIX, written by the primary authors (Ritchie, Thompson, Bourne, Aho, Kernighan, too many to list)
The BSTJ was critical for my understanding. Hard to find in 1979-80, but I had an inside connection that helped. Later, reading the kernel source was really useful.
Nice shirt. Loved the NeXT machines and the OS.
Really enjoyed this talk, thank you! When you get to the part about UNIX and 9k lines of code, it felt deeply like foreshadowing. One individual, you say? What if someone was thus inspired to make their own OS based on how lightweight UNIX was... They just needed a reason to not use UNIX itself after being inspired by Lions' book.
You can take a way the license, but not the inspiration. You certainly can never stop academic 'piracy', because students will often see that as a barrier to learning IMO.
I think the repeated issues caused by licensing with both Minix and Linux as well as UNIX show that licenses can't stand in the way of academic motivation.
That's why I agree: FOSS is driven by our desire to 'understand this', the desire to investigate always will drive us so why get in the way of helping each other understand. Maybe we don't strictly need to know things to do things, but in the process of chasing those mysteries we learn so much.
You mentioned the rapid dev paces several times, and one thing I've been trying to figure out is if system upgrades really are necessary after a certain point. It seems like many are indeed not well thought out and exist to add features that may cause other conflicts or to correct for something the last update broke. While that's acceptable, it gets under my skin... if we can't understand things and overcomplicate them, are they really serving their best purposes.
In the beginning of the video you mentioned approaching OS education via a limited toy OS approach; however something I keep returning to is the idea that for edge computing something as light as a toy could offer more advantages than something for more general use. That's part of what fascinates me about the little OSes for the ESP32 MCUs; I know it doesn't impress many people now, but that so much power can be packed into such a tiny chip and not-much-bigger board for the chip amazes me with possibility.
I don't know a lot about edge computing, but it sounds like the basis for the kind of distributed computing I need to improve on to build what I want to build. Something about cloud IoT deeply concerns me, so maybe I do need to write my own OSes.
I loved Unix, specifically I loved the whole everything's a file paradigm and the text based configuration files. When that changed I knew things were going to get much worse. Also the shell was so well developed, it was beautiful. System V and POSIX Compliance.
That's part of what makes Linux so great, is that they carried over those things.
@@anon_y_mousse
Linux is cool, but I think they made Linux overly complex to please the nerds.
@@justgivemethetruth Depends on if you use systemd I suppose. Otherwise it's pretty simple. If you haven't already, read the LFS book. Don't try to do it until you've read the whole book and it'll be a lot more clear to you.
@@anon_y_mousse
I started with Linux in the 1990's, back when you have to have the timings for your video card and configure it by hand, and the interrupts and DMAs for your network card, and configure all that stuff by hand. I MacOS was not so good or I had to use Linux for some kind of server, I'd go back to it, but it is too much trouble.
Plan 9 tho
Great talk. Thank You. Could You Please elaborate more about proprietary code in system 7 ?
The memories. Reports of Reinfelds and his research team porting UNIX to Interdata 7/32 at the University of Wollongong in Australia, was the first time I came across unix.
Love your 2001 screen saver in the background.
Thanks bodavis99
I was exposed to the PDP 11 (a PDP 11/73 and PDP 11/23) at Asnuntuck Community College. I was a lab assistant and my boss was the system manager. We were running RSTS/e with DCL. He constantly telling me he was offered UNIX for free and kicked himself for not taking up the offer.
My previous exposure had been a CDC Cyber 730 and a VAX 785 running VMS. There was a VAX running Ultrix, but that was for the upper class CS students, and not accessible to the general student population.
I love learning the rich history of UNIX and Linux (or GNU/Linux) and other os's. And as much as thongs like man and info facilitate information about commands and programs, I still haven't seen something beat out the help facility on VMS. That and EVE in EDT mode for an editor. I still have muscle memory for the keypad using a VT100 or VT220 keyboard.
Awesome Chris, its rare to find anyone who knows Cybers, I learned on a CDC 6400 and a Cyber 73. Yeah Ultix was always put to work and never shared with us meer mortal "students" Never worked long on VMS my career with DEC lasted 1 month, they had me writing documentation when I hired on as an developer. I guess they didnt know developers HATE documentation LOL
@@CyberGizmo I trained on the CDC3300, then the 6400, then the 6600, various Cyber models, trained on various PDP10s, PDP11, etc.
I tested "C" before it was released, and spotted its hundreds of fatal flaws, and sent feedback, much of which was dismissed.
EVE/EDT? I only had brief exposure to VMS, but I used RT-11 on a PDP-11 (actually an LSI-11 but who is counting?). My favourite editor was KED, the Keypad Editor. You did all the editing and text manipulation one handed using the numeric keypad and the 4 function keys on the top row. Incredible fast, incredible powerful. Fortunately, the editor in VMS also had a KED mode :) Unfortunately, noone seems to have ported KED to Unix or Linux :(
@@JanBruunAndersen Actually, emacs has an EDT mode, though I couldn't get the keypad mapped right. In EDT, gold-0 will open a line below the cursor (gold was PF1 on the VT100 and VT220 keyboards, same position as num lock).
You ROCK. You made my professional life POSSible. You, your buddies, the OS, and THE DOCUMENTATION.
Thanks,
Retired High-Tech Wageslave
PS VAX/VMS. is an amazing OS and still in use: go see why!
You won my subscription!
@DJ Ware : great content. I seem to understand you challenge and hope generation Z to reinvent a better operating system. Then I wonder about for example ways to instruct such operating system. Can we drop the UNIX shell? Perhaps we can and did. The Amiga one was not bad. But, is this video just about OS or also about anything on top of it?
I didn't believe I would understand this or be interested enough to watch the whole thing, but I did.
IIRC, it was Stallman who talked Torvalds into open sourcing the kernel. Stallman had been trying to find a backend to run his grand free (as in freedom) operating system frontend.
Thank very much for sharing all your experiences and knowledge. I love videos from people that really know about the environment and history. Meanwhile... I'm more an OS/400 user... Haha but nobody likes IBM legacy systems. Great video sir.
I am no expert on OS/400 but did some programming on it once, always found the machine kinda fun to use actually.
Maybe nobody likes IBM legacy systems, but they do like getting their paychecks on time.
@@JanBruunAndersen LOL yes we do :)
I falling in love with your channel 😅
Thank you for the informative talk. This from an old kernel developer. DG-Nova era.
John Lions was an operating system lecturer at the University of New South Wales in Sydney Australia
To think, we're closer to 2068 than 1968. 😐
At UC San Diego, we were studying A.Tanenbaum's book on Minix/OS design as early as 1984 (3 years before "1987")
I remember the VAX operating system, at UMASS Boston campus... in '76, If memory serves correctly. While taking Pascal, we all had accounts on the system. Oddly enough, the experience was helpful to my career later on at a regional bell company. Not sure of the dates without digging through piles of books. I still have the text.
I enjoyed the video very much. I've been using Slackware for a long time. I switched to it when it was very clear IBM OS/2 was not going anywhere. Geez I feel old just saying that.
19:31/19:36 - oof, can relate to that feeling. Not entirely sure I believe it (for you or for me), but I can definitely relate to feeling that way. Thanks; interesting video!
Great video like your NeXT t-shirt !
Thanks Caustic
I’m ordering the book!
You are NEVER too old.
Thank you for introducing John Lions to me. Never knew anything about him which is a shame actually.
Have you considered doing a Multics video? Interesting history there.
Hey DJ. I am very much interested in understanding/building Operating Systems. I have a deep curiosity for how things work at its lowest. Do you know where I can start my journey in becoming more knowledgeable about this topic? I am currently a software engineer looking for a better way to create an high performance system with a low carbon footprint. Thank you!
You remind me of my C++ teacher, in about 1999, who helped us understand how things worked undrr the hood including the incorpoation of assembly code in a .c file. His name was Dana Lasher NCState university
So what's the bare necessities of an operating system?
--Scheduling processes (synchronization)
--switching processes
--loading new processes into memory
--managing memory
--interprocess communication
--manage hardware interrupts
Nice t-shirt. I owned a monochrome 68040 slab way back in the day.
What's that screensaver or animation running behind you?
Nitpick: Dennis Ritchie is not dr - he has always been dmr. It's thought, by some, that the early preference for three-letter usernames may have had something to do with word lengths on ye olde 18-bit DEC machines that used the DEC SIXBIT character code (like the PDP-7 Unix was invented on): You could thus store a username in a single 18-bit word. For a time, those 3-letter usernames were pretty much globally unique, at least to the extent that Unix pioneers and similar talked to each other. There was only one (real) dmr, one rms, one dfb, etc. Computerphile has a video called TLU Three Letter Username Obsession.
Well aware of that, on occasion when I worked for AT&T I would email him as he was the one who taught me UNIX. I didnt want to use dmr on the video title as it gave too much away, its pretty widely known.
As usual, a fascinating talk! From your perspective, how should someone get started in OS development? Should the low-level language to start be Rust or is C knowledge still a prerequisite? Is assembly still relevant?
You can write an OS is any programming language, Rust is very popular, but I think I would choose a functional programming language like oCaML if i were building a new one today. Just an opinion though. And thank you for the kind words :)
I was a Bell Labs in the early 80’s. Owning a copy of the Lions books were grounds for dismissal. (Yes - I still have my ORIGINAL set on my bookshelf!)
15:54 an NT-based version of Windows. Windows NT was based on OS/2 1.3 and the development of OS/2 started in 1985, so I guess the baseline is from 1985. The baseline of Linux is actually newer, it's really from 1991. It is intended to be compatible with Unix, but doesn't contain any Unix code for copyright reasons.
It is interesting to note that in an even earlier operating system - George 2 (and 1) on the ICL (then ICT) 1900 - written totally in assembler, there was an area of the code that started wih the comment “you are advised in your own best interests not to modify the following code without contacting ~named person”. Ie “you are not expected to understand this”. The very interesting thing was this did a very similar function (an efficiency hack on peripheral assignment.
And no, though I did many mods to G2, I stayed well clear of that section 😀
Thanks for the story. The fundamental development problem is economic. A company lab has to pay its bills and make a profit to continue. Unix in a lab has to be supported by profit making products elsewhere. Someone needs an income to participate in open source too. Hopefully this development can continue.
He has a NEXT tee shirt on.
The Personnel Dept. said we need more eunuch programmers.
i do yep
I used Andrew Tanenbaum's Structured Computer Organization book at University of Queensland ITEE in 1980s doing my BSc. We used PDP-11s in the lab for learning operating system programming. This was before Linux.
haha yeah about 10 years before...I wonder how Linus feels about Andrew giving him and 'F' for Linux (some early heated usenet group messages between the two)
Thanks a lot, and I hope you teach me
Thank you for creating content. I use linux and I'm pissed off of the fact that I don't know how it works but I've started exploring and I find Operating Systems as one of the most interesting topics and I spend most of my free time trying to understand it. Can you suggest a good book on Operating Systems?
There is a free one called The Linux Operating System by William Stallings written in 2018 Its an extract of his book called Operating Systems: Internals and Design Principles, Sixth Edition. You should be able to find it in a Google Search. I do not have a link handy, but I know its currently used in a few OS courses
@@CyberGizmo Thank you very much!
My thought on the syntax change is that we nowadays forget that Ritchie and colleagues were still developing the computer language while in parallel using it to develop the OS.
C and UNIX few up together.
According to my boss at the time, I was one of the first 500 users of linux in the fall of 1991. I worked at a college MIS (IT) department and my boss was either a contributor or betatester of Torvald's early release and he gave me access to the floppy images. I was an admin for NeXT step, DEC VAX VMS, Ultrix, and Sun Sparc OS4 systems at the time and I give credit to linux and GNU for helping standardize the configuration of unix like systems over the years. They would have 30 students on a machine with 32MB of ram and a couple hundred megabytes of drive space. I believe that linux advanced the unix style system design by allowing everyday people like me run it at home and write more features and fix bugs. I actually have about 30 lines of code in the linux kernel circa 1998 in the 2.5 kernel in the netfilter section. I was not much of a programmer or CS engineer but a basic user like me had access to the source and the ability to contribute after finding a bug in the way packets were handled during certain firewall rules. That was fantastic! Without UNIX, much of the innovation in operating systems today would not have been possible. I was not aware of the reason why Bell Labs released System 6 source code but now it makes sense.
Great story, Mike. OMG I remember when Sun first showed OpenStep at the annual Sun Conference, that rooms was packed wall to wall, so I went down to the exhibit hall and found two developers and got the info and demo from them directly...hehe sometimes it pays not to attend the sessions...
One of the first operating systems I learned in the 70s during my college years
"Untold" is quite a stretch. Lot's of people are familiar with "Commentary on UNIX" by John Lions. I have a physical copy of it in my office.
I've heard all this before, but you still made it entertaining. Though, I thought only the British mispronounced assembly. Assembler assembles assembly. Surely you've heard that before.