BBS: The Documentary is essential viewing, if you haven't already seen it. Check out www.bbsdocumentary.com/ or watch the entire documentary through Jason Scott's playlist at ruclips.net/video/Dddbe9OuJLU/видео.html. You can follow Jason on Twitter at twitter.com/textfiles
@@Riskteven UC2 as well it had higher compressed then both arc and zip back then but never got traction because it wasn't super fast. I still have a copy kicking around on a floppy somewhere.
You didn't mention Gary Conway who contributed largely to the code and creation of PKZIP. Who is literally a co-creator of the .ZIP format. He is still alive and responds when reached out to and could offer a lot of first hand knowledge for this video.
In 1989 I worked for a Louisville Kentucky company called Infinity Design Concepts and we co-developed the ZIP format and wrote an entire assembly program compatible with PKZIP and PKUNZIP. My boss Gary spoke to Phil Katz during ZIP development, especially when we found that his PKUNZIP program would not unzip files compressed by our beta version program. It was then discovered that for extra speed PKZIP used only a pre-defined set of compression tables while we were generating dynamic tables based on the input file. At that point, PK provided us the specs of his fixed tables so our later released program also produced files PKZIP could still unzip. I was told I was one of just a few people in the world who understood how that data compression worked. 🙂 Cool history, right?
@@KowboyUSA It's been out for a while now. I find it to be far more buggy and less compatible with plugins and file formats than 5.66 was, so I prefer 5.66.
I once found this compressor which had an incredible ratio. I ran a few tests and they worked fine. It was beating zip by a factor of 10 or more (this was ages ago so not sure). I decided to take the plunge and store some data using it, stashed the disks and didn't think about it in a year or so. When I needed these files again, in a new computer, I installed the utility and got my disks and started the decompression process.... and it failed time and time again. I researched again, and found that it was just hiding a copy in the hard disk and writing the path in the "compressed" file. A scam. I lost that data. Does anyone remember that one? I forgot it's name.
ZIP has become so successful, it is also being used in places you might not recognize. It has become a common universal container for all kinds of document formats, e.g.: * ODF files (ISO 26300 office documents), the native format of LibreOffice * ZAE files (containing Collada DAE data and associated image textures etc) * Java JAR (class library) files and their offshoot, Android APK (application package) files Languages like Python and Java include ZIP handling routines in their standard libraries.
A long time ago my ‘daily driver’ CAD software went from a project folder to a single file format. It took me two minutes to figure out they just zipped the project files!
Crazy. My dad used to tell me the story of taking an interview with Phil to join him as a programmer, as they were both UWM alumni and knew each other beforehand, and how the interview was conducted inside their home with Phil and his mom. He said the whole thing came off very awkward and weird, and that Phil seemed to have a very hard time interacting with people so most of the interview was conducted by his mom. I'm not sure at what time in the story this interview took place, but he declined their offer, and after seeing all this that was definitely the right decision!
Ytrearneindre If Phil had decided to “borrow” some code from an even bigger company, they’d sue your pants off. He might have been a bit of a liability.
This video was pure propaganda for SEA. It makes it seem like ZIP was just a knockoff of something SEA invented. The deflate algorithm (in ZIP 2.0) is ENTIRELY invented by Phil Katz and THAT is why ZIP is popular today. Because pretty much every other compression algorithm up to then is either under a cloud of patent encumberment or not as good. Even now deflate is used everywhere because it is so much faster than anything which is tighter. MOREOVER Phil Katz had the foresight to make ZIP into a streaming compatible format. Many file formats need to have offsets computed and written into headers which means the file can't be created incrementally, it all has to be stored on a scratch disk first. A huge advantage in the age of the internet. PKARC was a lot faster than ARC because Phil Katz was a great programmer and rewrote SEA's code. Yes he used their PUBLICLY AVAILABLE code as a framework for his program but the reason the BBS community rallied around him was that his program was so much better. He could have rewritten the whole thing from scratch but he didn't see a point to it. Hard to see this as not SEA being an "indian giver."
It's hilarious (and sad) that how the Internet community could be turned against somebody who they view as a threat. Countless lives have been (and still is) ruined by them, while megacorporations survive and thrive...
@@ZlothZloth "The internet community or the BBS community?" Both; I remember the debate raging across USENET at the time as well. And FidoNet had several nodes with integrated internet portals as well, sort of running "betwixt and between" both worlds at times.
Not hust the "Internet community". Mainstream media also turn against people, often creating "pop justice" which is wrong more times that it being right. And, instead of apologizing, the media moves on to the next story. Disgusting! Is FidoNet still around?
@@MrJest2 I don't think dial-ups will ever disappear. I may (once I get my desktop up and running) get a phone modem and o.n...c......e...... .........a.............g..............a.............i............n............... .........................e...........................x..................p..........................e....................r........................i.........................e.........................n......................c..........................e...............CARRIER LOST. ARGH!!!!!!!!!!!!!!!!!!
ZIP did win... until RAR came along, fueled by warez. Then as the need to split archives lessened due to broadband everyone just started using whatever they wanted since it didn't matter anymore, I'd argue 7z is probably the current winner though.
@@mattpowell8369 More efficient, better compression algorithms, what more is needed? Those are the main points of file compression xD Oh and resource friendly :)
15:58 It's so bizzarre seeing a mid-90's style website with Coronavirus resource links. It's like a time traveller did it for a joke to mess with us in the future
Virus's have existed for like probably hundreds of thousands of years, probably million. We've only been able to sequence them like in the last 50 years.
I went to the website out of curiosity, and it is still functioning. Good to know that there are people out there who still stick to the 90s aesthetic.
A "corona virus" is nothing new, it was named based on its visible (microscopic) structure and is the carrier structure for certain illnesses including the "common cold". It has been known to science for many decades.
I was at a Black Angus sometime around 2005 or so. They had a trivia game (I think it was called something like NTN), Anyways, I was there at around 10 P.M. The trivia game was shutting down. As it was shutting down, I noticed that it went into a DOS window (Command Prompt for you kiddies out there). Since I'm a computer nerd, I started watching. I forget whether it was PKZIP or PKUNZIP (I'm thinking it was PKZIP, but I honestly don't remember) came up. Whe I saw the PK??ZIP, I was like "WOW! Long time since I seen that!".
I remember my dad had a friend with the Registered version of Wolfeinstein 3D. He could not split the files on more than 1 floppy so he had to compress it to fit 1 1.44mb floppy. PkZip compression still wasn't enough so he used ARJ and it fit :D I was happy with the result.
ARJ was the bomb. It slaughtered zip in compression ratio back in the day until one day out of nowhere the new version of pkzip "magically caught up". More phil fishiness in my opinion. I never knew why pkzip was popular at all until now.
@@CommodoreGreg I was an ARJ fan, till the day I discovered it couldn't recover from single errors, and a single error in one floppy disk out of a 30-something floppies archive rendered the whole thing unrecoverable. I wasn't happy that day
@@alicewyan Any archiver would fail to decompress files from a corrupted archive. Sometimes the ONE single affected file is lost (actually deleted by the decompressor to prevent you from using a corrupt file but that could be overidden). If the archive headers were corrupted, the chances at recovery were lower. Adding effective data recovery information would increase the file size so much that it would no longer be a file compression program. Have a look how CD-ROMs are designed to handle unreadable spots, scratches, fingerprints, etc. Each data bit is recorded on the medium 6 times at different places using different encodings. A typical 700 MB CD has a raw capacity of over 4 GB worth of pits. In my experience from the old days, in average, 1 floppy in each box of 10 was faulty. Truly an abysmal format. Even the good ol' Verbatim 3.5" 1.44 MB floppies were showing CRC errors sometimes. Relying on 30 disks in a row to be fully functional was quite foolish. Always verify your data after writing the data on them. It takes less time than going back and forth to make another copy. ;)
@@vladimirarnost8020 sure, but multivolume PKZIPs and RARs were able to recover from failures somewhat, whereas ARJ wasn't. A faulty floppy would cause PKZIP or RAR to lose a file or two, a faulty floppy caused ARJ to be unable to use the whole archive.
I think it was a case that SEA was being a bit more professional than PK. The BBS community was for the large part a group of hobbiests. Where piracy was really common then it felt safer to run a BBS using software that you didn't feel that some company is going to give you legal action for a Hobie that costs a mid range PC and a telephone line.
ARJ deserves some recognition in the compress utility history, and so RAR format. It would be great a follow up to this one with other formats in the mix. Great video by the way! Thank you!
I ran a bbs back in the day when I was in high school, as did several of my friends. Pkzip, arc, and several other tools were the lifeblood of our file transfer sections. Great vid!
The Zip format was fundamentally different. ARC had minimal directory/header information. Zip had both local and global directories for redundancy, it stored directory trees and the files could span several floppies. One could see that they had put some thought of the format instead of starting with the compression and then adding the file format as an afterthought. I was a very early registered user of the PKZIP.
one of the cool things about ZIP, is if one disk in a multi-disk set gets corrupted, there's a chance PK's recovery tool can restore the data from the checksums, basically the same way a RAID works. To my knowledge, none of other ZIP archivers has this recovery feature. With ARJ and RAR, if one file/disk in a set goes bad, the entire set is lost.
love your videos! just noticed that at 5:50 you were talking about university of Wisconsin Milwaukee while showing a picture of University of Wisconsin Madison.
I was looking to see if anyone else noticed that. I give him some slack since Peter is from the UK, but with so many clips available of Milwaukee on Google nowadays I do find it somewhat irritating.
Nice summary of the compression war of the 80s. I was there and running a BBS at the time so I definitely saw all this. I did learn a thing or two. I'm a programmer and I was just thinking recently about how the late Katz's legacy seems set to live forever - the ZIP format is now baked into a TON of stuff at a very low level right up to the latest web technologies.
All these years later, and I did not know until I saw this video that Katz's PKARC code contained those comments and misspellings showing parts had been lifted directly from SEA's code. I thought it was 100% "clean room" code reimplementing the ARC file specifications from scratch. It makes a difference. But Katz's death was still tragic.
Nicole Wren How? It’s just that he talks about stuff that happened when I was very young. I got internet in 1996, when I was 11. BBS were on their way out by then for example.
@@svankensen Just to be curious of other people's feelings, you should never comment on somebody's appearance (or in this case, their voice) unless it's positive and in a non-sexual manner. It's considered backhanded because it translates to "you sound a decade older than you are" which isn't flattering. I understand this isn't what you meant to say, but I wanted to answer your question :)
ARJ certainly was the best back in the BBS days. Along with ZIP. These days its still ZIP due to it being inbuilt into Windows. Along with RAR for handeling mutiple formats. Including ZIP. Which is still my favourite. Just as Macafee antivirus, norton utilities and xtree gold. were the goto software back in the 80's and 90s.
After I discovered that the doom installer was just merging a split self extracting zip executable, I wrote my own utility to split them. Yay! Here you go! all 50 disks for file X. lol Such a pain in the butt. I do not miss those days.
ARJ multi-disk was actually sane. It could be used to "sneakernet" an arbitrary volume of data from one computer to another with as little as three floppy disks -- one reading the first volume, one transporting the second volume, and one writing the third volume; by which time the first floppy would be ready to receive the fourth volume.
I got my name attached to the credits of a big Census data project because I was the on-site techie for the institute doing the data crunching, and I figured out how to use ARJ to make “installer disks” for the multi-megabyte databases they needed to send out.
I remember those days. From the sidelines, SEA came off as a large company aggressively suing Phil Katz for making a better ARC program. They even went to the length of suing him for using the term ARC in one of his user manuals after he had stopped making his ARC compatible software. It really pissed off the BBS community which resulted in BBS operators recompressing all of their existing files using PKZip. The idea of making the PKZip format public domain was what put the nail in SEA's coffin though. It guaranteed that the PKZip would be used by a large portion of the software development community, and become the defacto standard from then on. Thanks for the video!
What you're saying is all true. This video is propaganda. It omits that PKARC (and then ZIP) were much faster and the deflate algorithm (released in 1991 in beta form and 1993 in PKZIP 2.04g) was better than anything else out there AND patent-free which is why it is still used today. The fact the Phil Katz used immaterial amounts of FREELY AVAILABLE source code in his program is technically copyright infringement, but it sure was a dick move on SEA's part to release it and then turn around and sue a guy who used it. Why the fuck did they release it in the first place then?
I remember using PKZIP, PKUNZIP, PKSFX, etc. I went to UWM in the 80's, and my "Degrees of Phil Katz" number is _2._ I also remember walking past the PKWARE offices (after he'd passed) in the Grand Avenue Mall, downtown Milwaukee many times.
Fun note based on the graphic of all the file formats at the end... if you use Microsoft Office, you use ZIP files even more than you might realize. All of the "modern" file formats, introduced in 2007 (.docx, .xlsx, .pptx, etc.) are just .zip files with a bunch of textual .xml files in it (the 'x' in the file extension) and possibly other resources (like picture files).
Fascinating video! I well remember ARC and of course zip (started in computing in 1984 with an OU degree) but never knew the background until now. Well done !
@@Mnnvint It's basically associated with the Amiga and with Japan. LHA was written by a Japanese person and was the first archiver to really have proper Japanese documentation and promotion, therefore in Japan LHA continued to be the format of choice well into the Win95 era and was eventually built into Windows Explorer such that you can open them like folders (just like .zip opens in western Windows versions), a feature that remained even in Windows 7 (Japanese edition). This was kind of frustrating for us Amigans though as the Amiga version of LHA forked into a European Amiga version, and only supported compression methods up to lh5, whereas lh7 became standard in Japan. It was always super frustrating when you'd download a Japanese .lha file and you couldn't uncompress it (you could see the files in the header, but if they were lh6 or lh7 it would just say unsupported compression method when you tried to extract them)
Later on the Amiga got LZX which beat LHA, ZIP, ARJ, RAR. The LZX author went on to work for microsoft and the compression algorithm was/is used in their .CAB files.
@@NozomuYume In 2003, I was working with some online friends on hacking a Japanese PC fighting game into English. None of us could figure out the proprietary file format that images inside of this game were compressed in. After finally finding someone who really knew how to program and reverse engineer, they quickly realized the image format was simple LHZ with a custom header format. After building a new tool, he was able to get even better compression ratios then the original! I never really knew until reading NozomuYume's comments here that it was a very Japanese style of compression! XD 17 years later and I am still learning more about a fan game translation project I worked on 17 years ago!
I've always wanted to know more about the ARJ file compression format that seemed to be fairly popular in the early 90's. It had better compression that PKZIP and handled archives split across multiple floppies very well, much like RAR does today.
I have great fond memories of downloading games from local BBSes to play on my C64 using my 300 baud modem. Yep, can't tell you how many times me and my friend waited an hour or two to download a game only to find that it didn't work! Moving up to that light-speed 1200 baud modem was like going to heaven... I really miss the whole color-ASCII environment of those BBSes, too. Some were even animated. Good times.
Good coverage. I remember all of this. I had nearly forgotten about the ARC drama and even the format, but this refreshed my memory. PKARC and PKZIP simply trounced SEA’s software, legal issues aside. Phil Katz (Raymond Lau on the Mac side, via his StuffIt software) drove compression software to the state we k ow today. There was some similar work going on in the *nix world, but that wasn’t in the public view much.
5:50 talk about University of Wisconsin Milwaukee but shows stock footage of the state capital in Madison Wisconsin (home to University of Wisconsin Madison) Being from Madison, myself, I'm not at all upset, but I'm sure UW Milwaukee alumni would be offended
I work with a guy who just graduated from college at Madison. He is an assistant (jr) manager/product specialist for the factory/office. I know for a fact he would care. He has Madison swag (mostly badger related stuff including a football helmet) posted all around his cubical. That said, we are technically within driving distance to Milwaukee if you are willing to drive 7 hours+. Oh well. This is a British video so I give it a pass.
I'm glad I wasn't the only one to go huh when that clip popped up. Although it's not nearly as pretty on the outside as the MN capitol, it's a very distinctive building (and really pretty on the inside as long as you don't have a major fear of heights, as it's ledges with falls-to-your-death everywhere above the main floor as anybody who has been there knows) and can't stand in for Milwaukee (eww, why shit on Madison like that?).
I'm wondering if it was assumed Milwaukee is the Capitol of Wisconsin being the largest city. Otherwise I can see UW System which is based in Madison but it's complicated for anyone outside the Midwest to tell. (UW-Stout Grad myself, brother went to UW-Madison)
When the filesize is too large, YOUMUSTZIPIT! When the floppy is too small, YOUMUSTZIPIT! ZIP IT. ZIP IT GOOD. (Full parody pending, I may just nap all day)
@@ChannelReuploads9451 I used UC2 a lot as well. In fact. I remember it had a cool feature where you could give a special dictionary comprising/representative of your files and it'd use that to compress/decompress at a much better ratio.
.tar.gz is what we call the impostor of archive formats, only used by nolifer linux fans. The 3 primary formats you will be finding online are .zip, .rar and .7z.
@@BurnedPinguin8630 ...like how .7z is the impostor format of 7zip nolifer fans. seriously, who the fuck is "we"? can we get an actual vote in here? :P
I was an ARJ fan in the early to mid 1990s. Part of it was better compression, part of it was that it split files across floppies better, and part of it was that you only loaded each floppy disc once (ZIP required reinserting some of the discs twice). However, I think ARJ turned some people off because they pushed Christianity on its users. It wasn't a heavy push but it did feel odd and out-of-place. This wasn't the main reason ARJ didn't take off but it didn't help either.
@@gravis778 bzip (that is, bzip2) is interesting too, because it's based on a completely different approach to compression, which was discovered more or less by accident.
They all had their own unique features. arj could make a multi floppy archive, LHA could make a self extracting file with simply an extra command like option. Also LHA could pack subdirectories.
This explains a great deal for me. I got online right about the time all of that had just finished going down; 92 to 93. I remember seeing ARC files around but also that they were depreciated.
Zip was great for the time, I even had it on my calculator back in the day. Too bad there was the drama over Arc. Now, 7Zip is the superior format, offering a lot better compression than Zip, and is open source.
Thank you not only for a awesome video on a time I lived through like many...at the time I was like who cares as long as my archives uncompress...but later the details that I never knew that much about are very interesting...then the bbs documentary...what I find...toally off my radar...I spent this last weekend watching the whole thing and i am almost done.
I'm not so sure this is completely right. ZIP did not gain dominance during DOS times. Only after Win95 came along and command-line DOS archivers were no longer relevant it became the de-facto standard. It also had a big market share for commercial (non-BBS-related) uses. For the DOS+BBS era e.g. ARJ was in much wider use after it took over from LZH/LHA. Which itself had a smaller share in the fragmented post-ARC pre-ARJ era. However, it's not as if there only was one use case for compression, and naturally different use cases had different formats. For Fido there was the Nodelist distribution, which (as far as I know) want from ARC to ZIP with no experimentation as it was a globally generated file. Then there was message distribution, where stuff got put into archives just for individual automated transfers. For those you more or less had to coordinate with your links on what format to use, but especially in later times most BBS could accept archives in a dozen different formats---the unarchivers used were free so it was just a line in a config file. And then there were files that were offered on BBSs. For those everyone would chose the archiver as they wanted. Some BBSs re-archived files into a format to their liking, others left them as they got them. And even others offered the same file in multiple formats. (I operated a BBS from 93 to 99.)
Henry, I remember being introduced to SEA arc around 84/85. It was only about a year later that someone pointed me to the PK versions of arc and unarc. They just did everything faster on the same PCXT (8088 @4.77MHz) (thats 0.00477GHz)
Great video! I bought SEA's software and even splurged on the source code. I used it to learn 8086 assembler. When the poop hit the fan with Katz, I sided with SEA. I even spoke to one of the guys and the wife a few times on the phone. Katz was a thief and a liar. I never bought into that crap that all software should be free and embraced the shareware model, but I refused to give Katz a penny and freely distributed pkware whenever I could. His format change was bullshit.
The strange thing is that in the early nineties when I got into the IBM PC platform, the ARJ format was king, and ZIP files were few and far in between...
Omg... I had no idea! Great video, thanks. pkz204g.exe was always handy on a disk in my DOS days. I had completely forgotten about ARC. It was ubiquitous back in the BBS days. I ran a board on WWIV back in the late 80s and early 9os. Good times!
I still use the DOS version of PKZip 2.50 often on my older offline PCs. Its advantage over the more common 2.04g is that it understands long filenames, but only when run inside Windows 95 / 98.
A Gamer Aaron There was an exciting feeling during that time because you were just discovering Japanese exclusive games the west never got. Because we were kids back then we didn’t have the funds to procure these games the standard way. Nowadays it’s no big deal, either you import a game or just buy it digitally from the Japanese store on your respective console eshop, PSN etc. That feeling of excitement playing non English Roms are long gone.
@@songoku9348 I think part of it was also bandwidth/data storage restrictions too. It took 20 minutes to download a SNES rom. I can download the entire SNES library in less than that now. And I'm about to buy a separate SSD for downloading the entire PSX library. Now individual roms really aren't special.
What?? He's talking about ZIP? Did you miss the fact that ZIP was a new file format (with advantages over ARC which why it is still used today) and how all the code was written from scratch? Did you think that SEA invented the idea of a file compressor? Or that SEA invented the compression algorithms they used? Did you miss how Phil Katz's programs ran faster than any competing software? His software was faster than Info-Zip which is still used today. At the time he video was written Phil had just completed the DEFLATE algorithm which is STILL used today even outside of ZIP. The man was a genius.
I had no idea that's what happened to Phil. It's a sad thing. I would have liked him to know how great I thought PKLITE was. It was a win-win. It would compress your executables and they not only took up less space, but they loaded faster, in spite of the fact they had to decompress during the load. RIP Phil Katz.
Oh man, what have you done. I had completely forgot about all the zip formats and now some memories about pkzip and pkarch are resurfacing from the corners of my memory.
6:05 "Getting the job done with the fewest number of instructions and therefore running time." Fewer instructions does not necessary mean quicker execution time.
Thanks for this video, I knew Phil socially at the time this all happened, and this is as I remember it. My only caveat is I don't consider the Hendersons to be what I'd call "reliable narrators", but that's not your fault, that's on them. Somewhere there must exist text archives of the data compression and "bullroar" discussion boards from the old Exec-PC BBS, based in metro Milwaukee where we both lived but with a global userbase. Everything going through Phil's mind at the time, either with respect to his software or just his social interactions, would all be in those files. We'd listen to him get into the weeds of the technical aspects of data compression, and then we'd tease him about his celebrity crushes ... good times. If those could be found they would be a tremendous primary resource for any historians studying this stuff. And thanks for the link to the BBS documentary, I've been meaning to watch that.
6:10 Peter: As a Minnesota Resident, I can't help but notice you/editor *misspelled Swissconsin's Milwaukee as "MILWAUKE"...* Sorry for coming off as a grammar N@zi...
I studied Huffman coding and also image redundancy, and decided to try writing my own image compression format in QuickBasic. My goals were simple, to make something with comparable compression ratios to PNG, but to also make it reasonably fault tolerant as well. I didn't use any standard ZIP libraries or any such thing, I actually wrote my own variation of a self adapting Huffman tree algorithm myself. Though through different iterations of my program I figured it was best to drop the self adapting nature of it as it more or less defeated the main goal of fault tolerance, so there's probably still some old commented code in it left out for the self adapting tree. Each iteration of the program, I had to be especially careful that the encoder and the decoder were doing EXACTLY the same thing, as I found that it was all too easy for two equal weight tree branches to randomly pick either 1 or 0 depending on the exact technique I was testing at the time. It was a neat project for whatever it was worth, but certainly wouldn't be ZLib compatible. Only my program knows what the hell it's doing ya know. I also learned some rather weird quirks of QuickBasic itself while making it, I had to make some special fix code in one of the routines that kept on corrupting one particular variable that wasn't even supposed to be written to on the faulty line. Any which way, I managed to make it work, with differentials and an internal reset every 16 scanlines. If it found bad data anywhere within a 16 scanline section, it would simply substitute the differential from the last recognized good scanline and decode with that instead, up until the next reset scanline ya know. It was pretty robust too, I could literally mangle the encoded file with a hex editor randomly and it would decode whatever was intact, with only minimal artifacts in damaged areas. Anyways, thank you for coming to my Ted Talk, but yeah as neat as Huffman coding is, it's also a bit sensitive to the exact sort algorithm and techniques used in building the Huffman Tree.
Nerd, your stare during the intro unnerved the ever living hell out of me. Sip on a coffee or a cuppa tea next time, like it's an 1980's interview show...
BBS: The Documentary is essential viewing, if you haven't already seen it. Check out www.bbsdocumentary.com/ or watch the entire documentary through Jason Scott's playlist at ruclips.net/video/Dddbe9OuJLU/видео.html. You can follow Jason on Twitter at twitter.com/textfiles
love from ' #BHAI ' (BRO)
A a q 8 tqqQ
And after watching BBS: The Doc everyone make sure to check out another Jason Scott doc, Get Lamp.
@@Riskteven UC2 as well it had higher compressed then both arc and zip back then but never got traction because it wasn't super fast. I still have a copy kicking around on a floppy somewhere.
Remember using PKware back in the day, although also in early 90's using ARJ and been an early adaptor of RAR
You didn't mention Gary Conway who contributed largely to the code and creation of PKZIP. Who is literally a co-creator of the .ZIP format. He is still alive and responds when reached out to and could offer a lot of first hand knowledge for this video.
Conway’s game of life…
That name gives me GOL vibes
@@Versuffe not the same Conway, but yeah! That one’s John ;)
@@Vlr rip
@@Vlr Lots of amazing Conways in computer Science. John Conway, Gary Conway, Lynn Conway. The Conway name is blessed
you've been killing it lately with these videos.. great stuff
Interesting seeing you here
Nice seeing you here, mate!
Well well well, good to see that you watch these too. Makes me feel better 😂
Meeeeeeooooooowww
Agreed! And so do you!
In 1989 I worked for a Louisville Kentucky company called Infinity Design Concepts and we co-developed the ZIP format and wrote an entire assembly program compatible with PKZIP and PKUNZIP. My boss Gary spoke to Phil Katz during ZIP development, especially when we found that his PKUNZIP program would not unzip files compressed by our beta version program. It was then discovered that for extra speed PKZIP used only a pre-defined set of compression tables while we were generating dynamic tables based on the input file. At that point, PK provided us the specs of his fixed tables so our later released program also produced files PKZIP could still unzip. I was told I was one of just a few people in the world who understood how that data compression worked. 🙂 Cool history, right?
u such smarty, wow.
whos a good boi?
@@starflow90210 🥺
wow thats awesome I love hearing stories about developers and their journey to create what they create
In 1989, I was born
Really? Was LZ77 / LZSS considered that complicated?
Winzip, it really whips the Llamas ass.
Oh wait...can we get a WINAMP video?
I second this request.
The _Llama's Ass Whipper Academy_ recently released a 5.8 beta.
@@KowboyUSA It's been out for a while now. I find it to be far more buggy and less compatible with plugins and file formats than 5.66 was, so I prefer 5.66.
IF so.. include Null Installer.
Ooh, and Winamp is allegedly getting revived too.
I once found this compressor which had an incredible ratio. I ran a few tests and they worked fine. It was beating zip by a factor of 10 or more (this was ages ago so not sure).
I decided to take the plunge and store some data using it, stashed the disks and didn't think about it in a year or so.
When I needed these files again, in a new computer, I installed the utility and got my disks and started the decompression process.... and it failed time and time again.
I researched again, and found that it was just hiding a copy in the hard disk and writing the path in the "compressed" file.
A scam. I lost that data. Does anyone remember that one? I forgot it's name.
It's not just one: There were a few programs like that.
@@vylbird8014 oh man
RIP those 10 GB of porn
I remember hearing about that data compression scam, but also forgot its name.
@@Etcher lol Mandy. A blast from the past.
ZIP has become so successful, it is also being used in places you might not recognize. It has become a common universal container for all kinds of document formats, e.g.:
* ODF files (ISO 26300 office documents), the native format of LibreOffice
* ZAE files (containing Collada DAE data and associated image textures etc)
* Java JAR (class library) files and their offshoot, Android APK (application package) files
Languages like Python and Java include ZIP handling routines in their standard libraries.
Yep, so does .Net.
Send any Word .docx file to Notepad. The first two letters will be "PK".
funny how comments give more details about zip’s real domination than the video which supposed to address it :)
Microsoft Office files are also zips. Appx package files from the Microsoft Store are zips.
... and coming from a Unix background, I find Java's jar utility more intuitive to use than Zip!
A long time ago my ‘daily driver’ CAD software went from a project folder to a single file format. It took me two minutes to figure out they just zipped the project files!
Crazy. My dad used to tell me the story of taking an interview with Phil to join him as a programmer, as they were both UWM alumni and knew each other beforehand, and how the interview was conducted inside their home with Phil and his mom. He said the whole thing came off very awkward and weird, and that Phil seemed to have a very hard time interacting with people so most of the interview was conducted by his mom. I'm not sure at what time in the story this interview took place, but he declined their offer, and after seeing all this that was definitely the right decision!
Are you sure? 13:40 "..this led PKware to become a multi-million dollar company"
Ytrearneindre If Phil had decided to “borrow” some code from an even bigger company, they’d sue your pants off. He might have been a bit of a liability.
Was Phil like Terry Davis from TempleOS ?
He should have taken the job.
This video was pure propaganda for SEA. It makes it seem like ZIP was just a knockoff of something SEA invented. The deflate algorithm (in ZIP 2.0) is ENTIRELY invented by Phil Katz and THAT is why ZIP is popular today. Because pretty much every other compression algorithm up to then is either under a cloud of patent encumberment or not as good. Even now deflate is used everywhere because it is so much faster than anything which is tighter. MOREOVER Phil Katz had the foresight to make ZIP into a streaming compatible format. Many file formats need to have offsets computed and written into headers which means the file can't be created incrementally, it all has to be stored on a scratch disk first. A huge advantage in the age of the internet.
PKARC was a lot faster than ARC because Phil Katz was a great programmer and rewrote SEA's code. Yes he used their PUBLICLY AVAILABLE code as a framework for his program but the reason the BBS community rallied around him was that his program was so much better. He could have rewritten the whole thing from scratch but he didn't see a point to it. Hard to see this as not SEA being an "indian giver."
It's hilarious (and sad) that how the Internet community could be turned against somebody who they view as a threat. Countless lives have been (and still is) ruined by them, while megacorporations survive and thrive...
The internet community or the BBS community? Or maybe the AOL/Compu$erve/GEnie communities?
@@ZlothZloth "The internet community or the BBS community?" Both; I remember the debate raging across USENET at the time as well. And FidoNet had several nodes with integrated internet portals as well, sort of running "betwixt and between" both worlds at times.
Not hust the "Internet community". Mainstream media also turn against people, often creating "pop justice" which is wrong more times that it being right. And, instead of apologizing, the media moves on to the next story. Disgusting!
Is FidoNet still around?
@@rricci Yep. There's still a small but passionate FidoNet community, and still a fair number of dial-up BBSs out there, too.
@@MrJest2 I don't think dial-ups will ever disappear. I may (once I get my desktop up and running) get a phone modem and o.n...c......e...... .........a.............g..............a.............i............n............... .........................e...........................x..................p..........................e....................r........................i.........................e.........................n......................c..........................e...............CARRIER LOST. ARGH!!!!!!!!!!!!!!!!!!
ZIP did win... until RAR came along, fueled by warez. Then as the need to split archives lessened due to broadband everyone just started using whatever they wanted since it didn't matter anymore, I'd argue 7z is probably the current winner though.
@@mattpowell8369 More efficient, better compression algorithms, what more is needed? Those are the main points of file compression xD Oh and resource friendly :)
@@mattpowell8369 Mainly far better compression and it's free. If you're command line oriented then 7Zip all the way.
@@alanbourke4069 On top of that, it supports a lot of archive and compression formats. Great way to get into tar/bzip/gzip files on Windows.
Oh, I remember the point of RAR was that you could split a large archive into multiple parts to save it on multiple _diskettes_
@@FindecanorNotGmail ZIP and ARJ had that ability loooong before RAR existed.
15:58 It's so bizzarre seeing a mid-90's style website with Coronavirus resource links. It's like a time traveller did it for a joke to mess with us in the future
Holy crap I didn't even see that. I thought that was an archived copy of their 90s site.
Virus's have existed for like probably hundreds of thousands of years, probably million. We've only been able to sequence them like in the last 50 years.
I went to the website out of curiosity, and it is still functioning. Good to know that there are people out there who still stick to the 90s aesthetic.
@@rubansrirambabu7771 neat, good to know 2 years later!
A "corona virus" is nothing new, it was named based on its visible (microscopic) structure and is the carrier structure for certain illnesses including the "common cold". It has been known to science for many decades.
PKUnzip. Now that's a name I've not heard in a long time. A long time.
I was at a Black Angus sometime around 2005 or so. They had a trivia game (I think it was called something like NTN), Anyways, I was there at around 10 P.M. The trivia game was shutting down. As it was shutting down, I noticed that it went into a DOS window (Command Prompt for you kiddies out there). Since I'm a computer nerd, I started watching. I forget whether it was PKZIP or PKUNZIP (I'm thinking it was PKZIP, but I honestly don't remember) came up. Whe I saw the PK??ZIP, I was like "WOW! Long time since I seen that!".
That would be during the time of the (ARC) Clone Wars
Norton Utilities... Desqview... Lotus 123, Netscape, Wordperfect...
Yes, and it's pronounced phonetically - "p-kun-zip", if you ask me and my old school buddies XD
@@jacklewis100 Wrote several history papers on Wordperfect. It was way better than Word back then. Word suuuucked back then.
I remember my dad had a friend with the Registered version of Wolfeinstein 3D. He could not split the files on more than 1 floppy so he had to compress it to fit 1 1.44mb floppy. PkZip compression still wasn't enough so he used ARJ and it fit :D
I was happy with the result.
ARJ was the bomb. It slaughtered zip in compression ratio back in the day until one day out of nowhere the new version of pkzip "magically caught up". More phil fishiness in my opinion. I never knew why pkzip was popular at all until now.
@@CommodoreGreg I was an ARJ fan, till the day I discovered it couldn't recover from single errors, and a single error in one floppy disk out of a 30-something floppies archive rendered the whole thing unrecoverable. I wasn't happy that day
arj was a good one! with multiple volume option. I wonder if you can do 4.7 GB arj and burn it onto several dvds.
@@alicewyan Any archiver would fail to decompress files from a corrupted archive. Sometimes the ONE single affected file is lost (actually deleted by the decompressor to prevent you from using a corrupt file but that could be overidden). If the archive headers were corrupted, the chances at recovery were lower.
Adding effective data recovery information would increase the file size so much that it would no longer be a file compression program.
Have a look how CD-ROMs are designed to handle unreadable spots, scratches, fingerprints, etc. Each data bit is recorded on the medium 6 times at different places using different encodings. A typical 700 MB CD has a raw capacity of over 4 GB worth of pits.
In my experience from the old days, in average, 1 floppy in each box of 10 was faulty. Truly an abysmal format. Even the good ol' Verbatim 3.5" 1.44 MB floppies were showing CRC errors sometimes. Relying on 30 disks in a row to be fully functional was quite foolish. Always verify your data after writing the data on them. It takes less time than going back and forth to make another copy. ;)
@@vladimirarnost8020 sure, but multivolume PKZIPs and RARs were able to recover from failures somewhat, whereas ARJ wasn't. A faulty floppy would cause PKZIP or RAR to lose a file or two, a faulty floppy caused ARJ to be unable to use the whole archive.
I feel you really managed to cram a lot of information into a tight format. I really wish there was a word for it...
lol
Just Zip it already! :)
@@Meepswonder nah bro just rar it or 7z it
@@CanonOverseer just ARC it already! no space is left!
Another lesson about the importance of good PR.
I think it was a case that SEA was being a bit more professional than PK. The BBS community was for the large part a group of hobbiests. Where piracy was really common then it felt safer to run a BBS using software that you didn't feel that some company is going to give you legal action for a Hobie that costs a mid range PC and a telephone line.
@@toddfraser3353 Honestly I'd have posted to the BBS groups the exact details of what was going on.
ARJ deserves some recognition in the compress utility history, and so RAR format. It would be great a follow up to this one with other formats in the mix. Great video by the way! Thank you!
I ran a bbs back in the day when I was in high school, as did several of my friends. Pkzip, arc, and several other tools were the lifeblood of our file transfer sections. Great vid!
The Zip format was fundamentally different. ARC had minimal directory/header information. Zip had both local and global directories for redundancy, it stored directory trees and the files could span several floppies. One could see that they had put some thought of the format instead of starting with the compression and then adding the file format as an afterthought.
I was a very early registered user of the PKZIP.
one of the cool things about ZIP, is if one disk in a multi-disk set gets corrupted, there's a chance PK's recovery tool can restore the data from the checksums, basically the same way a RAID works. To my knowledge, none of other ZIP archivers has this recovery feature. With ARJ and RAR, if one file/disk in a set goes bad, the entire set is lost.
love your videos! just noticed that at 5:50 you were talking about university of Wisconsin Milwaukee while showing a picture of University of Wisconsin Madison.
I was looking to see if anyone else noticed that. I give him some slack since Peter is from the UK, but with so many clips available of Milwaukee on Google nowadays I do find it somewhat irritating.
@@dlinkster yeah I dont blame him, I'm sure he just searched university wisconsin and madison popped up cause it's the bigger school
@@D1nomite1 I also suspect that ostensibly no one in the UK has heard of Madison, Milwaukee is a lot more famous.
So many new videos lately, each better than the next. Your channel is several types of awesome!
I like the fact that Phil Katz wasn't hypocritical and made ZIP free for all developers to use.
Nice summary of the compression war of the 80s. I was there and running a BBS at the time so I definitely saw all this. I did learn a thing or two. I'm a programmer and I was just thinking recently about how the late Katz's legacy seems set to live forever - the ZIP format is now baked into a TON of stuff at a very low level right up to the latest web technologies.
I see tar or tar.gz more often in web compression
All these years later, and I did not know until I saw this video that Katz's PKARC code contained those comments and misspellings showing parts had been lifted directly from SEA's code. I thought it was 100% "clean room" code reimplementing the ARC file specifications from scratch. It makes a difference. But Katz's death was still tragic.
It’s weird seeing you in the videos and finding out you look about my age. I always assumed you were a decade older. Great content as always!
... what?
That’s... pretty backhanded
Nicole Wren How? It’s just that he talks about stuff that happened when I was very young. I got internet in 1996, when I was 11. BBS were on their way out by then for example.
@@svankensen Just to be curious of other people's feelings, you should never comment on somebody's appearance (or in this case, their voice) unless it's positive and in a non-sexual manner. It's considered backhanded because it translates to "you sound a decade older than you are" which isn't flattering.
I understand this isn't what you meant to say, but I wanted to answer your question :)
Charlotte L Ah, that makes sense, yeah. Glad someone questioned it so that I could clarify.
2:39 Actual and correct example of the current year’s most abused phrase “expanded exponentially”
5:55, the captions still have your pronunciation guide in there for UW Milwaukee
if you open a .zip file as a text file, it has the letters "PK" at the start and the end.
No, only the start. The last thing in the file is the index, which is why opening a truncated archive is so difficult.
@@vylbird8014 oh, ok, i didn't know that
@@vylbird8014 Each file has also a local directory entry so it is possible to fix truncated archives to the extent it at all is possible.
I thought you were gonna say it started in 197z
I prefered ARJ back in the days, as it could compress to multiple disks, which PKZIP did not support yet.
ARJ certainly was the best back in the BBS days. Along with ZIP. These days its still ZIP due to it being inbuilt into Windows. Along with RAR for handeling mutiple formats. Including ZIP. Which is still my favourite. Just as Macafee antivirus, norton utilities and xtree gold. were the goto software back in the 80's and 90s.
After I discovered that the doom installer was just merging a split self extracting zip executable, I wrote my own utility to split them. Yay! Here you go! all 50 disks for file X. lol Such a pain in the butt. I do not miss those days.
ARJ multi-disk was actually sane. It could be used to "sneakernet" an arbitrary volume of data from one computer to another with as little as three floppy disks -- one reading the first volume, one transporting the second volume, and one writing the third volume; by which time the first floppy would be ready to receive the fourth volume.
@Dave doom 2 is not compressed. Quake 3 on the other hand is just a zip file renamed to pak.
I got my name attached to the credits of a big Census data project because I was the on-site techie for the institute doing the data crunching, and I figured out how to use ARJ to make “installer disks” for the multi-megabyte databases they needed to send out.
I remember those days. From the sidelines, SEA came off as a large company aggressively suing Phil Katz for making a better ARC program. They even went to the length of suing him for using the term ARC in one of his user manuals after he had stopped making his ARC compatible software. It really pissed off the BBS community which resulted in BBS operators recompressing all of their existing files using PKZip. The idea of making the PKZip format public domain was what put the nail in SEA's coffin though. It guaranteed that the PKZip would be used by a large portion of the software development community, and become the defacto standard from then on.
Thanks for the video!
What you're saying is all true. This video is propaganda. It omits that PKARC (and then ZIP) were much faster and the deflate algorithm (released in 1991 in beta form and 1993 in PKZIP 2.04g) was better than anything else out there AND patent-free which is why it is still used today. The fact the Phil Katz used immaterial amounts of FREELY AVAILABLE source code in his program is technically copyright infringement, but it sure was a dick move on SEA's part to release it and then turn around and sue a guy who used it. Why the fuck did they release it in the first place then?
@@ssl3546 The video explicitly mentioned that PKARC was much faster.
Imagine if Technologic went “Drag and drop it, arc, unarc it”. Daft Punk would be ruined if SEA had prevailed.
Idk, it's kind of catchy tbh
I remember using PKZIP, PKUNZIP, PKSFX, etc. I went to UWM in the 80's, and my "Degrees of Phil Katz" number is _2._ I also remember walking past the PKWARE offices (after he'd passed) in the Grand Avenue Mall, downtown Milwaukee many times.
Fun note based on the graphic of all the file formats at the end... if you use Microsoft Office, you use ZIP files even more than you might realize. All of the "modern" file formats, introduced in 2007 (.docx, .xlsx, .pptx, etc.) are just .zip files with a bunch of textual .xml files in it (the 'x' in the file extension) and possibly other resources (like picture files).
The BBS documentary was so great! Took me right back to the mid-90s, for some serious nostalgia.
@Nostalgia Nerd “cut & shut” refers to welding halves of 2 written off cars together. The phrase you were looking for was a “cut & dry” case.
Or "open and closed."
Formedras yeh, that too
or "open and shut"
@@TheHackysack Yeah. That's actually what I meant. Thanks for the correction.
Fascinating video! I well remember ARC and of course zip (started in computing in 1984 with an OU degree) but never knew the background until now. Well done !
I uploaded and downloaded files compressed with LHZ and later LHA. As I recall, working with ZIP files was a nuisance on the Amiga...
I associate LHA with the Amiga too.
@@Mnnvint It's basically associated with the Amiga and with Japan. LHA was written by a Japanese person and was the first archiver to really have proper Japanese documentation and promotion, therefore in Japan LHA continued to be the format of choice well into the Win95 era and was eventually built into Windows Explorer such that you can open them like folders (just like .zip opens in western Windows versions), a feature that remained even in Windows 7 (Japanese edition).
This was kind of frustrating for us Amigans though as the Amiga version of LHA forked into a European Amiga version, and only supported compression methods up to lh5, whereas lh7 became standard in Japan. It was always super frustrating when you'd download a Japanese .lha file and you couldn't uncompress it (you could see the files in the header, but if they were lh6 or lh7 it would just say unsupported compression method when you tried to extract them)
Later on the Amiga got LZX which beat LHA, ZIP, ARJ, RAR. The LZX author went on to work for microsoft and the compression algorithm was/is used in their .CAB files.
LHA was the foundation that was to become LZX. LZX being MUCH faster, and more efficient than LHA, while retaining the LHA support.
@@NozomuYume In 2003, I was working with some online friends on hacking a Japanese PC fighting game into English. None of us could figure out the proprietary file format that images inside of this game were compressed in. After finally finding someone who really knew how to program and reverse engineer, they quickly realized the image format was simple LHZ with a custom header format. After building a new tool, he was able to get even better compression ratios then the original! I never really knew until reading NozomuYume's comments here that it was a very Japanese style of compression! XD 17 years later and I am still learning more about a fan game translation project I worked on 17 years ago!
Astonishingly good work again! Keep it up!
Love it. I did always wonder.
I still use .zip, I even owned a legit version of pkzip back in the day
Great stuff, I'll definitely be checking out the BBS doc. Thanks.
I've always wanted to know more about the ARJ file compression format that seemed to be fairly popular in the early 90's. It had better compression that PKZIP and handled archives split across multiple floppies very well, much like RAR does today.
i still use ARJ for some things
I always liked the word “ARJive”, but there’s probably millions of people out there who insist it is pronounced “Aryive”
@@circattle It will always be an Arrr-Jay file for me :))
I have great fond memories of downloading games from local BBSes to play on my C64 using my 300 baud modem. Yep, can't tell you how many times me and my friend waited an hour or two to download a game only to find that it didn't work! Moving up to that light-speed 1200 baud modem was like going to heaven...
I really miss the whole color-ASCII environment of those BBSes, too. Some were even animated. Good times.
When I started with computers in ~1999 I Thought Winzip was made by the same company as Winamp.
And that WinDo'hs programme too.
Whoever made winrar is a homie is all I know. Thanks for not being free but letting me use it for free all this time. I appreciate it
Pkunzip.exe was an essential program back in the day.
PK Fire
PK Thunder
"Okay"
i still have pkunzip.exe on my ibm 380xd ngl
@@patrickglaser1560 who is Sum F?
@@patrickglaser1560 well, hopefully I'm nothing like him, because I'm not a troll, just trying to make meaningful comments
@@patrickglaser1560 you can't comment or your comments don't show up for anyone else?
Man you've released some brilliant vids recently! Very entertainly educational. Cheers
Interestingly, during my BBS days, around 1993-4, it seemed to me that almost everyone preferred Robert Jung's ARJ.
Good coverage. I remember all of this. I had nearly forgotten about the ARC drama and even the format, but this refreshed my memory. PKARC and PKZIP simply trounced SEA’s software, legal issues aside. Phil Katz (Raymond Lau on the Mac side, via his StuffIt software) drove compression software to the state we k ow today. There was some similar work going on in the *nix world, but that wasn’t in the public view much.
5:50 talk about University of Wisconsin Milwaukee but shows stock footage of the state capital in Madison Wisconsin (home to University of Wisconsin Madison)
Being from Madison, myself, I'm not at all upset, but I'm sure UW Milwaukee alumni would be offended
I work with a guy who just graduated from college at Madison. He is an assistant (jr) manager/product specialist for the factory/office. I know for a fact he would care. He has Madison swag (mostly badger related stuff including a football helmet) posted all around his cubical. That said, we are technically within driving distance to Milwaukee if you are willing to drive 7 hours+. Oh well. This is a British video so I give it a pass.
Scrolled down for this comment
I'm glad I wasn't the only one to go huh when that clip popped up. Although it's not nearly as pretty on the outside as the MN capitol, it's a very distinctive building (and really pretty on the inside as long as you don't have a major fear of heights, as it's ledges with falls-to-your-death everywhere above the main floor as anybody who has been there knows) and can't stand in for Milwaukee (eww, why shit on Madison like that?).
I'm wondering if it was assumed Milwaukee is the Capitol of Wisconsin being the largest city. Otherwise I can see UW System which is based in Madison but it's complicated for anyone outside the Midwest to tell. (UW-Stout Grad myself, brother went to UW-Madison)
@@matthewjbauer1990 Just to clarify, Milwaukeee and Madison are only about 80 miles apart. Maybe you did not mean to imply that you are in Madison.
Yip. I remember PKZIP and PKARC. Great video! Brought back many memories of BBSs and, er, multiple floppy archives!!
When the filesize is too large, YOUMUSTZIPIT!
When the floppy is too small, YOUMUSTZIPIT!
ZIP IT. ZIP IT GOOD.
(Full parody pending, I may just nap all day)
still pending?
IT’S NOT TOO LATE
@@cmyk8964
*_TO ZIPIT! ZIPIT GOOD!_*
@@cmyk8964 STILL NOT
So knowledgeable and interesting. Even throwing things in that are quite simply over the heads of mere mortals, yours truly included, fantastic!
I used LHA and ARJ at the time, with better compression. ARJ had multi-disk spanning alongside better compression, so it was my go to choice.
LHA and LZX also had multiple disk spanning. You could tell it to split the archive at a certain size,
@@ChannelReuploads9451 I used UC2 a lot as well. In fact. I remember it had a cool feature where you could give a special dictionary comprising/representative of your files and it'd use that to compress/decompress at a much better ratio.
Farhan Yousaf heh i was just wondering if anyone else was using uc2 back then! It was slower, but for certain files much better.
Thanks for bringing back the nostalgia. I remember these times will with BBS and ARC. Appreciate reliving my late teen years.......
You know the shortest compression joke? PKUNZIP.ZIP
Reminds me of a Reddit post about screwdriver packaging that you need a screwdriver to open.
@@cst1229 Wasn't it scissors in a blister packaging that required scissors to open?
All your videos have the best sound quality. Good video and content also.
You should cover the saga of PGP next. Funny how the legal environment was a lottery back in the day.
"This T-shirt is a munition." Oh yeah, those were the days...
Just came across your channel love it! I have my OS/2 warp box next to me at my office.
Nerd: How .zip won
Me: *Laughs in .tar.gz*
I too was wondering about TAR in the story.
_tar -zvfx_ goes brrrrrr
.tar.gz is what we call the impostor of archive formats, only used by nolifer linux fans. The 3 primary formats you will be finding online are .zip, .rar and .7z.
@@Riskteven most likely it started with the casette storage devices. You know, those things that data centers would use as a cheap backup solution
@@BurnedPinguin8630 ...like how .7z is the impostor format of 7zip nolifer fans.
seriously, who the fuck is "we"? can we get an actual vote in here? :P
Epic. I just thought about that a couple of years ago and researched myself a little to freshen up my memories on this topic. Thanks!
Looks like "feelings over facts" has been a huge part of the technology field far longer than I thought.
That's just humanity as a whole, honestly.
Excellent series of videos lately!
You should have included other compression tools
LHA (sometimes called LHArc
And ARJ. Both had better compress than .arc or .zip
I completely agree, ARJ, RAR, GZIP, TAR, LHA.. Hopefully in a future episode
I was an ARJ fan in the early to mid 1990s. Part of it was better compression, part of it was that it split files across floppies better, and part of it was that you only loaded each floppy disc once (ZIP required reinserting some of the discs twice).
However, I think ARJ turned some people off because they pushed Christianity on its users. It wasn't a heavy push but it did feel odd and out-of-place. This wasn't the main reason ARJ didn't take off but it didn't help either.
ARJ!
Yes. That was the archiver of choice of myself and people around me back then... pkZIP was a joke
@@gravis778 bzip (that is, bzip2) is interesting too, because it's based on a completely different approach to compression, which was discovered more or less by accident.
They all had their own unique features. arj could make a multi floppy archive, LHA could make a self extracting file with simply an extra command like option. Also LHA could pack subdirectories.
Love this series of videos Peter!
We have used ARJ format which offered higher compression than ZIP back then at least until RAR came out...which had even better compression ratio 🙂
This explains a great deal for me. I got online right about the time all of that had just finished going down; 92 to 93. I remember seeing ARC files around but also that they were depreciated.
Zip was great for the time, I even had it on my calculator back in the day. Too bad there was the drama over Arc. Now, 7Zip is the superior format, offering a lot better compression than Zip, and is open source.
The origins of what we use today, well executed and informative. Love it
I like using p7zip, 7zip’s command line version. Mainly because when it’s done, it tells you “Everything is Ok.”
I mostly use tar cause linux
Great video again, amazing animations, good job!
2:52 1985 or 1995? The voiceover and subtitle is different.
I know i could just search it up but I'm a very lazy human being
85
Thank you not only for a awesome video on a time I lived through like many...at the time I was like who cares as long as my archives uncompress...but later the details that I never knew that much about are very interesting...then the bbs documentary...what I find...toally off my radar...I spent this last weekend watching the whole thing and i am almost done.
I'm not so sure this is completely right. ZIP did not gain dominance during DOS times. Only after Win95 came along and command-line DOS archivers were no longer relevant it became the de-facto standard. It also had a big market share for commercial (non-BBS-related) uses.
For the DOS+BBS era e.g. ARJ was in much wider use after it took over from LZH/LHA. Which itself had a smaller share in the fragmented post-ARC pre-ARJ era.
However, it's not as if there only was one use case for compression, and naturally different use cases had different formats. For Fido there was the Nodelist distribution, which (as far as I know) want from ARC to ZIP with no experimentation as it was a globally generated file. Then there was message distribution, where stuff got put into archives just for individual automated transfers. For those you more or less had to coordinate with your links on what format to use, but especially in later times most BBS could accept archives in a dozen different formats---the unarchivers used were free so it was just a line in a config file. And then there were files that were offered on BBSs. For those everyone would chose the archiver as they wanted. Some BBSs re-archived files into a format to their liking, others left them as they got them. And even others offered the same file in multiple formats.
(I operated a BBS from 93 to 99.)
Henry, I remember being introduced to SEA arc around 84/85. It was only about a year later that someone pointed me to the PK versions of arc and unarc. They just did everything faster on the same PCXT (8088 @4.77MHz) (thats 0.00477GHz)
Awesome work, as always. I would like you to do an episode on Gary Kildall and the tragedy of CP/M sometime.
It'd be interesting if you look into the history of usenet since that's been around a decade longer than the Internet
Is it still around? I thought all those pre-internet alternative was not used anymore.
Great video! I bought SEA's software and even splurged on the source code. I used it to learn 8086 assembler. When the poop hit the fan with Katz, I sided with SEA. I even spoke to one of the guys and the wife a few times on the phone. Katz was a thief and a liar. I never bought into that crap that all software should be free and embraced the shareware model, but I refused to give Katz a penny and freely distributed pkware whenever I could. His format change was bullshit.
The "X" is meant to be pronounced as the letter - it's an acronym for "eXtract".
Awesome video. You say have a great evening but I watched this at 4 in the morning.
The strange thing is that in the early nineties when I got into the IBM PC platform, the ARJ format was king, and ZIP files were few and far in between...
Omg... I had no idea! Great video, thanks. pkz204g.exe was always handy on a disk in my DOS days. I had completely forgotten about ARC. It was ubiquitous back in the BBS days. I ran a board on WWIV back in the late 80s and early 9os. Good times!
I still use the DOS version of PKZip 2.50 often on my older offline PCs. Its advantage over the more common 2.04g is that it understands long filenames, but only when run inside Windows 95 / 98.
Holy hell dude, this was a great listen while vacuuming the floor at my work! This is quite an amazing educational vid
Ah yes, the good old days of downloading SNES roms in zips. Eagerly waiting in excitement when loading them in zsnes.
A Gamer Aaron There was an exciting feeling during that time because you were just discovering Japanese exclusive games the west never got. Because we were kids back then we didn’t have the funds to procure these games the standard way. Nowadays it’s no big deal, either you import a game or just buy it digitally from the Japanese store on your respective console eshop, PSN etc.
That feeling of excitement playing non English Roms are long gone.
@@songoku9348 I think part of it was also bandwidth/data storage restrictions too. It took 20 minutes to download a SNES rom. I can download the entire SNES library in less than that now. And I'm about to buy a separate SSD for downloading the entire PSX library.
Now individual roms really aren't special.
Wait, that’s what the Z in ZSNES stands for?
That documentary was well worth the 5 hours!
7-Zip / 7z format now carries the torch for open compression
Your videos bring me back to a more simple time, I miss the earlier days of pc
14:00 "...our product..."
"...our compression technology..."
Sure, Phil, whatever you say...
:-(
What?? He's talking about ZIP? Did you miss the fact that ZIP was a new file format (with advantages over ARC which why it is still used today) and how all the code was written from scratch? Did you think that SEA invented the idea of a file compressor? Or that SEA invented the compression algorithms they used? Did you miss how Phil Katz's programs ran faster than any competing software? His software was faster than Info-Zip which is still used today. At the time he video was written Phil had just completed the DEFLATE algorithm which is STILL used today even outside of ZIP. The man was a genius.
@@ssl3546 But still, he was a royal class asshole, who built his program off of stolen code... Huh, he kinda sound a bit like Steve Jobs.
loving the updated look f your channel!
I had no idea that's what happened to Phil. It's a sad thing. I would have liked him to know how great I thought PKLITE was. It was a win-win. It would compress your executables and they not only took up less space, but they loaded faster, in spite of the fact they had to decompress during the load. RIP Phil Katz.
Oh man, what have you done. I had completely forgot about all the zip formats and now some memories about pkzip and pkarch are resurfacing from the corners of my memory.
6:05 "Getting the job done with the fewest number of instructions and therefore running time."
Fewer instructions does not necessary mean quicker execution time.
Due to instruction level parallelism and out-of-order execution, and speculative execution and a myriad of CPU tricks
Thanks for this video, I knew Phil socially at the time this all happened, and this is as I remember it. My only caveat is I don't consider the Hendersons to be what I'd call "reliable narrators", but that's not your fault, that's on them.
Somewhere there must exist text archives of the data compression and "bullroar" discussion boards from the old Exec-PC BBS, based in metro Milwaukee where we both lived but with a global userbase. Everything going through Phil's mind at the time, either with respect to his software or just his social interactions, would all be in those files. We'd listen to him get into the weeds of the technical aspects of data compression, and then we'd tease him about his celebrity crushes ... good times. If those could be found they would be a tremendous primary resource for any historians studying this stuff.
And thanks for the link to the BBS documentary, I've been meaning to watch that.
ARJ was also quite popular before ZIP took over
Arj was much later.
LZH was around in 88 and was popular, too. ARJ was mid 90s
Great video! Keeps them coming!
6:10 Peter: As a Minnesota Resident, I can't help but notice you/editor *misspelled Swissconsin's Milwaukee as "MILWAUKE"...* Sorry for coming off as a grammar N@zi...
All the love in the world ain't enough
It isn't misspelled, it just fades out a little quickly. Here's a snapshot I made as it started fading out. i.imgur.com/1j0tBSI.png
amazing videos! keep em coming!
Is it just me or does every single one of those clips from the documentary look as if it was filmed on a “Roseanne“ set from some parallel universe?
I studied Huffman coding and also image redundancy, and decided to try writing my own image compression format in QuickBasic. My goals were simple, to make something with comparable compression ratios to PNG, but to also make it reasonably fault tolerant as well.
I didn't use any standard ZIP libraries or any such thing, I actually wrote my own variation of a self adapting Huffman tree algorithm myself. Though through different iterations of my program I figured it was best to drop the self adapting nature of it as it more or less defeated the main goal of fault tolerance, so there's probably still some old commented code in it left out for the self adapting tree.
Each iteration of the program, I had to be especially careful that the encoder and the decoder were doing EXACTLY the same thing, as I found that it was all too easy for two equal weight tree branches to randomly pick either 1 or 0 depending on the exact technique I was testing at the time.
It was a neat project for whatever it was worth, but certainly wouldn't be ZLib compatible. Only my program knows what the hell it's doing ya know. I also learned some rather weird quirks of QuickBasic itself while making it, I had to make some special fix code in one of the routines that kept on corrupting one particular variable that wasn't even supposed to be written to on the faulty line.
Any which way, I managed to make it work, with differentials and an internal reset every 16 scanlines. If it found bad data anywhere within a 16 scanline section, it would simply substitute the differential from the last recognized good scanline and decode with that instead, up until the next reset scanline ya know.
It was pretty robust too, I could literally mangle the encoded file with a hex editor randomly and it would decode whatever was intact, with only minimal artifacts in damaged areas.
Anyways, thank you for coming to my Ted Talk, but yeah as neat as Huffman coding is, it's also a bit sensitive to the exact sort algorithm and techniques used in building the Huffman Tree.
Nerd, your stare during the intro unnerved the ever living hell out of me. Sip on a coffee or a cuppa tea next time, like it's an 1980's interview show...
What a non thing
Omg you actually covered it! Dude thank you.