While working for Undisclosed Big Old Transport Company as a "SQL developer," the oldest stored proc I had to touch was older than me. First written in 1989, last updated in 1997 by a guy three "generations" before me in that role
I'm assuming that the procedure wasn't bad if it lasted that long, but maybe it also forced some behaviors in the calling code that weren't so ideal? Interesting how long something can last but why mess with something that works? I'd hate to be you if you had messed with it, hahaha.
Never underestimate the value of DELETE ME in an item name. So many times has that saved me from having to spend 30 minutes digging to see if I still need a file or a configuration item or a whatever... past self has told me it's OK. I trust you past self. Thank you.
The big problem IMHO with old gigantic, monolithic projects is once you grow past certain size, it becomes impossible to grok the whole thing at once. This is where good design, architecture and people like Linus comes in. Good design keeps dependecies sane, individual parts grokkable and Linus keeps devs from undermining the design.
@@censoredeveryday3320 Yeah, what does he know, right? Having been a scientific programmer in C++, I can only applaud his decision. I see so many people without formal training, no knowledge of how to proof correctness in programming, and no habit of using patterns that allow the validation of the correctness of your program. And they're working on fricking kernels. Rust is a necessity for the future. Very few people are getting educated on C anymore, and self-taught people are not getting the formal CS training you need to do it competently on a large and sensitive codebase such as the Linux kernel.
The C language is not intended for writing large programs-according to the creators of the language. In a 1978 paper, Ken Thompson estimated that the UNIX kernel consisted of about 10,000 lines of C code (plus a small amount of assembly language). Other UNIX components were of comparable size; in another 1978 paper, Dennis Ritchie and colleagues estimated the size of the PDP-11 C compiler to be 9660 lines. By today's standards these are really small programs. They thought no one would write programs for 100,000 or even 1,000,000 lines of code in C-this is madness.
What you said, I did not know but have always suspected. The tool has to be correct for the scale of the project, but so does the design of how to use it. I'm not sure we have either for a 25 million line code base, but it's massively popular and reliable in production despite its nightmarish maintenance problems. Some day it has to become a Weekend At Bernie's affair though. Think of how fast it will burn through maintenance coders and all the while The They are discouraging kids from become coders because "AI is going to displace you and leave you homeless with no other skills".
Well, at that point 20MB wasn't a trivial amount of storage space to screw around with nor cheap, so going crazy with the LOC wasn't in their best interest. Have a feeling that C was very much designed around the storage and storage constrained programming paradigms of the time, rather than strictly not being for large programs. After all, logic is logic, and logic doesn't really care about how many LOC there are, just if the whole thing works or not.
What language has to do with the size and complexity of the project? Proper type definitions and module/artifact separation allows you to split a product of ANY complexity into a reasonable chunks (let's say under 10kloc). The problem is that such projects never have requirements written in advance and built up on top of poor foundations.
I think maintainability depends more on the quality of design and implementation then LOC or the language. I'm working on a database started somewhere around 1980. It has above 2 million LOC in C and we don't have such problems. We can maintain it and still bring in new features with only a few devs.
Yeah. I worked in code bases like that. Nobody understood how it all worked and it was all strongly coupled so each time you changed something you ran a huge suite of tests and see how much did fail. If 1k+ failed then it was a small mistake and fixing it was easy. If few failed you were ducked since it was a complex set of conditions to cause it.
"The fact thet this product even works is nothing sort of a miracle". This actually sums up nicely ANY modern CPU or GPU or ASIC, made up from billions of basic units - often integrated from different design houses running at 2GHz and up...
I worked at Oracle and on that code base for several years and yes it's all very fragile. Even the build system (called ADE) was this horrible Clearcase monster that used NFS to mount up different volumes to pull in various dependencies and other associated files. The bug tracking system is likely still the same and written in the 90s with web 1.0.
@@Blaisem not necessarily. E.g. he brings the q of macros with dozens of flags. It is indeed the case, there can be multiple level deep macros with non trivial conditional logic. But it's not hard to understand how they worked and most of them were very well documented. Sometimes the comments for a macro are hundreds of lines of well written docs. Also, you are developing a highly robust system with extremely complex logic across hundreds of weird features. In plain C. You need some kind of domain specific language on top of C, it makes your life easier not harder. That's just one example. Don't get me wrong, some code was extremely bad. But overall it was an o.k. experience.
@@kephir4eg great comments especially about the domain-specific language to control C with all its macros and flags. I suppose this gives Rust people some hope that they could do a better job but damn, 25 million lines, some very accomplished people have to ride herd over that.
Good story, and a typical programmer perspective. He should have mentioned that Oracle created the market for commercial relational databases, which replaced stuff even more crap. Oracle became ubiquitous and one of the most profitable pieces of software ever written. One might even conclude that code quality, as understood by programmers, has little to nothing to do with customers buying and using the code, and making a shitload of money from it.
That's really crazy. I've seen a lot of companies running on very bad code but still making a lot of money from being good at selling their shit, and a few companies with good software but bad sales and/or management that failed to become successful. I am always wondering if I'll ever find a company which is good at both.
@@Fanmade1b Oracle the second largest software company, after Microsoft, by market cap. The next one is SAP, also reported to have a giant crap codebase.
@@gregjor yeah. I've actually had to deal with code from all of them. SAP has a special place in my heart. One of the companies I worked for which actually had good code was developing an SAP and a PHP (Laravel) application in parallel, with both doing essentially the same, but with different target groups and implementations. One day at lunch, the SAP project managers were talking about a new feature that was requested by their customers. Their application basically matched ours in the affected place, so we (at this time PHP devs) asked them to explain that request to us. It was basically just one button to run a small series of actions which were always the same in a specific use case. We liked the idea, because even though our customers haven't actually asked for it, we could imagine that it could be useful for them and that page in the UI was pretty empty anyway, so that button wouldn't hurt even if it would've been rarely used. Since it was such a small task, we didn't even create a ticket for it and deployed it on the same afternoon (or the next day, I don't really remember exactly). The customers actually liked it and they started to use it right away. A few months later we heard one of the SAP guys mention that use case again and we asked what it was about. He then answered that they tried to implement that functionality, but it didn't work and they had to abandon that part of the project. But the customers already payd five figures for it and everybody was blaming each other. I don't know if it came to a lawsuit, but at that time it already sounded close. Well, even though our customers liked our PHP version a lot and they even wanted to have it for other facilities (it was about work safety, mainly used in the oil industry), our boss wanted to focus more on the SAP version, since there was more money in it. I quit a few months after he made that decision public and it took about two more years for the company to close down. So there we have a case of good software making some money and bad software making a lot of money at first and then none at all after it became a complete mess.
When I was in the Marines, from 1999-2007, Oracle hosted a company party in one of our hangers, and invited a few hundred of us to join. We drank all of the beer within the first 30 minutes, and they had to order more. They were pissed, and we didn't care. I've always hated working with Oracle products.
Bots seemingly switched to talking about bots to avoid being banned. Any video I click nowadays, the first comments are always about bots and there are no other comments
Say what you want about the company but the database itself is even constantly evolving. In the releases after 12.2 there were tons of new features all around; Oracle contributes to the SQL standard a lot lately. I guess they must have improved something. And the quality is crazy good, imagine pushing an update to critical systems of 90 somewhat percent of fortune 500 companies. Amazing piece of engineering.
The description in the post in the video and "Amazing piece of engineering" are not compatible. If I knew this about the codebase of a product I was critically dependent on, I wouldn't use it. Everyone bends over backwards coping because of organizational inertia: Oracle is trapped supporting DB, and clients are trapped using DB. So everyone acts like it's actually great and a miracle until a failure nukes 5% of the world economy.
@@nERVEcenter117 Well Oracle is not "trapped". Look at some features of the last 5 years: - new JSON format that easily beats mongo in performance - vector storage and indexes - invented new data access types for graph-relational and json-relational auto mapping - introduced domains - introduced a data descriptive language - AI utilities (chunking and other stuff lang chain would do) - Import AI models into the database and run them there - huge high availability improvements - Multi lingual engine to run JavaScript, Python or Java inside the database - leadership position of standardising JSON SQL functionalities, etc. You can argue whether you need these features but they definitely are not held-back fighting against their own source code. And look up how many times Oracle "nuked up" or anything like that, you will have a hard time finding anything. They are in the game too long for agile quality issues. Also just run on-prem and test patches?!
@@nERVEcenter117if you do use oracle extensively, you cannot not be amazed at what the product does. It is a great piece of engineering, probably unmatched.
2:00 Unix is not a single product either. It's a collection of a lot of different products and it runs on all kinds of different architectures (non-MMU-Unix anybody?). This modularity means some code has been changed A LOT, while some code hasn't changed (because there was no reason to change it). Oracle is a monolith and I wouldn't be surprised if there are some pieces of 1977 code still in the codebase, and you can't just rip them out and replace them with your own shitty code (Rust mentioned, well, I didn't mention Rust, but you know what I mean).
Not really. There is no one "Unix" ; and what is and isn't' a "Unix" depends on who you are asking to at any given moment. Depending on your definition of what a Unix is, it will or will not be a single product. For example...if you are talking about the kernel...it's a single product. If you are talking about GNU Linux...it isn't a single product. If you are talking about any of the mainline BSDs, it's a single product. Ad infinitum.
@@thecollector6746 technically, an OS has to be certified as Unix according to Single UNIX Specification (SUS). Therefore, Unix is a commercial product, since certification to use Unix trademark costs and then certified Unix OS is being sold. GNU's Not Unix, generally Linux Is Not UniX, Free/Net/OpenBSD are not as well. Those are Unix-like systems. Unix has a strict definition. macOS is a Unix, certified to Unix 03 specification. AFAIK only two chinese Linux distros ever had Unix certification: Inspur K-UX 2.0 & 3.0 and Huawei EulerOS 2.0.
From your paycheck, to all the essential public and private services you rely on, a large part of your life lives in Oracle databases that are secure, highly available, and have been upgraded and evolved countless times over those 40+ years. Sounds like this guy just doesn't really want to work in a huge, established C code base.
13:22 just wanna point out that you can do structural typing in C and is done for Unix socket API (like casting struct sockaddr_in* tosttruct sockaddr*), but it relies on potentially unsafe casts and you need to know what you're doing. This is allowed because the standard mandates that the order of struct members is the same as they are declared, and the standard mandates this for this exact use case too lol. in actual C code, void* is extremely rare, because it normally sucks. the only time I see void* is for some 3rd party callback API to pass the arguments to a callback function (along the lines of qsort in the stdlib)
LOL - As a dev, having to target multiple engines Oracle, DB2, MS SQL, My SQL, Informix Etc. Oracle is the most over rated, don't get me wrong Oracle is very good but its really about collecting a pay check by the ever shrinking and outsourced Oracle DB team as the total cost of Oracle is out of this world.
So Oracle started a month before I was born. And their code is like DNA. Optimised over generations and tested again the environment. It does not make sense anymore, but it works.
For a 50 year old product, this is not shocking. The coding styles alone have changed so much in that time. Half of those 12mloc are probably tests anyway. I wonder when was the last time they have done a complete rewrite.
I doubt there's any engineer on their team capable of leading a feat like that. How many decades would one have to work on that software to understand the nuances and use cases of hundreds of flags to know how you would need to simplify and/or remove any of them?
Controversy incoming. I love templating in Cpp. My last personal websocket server was made fully with templates. It’s not hard unless you do not give yourself clues. The error stack is occasionally useful but rarely rely on it because templates do make that extremely messy.
It's actually quite cool to be able to just give up on a piece of code. I've never ever done that in my life. I've always felt that if I would do that it would end my career immediately. Once, I cried on the toilet after trying to fix a bug for 2 weeks straight. I felt like if I can't fix that bug then I am just now worthy to be a software engineer. After crying I found new inspiration and fixed the bug that day :) That was over 20 years ago.
FYI it’s not this video why I’m subbing, but the one I watch about the ugly codebase plus knowing I’ve liked other videos of yours. Subbed within the first few minutes of this one.
Where I work you can't merge with a red x, it's actually kinda smart because it forces you to always run the tests beforehand so you know which ones to delete before making a pull request.
I love the comment about Linus. You really need a man like that stop the code crap. Sugar-coating is okay when you performance manage a secretary. But dependable code needs strict and serious controls.
You need direct honest feedback and can do it even with folks who missed something basic without being horrible. No one is perfect, don't expect perfection, but don't allow mistakes.
This is why you fix technical debt as you go along and it's super important to report and actually fix the debt. If they had clean codebase, all new features would be implemented in hours instead of weeks
Ive worked on a database software for about 1.5 years, the experience is similar except there are only a core set of tests required to pass for merging (about 20min to run in the code gate, and it's required to run a little bit larger set locally but I never did) while other tests are maintained by specific teams or testers
In case people don't know. If you got complex macro behavior clang and gcc have -E which just processes the macros and spits out what the file looks like to the actually compiler. Remember macros are literally text macros like you would find in a text editor. For Example, #include literally just places the contents of the other file as plain text in your file. Then processes the macros in that text as if they were in your file.
Your tester should be tested by the last good version of your tester. Sort of like the compiler written in the language it compiles. The best part about this is when you implement a language feature and then realize this language feature is the prefect feature for implementing it self. So you implement it twice, once to get it into the compiler and then a 2nd time to use it on itself. The tester version of this would be adding the feature to the tester and then using that feature to test the feature.
Interned on IBM DB2 10+ years ago. Was not much different with 30M loc. Takes weeks to consult expert developers from different teams to understand some magic feature in order to implement a minimal first version on some new feature. That was before DB2 went for column based. Can’t imagine what’s it like now.
Part of it is how it feels like to work on a big ORACLE & PL/SQL codebase running on an ORACLE DB. For example, there was this one big...Query. With subqueries inside it. I only could understand it after color-coding parts which worked together and staring at it afterwards for some hours
I read a long time ago that early versions of this DB were shitty as hell. In that time Oracle Inc. used IBM DB inside their infrastructure. Only when they migrated to their own Oracle DB, they realised why the clients weren't happy about their product. Through their own usage Oracle DB become a pretty good DB, so far as I recall the first usable version of DB was this one, where they implemented table triggers.
The thing I don't understand about projects like this enormous C Oracle project are... is there really a good excuse for that? What is keeping them from introducing encapsulation where they can start adding new code in a way that won't break existing code? Start migrating ridiculous code blob monoliths to something resembling reasonable separation of concerns or encapsulation or anything that would make it less insane? Is it purely incompetent leadership or are they somehow trapped in a 25 million line C hell?
The problem with something like Oracle is that you're plugging in a myriad of features to a relatively limited usage path. For example, I bet 12 million of those lines are just the computations for query planning and optimizing. The database has to make decisions about partitions, indexes, parallelization, and join algorithms just to start. Then it also needs to make advanced decisions like "do I apply a star transform here?" or "can I use a bloom filter to prune partitions?" Each of those having massive alternate paths to follow. All of that needs to pull together into a final query plan that the database is capable of executing. Is there any wonder there are thousands of flags? It's not necessarily bad architecture. It's just the complexity of the task being performed.
I don't know if I'd phrase it as "incompetence." Rearchitecting a 25 million code project _into something better_ would be nothing short of a super computer level of intellect.
Oracle still needs their flagship db to scale to insane sizes while maintaining at least reasonable performance, encapsulation and separation of concerns is in conflict with scalability and performance at fundamental level. Plus refactoring enormous C codebase including the insane amount of regression testing is incredibly costly, so it probably aint happening anytime soon.
Literally have kernel-level malware all the way down to the boot sequence, and its main exe is pulseaudio. Ion even have speakers on my monitor lmaooooo wtf I didnt expect to hear that
As much as Primeagen claims to be old, he's actually pretty damn young. He's among the whippersnappers who completely missed the Unix Wars. All they know is Darth Windows and the scrappy (but well funded by their own interests) Linux rebellion.
I kinda understand him. I started a job in company where whole system infrastructure has been built by one person for 30 years in C. You have to learn so much just to understand basic and transitioning to other languages or separating from it is a slow process. 😂
Next time we hear Oracle is rewriting everything in Rust. Well the same problem was with Apple's Final Cut code, so Apple just rewrote it. New Final Cut had way less features and people were upset but that was the way to go.
Word to the wise. Linus and Theo might have been able to get away with being a jerk to other maintainers in the interests of code quality, but most people aren't a quarter as smart as Linus/Theo, and should probably default to being pleasant.
Actual unix distributions (Solaris, AIX, zOS, HPUX) still are almost always commercial and closed source, BSDs and Darwin are the two big outliers in that. Prime just doesn’t remember the early UNIX wars nor the massive BSD legal shenanigans.
I think this is standard enterprise development if you work on anything that has a relevant user base and is 10+ years old. Our mutant is c++ we strive to keep it modern but my day to day work is similar fix bug/add feature, see 100 failed regressions, try to understand why, try to fix them, repeat
I was asked to clean such a code🤣, it was then i realized how out of depth i was. whenever we have to debug some issue, we cant rely on comment on what the stored proc is supposed to do. because it might be doing something completely different.
Is this related to the oracle like SGD that after retiring the OSGD I can no longer work over two screens when remotely accessing the drives of my organisation? It was such a breeze previously and suddenly I am restricted to just one screen when I need to work on like 8 windows.
2:30 the only way linus gets away with this type of communication is that he knows he's irrplaceable, if this was a corporate linus would be plastered by hr so fast no matter if he was right or wrong (unless he was politically connected) or they would have had workplace bullying lawsuites Which begs the question is the linux project as good as long as linus... lives? What happpens to it when he's gone?
well, oracle would have enough money to potentially just start a new decision, even with 10000 engineers, that copies the code base and refractors it into something maintainable.
While working for Undisclosed Big Old Transport Company as a "SQL developer," the oldest stored proc I had to touch was older than me. First written in 1989, last updated in 1997 by a guy three "generations" before me in that role
Could've been me ;-) My SQL career started in 1988.
I'm assuming that the procedure wasn't bad if it lasted that long, but maybe it also forced some behaviors in the calling code that weren't so ideal? Interesting how long something can last but why mess with something that works? I'd hate to be you if you had messed with it, hahaha.
You're seriously too young to touch that codebase. Wait until you've got your basic 25 years of professional experience
If you don't understand the word "generation" then you probably shouldn't be touching that codebase.
It’s very normal to have 30-40 year old code based in critical systems.
You mentioned the word ORACLE. A team of licensing experts will now audit your channel to determine how much money you owe Larry Ellison.
I think I heard mention of Java also. That'll up the cost for him
The same letter used in multiple sentences and in multiple lines and paragraphs? That's an extra seat. More money please.
😂 too real... Just dealt with this
Meanwhile, if you mention the word G*Og*e, your account gets shadowbanned.
And chat. After he acquisitions of course.
I love the "DELETE ME DELETE ME" in the description instead of the article source
Skill issue
Someone get Flip another coffee
Just a bug in the templating library Prime's brain uses. Or maybe the worker queue entry got lost. Or the code got commented out. A true mystery.
Its been confirmed. Prime is an AI 😂
Never underestimate the value of DELETE ME in an item name. So many times has that saved me from having to spend 30 minutes digging to see if I still need a file or a configuration item or a whatever... past self has told me it's OK. I trust you past self. Thank you.
"Psh, I could rewrite it in a weekend from scratch"
in scratch!
🤣 We get the joke
BLOW, Jonathan.
Someone clearly went to the Elon Musk school of programming.
@@ArrKayLondonElongate U
The big problem IMHO with old gigantic, monolithic projects is once you grow past certain size, it becomes impossible to grok the whole thing at once. This is where good design, architecture and people like Linus comes in. Good design keeps dependecies sane, individual parts grokkable and Linus keeps devs from undermining the design.
What Linus had bring? Hardly u can find linux kernel devs... that's why is rust crap in linux kernel...
Theo DeRadt is even better.
And then he allowed Rust into the kernel....
@@censoredeveryday3320 Yeah, what does he know, right?
Having been a scientific programmer in C++, I can only applaud his decision. I see so many people without formal training, no knowledge of how to proof correctness in programming, and no habit of using patterns that allow the validation of the correctness of your program. And they're working on fricking kernels.
Rust is a necessity for the future. Very few people are getting educated on C anymore, and self-taught people are not getting the formal CS training you need to do it competently on a large and sensitive codebase such as the Linux kernel.
The C language is not intended for writing large programs-according to the creators of the language. In a 1978 paper, Ken Thompson estimated that the UNIX kernel consisted of about 10,000 lines of C code (plus a small amount of assembly language). Other UNIX components were of comparable size; in another 1978 paper, Dennis Ritchie and colleagues estimated the size of the PDP-11 C compiler to be 9660 lines. By today's standards these are really small programs. They thought no one would write programs for 100,000 or even 1,000,000 lines of code in C-this is madness.
What you said, I did not know but have always suspected. The tool has to be correct for the scale of the project, but so does the design of how to use it. I'm not sure we have either for a 25 million line code base, but it's massively popular and reliable in production despite its nightmarish maintenance problems. Some day it has to become a Weekend At Bernie's affair though. Think of how fast it will burn through maintenance coders and all the while The They are discouraging kids from become coders because "AI is going to displace you and leave you homeless with no other skills".
Well, at that point 20MB wasn't a trivial amount of storage space to screw around with nor cheap, so going crazy with the LOC wasn't in their best interest. Have a feeling that C was very much designed around the storage and storage constrained programming paradigms of the time, rather than strictly not being for large programs. After all, logic is logic, and logic doesn't really care about how many LOC there are, just if the whole thing works or not.
What language has to do with the size and complexity of the project? Proper type definitions and module/artifact separation allows you to split a product of ANY complexity into a reasonable chunks (let's say under 10kloc). The problem is that such projects never have requirements written in advance and built up on top of poor foundations.
@@ivanjermakov, you are saying it like generations of programmers had no clue.
I think maintainability depends more on the quality of design and implementation then LOC or the language. I'm working on a database started somewhere around 1980. It has above 2 million LOC in C and we don't have such problems. We can maintain it and still bring in new features with only a few devs.
these bot comments are blazingly fast
Beep boop schizo man!!!!!!
You are blazingly fast 🥔
@@InvalidPersistentName 🥔🥔🥔
They’re written in Rust
@@andguythat's not even funny as rust isn't really as fast as most of its competition. But cultists be culting in the comment section.
Yeah. I worked in code bases like that. Nobody understood how it all worked and it was all strongly coupled so each time you changed something you ran a huge suite of tests and see how much did fail. If 1k+ failed then it was a small mistake and fixing it was easy. If few failed you were ducked since it was a complex set of conditions to cause it.
"The fact thet this product even works is nothing sort of a miracle". This actually sums up nicely ANY modern CPU or GPU or ASIC, made up from billions of basic units - often integrated from different design houses running at 2GHz and up...
I worked at Oracle and on that code base for several years and yes it's all very fragile. Even the build system (called ADE) was this horrible Clearcase monster that used NFS to mount up different volumes to pull in various dependencies and other associated files. The bug tracking system is likely still the same and written in the 90s with web 1.0.
I worked on that code for 6 years. It's not half as bad as it looks. One of the better things I worked on in my career.
Was the article dishonest then?
@@Blaisem not necessarily. E.g. he brings the q of macros with dozens of flags. It is indeed the case, there can be multiple level deep macros with non trivial conditional logic. But it's not hard to understand how they worked and most of them were very well documented. Sometimes the comments for a macro are hundreds of lines of well written docs. Also, you are developing a highly robust system with extremely complex logic across hundreds of weird features. In plain C. You need some kind of domain specific language on top of C, it makes your life easier not harder. That's just one example. Don't get me wrong, some code was extremely bad. But overall it was an o.k. experience.
@@kephir4eg great comments especially about the domain-specific language to control C with all its macros and flags. I suppose this gives Rust people some hope that they could do a better job but damn, 25 million lines, some very accomplished people have to ride herd over that.
Good story, and a typical programmer perspective. He should have mentioned that Oracle created the market for commercial relational databases, which replaced stuff even more crap. Oracle became ubiquitous and one of the most profitable pieces of software ever written. One might even conclude that code quality, as understood by programmers, has little to nothing to do with customers buying and using the code, and making a shitload of money from it.
That's really crazy. I've seen a lot of companies running on very bad code but still making a lot of money from being good at selling their shit, and a few companies with good software but bad sales and/or management that failed to become successful.
I am always wondering if I'll ever find a company which is good at both.
@@Fanmade1b Oracle the second largest software company, after Microsoft, by market cap. The next one is SAP, also reported to have a giant crap codebase.
@@gregjor yeah. I've actually had to deal with code from all of them.
SAP has a special place in my heart.
One of the companies I worked for which actually had good code was developing an SAP and a PHP (Laravel) application in parallel, with both doing essentially the same, but with different target groups and implementations.
One day at lunch, the SAP project managers were talking about a new feature that was requested by their customers. Their application basically matched ours in the affected place, so we (at this time PHP devs) asked them to explain that request to us.
It was basically just one button to run a small series of actions which were always the same in a specific use case.
We liked the idea, because even though our customers haven't actually asked for it, we could imagine that it could be useful for them and that page in the UI was pretty empty anyway, so that button wouldn't hurt even if it would've been rarely used.
Since it was such a small task, we didn't even create a ticket for it and deployed it on the same afternoon (or the next day, I don't really remember exactly).
The customers actually liked it and they started to use it right away.
A few months later we heard one of the SAP guys mention that use case again and we asked what it was about. He then answered that they tried to implement that functionality, but it didn't work and they had to abandon that part of the project. But the customers already payd five figures for it and everybody was blaming each other. I don't know if it came to a lawsuit, but at that time it already sounded close.
Well, even though our customers liked our PHP version a lot and they even wanted to have it for other facilities (it was about work safety, mainly used in the oil industry), our boss wanted to focus more on the SAP version, since there was more money in it.
I quit a few months after he made that decision public and it took about two more years for the company to close down.
So there we have a case of good software making some money and bad software making a lot of money at first and then none at all after it became a complete mess.
Apple, I’d assume
When I was in the Marines, from 1999-2007, Oracle hosted a company party in one of our hangers, and invited a few hundred of us to join.
We drank all of the beer within the first 30 minutes, and they had to order more. They were pissed, and we didn't care. I've always hated working with Oracle products.
Just sounds like you're lousy drunks
Isn't that just your fault💀💀
@@69k_gold, what's my fault? That we drank all the beer, and didn't care because fuck Oracle?
I hope at least you like their beers
What did they expect hundreds of Marines to do? NOT drink the free beer? Are they stupid?
Still better than being a frontend dev.
So true
it's not even a job frontend dev
This is gonna be good. The bots seem to love it.
Bots seemingly switched to talking about bots to avoid being banned. Any video I click nowadays, the first comments are always about bots and there are no other comments
Say what you want about the company but the database itself is even constantly evolving. In the releases after 12.2 there were tons of new features all around; Oracle contributes to the SQL standard a lot lately. I guess they must have improved something.
And the quality is crazy good, imagine pushing an update to critical systems of 90 somewhat percent of fortune 500 companies.
Amazing piece of engineering.
The description in the post in the video and "Amazing piece of engineering" are not compatible. If I knew this about the codebase of a product I was critically dependent on, I wouldn't use it. Everyone bends over backwards coping because of organizational inertia: Oracle is trapped supporting DB, and clients are trapped using DB. So everyone acts like it's actually great and a miracle until a failure nukes 5% of the world economy.
@@nERVEcenter117 Well Oracle is not "trapped". Look at some features of the last 5 years:
- new JSON format that easily beats mongo in performance
- vector storage and indexes
- invented new data access types for graph-relational and json-relational auto mapping
- introduced domains
- introduced a data descriptive language
- AI utilities (chunking and other stuff lang chain would do)
- Import AI models into the database and run them there
- huge high availability improvements
- Multi lingual engine to run JavaScript, Python or Java inside the database
- leadership position of standardising JSON SQL functionalities, etc.
You can argue whether you need these features but they definitely are not held-back fighting against their own source code.
And look up how many times Oracle "nuked up" or anything like that, you will have a hard time finding anything. They are in the game too long for agile quality issues. Also just run on-prem and test patches?!
Oracle would be more impressive if Postgres didn't exist.
@@nERVEcenter117if you do use oracle extensively, you cannot not be amazed at what the product does. It is a great piece of engineering, probably unmatched.
2:00 Unix is not a single product either. It's a collection of a lot of different products and it runs on all kinds of different architectures (non-MMU-Unix anybody?). This modularity means some code has been changed A LOT, while some code hasn't changed (because there was no reason to change it). Oracle is a monolith and I wouldn't be surprised if there are some pieces of 1977 code still in the codebase, and you can't just rip them out and replace them with your own shitty code (Rust mentioned, well, I didn't mention Rust, but you know what I mean).
Not really. There is no one "Unix" ; and what is and isn't' a "Unix" depends on who you are asking to at any given moment. Depending on your definition of what a Unix is, it will or will not be a single product. For example...if you are talking about the kernel...it's a single product. If you are talking about GNU Linux...it isn't a single product. If you are talking about any of the mainline BSDs, it's a single product. Ad infinitum.
@@thecollector6746 technically, an OS has to be certified as Unix according to Single UNIX Specification (SUS). Therefore, Unix is a commercial product, since certification to use Unix trademark costs and then certified Unix OS is being sold. GNU's Not Unix, generally Linux Is Not UniX, Free/Net/OpenBSD are not as well. Those are Unix-like systems. Unix has a strict definition. macOS is a Unix, certified to Unix 03 specification. AFAIK only two chinese Linux distros ever had Unix certification: Inspur K-UX 2.0 & 3.0 and Huawei EulerOS 2.0.
From your paycheck, to all the essential public and private services you rely on, a large part of your life lives in Oracle databases that are secure, highly available, and have been upgraded and evolved countless times over those 40+ years. Sounds like this guy just doesn't really want to work in a huge, established C code base.
13:22 just wanna point out that you can do structural typing in C and is done for Unix socket API (like casting struct sockaddr_in* tosttruct sockaddr*), but it relies on potentially unsafe casts and you need to know what you're doing. This is allowed because the standard mandates that the order of struct members is the same as they are declared, and the standard mandates this for this exact use case too lol.
in actual C code, void* is extremely rare, because it normally sucks. the only time I see void* is for some 3rd party callback API to pass the arguments to a callback function (along the lines of qsort in the stdlib)
LOL - As a dev, having to target multiple engines Oracle, DB2, MS SQL, My SQL, Informix Etc. Oracle is the most over rated, don't get me wrong Oracle is very good but its really about collecting a pay check by the ever shrinking and outsourced Oracle DB team as the total cost of Oracle is out of this world.
The best comment I put in the code: "I am sorry."
So Oracle started a month before I was born. And their code is like DNA. Optimised over generations and tested again the environment. It does not make sense anymore, but it works.
For a 50 year old product, this is not shocking. The coding styles alone have changed so much in that time. Half of those 12mloc are probably tests anyway. I wonder when was the last time they have done a complete rewrite.
I doubt there's any engineer on their team capable of leading a feat like that. How many decades would one have to work on that software to understand the nuances and use cases of hundreds of flags to know how you would need to simplify and/or remove any of them?
I love Linus' unhingedness, God bless his soul lol
There must be a lot of requirements after almost 50 years of product history.
Controversy incoming. I love templating in Cpp. My last personal websocket server was made fully with templates. It’s not hard unless you do not give yourself clues. The error stack is occasionally useful but rarely rely on it because templates do make that extremely messy.
Oracle 23ai was released as stable just recently. And it's still amazing that it's that stable 😂
I worked at Oracle in 2002. When I was there it took 12hrs to compile.
It's actually quite cool to be able to just give up on a piece of code. I've never ever done that in my life. I've always felt that if I would do that it would end my career immediately. Once, I cried on the toilet after trying to fix a bug for 2 weeks straight. I felt like if I can't fix that bug then I am just now worthy to be a software engineer. After crying I found new inspiration and fixed the bug that day :) That was over 20 years ago.
One trick I've used when there is complex preprocessor code that I can't tell what is going on is run the processor and see the code it generates.
I worked 25 yrs oracle dba and application development (from 1983). Now it seems to haved morphed into the inflexible Sap mode. No more fun user exits
I wanna wish you very luck programmers working on this legacy code in the year 2077!
This is excellent content. Idk why I didn’t sub sooner. I need more entertaining and productive coding content in my life
FYI it’s not this video why I’m subbing, but the one I watch about the ugly codebase plus knowing I’ve liked other videos of yours. Subbed within the first few minutes of this one.
Where I work you can't merge with a red x, it's actually kinda smart because it forces you to always run the tests beforehand so you know which ones to delete before making a pull request.
To be fair, "Do not break user space" is like the first commandment of Kernel maintainers, so it's kinda understandable Linux flipped.
I love the comment about Linus. You really need a man like that stop the code crap. Sugar-coating is okay when you performance manage a secretary. But dependable code needs strict and serious controls.
You need direct honest feedback and can do it even with folks who missed something basic without being horrible. No one is perfect, don't expect perfection, but don't allow mistakes.
Linus defenestrating people is a funny term. He is "throwing them out of windows", so... into Linux?
This is why you fix technical debt as you go along and it's super important to report and actually fix the debt. If they had clean codebase, all new features would be implemented in hours instead of weeks
Ive worked on a database software for about 1.5 years, the experience is similar except there are only a core set of tests required to pass for merging (about 20min to run in the code gate, and it's required to run a little bit larger set locally but I never did) while other tests are maintained by specific teams or testers
Googled what defenestrate meant and here's the first definition:
1.
RARE
throw (someone) out of a window.
In case people don't know. If you got complex macro behavior clang and gcc have -E which just processes the macros and spits out what the file looks like to the actually compiler. Remember macros are literally text macros like you would find in a text editor. For Example, #include literally just places the contents of the other file as plain text in your file. Then processes the macros in that text as if they were in your file.
Your tester should be tested by the last good version of your tester. Sort of like the compiler written in the language it compiles. The best part about this is when you implement a language feature and then realize this language feature is the prefect feature for implementing it self. So you implement it twice, once to get it into the compiler and then a 2nd time to use it on itself. The tester version of this would be adding the feature to the tester and then using that feature to test the feature.
whaaaat? HP-UX, AIX, (...), are no commercial products????!!!!!
Interned on IBM DB2 10+ years ago. Was not much different with 30M loc. Takes weeks to consult expert developers from different teams to understand some magic feature in order to implement a minimal first version on some new feature. That was before DB2 went for column based. Can’t imagine what’s it like now.
This seems just like a giant vent, but no real problem solving. Good luck to anyone in software development.
Part of it is how it feels like to work on a big ORACLE & PL/SQL codebase running on an ORACLE DB. For example, there was this one big...Query. With subqueries inside it. I only could understand it after color-coding parts which worked together and staring at it afterwards for some hours
Shoud've programmed Oracle DB in JDSL 🤷
thumbnail - POLAND MOUNTAIN
I read a long time ago that early versions of this DB were shitty as hell. In that time Oracle Inc. used IBM DB inside their infrastructure. Only when they migrated to their own Oracle DB, they realised why the clients weren't happy about their product. Through their own usage Oracle DB become a pretty good DB, so far as I recall the first usable version of DB was this one, where they implemented table triggers.
The thing I don't understand about projects like this enormous C Oracle project are... is there really a good excuse for that? What is keeping them from introducing encapsulation where they can start adding new code in a way that won't break existing code? Start migrating ridiculous code blob monoliths to something resembling reasonable separation of concerns or encapsulation or anything that would make it less insane? Is it purely incompetent leadership or are they somehow trapped in a 25 million line C hell?
The problem with something like Oracle is that you're plugging in a myriad of features to a relatively limited usage path. For example, I bet 12 million of those lines are just the computations for query planning and optimizing. The database has to make decisions about partitions, indexes, parallelization, and join algorithms just to start. Then it also needs to make advanced decisions like "do I apply a star transform here?" or "can I use a bloom filter to prune partitions?" Each of those having massive alternate paths to follow.
All of that needs to pull together into a final query plan that the database is capable of executing. Is there any wonder there are thousands of flags? It's not necessarily bad architecture. It's just the complexity of the task being performed.
I don't know if I'd phrase it as "incompetence." Rearchitecting a 25 million code project _into something better_ would be nothing short of a super computer level of intellect.
Oracle still needs their flagship db to scale to insane sizes while maintaining at least reasonable performance, encapsulation and separation of concerns is in conflict with scalability and performance at fundamental level. Plus refactoring enormous C codebase including the insane amount of regression testing is incredibly costly, so it probably aint happening anytime soon.
Flag Abuse + Macro Abuse = OOP Inheritance Abuse? A lot of nice little tools explode when we scale with them.
wait. so oracle was available to the mini computers? can you imagine running a server on one of those s-100 bus systems...
Having used oracle software, I’m not surprised.
Literally have kernel-level malware all the way down to the boot sequence, and its main exe is pulseaudio. Ion even have speakers on my monitor lmaooooo wtf I didnt expect to hear that
Im convinced Larry Ellison life force is sustained by the tears and misery of developers forced to interact with oracle products
What do you mean Unix is not a commercial product?
As much as Primeagen claims to be old, he's actually pretty damn young. He's among the whippersnappers who completely missed the Unix Wars. All they know is Darth Windows and the scrappy (but well funded by their own interests) Linux rebellion.
I kinda understand him. I started a job in company where whole system infrastructure has been built by one person for 30 years in C. You have to learn so much just to understand basic and transitioning to other languages or separating from it is a slow process. 😂
What are you talking about? The Linux kernel is 30 million lines of code.
Isn't most of that just drivers?
@@saharatulyes, the core is pretty small in relation
Why all great products started before Uncle Bob manifestos?
I still remember working on a legacy on-prem product which was on OSGi. What the heck... is OSGi was my first reaction.
Thor said "Caboose" is something you did prime.
If you're an Oracle employee, you can't say publicly that there's a bug, or they'll fire you.
This explains the price of Oracle DB, which is a great product btw.
Can't even imagine what it would be like to work on such a large codebase.
Next time we hear Oracle is rewriting everything in Rust.
Well the same problem was with Apple's Final Cut code, so Apple just rewrote it. New Final Cut had way less features and people were upset but that was the way to go.
It's paid for Larry's yacht at least...
Word to the wise. Linus and Theo might have been able to get away with being a jerk to other maintainers in the interests of code quality, but most people aren't a quarter as smart as Linus/Theo, and should probably default to being pleasant.
The funny thing is rhat Unix was a commercial product during its early life through the 70s to 80s.
Actual unix distributions (Solaris, AIX, zOS, HPUX) still are almost always commercial and closed source, BSDs and Darwin are the two big outliers in that. Prime just doesn’t remember the early UNIX wars nor the massive BSD legal shenanigans.
Oracles first major “enterprise” deals were with the CIA 😉
I think this is standard enterprise development if you work on anything that has a relevant user base and is 10+ years old. Our mutant is c++ we strive to keep it modern but my day to day work is similar fix bug/add feature, see 100 failed regressions, try to understand why, try to fix them, repeat
I was asked to clean such a code🤣, it was then i realized how out of depth i was.
whenever we have to debug some issue, we cant rely on comment on what the stored proc is supposed to do. because it might be doing something completely different.
2:00 I agree with him on this one. I rather have a mean lead maintainer than whatever this Oracle C code base abomination is.
Mauro - it's probably pronounced MOW-RO - think in these subunits: MA U RO
Whats wrong with pulseaudio? Ive used it for tons of stuff without any issue. Added USB audio playing to my router just fine with it.
Is this related to the oracle like SGD that after retiring the OSGD I can no longer work over two screens when remotely accessing the drives of my organisation? It was such a breeze previously and suddenly I am restricted to just one screen when I need to work on like 8 windows.
2:30 the only way linus gets away with this type of communication is that he knows he's irrplaceable, if this was a corporate linus would be plastered by hr so fast no matter if he was right or wrong (unless he was politically connected) or they would have had workplace bullying lawsuites
Which begs the question is the linux project as good as long as linus... lives? What happpens to it when he's gone?
well, oracle would have enough money to potentially just start a new decision, even with 10000 engineers, that copies the code base and refractors it into something maintainable.
How long does it take to fix a bug and push a change to CrowdStrike?
What happens to the kernel when Linus Dies? Will there be someone prevent the kernel going to shit?
Linus is right -> do not break your users' operations (your customers).
If I ever worked for Oracle I will just pretend I'm an techpriest from 40k
Makes me feel good about my current legacy code base…
So ummm. What’s the longest line of Assembly Language today?
I thought he was talking about LInus tech tips. lol
They clearly need a rewrite or else they'll suffice themselves into oblivion.
the red check in CI would like to talk to the boy who cried wolf
Why don't you ask AI - when Oracle DB was created? It's faster than search.
Linus is still somewhat of a teddy bear compared to Theo DeRaat (OpenBSD)
At least there have test cases
While it may the be the most unmaintainable code base, at least it can support more workloads and critical feature set then AWS Aurora.
Multiply issues by a 1000 and you get Windows
Imagine doing a rewrite of that stuff
When did "the reason is is X" become grammatically correct?
If ONLY they use Agile Development from the start! 🤣🤣🤣
The crazy thing is the oracle devs think the mysql code is bad.
As a database developer I can say db developer are the real deal.
I refuse to believe that Oracle has got any tests in their code bases.
How can I find a link to the original blog post?
Oh yeah this was a great post.
lawnmower can't steer a ship