Hi Moshix! Interesting observation -- yes, I can see how the idea of having those sets of multivalue fields to have complete "subrecords" embedded in them does share a kind of hierarchical approach. And now that you mention it, consider the history: The predecessor of Pick was implemented on an S/360 in 1965 to be a database for managing the parts inventory for a helicopter... and IMS was first built on an S/360 in 1966 to manage the inventory of parts in the BOM for the Saturn V rocket and Apollo spacecraft.
Indeed! You’re right. Didn’t see the immediate historical link between the two. Interestingly, all Pick books mention to not make the part number the ID, because part numbers tend to change. And DL/1 manuals say the thing…
Great video, thanks! I know jack about DBs (my focus are OSes themselves), so pls excuse my layman question: is there any (and is it possible at all to have) Multivalue DB that is also immutable?
There are none that I know of; I'd be surprised if there are. Is it possible? You know what they say... "it's just software, anything's possible!" :-) I can imagine you could implement a multivalue-type environment where the underlying system enforces immutability by simply not allowing you to edit or delete existing keys; or do something where if you edit a key, say key 1, it interally always and automatically versions keys, so if the existing key 1 was version 1, instead of changing it, you'd really be saving a new "key 1 version 2". The system could transparently return the latest version of a key when you request it, but have a way to explicitly request previous versions of a key if needed. Furthermore, an implementation of an immutable multivalue database system could automatically include an attribute that is the hash of the (new entry + previous entry's hash) to maintain the Merkel tree ensuring the integrity of the data.
@@MatthewMainframes Thanks for your kind answer! IDK know why exactly, but I’m thinking about ZFS snapshots now, and how little overhead is there even for a lot of data, and that this imaginary multival.immudb maybe can use similar tech? Maybe @moshixmainframechannel has some ideas, at the end of the day, thinking about his project immudb made me ask the question in the first place 😊
There wasn't a Y2K problem per se, but there was a day 10000 problem because Pick counted days from an arbitrary date, and when it went from 4 to 5 digits a lot of software blew up, or could.
Ah, yes. I had read something about that in my research preparing for the videos. At first it didn't make sense to me because the values are all stored as effectively unlimited-length strings in the database... but then I realized that it's probably the internal representation of the date in-memory in the kernel that only had 4 bytes reserved for it in some implementations.
@@MatthewMainframes Henry Eggers reminded me the issue wasn't with the database, but with one of the "conversions", specifically the iconv to convert the date from external representation, such as May 1, 1970 or such to an internal number. All dates are stored as numbers, so an integer sort on an attribute with a date will result in the date order for the sort / selection. All dates are encoded with "0' being 12/13/1967. The converter works surprisingly well. You have to account for hiccups in dates such as the Gregorian calendar adjustments, but I had subroutine which worked with number < 0 AD, so worked for such uses as historical documentation. Of course forward for such as insurance or policy ranges. Anyway the problem arose with a lot of code allowing only for 4 digits of precision internally for this field.
Yep, the Part 2 video coming out tomorrow will be using ScarletDME extensively :-) We're very fortunate that Ladybridge Systems did release a snapshot of the code under the GPL at one point in time, but yes, they weren't really doing to to try to build a real open source community around it. I think at the time even they described it as an experiment for people to be able to try new ideas and then give the new features back to Ladybridge for incorporation in the commercial version. I don't think that concept ever really went anywhere. But at least it did give us the foundation for free software multivalue system and the work that Gene and others have done on ScarletDME to bring it up into the 64-bit world is much appreciated.
![[049] Multivalue DB Part 2 - OpenQM / ScarletDME](http://i.ytimg.com/vi/6n0pZGwhMrY/mqdefault.jpg)
![[049] Multivalue DB Part 2 - OpenQM / ScarletDME](/img/tr.png)
![[050] Multivalue DB Part 3 - Programming](http://i.ytimg.com/vi/KrtD-hlXPOc/mqdefault.jpg)
Very helpful !! Thank you. Mutivalue databases are very similar to hierarchical databases like IBM’s IMS DL/1
Hi Moshix! Interesting observation -- yes, I can see how the idea of having those sets of multivalue fields to have complete "subrecords" embedded in them does share a kind of hierarchical approach.
And now that you mention it, consider the history: The predecessor of Pick was implemented on an S/360 in 1965 to be a database for managing the parts inventory for a helicopter... and IMS was first built on an S/360 in 1966 to manage the inventory of parts in the BOM for the Saturn V rocket and Apollo spacecraft.
Indeed! You’re right. Didn’t see the immediate historical link between the two. Interestingly, all Pick books mention to not make the part number the ID, because part numbers tend to change. And DL/1 manuals say the thing…
@@moshixmainframechannel & @MatthewMainframes You two guys are amazing! Thanks for sharing your knowledge, you guys rule! 👍
@@MatthewMainframes GIM was implemented by TRW and was marketed thru the 90s. It was the Don Nelson that Pick took and used to implement his system.
This is awesome! I work with Rocket Universe and I'm learning so much from you!! Thanks!
Glad the videos are helpful!
interesting. and completely new to me. :)
Awesome. I saw some red color in there! I assume it can do colored text? (The important stuff...) lol.
Great video, thanks! I know jack about DBs (my focus are OSes themselves), so pls excuse my layman question: is there any (and is it possible at all to have) Multivalue DB that is also immutable?
There are none that I know of; I'd be surprised if there are.
Is it possible? You know what they say... "it's just software, anything's possible!" :-)
I can imagine you could implement a multivalue-type environment where the underlying system enforces immutability by simply not allowing you to edit or delete existing keys; or do something where if you edit a key, say key 1, it interally always and automatically versions keys, so if the existing key 1 was version 1, instead of changing it, you'd really be saving a new "key 1 version 2". The system could transparently return the latest version of a key when you request it, but have a way to explicitly request previous versions of a key if needed. Furthermore, an implementation of an immutable multivalue database system could automatically include an attribute that is the hash of the (new entry + previous entry's hash) to maintain the Merkel tree ensuring the integrity of the data.
@@MatthewMainframes Thanks for your kind answer! IDK know why exactly, but I’m thinking about ZFS snapshots now, and how little overhead is there even for a lot of data, and that this imaginary multival.immudb maybe can use similar tech? Maybe @moshixmainframechannel has some ideas, at the end of the day, thinking about his project immudb made me ask the question in the first place 😊
There wasn't a Y2K problem per se, but there was a day 10000 problem because Pick counted days from an arbitrary date, and when it went from 4 to 5 digits a lot of software blew up, or could.
Ah, yes. I had read something about that in my research preparing for the videos. At first it didn't make sense to me because the values are all stored as effectively unlimited-length strings in the database... but then I realized that it's probably the internal representation of the date in-memory in the kernel that only had 4 bytes reserved for it in some implementations.
@@MatthewMainframes Henry Eggers reminded me the issue wasn't with the database, but with one of the "conversions", specifically the iconv to convert the date from external representation, such as May 1, 1970 or such to an internal number.
All dates are stored as numbers, so an integer sort on an attribute with a date will result in the date order for the sort / selection.
All dates are encoded with "0' being 12/13/1967. The converter works surprisingly well. You have to account for hiccups in dates such as the Gregorian calendar adjustments, but I had subroutine which worked with number < 0 AD, so worked for such uses as historical documentation.
Of course forward for such as insurance or policy ranges.
Anyway the problem arose with a lot of code allowing only for 4 digits of precision internally for this field.
ScarletDME is an opensource version of OpenQM which wasn't that friendly to open source models it claimed to be.
Yep, the Part 2 video coming out tomorrow will be using ScarletDME extensively :-)
We're very fortunate that Ladybridge Systems did release a snapshot of the code under the GPL at one point in time, but yes, they weren't really doing to to try to build a real open source community around it. I think at the time even they described it as an experiment for people to be able to try new ideas and then give the new features back to Ladybridge for incorporation in the commercial version. I don't think that concept ever really went anywhere. But at least it did give us the foundation for free software multivalue system and the work that Gene and others have done on ScarletDME to bring it up into the 64-bit world is much appreciated.
@@MatthewMainframes Have you shared these to the FB Pick group? Or do you mind if I share links?
@@jw0stephens I don't use Facebook but yes, feel free to share away!