WOAH! I've been thinking so hard trying to figure out which one I should implement when I completely forgot that there is the option of both....Thank you for helping me see that.
The example at the end for me was a great explanation. Finally understand a simplistic implication. To further on this: a user is logged in, the cart is all products being purchased or a favourites section ,which will all be persisted to storage (HD). The transient (or constantly changing) data could be the session data (login duration day, week, month stats), product browsing history, of that user, but wouldn't this be also stored at a later point? The cart and favourites data is used by both user and supplier, but the session data is usually only used by supplier for analytics, targeted promotions, predictive data modelling (other product suggestions etc) etc...From the users POV might not be large but from a Product POV could be huge.
@@fallinginthed33pyou build db data storage plan so you don't have to worry about later , you got to solve new problems that comes not stuck in old mistakes too long
As @racc1337 mentioned... You shoot him writing thru a transparent surface and then mirror the video before posting... So he's actually right-handed (it doesn't help that he's wearing his watch on his writing hand just to mess with your head further :) and the write hand is the right hand 😆
As racc1377 said, a large glass pane then horizontally reversed in post, computer images slid in too. I've seen this done before but this one is done very well indeed. ..and very well delivered too. Excellent content imo :D ...and I learned from it too which is really the acid test!
Wow interesting last point! I remember facing dev situations and feeling like a limitation of some sort, but perhaps taking a NoSQL approach would have lifted that. Quite exciting. There's a couple of teasers in here that i'd like more information on. Like what is vertical scaling, what is horizontal scaling? And what accessing NoSQL looks like? But i definitely feel certain of the mode of SQL and the mode of NoSQL enough to experiment with NoSQL.
A lot of people think NOSQL is a new concept. NOSQL predates Relational Databases by decades (1960s) and was called Hierarchical Databases. When Edgar F. Codd invented Relational Databases (early 1970s), computers weren't powerful enough to support them and it wasn't until 1979 the Oracle truly commercialized Relational Databases. Just like today Relational Databases need more computing power than the simpler NOSQL Database. This is because the main drawcard of RDBMs needs it, "Data Relationships". Without the relationships that SQL provides, one has to write the joins into their own code whether it be Python, Ruby, C++ , Go etc or rely on the same or similar SQL syntax to do perform joins (Hadoop, Splunk come to mind). SQL reduced the proliferation of code needed to do joins. Imagine having downstream systems using multiple programming languages, each one having to munge their own joins. RDBMs also provide other things such as ACID, which ensures your data is in a consistent state even if there are conflicts. Frankly once the data is normalised it easier to work with SQL plus ORMs.
This could have been easier to follow - in some parts you are using IT buzzwords to describe an IT buzzword concept - which is even more confusing. It would be a lot simpler to understand if you could use plain English and examples - so, for example, what does horizontal and vertical scaling actually mean, what does 'dynamic' as point 4 on the No SQL side of your diagram actually mean. In the first few sentences of the presentation you mention that it might be surprising that people ask questions about these differences between technologies, or that there is confusion about the differences, but in my mind it's not surprising at all when the meaning of the buzzwords/language used to actually explain the differences are unclear.
Yeah, I already read the quick list of differences and I was hoping, like you have said, that this video would explain those differences. I got nothing out of this video.
@@firstcommenter202 That sounds like an excellent follow up video. I will take note of this great suggestion the next time we produce more videos on this topic.
@dovasd Thanks for your comments. Yes we intended to "scratch" the surface with this video and dig a little deeper on other topics. There are certainly many options nowadays for either NoSQL & SQL Databases. We will have some profiles coming up soon on different types.
I am curious about "Vertical vs. Horizontal Scaling" 1. Why can I not scale SQL Databases Horizontally? You *could* use sharding and read replicas improve scaling needs - sure, it requires work, but SQL is constantly improving - isn't that right? 2. What I can see is - you can easily make changes in NoSQL e.g. unstructured - which is quite costly in SQL
The issue with relational and horizontal scaling is yes you could spread it via sharding but the issue comes in how you can keep the read and writes consistent across them all. You can split a database into multiple smaller databases but then that just leads to higher maintenance costs. You could also have one just for writes and several others as read only databases but again this would need managing with the services accessing the databases. With non relational yes you can throw into them what you want but unless you “lock down” a schema for the data then you can’t get consistent results as each insert is a new entry rather than an “upsert”. I guess each have their pros and cons but you would choose whichever is best for your end system which you would know once you’ve done the correct analysis
The visual presentation of this video is superb, but it's really let down by the audio, there's a very slight reverb/echo effect that lands perfectly in the uncanny valley of normal speech.
I think addressing performance of a query should be a major factor. I'm SQL trained and do not know how one performs extraction of data from a no-sql data pool other than perhaps by the Linux commands "awk" and "grep". Is there a performance savings by mapping everything into SQL vs. leaving stuff in a blob?
The problem I see with your explanation is that SQL is just a language. There are an ever increasing number of options to scale out RDBMS systems. Also it’s not like, for instance, Postgres is bad at handling JSON data with JSONB. There are some attributes that you are likely to always want to have, and some that may change per instance. So something like a table with a JSONB column is a nice approach. However, I wouldn’t go around installing Postgres on loads of mobile phones for kicks. It’s just not reasonable to use that amount of storage space. Just a thought.
how do they write the things in front of them? It means they are writing the other way around right? or what? I have been trying to watch the tutorial but my head is stuck at that
As a coder.. Code itself can be referred as a jungle that its own writer can navigate.. That is why convention is created, manual is needed, etc With unstructured nature of nosql, we can double that. Did wrong in here??
It works by giving you fast access to the data through memory access. There may be some files that are written to disk but that is not its main access method. It is more a function of how the software application works. Let me know if I helped answer your question or have more questions.
Another "ancient" computing technology has been resurrected with a new name ! Ack in the 1960s, IBM invented "indexed sequential access method" (ISAM) of accessing data (not 100% certain of the name IBM used or the timing). By the late 1970s, Digital Equipment Corporation (DEC) had duplicated the technology. I am not familiar with IBMs implementation so the rest of my comments will apply to DEC. It is important to understand that the underlying storage technology is one aspect and HOW THE DATA WAS ACCESSES is totally another ! On DEC machines, there as an extra "layer" added between the data and the application such that the data could be accessed via the primary or SECONDARY keys or sequentially ! While the primary key had to be unique, secondary keys did NOT have to be unique !! The biggest problem was that no "high level languages" (Fortran, COBOL, C) supported this concept. "Extensions" were added to each to support look up by primary or secondary keys. As mentioned, there was no "standard" query language. DEC invented Datatrieve. It was a bit quirky but it included a "report writer" similar in concept to IBM RPG which was immensely popular at the time. The basic query could be written in Datatrieve and that could be linked into another custom application. Sadly for DEC, ISAM never caught on in large volume. It also could not scale horizontally without a lot of additional programming. This was probably because it required proprietary language extensions. Building "forms" for data entry was a sperate product or required a lot of programming. As the world moved to "windows" based applications, DEC stayed in "character cell" interfaces as did IBM for much too long. Microsoft Access was one of the biggest "nails in the coffin" for ISAM. IBM and DEC had proprietary relational databases, but it was too Little to late.
Multiple questions come to my mind after looking at this very good video: thank you for that. - To come back to your illustration with an e-commerce website: don’t you need a structured DB (SQL) to identify the products hosted in a Non-SQL DB that you will present to the consumer? - Another comment: as No-SQL is in memory it limits its size where with SQL you can have huge size of DB. Correct? - Also I’m curious about performances: is there a kind of index with No-SQL DB? As I see it and you mention it at the end of the video, the combination of the 2 technologies is the key.
To come back to your illustration with an e-commerce website: don’t you need a structured DB (SQL) to identify the products hosted in a Non-SQL DB that you will present to the consumer? [ Possibly, but I have run complete backends off a NoSQL DB like MongoDB.. of course I had to make accomodations to ensure the data was in tact and standard ]
as No-SQL is in memory it limits its size where with SQL you can have huge size of DB. Correct? [ Correct, for SQL DB it will persist to disk. You are dependent on loading in memory for NoSQL DBs. This principle could also guide your balance of usage in your architecture. I could potentially host MYSQL on a VM Image, but something like MongoDB in the cloud is charged by the amount of memory used. The larger your JSON Docs, the more they consume. ]
is there a kind of index with No-SQL DB? [ Yes, you can create indexes in a No SQL DBs. There are so many options to choose from so if that is your guiding principle, you can let that guide which one you decide to go for. I know CouchDB has some great features that help with indexes and has some great abilities to facilitate offline sync use cases through it's native architecture with revisions of Documents just as an example of when you might let features guide you down a path of which DB in the category to use ]
Not always, it really depends on the needs of your application and the data will be queried. For example, if you are working with interconnected data, SQL will be acceptable, but Graph models are better. Another case, if you have to use many-to-many queries, then a relation model (read SQL is better). However, for certain applications, e.g., retrieving document for a web-site, a NoSQL database is better and if affords good locality of data.
Someone ignored the PRO/CONS and decided NoSQL-databases are better than SQL-databases (Which are tested and proven to be good enough for the most situations the past 30, 40+ years?). Poor company, who got a dev at the end, who chooses a database which are not good enough for the current problem. By the way good luck when your application is heavly relationed based, NoSQL would be a overkill.
Pretty poorly explained. You don't really seem to fully grasp what it is that you are talking about. As some of the other comments point out, you use buzzwords to explain buzzwords.
WOAH! I've been thinking so hard trying to figure out which one I should implement when I completely forgot that there is the option of both....Thank you for helping me see that.
The example at the end for me was a great explanation. Finally understand a simplistic implication. To further on this: a user is logged in, the cart is all products being purchased or a favourites section ,which will all be persisted to storage (HD). The transient (or constantly changing) data could be the session data (login duration day, week, month stats), product browsing history, of that user, but wouldn't this be also stored at a later point? The cart and favourites data is used by both user and supplier, but the session data is usually only used by supplier for analytics, targeted promotions, predictive data modelling (other product suggestions etc) etc...From the users POV might not be large but from a Product POV could be huge.
Each has its strengths and weaknesses and smart decision would be to employ both systems together with knowing where to store what....
SQL
- More pre-planning required
- Vertical scaling
- Better for multi-row transactions
NoSQL
- Horizontal scaling
- Unstructured
JSON it and worry about it later
@@fallinginthed33pyou build db data storage plan so you don't have to worry about later , you got to solve new problems that comes not stuck in old mistakes too long
I got lost in the thoughts about - how did you make this video ? how are you writing !!
Probably a glass pane and the image is mirrored in post
Mild blacklight, glass and some post when he makes mistakes(like the numbers coming in without him writing) black also helps make post work easier.
Notice he is writing with the left hand. That’s your answer probably
As @racc1337 mentioned... You shoot him writing thru a transparent surface and then mirror the video before posting... So he's actually right-handed (it doesn't help that he's wearing his watch on his writing hand just to mess with your head further :) and the write hand is the right hand 😆
As racc1377 said, a large glass pane then horizontally reversed in post, computer images slid in too. I've seen this done before but this one is done very well indeed. ..and very well delivered too. Excellent content imo :D ...and I learned from it too which is really the acid test!
Wow interesting last point! I remember facing dev situations and feeling like a limitation of some sort, but perhaps taking a NoSQL approach would have lifted that. Quite exciting. There's a couple of teasers in here that i'd like more information on. Like what is vertical scaling, what is horizontal scaling? And what accessing NoSQL looks like? But i definitely feel certain of the mode of SQL and the mode of NoSQL enough to experiment with NoSQL.
Thank you for your comment and your suggestions! 🙂 We'll see what we can do to incorporate that info into future videos. Stay tuned!
I appreciate your writing skills more
A lot of people think NOSQL is a new concept. NOSQL predates Relational Databases by decades (1960s) and was called Hierarchical Databases. When Edgar F. Codd invented Relational Databases (early 1970s), computers weren't powerful enough to support them and it wasn't until 1979 the Oracle truly commercialized Relational Databases. Just like today Relational Databases need more computing power than the simpler NOSQL Database. This is because the main drawcard of RDBMs needs it, "Data Relationships". Without the relationships that SQL provides, one has to write the joins into their own code whether it be Python, Ruby, C++ , Go etc or rely on the same or similar SQL syntax to do perform joins (Hadoop, Splunk come to mind). SQL reduced the proliferation of code needed to do joins. Imagine having downstream systems using multiple programming languages, each one having to munge their own joins. RDBMs also provide other things such as ACID, which ensures your data is in a consistent state even if there are conflicts. Frankly once the data is normalised it easier to work with SQL plus ORMs.
This could have been easier to follow - in some parts you are using IT buzzwords to describe an IT buzzword concept - which is even more confusing. It would be a lot simpler to understand if you could use plain English and examples - so, for example, what does horizontal and vertical scaling actually mean, what does 'dynamic' as point 4 on the No SQL side of your diagram actually mean.
In the first few sentences of the presentation you mention that it might be surprising that people ask questions about these differences between technologies, or that there is confusion about the differences, but in my mind it's not surprising at all when the meaning of the buzzwords/language used to actually explain the differences are unclear.
Yeah, I already read the quick list of differences and I was hoping, like you have said, that this video would explain those differences. I got nothing out of this video.
good job on these explainer videos IBM!
Thank you! this is the perfect video for explaining the difference to me
Good effort, but needs some more depth. The way is explained, makes a classic Oracle DB qualifying as NoSQP DB :)
can discuss performance issues, and the types of non-SQL DBs
But for me, it looked like NOSql seems to have upper hand here.
@@firstcommenter202 That sounds like an excellent follow up video. I will take note of this great suggestion the next time we produce more videos on this topic.
@dovasd Thanks for your comments. Yes we intended to "scratch" the surface with this video and dig a little deeper on other topics. There are certainly many options nowadays for either NoSQL & SQL Databases. We will have some profiles coming up soon on different types.
I am curious about "Vertical vs. Horizontal Scaling"
1. Why can I not scale SQL Databases Horizontally? You *could* use sharding and read replicas improve scaling needs - sure, it requires work, but SQL is constantly improving - isn't that right?
2. What I can see is - you can easily make changes in NoSQL e.g. unstructured - which is quite costly in SQL
The issue with relational and horizontal scaling is yes you could spread it via sharding but the issue comes in how you can keep the read and writes consistent across them all. You can split a database into multiple smaller databases but then that just leads to higher maintenance costs. You could also have one just for writes and several others as read only databases but again this would need managing with the services accessing the databases.
With non relational yes you can throw into them what you want but unless you “lock down” a schema for the data then you can’t get consistent results as each insert is a new entry rather than an “upsert”.
I guess each have their pros and cons but you would choose whichever is best for your end system which you would know once you’ve done the correct analysis
The visual presentation of this video is superb, but it's really let down by the audio, there's a very slight reverb/echo effect that lands perfectly in the uncanny valley of normal speech.
very good explanation of the key differences thank you
Thank you for your feedback! 😀
Nice video with great notes
Well explained, thanks ❤
I think addressing performance of a query should be a major factor. I'm SQL trained and do not know how one performs extraction of data from a no-sql data pool other than perhaps by the Linux commands "awk" and "grep". Is there a performance savings by mapping everything into SQL vs. leaving stuff in a blob?
Thanks for the clear clarifications
SQL requiring more memory & computing power is making me lean more toward NoSQL because that means hosting is cheaper for NoSQL apps.
every time I watch these videos I get distracted wondering "how tf did they write backwards?!"
He's writing normally, they just flip the video
Very clear! Thank you👌
Great explanation!
The problem I see with your explanation is that SQL is just a language. There are an ever increasing number of options to scale out RDBMS systems. Also it’s not like, for instance, Postgres is bad at handling JSON data with JSONB. There are some attributes that you are likely to always want to have, and some that may change per instance. So something like a table with a JSONB column is a nice approach. However, I wouldn’t go around installing Postgres on loads of mobile phones for kicks. It’s just not reasonable to use that amount of storage space.
Just a thought.
how do they write the things in front of them? It means they are writing the other way around right? or what? I have been trying to watch the tutorial but my head is stuck at that
The way he wrote is even more impressive 😮
well explained, thank you!!
As a coder..
Code itself can be referred as a jungle that its own writer can navigate..
That is why convention is created, manual is needed, etc
With unstructured nature of nosql, we can double that.
Did wrong in here??
Very good video.
Thank you very much!
I don't get what you mean by NoSQL is all in memory, isn't it also have to persist data on disk, like MongoDB...
It works by giving you fast access to the data through memory access. There may be some files that are written to disk but that is not its main access method. It is more a function of how the software application works. Let me know if I helped answer your question or have more questions.
Love it but the sound on this one was not good. Needs to be louder.
how can you compare "non sql" without proper definition? Time series dbs are no-sql nd they do not follow your observations?
Excellent explanation sir! Thank you very much! Cheers!
can we use NoSQL in Customer relationship management systems
I'm distracted wondering how this video was made, was the video flipped?
You've got the idea. You can see some of our backstage "secrets" on our Community page, you can check it out here 👉 ibm.co/3pT41d5
@@IBMTechnology Thanks for the BTS picture, there was no way I was going to be able to pay attention to the video until I saw the setup.
All these instructors are left-handed. The only conclusion is that the camera is recording this from a mirror. 😊
Good!
Thank you!
Another "ancient" computing technology has been resurrected with a new name !
Ack in the 1960s, IBM invented "indexed sequential access method" (ISAM) of accessing data (not 100% certain of the name IBM used or the timing). By the late 1970s, Digital Equipment Corporation (DEC) had duplicated the technology. I am not familiar with IBMs implementation so the rest of my comments will apply to DEC.
It is important to understand that the underlying storage technology is one aspect and HOW THE DATA WAS ACCESSES is totally another ! On DEC machines, there as an extra "layer" added between the data and the application such that the data could be accessed via the primary or SECONDARY keys or sequentially ! While the primary key had to be unique, secondary keys did NOT have to be unique !! The biggest problem was that no "high level languages" (Fortran, COBOL, C) supported this concept. "Extensions" were added to each to support look up by primary or secondary keys.
As mentioned, there was no "standard" query language. DEC invented Datatrieve. It was a bit quirky but it included a "report writer" similar in concept to IBM RPG which was immensely popular at the time. The basic query could be written in Datatrieve and that could be linked into another custom application.
Sadly for DEC, ISAM never caught on in large volume. It also could not scale horizontally without a lot of additional programming. This was probably because it required proprietary language extensions. Building "forms" for data entry was a sperate product or required a lot of programming. As the world moved to "windows" based applications, DEC stayed in "character cell" interfaces as did IBM for much too long. Microsoft Access was one of the biggest "nails in the coffin" for ISAM. IBM and DEC had proprietary relational databases, but it was too Little to late.
Multiple questions come to my mind after looking at this very good video: thank you for that.
- To come back to your illustration with an e-commerce website: don’t you need a structured DB (SQL) to identify the products hosted in a Non-SQL DB that you will present to the consumer?
- Another comment: as No-SQL is in memory it limits its size where with SQL you can have huge size of DB. Correct?
- Also I’m curious about performances: is there a kind of index with No-SQL DB?
As I see it and you mention it at the end of the video, the combination of the 2 technologies is the key.
To come back to your illustration with an e-commerce website: don’t you need a structured DB (SQL) to identify the products hosted in a Non-SQL DB that you will present to the consumer? [ Possibly, but I have run complete backends off a NoSQL DB like MongoDB.. of course I had to make accomodations to ensure the data was in tact and standard ]
as No-SQL is in memory it limits its size where with SQL you can have huge size of DB. Correct? [ Correct, for SQL DB it will persist to disk. You are dependent on loading in memory for NoSQL DBs. This principle could also guide your balance of usage in your architecture. I could potentially host MYSQL on a VM Image, but something like MongoDB in the cloud is charged by the amount of memory used. The larger your JSON Docs, the more they consume. ]
is there a kind of index with No-SQL DB? [ Yes, you can create indexes in a No SQL DBs. There are so many options to choose from so if that is your guiding principle, you can let that guide which one you decide to go for. I know CouchDB has some great features that help with indexes and has some great abilities to facilitate offline sync use cases through it's native architecture with revisions of Documents just as an example of when you might let features guide you down a path of which DB in the category to use ]
@@jamilspain7532 Thank you for all the answers. I really appreciate and highly valuable to me who's not an expert of DB.
@@janekovr4722 your welcome anytime
excellent explanation...thank you very much.
This guy's pretending to write backwards for the sake of the video.
Entertaining.
he's not pretending, he is really writing, but he isn't really writing backwards.
You don't even know how to flip a video, yet you came to see the difference between SQL and NoSQL
NOSQL was developed after SQL because of the lackings of SQL. I would always prefer NoSql
Not always, it really depends on the needs of your application and the data will be queried. For example, if you are working with interconnected data, SQL will be acceptable, but Graph models are better. Another case, if you have to use many-to-many queries, then a relation model (read SQL is better). However, for certain applications, e.g., retrieving document for a web-site, a NoSQL database is better and if affords good locality of data.
Someone ignored the PRO/CONS and decided NoSQL-databases are better than SQL-databases (Which are tested and proven to be good enough for the most situations the past 30, 40+ years?). Poor company, who got a dev at the end, who chooses a database which are not good enough for the current problem. By the way good luck when your application is heavly relationed based, NoSQL would be a overkill.
Wow it took me way too long to figure out how he was writing backwards lol.
You're living in the 50's
Pretty poorly explained. You don't really seem to fully grasp what it is that you are talking about. As some of the other comments point out, you use buzzwords to explain buzzwords.
not explained thoroughly
worst explanation ever