@@willi1978 Yes they are. If used correctly they can achieve 90% compression rate on time series data, using compression techniques like delta encoding and run length encoding.
@@arlandmv4053 as IlijaNL , timescale is cool. I'm working on migrating my company's crufty postgres db schema to a fresh timescale one. It compressed 100GB of timeseries data (4-5 years' worth) down to 14. And then their downsampling algorithm was able to turn half a year of measurements into 750 data points in 2.7 seconds. I tried it with 1.5 years and it came back in 3.3 seconds. It's insane.
AGE (A Graph Extension) for Postgres seems similar to EdgeDB. Using Cypher to query the graph, but iirc you can mix sql and cypher together to get the best of your existing model with graph on top
If you mix SQL and Cypher you get a DSL only one person in the world understands. Also input sanitization and regular expression handling becomes a giga cancer.
minddb is especially helpful to ai/ml researchers and fine-tuners because it's easier to retrieve the training data and then use the training data with a logical "sql" like syntax.
I am currently writing master thesis evaluating the extension Citus that distribute postgresql very efficiently and transparently, but still reusing most of the postgresql code. Definitely interesting as it also supports a lot of extensions, which is also a great property of postgresql.
He got 1.84 million subscribers. People like that make a lot of money on RUclips. Enough to pay people to do research for them, write a script and help with the editing.
That's probably the most important skill a developer can have today. You need to keep up on new technologies constantly to stay competitive in this industry.
Exactly, I now I won't share this publicly in a work environment, just in case someone I don't know sees it and makes them feel uncomfortable not knowing where it comes from.
I'd love to see a video on full text search engine "databases". Things like Elasticsearch, Typesense, algolia, and more. I have a project using Typesense as a full text search engine, but its not feasible for medium-size amateur projects considering its in-memory approach. I'm not sure what to go with. Keep up the good work!
Outside of jokes, the fact that your are showing off new technologies in a world where people seem to only give them a chance when they get "stable", makes your channel a TOP! Bonus for the way you present things and for your humor ;)
It's bad, the codebase is terrible, it is very slow, only works over http/https. I wrote a query server and a client, so I know what I am talking about. It has a lot of limitations, and I still regret I have used in a project. It is underrated because it sucks, MongoDB is way better.
Not quite I sure the idea behind database branching. Like does the data replicated to the branch as well? Or is it just the schema? If so what’s the benefit vs just checking that locally… Maybe you can create a video on that?
You will love to make a 100-sec video on DuckDB, given their cloud proposition is called MotherDuck, and they have Ducklings in their architecture. The script just writes itself
I'm always amused just how well Google/Amazon/MS have sold web devs on the 'infinitely scalable cloud' Kool-Aid. Seems most don't even know what a modern server (measured in hundreds of cores and terabytes of RAM) is capable of any more. Better usability (like SurrealDB) I do get excite for 🤘🤘 At least until AWS "adapts" it into a cloud offering and it dies as an open-source project.
I'm not sure I understand your message. Are you telling us, that developers nowadays don't think that sometimes your own server is better than cloud providers that scale the resources based on your project needs? And that developers straight up want to run their apps in the cloud rather on their own infrastructure?
@@pastelstoic8416 That'll just slow them down a bit. Maybe. I like those new licenses but Amazon will find a way to rip any good idea off. Interestingly, their retail is notorious for that as well, ex Sling: ruclips.net/video/HbxWGjQ2szQ/видео.html But hey, what's a poor guy that needs a third G-650 supposed to do 🤷♂
A lot of people just want to build an app and not want to have the very real effort of managing the IT of a physical server. I get maybe you or others similar don’t think it’s any effort. But it is. Along with managing reliability and stability of your own server. I mean props to you and others like you for doing it. But the. loud isn’t Kool-Aid, and rolling your own server isn’t a silver bullet.
I hardly never comment on RUclips, but it’s 4:51am, I can’t sleep and I’m just amazed by those videos!!! Love your channel! Always watch 100 seconds series as part of decision maker!!! Great job guy! 🎉
For scaling Postgres, there's also Citus which is a fully open source extension bringing distributed tables and columnar support. It's acquired by Microsoft and also served as a service under CosmosDB brand.
@@uziboozy4540 they have different approaches in the dsql area. For example, Citus keeps the PG as is, doesn't fork or anything, and installed on top of the regular pg as an extension, which is important for many. Also their approach for many db concepts like HA or replication are different. The list can go on. btw disclaimer I work on Citus.
@@uziboozy4540 Yes, different architecture. CitusDB is sharding on top of PostgreSQL. YugabyteDB is SQL on top of distributed DB. So different use cases. Datawarehouse need many analytic pushdowns, and no need for global transactions or HA -> CitusDB. OLTP need global transactions, foreign keys, HA and elasticity -> YugabyteDB
Switched from nosql to citus recently and love it. Nosql was just so painful to work with, having to know how data will be queried at the time of writing, having to denormalise data and write it many times by different keys, etc. Queries existed but had such bad performance that point reads were needed to make it scale well enough, at which point it's just key-value. We evaluated vitess as we run in kubernetes but it was a headache to manage and it was missing many features of mysql 8, while citus is kept up to date with postgres within weeks. Switching to citus (cosmos postgresql) has reduced the amount of data we store, made it much easier to work with to implement new features and improved performance dramatically. Huge success story for us.
Hadn't really thought of AI inside my DB, but it sounds interesting. Would love to learn a little more about potential usages and have a more precise example. ^^
I was expecting at least a mention of DGraph, which is a native GraphQL database (data is stored in graphs), is made in Go, and is pretty straightforward.
Great review of existing exotic DBs. Still, many will remain in "toy-state". Remember it takes around 10 years (best case) for a DB to mature so that a it will be used in a "real company" (not hobby projects). Also, you actually forgot the most interesting DB currently in development IMO, which is MotherDuck (DuckDB). This DB might actually be a game-changer.
I don't think it's a matter of time per se. It just takes one "killer" app to become really popular using it and then the technologies it uses also become equally popular.
@@leonardoraele Not sure. Take a look at FaunaDB. It's way ahead of its time. Absolutely checks all the boxes an more. But they are still struggling to get really big customers. NextDoor did use them but stopped, why?
Man... IT S AWESOME ! Thanks for sharing ! I think that I will try that SurrealDB "Develop easier.Build faster.Scale quicker." Sounds way to good but let's go !
Yeah, until they remove the free tier or increase the prices, or the company goes out of business. Then you're in for a worse time than ActiveX and Silverlight.
Great video! Curious if you have covered the popular Snowflake engine. As a data engineer I see snowflake as the gold standard. Integrating that with dbt and you have the hippest BI stack!
Neondb is postgres the only difference is they separated compute from storage and persist the storage in something like s3 and cache the important data in the VM , they also have a bunch of tricks to allocate more VMS to handle scaling while retaining the ability to scale down to 0, they even went as far as commiting the changes back into postgres but they probably won't be merged
I think SurrealDB is interesting but EdgeDB is written in Python which is a great language for many things but not for a database. Also, I think it is a bit unfortunate that EdgeDB uses some weird := colon-equals syntax.
@@michaelbitzer7295 It's a lot of Python. I code in Python myself, and I like it for some stuff, but I wouldn't want to use any database written in Python. It is absolutely terrible for performance, scaling and reliability.
Me until 2:08 : "I think I should share this to my company's tech team channel, I think my coworkers would find this informative and entertaining." Me after 2:09 : ".....nevermind."
Wow, so many things popping up pushing the boundaries and so much of docs to be digested to put out such although short but saturated videos on the topic. I'd love to watch about graph databases which in my view are solid-grounded and must be safe to use in production today and may be worth replacing SQL types manywhere.. Sorry, non-english from central Asia:))
One thing I hate about developer tools is developers love making developer tools so there's way too many choices and I get choice paralysis.
Just ignore everything with an on-demand pricing model and you already have removed 80% of the choices.
1 Planetscale 0:45
2 Yugabyte 1:32
3 Neon 1:51
4 Dolt 2:08
5 CockroachDB 2:43
6 Cloudflare D1 3:08
7 Xata 3:48
8 8Base 4:26
9 EdgeDB 4:47
10 SurrealDB 5:22
11 Fauna 5:55
12 Mem Graph 6:23
13 KeyDB 6:47
14 MeiliSearch 7:20
15 Mindsdb 7:49
Thanks for this!
@Fireship it'd be rad if you could include timestamps like these in the description so that they appear on the video itself 🙏
What can you do with this all?
Ha great recap thanks @NamalJayathunga
Hero we don’t deserve
Thank you very much Fireship for the feature. As always, we really appreciate it. Have a great 2023! ❤
I predict that surrealDB will be the gold standard in a few years.
This actually is the real account
Went on ur website to find out
I found SurrealDB through Fireship, just getting going with it but absolutely LOVE it. You guys are doing awesome work! :)
We are already a 1/3 into February now. Time flies, it feels like it is still the 1st of January
"For your next failed side project" - LOL I felt that one.
Bro, like a hot knife straight to the heart....
Yeah. me too🥲
I would have liked you to include some time series optimized databases. They are very essential for IoT projects
Are they so much better? Most of the time I saw time series data stored in relational dbs
@@willi1978 Yes they are. If used correctly they can achieve 90% compression rate on time series data, using compression techniques like delta encoding and run length encoding.
do you know of any? Id like to know some
timescaledb is probaby a good starting point (just extension of postgresql)
@@arlandmv4053 as IlijaNL , timescale is cool. I'm working on migrating my company's crufty postgres db schema to a fresh timescale one. It compressed 100GB of timeseries data (4-5 years' worth) down to 14. And then their downsampling algorithm was able to turn half a year of measurements into 750 data points in 2.7 seconds. I tried it with 1.5 years and it came back in 3.3 seconds. It's insane.
Thanks Fireship for featuring Memgraph!! Streaming graphs FTW
0:43 guillermo franchella cameo, nice one fireship
Rip Planet Scale free tier
AGE (A Graph Extension) for Postgres seems similar to EdgeDB. Using Cypher to query the graph, but iirc you can mix sql and cypher together to get the best of your existing model with graph on top
If you mix SQL and Cypher you get a DSL only one person in the world understands. Also input sanitization and regular expression handling becomes a giga cancer.
I am watching again this material 5 months later and still a lot of hints to check over. Thanks for your work - kind of IT tech wikipedia
minddb is especially helpful to ai/ml researchers and fine-tuners because it's easier to retrieve the training data and then use the training data with a logical "sql" like syntax.
I may be biased, but PostgresML is more scalable, more capable and several times faster than MindsDB. You can google it and find the benchmarks.
I am currently writing master thesis evaluating the extension Citus that distribute postgresql very efficiently and transparently, but still reusing most of the postgresql code. Definitely interesting as it also supports a lot of extensions, which is also a great property of postgresql.
JS: There is a new framework seems like every day!
DB: Hold my bear!
Now I don't watch your videos for software development purposes, at this point I just enjoy your memes.
the left, right, inner, fullouter joins meme killed me!! hahaha
I'm here after 12 seconds. You're welcome, Fireship.
What a blazingly fast comment
Much better comment than the guys who comment first, well done!
@@leonardoraele 20x performance increase!!!! 🔥😱👈👉👆👇👊🙌🤝🙃🔫
definitely need a fast database to allow for this speed
Game changing , ultra fast scalable comment section
Man your a learning machine or an alien, the way you skeem through all that and teach it to us like you made it :)
Seriously,
All I want from him is how to learn in 100 seconds
He got 1.84 million subscribers. People like that make a lot of money on RUclips. Enough to pay people to do research for them, write a script and help with the editing.
@@maythesciencebewithyou Thanks for the red pill.
That's probably the most important skill a developer can have today. You need to keep up on new technologies constantly to stay competitive in this industry.
wow. A few hours ago I was searching for a cool new db to use in my own project. And bam! This video released!
Same for me! Ship sails
Thanks for the shout out!
Your videos are educational and people of all ages use them to learn. Was the dolphin scene really necessary or useful?
It was necessary and useful. Cry about it
@@paradiseexpress3639 are you mentally ill my friend? What I said is not wrong and not crying and stop liking your own comments.
Exactly, I now I won't share this publicly in a work environment, just in case someone I don't know sees it and makes them feel uncomfortable not knowing where it comes from.
@@javiercandalaft and then you get kids like panda express above telling you it was necessary, lmao. Clearly they had no jobs before.
I'd love to see a video on full text search engine "databases". Things like Elasticsearch, Typesense, algolia, and more. I have a project using Typesense as a full text search engine, but its not feasible for medium-size amateur projects considering its in-memory approach. I'm not sure what to go with.
Keep up the good work!
Outside of jokes, the fact that your are showing off new technologies in a world where people seem to only give them a chance when they get "stable", makes your channel a TOP!
Bonus for the way you present things and for your humor ;)
You should do one of CouchDB - I think it is underrated as a NoSQL database for mid-size projects.
No, bad!
*sprays you with anti-intellectual spray*
BAD!!!!
It's bad, the codebase is terrible, it is very slow, only works over http/https. I wrote a query server and a client, so I know what I am talking about. It has a lot of limitations, and I still regret I have used in a project. It is underrated because it sucks, MongoDB is way better.
@@biomorphic I agree not good for larger projects, but I found the usability of it pretty straight forward with libraries like pouchDB.
@@biomorphic We have standards Biomorphic. . . .
*We have standards.*
big fan of PouchDB with CouchDB
Still waiting for that awesome sounding full Svelte(kit) course!
ArrangoDB is also a graph + document database with it's own programming like wish language.
Dude, you're klillin' it. So happy for you.
8base looks pretty cool. Definitely taking this for a spin.
Not quite I sure the idea behind database branching. Like does the data replicated to the branch as well? Or is it just the schema? If so what’s the benefit vs just checking that locally… Maybe you can create a video on that?
Very smart to put 8base on 8th place! 😁
I'm impressed by how funny you can make these vids. The Rust joke and the "image of two dolphins f**cking" made me lol.
You will love to make a 100-sec video on DuckDB, given their cloud proposition is called MotherDuck, and they have Ducklings in their architecture. The script just writes itself
I'm always amused just how well Google/Amazon/MS have sold web devs on the 'infinitely scalable cloud' Kool-Aid. Seems most don't even know what a modern server (measured in hundreds of cores and terabytes of RAM) is capable of any more. Better usability (like SurrealDB) I do get excite for 🤘🤘 At least until AWS "adapts" it into a cloud offering and it dies as an open-source project.
I'm not sure I understand your message. Are you telling us, that developers nowadays don't think that sometimes your own server is better than cloud providers that scale the resources based on your project needs? And that developers straight up want to run their apps in the cloud rather on their own infrastructure?
From all this video, surreal seemed something i want to explore what it is...
how is surreal better than postgres for example? ...
Luckily, that won't happen! Surreal has a "you aren't allowed to sell this, looking at you Amazon" clause in their license.
@@pastelstoic8416 That'll just slow them down a bit. Maybe. I like those new licenses but Amazon will find a way to rip any good idea off. Interestingly, their retail is notorious for that as well, ex Sling: ruclips.net/video/HbxWGjQ2szQ/видео.html But hey, what's a poor guy that needs a third G-650 supposed to do 🤷♂
A lot of people just want to build an app and not want to have the very real effort of managing the IT of a physical server. I get maybe you or others similar don’t think it’s any effort. But it is. Along with managing reliability and stability of your own server. I mean props to you and others like you for doing it. But the. loud isn’t Kool-Aid, and rolling your own server isn’t a silver bullet.
0:25 excellent illustration 😀
Is it blazingly fast?
Since Oracle owns mySQL it means that Oracle dominates the enterprise database and the open-source database world according to 0:24...
Your well-produced videos help increase the speed in which my failed-side-projects can ramp up.
02:00 made my day man, made my day.
I hardly never comment on RUclips, but it’s 4:51am, I can’t sleep and I’m just amazed by those videos!!! Love your channel! Always watch 100 seconds series as part of decision maker!!! Great job guy! 🎉
Being able to delete databases using drop is great :)
For scaling Postgres, there's also Citus which is a fully open source extension bringing distributed tables and columnar support. It's acquired by Microsoft and also served as a service under CosmosDB brand.
Yugabyte is pretty much Citus on steroids
@@uziboozy4540 they have different approaches in the dsql area. For example, Citus keeps the PG as is, doesn't fork or anything, and installed on top of the regular pg as an extension, which is important for many. Also their approach for many db concepts like HA or replication are different. The list can go on.
btw disclaimer I work on Citus.
@@uziboozy4540 Yes, different architecture. CitusDB is sharding on top of PostgreSQL. YugabyteDB is SQL on top of distributed DB. So different use cases. Datawarehouse need many analytic pushdowns, and no need for global transactions or HA -> CitusDB. OLTP need global transactions, foreign keys, HA and elasticity -> YugabyteDB
Switched from nosql to citus recently and love it.
Nosql was just so painful to work with, having to know how data will be queried at the time of writing, having to denormalise data and write it many times by different keys, etc. Queries existed but had such bad performance that point reads were needed to make it scale well enough, at which point it's just key-value.
We evaluated vitess as we run in kubernetes but it was a headache to manage and it was missing many features of mysql 8, while citus is kept up to date with postgres within weeks.
Switching to citus (cosmos postgresql) has reduced the amount of data we store, made it much easier to work with to implement new features and improved performance dramatically. Huge success story for us.
I just signed up for 8base, looks like fun to experiment with 😄
0:41 Saludos desde Argentina, great video :D
Thanks a lot ! Really helpful 👍🏻👍🏻
This was awesome thanks for putting it together.
What about ArangoDB ? it's multi paradigm document-key/value-graph, seems interesting
Branching is also available on PlanetScale
Really nice👍Thanks!
1:20 - Rip they realized this and killed the free tier.
Wonderful summary of databases i heard first time
that's an hell of a headache of a video! great research
Hadn't really thought of AI inside my DB, but it sounds interesting. Would love to learn a little more about potential usages and have a more precise example. ^^
Bigquery, Aurora already does some ML inside the DB. Check it out
I'll wait next week for the next release of a database
Lokking forward to see how surrealDB will evolve
We need a database to manage these databases.
We have been using Stardog recently
failed side projects? bold of you to assume im gonna even start them
Could you do a video on Turso and Sqlite?
Would love to see more content on edgedb
very interesting and useful video. thanks for sharing so many interesting projects, many I hadn't heard of yet
Title: futuristic databases
Actual video: database hosting solutions
I was expecting at least a mention of DGraph, which is a native GraphQL database (data is stored in graphs), is made in Go, and is pretty straightforward.
Those were some crazy ass databases fr
New databases have been "coming out" for the past 30 years, we're still using SQL.
Finally a list of "things that you've never heard of" that I actually never heard of.
These videos keep getting better. The goat!
Great review of existing exotic DBs. Still, many will remain in "toy-state". Remember it takes around 10 years (best case) for a DB to mature so that a it will be used in a "real company" (not hobby projects). Also, you actually forgot the most interesting DB currently in development IMO, which is MotherDuck (DuckDB). This DB might actually be a game-changer.
I don't think it's a matter of time per se. It just takes one "killer" app to become really popular using it and then the technologies it uses also become equally popular.
@@leonardoraele Not sure. Take a look at FaunaDB. It's way ahead of its time. Absolutely checks all the boxes an more. But they are still struggling to get really big customers. NextDoor did use them but stopped, why?
How DuckDB compares to Graph DBs, what i've found is graphs are still better.
This video came at the time I needed it the most and I'm late, 2mins late
Awesome video, holy information overload. More databases than I would ever want to learn.
The first 10 seconds speaks volumes
I used to spend long time to decide what DB i will go for, thanks fireship and I will spend an era
0:25 perfect depiction of Joins 😂
Futuristic databases that are actually old databases
Man... IT S AWESOME ! Thanks for sharing !
I think that I will try that SurrealDB "Develop easier.Build faster.Scale quicker." Sounds way to good but let's go !
Best part of waking up, is fireship in your notifications.
Neat.
MindsDB is the most interesting one but that Planetscale free tier is almost too good to be true... Makes me wanna do something with it.
Yeah, until they remove the free tier or increase the prices, or the company goes out of business. Then you're in for a worse time than ActiveX and Silverlight.
Edge, Surreal, Xata and MindsDB look pretty awesome.
EdgeDB is written in Python, I am not sure I would trust my data with that. Python isn't great for performance, scalability or reliability.
Last one is interesting!
TigerBeetle is another new entry that's really, really interesting. Very excited to see where it goes and how it evolves
Great video! Curious if you have covered the popular Snowflake engine. As a data engineer I see snowflake as the gold standard. Integrating that with dbt and you have the hippest BI stack!
I use Snowflake at my job and I love it. Unfortinately they aren't startup friendly
Neondb is postgres the only difference is they separated compute from storage and persist the storage in something like s3 and cache the important data in the VM , they also have a bunch of tricks to allocate more VMS to handle scaling while retaining the ability to scale down to 0, they even went as far as commiting the changes back into postgres but they probably won't be merged
Super happy with Meilisearch.
Most interesting for me were EdgeDB and SurrealDB.
I think SurrealDB is interesting but EdgeDB is written in Python which is a great language for many things but not for a database. Also, I think it is a bit unfortunate that EdgeDB uses some weird := colon-equals syntax.
@@fred.flintstone4099 EdgeDB is not fully written in python. The important bits are in C/Rust.
@@michaelbitzer7295 93.5% Python, 4.2% Cython, 2.2% Rust, Other 0.1%
@@fred.flintstone4099 your point is?
@@michaelbitzer7295 It's a lot of Python. I code in Python myself, and I like it for some stuff, but I wouldn't want to use any database written in Python. It is absolutely terrible for performance, scaling and reliability.
Mistake at 6:10, Couchbase supports native JOINS ;)
I’m really curious how you get to know all these possible options
None of this is new database. This is just a branch of database suppliers.
i love your channel very much. I've learnd so many useful things and now planetscale made my life 2.66 times easyer. thank you
0:42 I wasn't expecting to see Guillermo Francella in a video about databases.
Gonna need a database of all the database engines.
The picture of the dude with the beards and hair was hilarious when talking about joins, omg.
Me until 2:08 : "I think I should share this to my company's tech team channel, I think my coworkers would find this informative and entertaining."
Me after 2:09 : ".....nevermind."
Love seeing Guillermo Franccella's meme and "Te lo resumo así nomás" logo. 🇦🇷
The moment i saw the title, i knew right away that surreal would have a place in the list
Rightfully so!
“if you wanna look cool in front of your friends” was epic 😂
Wow, so many things popping up pushing the boundaries and so much of docs to be digested to put out such although short but saturated videos on the topic. I'd love to watch about graph databases which in my view are solid-grounded and must be safe to use in production today and may be worth replacing SQL types manywhere..
Sorry, non-english from central Asia:))
Your English is fine mate!
expectation: 15 futuristic databases I can use in the coming months
reality: tons of companies using excel as database on a local server
It's actually Microsoft Access, not Excel!
What about ArangoDB?
The guy at 0:42 is called Guillermo Francella, a famous argentinian, Im wondering how did you come across that gif
What do you think about TypeDB?
00:42 Francella cameo
Any recommendation to use with Go and Postgres ?
Great videos. Nun-db will soon be in this list ❤️