Appreciate the comparisons especially the section specifically on TimescaleDB. My feeling is that TimescaleDB is a lot easier to adopt when the company has a lot of old-school engineers who also want the relational data model that InfluxDB does not have.
Yeah that’s precisely why I wanted to try it, it feels like the path of least resistance. Although, if I have to pick one as the one I feel has the most potential to grow and dominate, I think it’ll be influx, it’s just my hunch though.
very, very, very good content and clear explanation! Great selection of behavior/characteristics to emphasize as a mature professional should be able to advise! thank you so much! also: nice shirt!
Nice explanation Gabriel. Thank you for your dedication. I saw some projects using the timescaleDB when fetching cryptocurrency data. I saw some comparison with influx, prometheus, benchmarks. I think is a good option!
TimescaleDB is the one I have little experience with, though it’s the one I wanted to use, it’s a pity that managed Postgres in AWS doesn’t support it and my company is too large to move all the parts to sign a new contract.
You dont miss cpu spikes in prometheus because it's a counter on used seconds total that you rate to get the graph you're describing. So maybe you should have mentioned why you loved counters!
Hey thanks for the comment! I haven't had the chance yet. My guess is that with SQL query language it's going to be more accessible for developers. One of the biggest barriers I have in my company with influx or prometheus is explaining how they must create their schema to avoid cardinality in tags/labels and the query language (flux/promql)
Hey brother, sorry about that. My life changed radically and I’ve been very bad at heading a company plus being a father plus RUclips. I really should organize things and get this done.
Great video! Quick question - to query the blockchain periodically and get token balances, would you recommend timeseriesDB or influxDB? It's somewhat similar a stock price ticker data, but frequency would be once a day/week/month.
I like influx a lot so I’m biased. In this case I think either one would work fine, I personally would go for influx. Even Prometheus gauges here would be fine since you are ok with just periodic data.
omg, I'm trying to learn the basics of a tsd and there is so much awful content out there. Then I find this video and this guy should be a teacher/professor as he breaks it down and explains it like I'm a 5 year old which is exactly what I was looking for. Sampling isn't explained very well even in the influx and grafana docs. Thank you for this video! (I even bought a udemy course on Grafana and although it's good, he assumes you already know all of what you are covering in this video)
They Tom thanks for the kind words and I’m glad it’s helping you. I had a very complete series planned for this topic using examples from my real job. Unfortunately I’m a bit of an introvert so it takes a lot of energy for me to record myself and talk to a camera, I’ll try to put a bit more effort in if people find these helpful
@@Gabzim I think the use of Grafana for BI is gaining popularity and observability is the new trend right now. TSDBs can be very confusing and intimidating and this comes from a guy that has been doing rdbms for 20+ years. People really just want a BI/reporting/visualization tool that is dead simple and they can generate pretty graphs/dashboads in minutes and not have to steep learning curve such as power bi/tableau/looker. Looking forward to your next videos.
I'd go with QuestDB, which was designed specifically for this use case. It is used now for every kind of time-series data, but fintech/trading/crypto are some of the more popular use cases amongst its users
if you mean via the push gateway i’d say “kind of”. you can push it to the push gateway but then that gets scraped by prometheus. if you mean you can push granular data in between scraping intervals i haven’t seen that.
Hey thanks! appreciate the compliment. I’m using Final Cut Pro, but it’s overkill for the kind of editing that I do. I should have gone for a simpler tool. I use like 0.5% of the features it has.
@@Gabzim i get you, but i think most tools out there are pretty overkills, although tbh Final Cut Pro is pretty simple and straightforward to use Keep the videos coming!
Because then each file would create a new metric with a single data point, after a while this would really wreck your performance and you’d need to accommodate for this in your queries. Labels have to have a low cardinality
@@Gabzim labels are used like that in loads of places though, and pretty common right? e.g. node exporter, so it is possible, but like with all prometheus metrics, you just have to be wary of cardinality - though it feels like the same could be said when comparing to some downsides of SQL based systems - just feels like in this case its maybe a bad example to give of something you can't do in prometheus, when you can? Side note - really like your drawings, it fits in really well with the video as you explain stuff.
Yes, you're both right. You can use labels like that, but it will blow up your number of timeseries unless you keep a short retention policy for it. Pod name could be similar depending on your cluster usage and people use that all the time.
That's ok because, when you query the data, the rate operation in prometheus takes this into account. Let's say you have a counter that goes like: 1, 3, 5, 8, 10, 2. That last 2 is "wrong" in the sense that a counter should always increase, so rate will look at the data like: 1, 3, 5, 8, 10, 12 (roughly speaking) and will calculate the rate from there.
![Jack Harlow - Hello Miss Johnson [Official Music Video]](http://i.ytimg.com/vi/Q86_nlRoIGw/mqdefault.jpg)
Very helpful.
There are many videos just saying install one or the other, but none I've watched so far have actually explained the differences.
Honestly quite helpful with the decision I was try to make. I'll be integrating Influx after reading some articles and enjoying this explanation.
It's impressive how within the first 10 minutes you clarified all my doubts about influx vs prometheus
So excited for the Flux videos!! 🙌🏾🙏🏾
Great video! Please keep it going with a advance tutorial/dive into InfluxDB. Looking forward for it
Appreciate the comparisons especially the section specifically on TimescaleDB. My feeling is that TimescaleDB is a lot easier to adopt when the company has a lot of old-school engineers who also want the relational data model that InfluxDB does not have.
Yeah that’s precisely why I wanted to try it, it feels like the path of least resistance. Although, if I have to pick one as the one I feel has the most potential to grow and dominate, I think it’ll be influx, it’s just my hunch though.
Really thanks for this master class! You've spoken for clear way
Fantastic tutorial!
Fantastic video. Helped with my second guessing
You are the champ, fantastic explanation!!, nice t-shirt by the way !!
I would need those next videos :D
Great video, I hope you will do more of them!
very, very, very good content and clear explanation! Great selection of behavior/characteristics to emphasize as a mature professional should be able to advise!
thank you so much!
also: nice shirt!
Thank you for taking the time and leaving a compliment! I’ll make more videos when I’m back from my trip
Very useful, you simply explained.... Thank you so much...... Do more video❤❤
Very educational! Thank you very much for sharing your expertise!
Crisp and precise
Nice explanation Gabriel. Thank you for your dedication. I saw some projects using the timescaleDB when fetching cryptocurrency data. I saw some comparison with influx, prometheus, benchmarks. I think is a good option!
TimescaleDB is the one I have little experience with, though it’s the one I wanted to use, it’s a pity that managed Postgres in AWS doesn’t support it and my company is too large to move all the parts to sign a new contract.
Having said that, my experience with influx so far is great. Nice performance, flux is easy once you get it. Curious to see what influx IOx can do.
Looking forward to you flux videoes
very useful and clearly explained! Thanks a bunch!
Btw its push vs poll not pull(Labelled in chapters of this video) which is major difference.
Super content!
You dont miss cpu spikes in prometheus because it's a counter on used seconds total that you rate to get the graph you're describing. So maybe you should have mentioned why you loved counters!
Great video
Great video!
This is great and helpful. Have you by chance looked at the AWS time-series database, Timestream? How would you compare them?
Hey thanks for the comment! I haven't had the chance yet. My guess is that with SQL query language it's going to be more accessible for developers. One of the biggest barriers I have in my company with influx or prometheus is explaining how they must create their schema to avoid cardinality in tags/labels and the query language (flux/promql)
Me gustó. Cuando puedas los TIPS de FLUX... Likes asegurados
Les debo ese video hace mucho. Estoy cerrando una release y me pongo a grabar
thank you!
appreciate it..thank you :)
Wow, man, where are you? This video is so good but you haven't posted anything after this one.
Hey brother, sorry about that. My life changed radically and I’ve been very bad at heading a company plus being a father plus RUclips. I really should organize things and get this done.
Really appreciate the kind words btw
Great video! Quick question - to query the blockchain periodically and get token balances, would you recommend timeseriesDB or influxDB? It's somewhat similar a stock price ticker data, but frequency would be once a day/week/month.
I like influx a lot so I’m biased. In this case I think either one would work fine, I personally would go for influx. Even Prometheus gauges here would be fine since you are ok with just periodic data.
Thank you
omg, I'm trying to learn the basics of a tsd and there is so much awful content out there. Then I find this video and this guy should be a teacher/professor as he breaks it down and explains it like I'm a 5 year old which is exactly what I was looking for. Sampling isn't explained very well even in the influx and grafana docs. Thank you for this video! (I even bought a udemy course on Grafana and although it's good, he assumes you already know all of what you are covering in this video)
They Tom thanks for the kind words and I’m glad it’s helping you. I had a very complete series planned for this topic using examples from my real job. Unfortunately I’m a bit of an introvert so it takes a lot of energy for me to record myself and talk to a camera, I’ll try to put a bit more effort in if people find these helpful
@@Gabzim I think the use of Grafana for BI is gaining popularity and observability is the new trend right now. TSDBs can be very confusing and intimidating and this comes from a guy that has been doing rdbms for 20+ years. People really just want a BI/reporting/visualization tool that is dead simple and they can generate pretty graphs/dashboads in minutes and not have to steep learning curve such as power bi/tableau/looker. Looking forward to your next videos.
@Tom N agree, I’ll try to make a couple this week. Thanks for the support.
Can you recommend which database should I use storing stock market data.?
I’d use influx for that.
Flux also has other features that will be very useful for analyzing that data. Stick around, I’ll make a video on that.
@@Gabzim thanks man! Great video btw 👍🏿
@@Gabzim thank you, waiting for the next
I'd go with QuestDB, which was designed specifically for this use case. It is used now for every kind of time-series data, but fintech/trading/crypto are some of the more popular use cases amongst its users
You can also push data with Prometheus
if you mean via the push gateway i’d say “kind of”. you can push it to the push gateway but then that gets scraped by prometheus. if you mean you can push granular data in between scraping intervals i haven’t seen that.
there's a remotewrite now too from various agents
Very good explanation! What software do you use to edit your video?
Hey thanks! appreciate the compliment. I’m using Final Cut Pro, but it’s overkill for the kind of editing that I do. I should have gone for a simpler tool. I use like 0.5% of the features it has.
@@Gabzim i get you, but i think most tools out there are pretty overkills, although tbh Final Cut Pro is pretty simple and straightforward to use
Keep the videos coming!
in your limitation example of prometheus, why wouldn't you store the filename as a label (or dimension depending on terminology)?
Because then each file would create a new metric with a single data point, after a while this would really wreck your performance and you’d need to accommodate for this in your queries. Labels have to have a low cardinality
@@Gabzim labels are used like that in loads of places though, and pretty common right? e.g. node exporter, so it is possible, but like with all prometheus metrics, you just have to be wary of cardinality - though it feels like the same could be said when comparing to some downsides of SQL based systems - just feels like in this case its maybe a bad example to give of something you can't do in prometheus, when you can? Side note - really like your drawings, it fits in really well with the video as you explain stuff.
Yes, you're both right. You can use labels like that, but it will blow up your number of timeseries unless you keep a short retention policy for it. Pod name could be similar depending on your cluster usage and people use that all the time.
What about we losing counter because of the service which generate the counter get restarted . Its with respect to the pull model of Prometheus
That's ok because, when you query the data, the rate operation in prometheus takes this into account. Let's say you have a counter that goes like: 1, 3, 5, 8, 10, 2. That last 2 is "wrong" in the sense that a counter should always increase, so rate will look at the data like: 1, 3, 5, 8, 10, 12 (roughly speaking) and will calculate the rate from there.
Which TSDB is suitable for tick data of stock market?
Of the three, influx is the one I’d personally recommend. You can’t push data to Prometheus so you’ll lose resolution.
In addition, flux has multiple functions that are useful for processing moving averages, predictions etc
the titles should be "influx vs prometheus".
Timescale isnt being discussed in a decent way here.
Valid point, sorry about that.
Hi man, what program did you use for drawing?
Ah seems to be something of ios.. Anyways great explanation keep it up
GoodNotes and I just record my screen
Good lecture, but a bit too sentimental. Could have packed all that info to make the vid shorter.
graphite
🎉😂❤🎉😮😂❤😊😊😊😮😂
pull not poll