Hi, yet another excellent tutorial, I have been looking for such a clear set of explanations for a while. Hopefully I will have some time over the weekend to apply. Thanks for that, much appreciated 👍🏻
Fantastic yet again! Thank you for the tutorial and keep them coming. Your teaching style is precise and very easily followed. I look forward to your next video! Cheers
Very kewl stuff. Started a proxmox server(HP 800 G5 TWR, had a couple from work) 2 months ago and been learning a lot. I just have 1 ubuntu VM running with up to 10 docker containers, so far. Mostly for plex and Arrs...I like how you explain the commands in your videos. Trust me I've been on many youtubers sites...hehe I need to be able to expose some of my services soon, so more learning to do. Thx
This was an excellent tutorial. I’m just starting to get into playing with Docker in my lab environment. Is there a way to use this setup to query SNMP hosts like routers and wifi APs and printers?? Thanks in advance!
Hey mate, glad you found it useful! So the same could be used in that situation I think, but instead of a node exporter you could use a SNMP exporter, I have not used it before though so kinda assuming here
ThankYou for making my life easy it was a hell, I was setting it up on EC2 Instance, tried various methods something was going wrong every time. But I want to run Grafana on port=3020 its was unable to reach. anyways I set it up on port=3000
Thank you for posting this. I've tried doing the influxdb, telegraph, grafana way to visualize data on my servers, but that's overly complicated. This was very easy and straight forward. The only question I have is is there a way to set Prometheus to store metrics on remote storage? I'm running my docker container on a nuc with a small hard drive and would love to have metric data living on a share hosted by my TrueNAS server.
Yeah so you can set the Prometheus data location to the shared drive, Docker supports NFS as it’s pretty critical for a lot of production environments and swarm
Hi Sir, Can I ask something? Why we are combining promethus and grafana?Instead we can use grafana only,right? What extra advantage we will get if we combine these 2?
Grafana by itself is just for the dashboards, you need to bring data into Grafana and we are using Prometheus to grab that data and store it, we then pass that data over to Grafana to create nice dashboards
Excellent Video !! but I'm getting same issue as simuman - " Got a line at the bottom saying "info msg="TLS is disabled." http2=false address=[::]:9100"> " and node_exporter data is not visible outside server, It's visible only inside the server
Thank you, this is wonderful. I have a question: I have multiple servers and I want to display them within one Grafana dashboard, meaning more than one control panel for different servers. Is this possible? Please explain the matter to me.
Thank you! So you can have your servers in one dashboard. When you have say a CPU tile watching the utilisation, you can change the query of that tile and specifically point to your server and then rinse and repeat for the others. By default I believe the tile is looking at what ever value you have selected for the server as a variable. If that makes sense?
@@Techdox Okay, but how can I identify the secondary servers? Do I need to add them? Or download Grafana again on each server? I searched for a solution but couldn't find one.
@@m7mmadomar you follow the same steps I showed in this video. All data from your servers come from Prometheus and Node exporter, once all those logs are in Grafana you can use them as you please. It’s just a matter of playing around with the Grafana queries
Quick questions guys, 1. how to setup so it could read event at different network/wifi let say I install node exporter at other device as agent, aleady setup allow port and even turn off firewall but noting avail. 2. related to point one, I try to install in server (linux) but only accessible via domain like domain/metrics give me data unlike point number one but when I setup config YAML with domain its error and not even show in local 9090 notes I used macos m2 as host installed / setup Prometheus & Node Exporter via brew on terminal. Thx.
Hi great video! I just have a query, since I'm just trying to monitor two servers. I create another job under the /prometheus/prometheus/prometheus.yml. The problem is that I cannot see the second job. And the first one is named as your job "elzim" I already change ip's and job names but still cannot get the second one running. I restarted the node exporter and both grafana to no avail. Edit: I'm now able to see the second job, but not the second "ip/host" in grafana. And one of the jobs is still stuck with the "elzim" name I have the current config. - job_name: server1 static_configs: - targets: ['x.x.x.x:9100'] - job_name: server2 static_configs: - targets: ['y.y.y.y:9100'] I will appreciate any advice. Thank you!
great video! i have one problem (same issue i had when running it under windows), when i edit the yml, adding a target and restart prometheus it simply crashes, i have already checked the yml for any errors. maybe someone here knows the issue?
@@Techdox the Prometheus container yes, i fixed it by completely reinstalling the lxc container and creating the yml from scratch. there was something weird going on with the container (lxc) and/or with the yml. Thanks for the reply! Greatest video on that topic :D
@@arjuna5051 you can, depends how it’s going to be accessed, making sure you setup users and with the permissions they need etc if it’s public you can put it behind Cloudflare zero trust SSO. There’s a few ways to go about it
Hi, bit late to the party I know, followed along and everything went great until I started the node_exporter service. Got a line at the bottom saying "info msg="TLS is disabled." http2=false address=[::]:9100"> Anybody know what I've done wrong?
@@Techdox Says that service status is running, but nothing coming through to Grafana. Just completely rebuilt server today and started from scratch and same message have not got to setting up dashboard yet will let you know. Yeah, completed all the steps shown in video, still same issue. Name of job does not show in Grafana and nothing coming through to dashboard obviously as not seeing job. Looks like data is being captured as when I run the curl statement seems to have some values showing. Should there be an IP address in the "listening on" line when you run the systemctl status command of the service. Mine is showing "Listening on" address=[::]:9100. Not sure what the issue was, but found that after recopying the prometheus.yml file it all came to life. All good.
Hey, im following the steps exactly, but when I docker compose up prometheus, I get a fail start. When I look at logs, I see: "field tergets not found in type struct { Targets []string \"yaml:\\\"targets\\\"\"; Labels model.LabelSet \"yaml:\\\"labels\\\"\" }" ts=2024-07-05T20:20:39.657Z caller=main.go:537 level=error msg="Error loading config (--config.file=/etc/prometheus/prometheus.yaml)"" Following it exact, I have even tried changing the path, but now working Seems the dir might not be passing throught?
Any idea why my Prometheus container is not hot-reloading? I was racking my brain trying to figure out why it wouldn't see my custom exporter after updating the prometheus.yml file, trying all kinds of ip/localhost/dns until I decided to remove the default metrics and saw no changes on the Prometheus web-app Sure enough, taking down the container and restarting it updated the metrics it could see. I'm using the same volume/binds as yours in the docker-compose.yml files
Truly, the most comprehensive and straight to the point tutorial.. 30 minutes that flew by, feels like 5. Thanks so much for such great work.
you are the goat. i was so close to giving up but this video made it so simple and easy. 📈🔥
I cannot express how greatful I am for this guide. This is the only one that worked first try with no problems :)
Glad you found it helpful :)
So satisfying to see this is working on my home's lab! Thank you much, bro! --- Greetings from Argentina
Hi, yet another excellent tutorial, I have been looking for such a clear set of explanations for a while. Hopefully I will have some time over the weekend to apply. Thanks for that, much appreciated 👍🏻
I agree another great tutorial 👍 may dig out an old pi to test before live environment
✅clear points
✅chapters in video
✅accompany blog posts
Concise, yet thorough. Wonderful explanations!
Gracias hombre.
praise you dude! just started to monitor my homelab, and this came at the right time. keep up the good work :D
Thanks! Appreciate it
Thanks. A tip for your video. Add 2 small separation lights behind you. Preferably tuned to the background color. Continued success
Great walktrough, followed step by step and everything is working.
Underrated channel! Looking forward to your growth!
Thank you!
Just discovered it... Liking & Subscribing NOW!!!
Great video!!!
Fantastic yet again! Thank you for the tutorial and keep them coming. Your teaching style is precise and very easily followed. I look forward to your next video! Cheers
Thank you :)
Very kewl stuff. Started a proxmox server(HP 800 G5 TWR, had a couple from work) 2 months ago and been learning a lot. I just have 1 ubuntu VM running with up to 10 docker containers, so far. Mostly for plex and Arrs...I like how you explain the commands in your videos. Trust me I've been on many youtubers sites...hehe I need to be able to expose some of my services soon, so more learning to do. Thx
This is ace. Thanks for posting. Very clear easy to follow. You have to like a nice graph and grafana certainly delivers
Agreed! Something satisfying when you get that dashboard setup and start to see the metrics flow in
Just what I needed Thank you !!!! super to the point !!!
Love this, thanks man, you are a life saver, just beautifully amazing
Excellent video and continue to keep up the good work.
Thanks
You are very kind! Thank you
really great tutorials, and helpfull person!! hope ur channel gonna be successfull!
Thank you Bro very useful this tutorial.
Excellent video. could you tell me what you use to complete docker options and other commands in terminal? iTerm2 ;) thanks
This was an excellent tutorial. I’m just starting to get into playing with Docker in my lab environment.
Is there a way to use this setup to query SNMP hosts like routers and wifi APs and printers??
Thanks in advance!
Hey mate, glad you found it useful!
So the same could be used in that situation I think, but instead of a node exporter you could use a SNMP exporter, I have not used it before though so kinda assuming here
ThankYou for making my life easy it was a hell, I was setting it up on EC2 Instance, tried various methods something was going wrong every time.
But I want to run Grafana on port=3020 its was unable to reach. anyways I set it up on port=3000
Thank you for posting this. I've tried doing the influxdb, telegraph, grafana way to visualize data on my servers, but that's overly complicated. This was very easy and straight forward. The only question I have is is there a way to set Prometheus to store metrics on remote storage? I'm running my docker container on a nuc with a small hard drive and would love to have metric data living on a share hosted by my TrueNAS server.
Yeah so you can set the Prometheus data location to the shared drive, Docker supports NFS as it’s pretty critical for a lot of production environments and swarm
Thanks!
Thank you for the support! Have a great day
Great tutorial .. clearly im doing something wrong somehow as Prometeus just loops in a restarting loop
Your Prometheus config file could have incorrect syntax
@@Techdox or I just can't spell Prometheus lol
@@Techdox I am however having issues with the node exporter user and running as a service. I'll post in discord as I really want to get this going
nice video but I am getting error while creating Data source . 403 Forbidden - There was an error returned querying the Prometheus API.
Hi Sir,
Can I ask something?
Why we are combining promethus and grafana?Instead we can use grafana only,right?
What extra advantage we will get if we combine these 2?
Grafana by itself is just for the dashboards, you need to bring data into Grafana and we are using Prometheus to grab that data and store it, we then pass that data over to Grafana to create nice dashboards
why alternative tools like Kibana, Microsoft Azure Monitor, and others are not being considered or used.@@Techdox
@@000shahul Azure monitor is not a self hosted solution. Kibana is just another product I have yet to make content around.
Excellent Video !! but I'm getting same issue as simuman - " Got a line at the bottom saying "info msg="TLS is disabled." http2=false address=[::]:9100"> "
and node_exporter data is not visible outside server, It's visible only inside the server
Jump into the discord, will be happy to help
I see only a single endpoint in Prometheus after setting it up. I’m not able to get the metrics on grafana. Please help!
Did you restart your Prometheus container so it can pick up the new exporters?
@@Techdox works now thanks a lot!
Thank you, this is wonderful.
I have a question: I have multiple servers and I want to display them within one Grafana dashboard, meaning more than one control panel for different servers. Is this possible? Please explain the matter to me.
Thank you! So you can have your servers in one dashboard. When you have say a CPU tile watching the utilisation, you can change the query of that tile and specifically point to your server and then rinse and repeat for the others.
By default I believe the tile is looking at what ever value you have selected for the server as a variable.
If that makes sense?
@@Techdox Okay, but how can I identify the secondary servers? Do I need to add them?
Or download Grafana again on each server?
I searched for a solution but couldn't find one.
@@m7mmadomar you follow the same steps I showed in this video. All data from your servers come from Prometheus and Node exporter, once all those logs are in Grafana you can use them as you please.
It’s just a matter of playing around with the Grafana queries
Alright, I am not very experienced in this matter. I will try.@@Techdox
Now that this works, can you access it outside your local network trough a Cloudflare tunnel?
100% just expose Grafanas port via your Cloudflare tunnel
Quick questions guys,
1. how to setup so it could read event at different network/wifi let say I install node exporter at other device as agent, aleady setup allow port and even turn off firewall but noting avail.
2. related to point one, I try to install in server (linux) but only accessible via domain like domain/metrics give me data unlike point number one but when I setup config YAML with domain its error and not even show in local 9090
notes I used macos m2 as host installed / setup Prometheus & Node Exporter via brew on terminal. Thx.
If you can, jump into the discord and I can try help there :)
Hi great video!
I just have a query, since I'm just trying to monitor two servers.
I create another job under the /prometheus/prometheus/prometheus.yml. The problem is that I cannot see the second job. And the first one is named as your job "elzim" I already change ip's and job names but still cannot get the second one running. I restarted the node exporter and both grafana to no avail.
Edit: I'm now able to see the second job, but not the second "ip/host" in grafana. And one of the jobs is still stuck with the "elzim" name
I have the current config.
- job_name: server1
static_configs:
- targets: ['x.x.x.x:9100']
- job_name: server2
static_configs:
- targets: ['y.y.y.y:9100']
I will appreciate any advice. Thank you!
This is my one - paste.techdox.nz/?d51044dfb4c2bab7#2zZqTg2vwUtfyWV7e4xzjG2mryqqc5YNn2AsmY8vKM2h
great video!
i have one problem (same issue i had when running it under windows),
when i edit the yml, adding a target and restart prometheus it simply crashes, i have already checked the yml for any errors.
maybe someone here knows the issue?
The container crashes? Are you sure you don't have the yaml formatted incorrectly? That will break things
@@Techdox the Prometheus container yes, i fixed it by completely reinstalling the lxc container and creating the yml from scratch. there was something weird going on with the container (lxc) and/or with the yml.
Thanks for the reply!
Greatest video on that topic :D
What terminal/shell are you using?
Hey! I’m using iTerm2 with OHMYZSH and the Dracula theme
Can I use this tutorial in a production environment? How can we make this more secure? And please make a video about integrating loki with these.
@@arjuna5051 you can, depends how it’s going to be accessed, making sure you setup users and with the permissions they need etc if it’s public you can put it behind Cloudflare zero trust SSO. There’s a few ways to go about it
Thanks. Your duplicati video really saved me when duplicati on one of my server stopped working and I couldn't troubleshoot it or reinstall.
By the way what terminal emulator do you use, I like the path preview popping up!
I’m using iterm2 but the path preview etc is a service called fig
Thanks a lot. Have applied your tutorial to my servers (mini Lenovo sff and qnap) successfully, amazing graphs! Thanks a bunch 🙏🏻
does it works for raspberry pi 4 64bit ARM?
Sure does, check out the image on DockerHub under tags to check support - hub.docker.com/r/grafana/grafana/tags
I have a question, on grafana how can you add more host?
Like more systems to monitor? You deploy node exporters on them
@@Techdox i have try this but it only appears on prometheus, but on grafana those not
@@Jpes211 jump into the discord and I can help. Link should be in my videos latest description
@@Techdox hi, sorry I wasn't on my pc, do you have time today?
@@Jpes211 yup, Discord is always open :)
Hi, bit late to the party I know, followed along and everything went great until I started the node_exporter service. Got a line at the bottom saying "info msg="TLS is disabled." http2=false address=[::]:9100"> Anybody know what I've done wrong?
Is the service running? I think that’s just a generic info message
@@Techdox Says that service status is running, but nothing coming through to Grafana. Just completely rebuilt server today and started from scratch and same message have not got to setting up dashboard yet will let you know. Yeah, completed all the steps shown in video, still same issue. Name of job does not show in Grafana and nothing coming through to dashboard obviously as not seeing job. Looks like data is being captured as when I run the curl statement seems to have some values showing. Should there be an IP address in the "listening on" line when you run the systemctl status command of the service. Mine is showing "Listening on" address=[::]:9100.
Not sure what the issue was, but found that after recopying the prometheus.yml file it all came to life. All good.
Hey, im following the steps exactly, but when I docker compose up prometheus, I get a fail start.
When I look at logs, I see:
"field tergets not found in type struct { Targets []string \"yaml:\\\"targets\\\"\"; Labels model.LabelSet \"yaml:\\\"labels\\\"\" }"
ts=2024-07-05T20:20:39.657Z caller=main.go:537 level=error msg="Error loading config (--config.file=/etc/prometheus/prometheus.yaml)""
Following it exact, I have even tried changing the path, but now working
Seems the dir might not be passing throught?
Any idea why my Prometheus container is not hot-reloading?
I was racking my brain trying to figure out why it wouldn't see my custom exporter after updating the prometheus.yml file, trying all kinds of ip/localhost/dns until I decided to remove the default metrics and saw no changes on the Prometheus web-app
Sure enough, taking down the container and restarting it updated the metrics it could see.
I'm using the same volume/binds as yours in the docker-compose.yml files
Can it hot reload? I have had to restart it each time I make a new connection. I think I showcased that in the video