Keep an eye on your network - Observium Tutorial
HTML-код
- Опубликовано: 14 июл 2024
- Thanks to Linode for sponsoring this video. Visit linode.com/CraftComputing to get a $100 60-Day credit when signing up for a new account.
A wise man once said "You can't know what's wrong if you never knew what right looks like". That was me. Keeping a watchful eye on your servers and network gear is quite a challenge, but where do you even begin? Today, I'm going to show you how to setup Observium to monitor and log activity for any device on your network. Best of all, its 100% free.
But first... What am I drinking???
From Big Ditch Brewing (NY) comes... a beer that tastes like beer in the form of A Beer That Tastes Like Beer. It's definitely a beer, and it tastes like a beer, but I'm not sure if it's the beer-like beer they were going for.
Copy/Paste instructions can be found here: drive.google.com/drive/folder...
Links to items below may be affiliate links for which I may be compensated
Hardware from my server rack
- Network -
UniFi UDM-Pro: amzn.to/2RV7gPg
UniFi G3 FLEX Camera: amzn.to/2EsGFpr
UniFi G4 PRO Camera: amzn.to/32W9RhU
UniFi USW-24-POE Gen2: amzn.to/3iZcCEK
Mikrotik CRS317-1G-16S+ on Amazon: amzn.to/3p5k5Gp
Mikrotik Desktop 5-port 10Gb Switch: amzn.to/2TvVezP
Mikrotik CRS328-24P-4S+ 24-Port PoE + 10Gb Switch: amzn.to/3fHpGjf
Ubiquiti UniFi UDM-Pro Router/Firewall: amzn.to/3fA3Spm
10Gb DAC cables: amzn.to/3wLrWeJ
HP NC523SFP Dual 10-Gigabit PCIe adapter: amzn.to/3fCjEjB
Asus XG-C100F 10Gbps PCIe 3.0 Network Card: amzn.to/3g1Ulqj
- HP DL80 Gen9 -
(Barebones) 2U: ebay.to/3ebduG4
HP Gen8 - Gen10 3.5" Drive Trays: amzn.to/3egChsf
Xeon E5-2660v3 - ebay.to/3i8JmMG
Xeon E5-2678v3 - ebay.to/3eh1yTb
- Supermicro Ultra Servers -
www.supermicro.com/en/product...
- Hyve Zeus 1U Servers -
Hyve Zeus Dual 2011-0 1U Barebones: ebay.to/2JxBFCI
Hyve Zeus Rails: ebay.to/3qxBJCU
Xeon E5-2650 v2: ebay.to/3lVJUWk
nVidia Quadro P400: ebay.to/2LyaiJg
nVidia Quadro P1000: ebay.to/3rIwf8n
HP Dual 10Gb SFP+ Card: ebay.to/3fpjmLk
1U PCIe Riser (the right one...): ebay.to/35TJLxG
- Chenbro NR12000 -
(Barebones) 1U 12-Bay: ebay.to/3s6X6eM
Chenbro NR12000 Rails: ebay.to/2O4NLF1
Hard Drive Rubber Grommets: amzn.to/35RPQKU
Drive Mount Screws (6-32 3/8" Truss Head): amzn.to/35Vn2kE
HGST 6TB SATA: amzn.to/3r32v57
PCIe 1U Riser: ebay.to/35TJLxG
- Storage Server -
Supermicro 846 Chassis: ebay.to/3pAilW1
Supermicro H11SSL-i SP3 Motherboard: amzn.to/3xtSkKG ebay.us/N73DGi
32GB DDR4-2666 ECC RDIMM (x8) - amzn.to/3zDledg ebay.us/rZwRv8
8TB HGST He8 Helium SAS: amzn.to/2X7Xb7C
Seagate FureCUDA 1TB NVMe: amzn.to/3iOOIij
Seagate IronWolf 110 1.92TB SATA SSD: amzn.to/3cNdKKG
EVGA Supernova T2 1600W: amzn.to/2U9cjjb
Noctua NH-U12S TR4-SP3: amzn.to/3q1lZby
Fractal Design Define 7: amzn.to/3iQJeUe
AMD Epyc 7601: amzn.to/3iKYNg5 ebay.us/D9ioE9
- Cloud Gaming Server -
AMD Epyc 7742: amzn.to/3hYoKIe
AsRock RACK ROMED8-2T Motherboard (7002/7003 Support): amzn.to/3xdxoal
256GB (8x32GB) DDR4 ECC-REG 2666: amzn.to/2TxxVpA
be quiet! Dark Power 12 1500W 80+ Titanium: amzn.to/3kTp0de
Asus Hyper m.2 x16 V2: amzn.to/3xdxtLb
Noctua NH-U12S TR4-SP3: amzn.to/3eOOftu
Noctua NH-U9 TR4-SP3: amzn.to/3eRAhHt
InWin R400N 4U Server Chassis: amzn.to/3BFYUjQ
- UPS Unit -
APC Smart-UPS RM SMT1500RM2U: amzn.to/3CIqc9G
- Air Conditioning -
Tripp Lite SRCOOOL7KRM on Amazon: amzn.to/34SYN6I
Duct Kit for SRCOOL7KRM: amzn.to/32L6Wrg
Grab yourself a Pint Glass at craftcomputing.store
Follow me on Twitter @CraftComputing
Support me on Patreon or Floatplane and get access to my exclusive Discord server. Chat with myself and the other hosts on Talking Heads all week long.
/ craftcomputing
www.floatplane.com/channel/Cr...
0:00 Intro
2:50 Observium Install
4:31 Linux SNMP v2c
7:44 Windows SNMP v2c
9:31 Linux SNMP v3
11:35 Windows ~~SNMP v3~~ WMI
15:51 Wrap up
18:02 Beer That Tastes Like Beer
Music:
Intractable by Kevin MacLeod
Link: incompetech.filmmusic.io/song...
License: filmmusic.io/standard-license 
Наука
I use Observium already, and wanted to note that my network printer and Observium even work together to accurately report levels and lifespan of internal components like the toner level and drum/fuser life remaining.
You can thank Tom for his printer obsession! 😅
"SNMP: Four lies in one acronym" -Michael Lucas
So last I looked, WMI isn't going away, one of the management tools for WMI is going away.
SNMP, however, is going away. Microsoft has already started the process. Not so much that they're not using SNMP with new features in Windows moving forward, but SNMP support was actively removed from their DHCP service. The last version of Windows you can query the DHCP service with SNMP was Server 2012 r2, iirc. 2016 and up, SNMP provides no information on DHCP.
@@praecorloth MS likes vendor lock-in, and SNMP is too standardized, making it too easy to switch your OS while keeping your monitoring tools. Thus, they must make a “better” service that works well enough that product managers believe the hype and sysadmins are not literally rioting at having to use it.
No notification (because i have those turned off) but literally like "oh hey, i forgot something and i know Jeff made a video"... Now i have no idea what i was looking for because here i am watching this one. Thanks Jeff... :-P
This always happens to me...
I have a whole playlist of the videos Jeff has posted that I find useful and may need to refer back too. Of course I may have to start a second because it's getting large
Thank you for going into so much detail regarding SNMP and WMI lifecycle and setup. Really appreciated.
Thanks for this tutorial - especially for all the DCOM configuration notes. All in one digestible section. Nice. Cheers 😊
I'd love to see what you look for when diagnosing common network anomalies.
I would say that depends entirely on the network.
The basic diagnostic mentality still stands regardless if it is a car, network, computer or something complex.
You have a box that is too big to explore, too big to do something meaningful with, so you basically have to start a process of elimination.
Basically find a way to make the box smaller but cutting out potential "troublemakers" and simplifying it.
There are rarely a common smoking gun that anyone can point at, there are common ones, but they can change pending on how the network is set up.
Home A and Home B, and even Business A and business B can have wildly different causes of their problems, which is why making it smaller is what you have to work on.
If it is a bad cable that seems to only act up on hot days because it is hidden deep inside the wall, or it is a service that breaks, you will often find it after eliminating what is working in the chain.
@@CMDRSweeper And if your issue is about network hardware card ? For instance one the NYC, adpater integrated or not you can have is bugged. Physically working but not responding as it's intended ? How physically can you detect this ? Intern software dedicated to theses components or do we have other things to consider ?
@@Apaurie Well you deploy the same service to a different machine, this is in the phase where you try to rule out if it is hardware related or software related as you make the box smaller.
In your example you will notice that it seems to work and your box is pointing to the hardware which could be a NIC, or something else, but you now have narrowed it down.
Of course, it all depends on what tools you have available to you, but you have to find a way to isolate probable causes to tackle them.
An example of a cause I tracked down was that Realtek network cards with my switch have a tendency to "kill" ports on it under Linux.
Basically what happens is that the port goes dead on the switch as if nothing was connected, reconnecting the cable does nothing.
How did I arrive at a firmware bug? Well I put a different NIC in from a different vendor and it worked for days.
But part of the diagnostic was that I did try to connect directly to the Realtek based NIC and it would respond fine, even if I swapped ports it would work for 2-3 days again, but that old port was dead.
Only commonality I had was that it was running Realtek under Linux, which had reports of being buggy, so I swapped the NIC to get around the problem.
A rather excellent intro into snmp. Thanks for sharing chap!
I have to be the super pedantic one: "depreciated" and "deprecated" are two very different words.
I was just going to write this :D
I usually try not to be pedantic about language, but this one is my pet peeve
YES! Thank you
I'm dumb...can you explain the difference?
@@isaackvasager9957 Deprecation (no i) in programming is the act of no longer using or developing something. Depreciation is a reduction in value or belittling of something. Microsoft isn't depreciating the newer version of SNMP, they're just not using it going forward. They're deprecating usage of SNMP in favor of WMI.
@@Blooest gotcha, thank you.
Ty for the run down on Observium. I would love a vid diving into using this to monitor networking equipment and printers.
Thanks for sharing linode with us
Great content as always. Would love to see a more in-depth video on this. Thanks again for the great content.
Just follow his text in the link...copy and paste
I did change the line to one line though sudo apt update && sudo apt install snmpd -y
easy peasy to install on LAN devices to see their status's...but like he said DO NOT open it to the WAN...unless you use the encrypted v3
agree
Thanks for the video, love the Defiant wallpaper.
Great tutorial! You've got a new subscriber!
Thanks, have been looking for something like this!!
Hello. Perfect start to the week
Loved the run down of Observium… you had the beer sitting there and I didn’t even realize it was Big Ditch. Buffalo guy here so like my eyes perked up when you said it. Never tried that one but Big Ditch is known in Buffalo for Hayburner for IPA’s. Past the season now but for fall there Cinnamon Apple Amber Ale is outstanding.
Another +1 for the Hayburners!! (Full disclaimer, another Buffalo guy here too...)
Hell yeah! Thanks! I thought they'd sunset on Observium, but glad a video is made!
Great video - I do this for a living. Used a handful of different enterprise monitoring solutions over the past few years. This free solution looks pretty robust though.
That shirt is amazing.
Bonus points for the Defiant wallpaper!
Love the content, keep it up!
Thanks for this, I did watch your video some time ago, but only got around to trying this software recently. Thank-you! I've used a number of enterprise packages professionally, but this product just makes things so much easier for my home network. Don't get me wrong, I've tried a few home-lab packages, but they all just turned into an Admin mess, so this ticks all the boxes for me.
that T-Shirt is 10/10
Would be great to spend a little time showing what the Observium software looks like and how easy it is to work with. I skimmed the video twice and nearly all the content was how to setup the clients to report data.
@Scott Myers flashed on the screen for 2 seconds. Try harder to troll next time.
Love that T-Shirt 🤣🍻
I have to elaborate slightly as I felt very similar when I had that beer before sending it to you. I agree the description on the can wasn't quite there and your review was spot on. Now for the price reveal, for a $8 for the 4 pack of 16oz cans it makes a decent lawn mower beer, they have a whole series of "beer that tastes like beer" and I need to try the rest to see which ones they really nail off any.
Like I said, I was definitely being overly critical because of the claims on the can. It's a beer I'd be happy to have after any amount of yard work. It's just not 'what a West coast IPA should taste like'.
@@CraftComputing completely agree! Great video as well!
Nice moment to post this video. I can use it for my homework for Wednesday hahha
Solid tutorial
Awesome shirt 👍
I’d love to see more about how to troubleshoot network issues. Like when a switch seems to be randomly eating packets. When I started having this issue recently I had literally no idea where to start other than moving Ethernet cables around to identify it was definitely the switch.
Ohhh Big Ditch. I've been to their brewery, good beers there!
Wow, great t-shirt.
Fun fact I used to work at an ISP after the creator of observium and worked there. So we had a very very early version installed.
I think it would be useful to see a walk through of the Observium and features of it
You should make a video on setting WDS up. That and SCCM if you use it. I learn much more from your videos than any others I find on the topic at hand.
I have a pair of WDS videos. Looking at writing a new one for Windows 11, but it's going to be a while out. They changed a lot of things and haven't yet released documentation.
Gotta admit, I love the shirt.
Apparently the Craft Computing secret lair is on the island of San Miguel off the coast of Portugal.
....You got me.
Nailed it with that Shirt!
I would recommend SHA-256 instead of MD5. Or is there preventing it?
My thoughts exactly!
SHA-256 supremacy!
MD5 is no longer secure.
Thanks Jeff! Can't wait for the Proxmox integration of Observium as I get an error when restarting the snmpd service 🙂
Edit : Got it working!
Love the Shirt.
Long time Nagios user .. will give Observium a close look.
watching this video I feel like I'm learning a new language by watching a TV channel in that language... I run all Windows servers at home (shared drives) and have run Linux servers but only for sendmail in 2001. have the server room itch!
Amazing video. Could you create one explaining how to create email alerts for device offline, high cpu, network overload , etc ?
Shoutout for Zabbix. Would love to see Jeff do a video on that.
"let's just throw in 6 high performance cpu cores and 8gb of ram to a vm that will use less than 10% of these resources"
meanwhile my homelab has 4 weak threads and 8gb ram total
on the positive side, your homelab is much more efficient
Do you have a current video of all the various software and services that you have running, along with a brief description of why and what it does
Loved this but learned the hard way to install SNMPD service on the linux first as Obersvium wouldn't pick them up. Honestly thinking this will replace PRTG for my network monitor. Internally only of course.
Outside of network devices I've never been a fan of SNMP. Was actually offered a job at a big tech company that made a software build around SNMP for monitoring data centers for large clients. Decided not to take that job and a few years later they let a lot of people go along with a number of other companies that were in that industry.
IMO stuff like elastic search, Prometheus (favorite for metrics), Elasticsearch w/ beats, influxDB, Graphite, or other time series database (TSDB) solution.
Holly shit.... when you opened that home page the map is showing my town.... Freaky stuff man 🤨🤨
Great video as usual! Can you do a tutorial for phpIPAM with SSL? Thanks!
Did anybody else try this script, it has been painful for me, still trying to get a running instance.
UPDATE1: Ubuntu 20.04 is terrible, 18.04, works like butter
UPDATE2: Observium Rocks!! Thanks for bringing this to us Jeff!!
18.04 might drop off our minimum versions at some point soon. 20.04 should work fine, what issues did you have?
pondered my orb to this
That t-shirt is 🔥🔥🔥 Where did you buy it? 🤩
Auto discovery is a main concern of mine. I was wondering if there is a more in depth video you could do about that. How to properly setup the config.php file and discovery.php -h commands
Awesome! I can tell you it will work on a Raspberry Pi 4B 8GB, have configured 9 devices..but BOY HOWDY does it bog it down! Definitely think this is for beefier machines
But it does work...even got it to connect to my pfSense machine on the LAN port of course :)
How bad is the bog down? I've been needing to condense my server down to a lower powered solution since I moved into an apartment. I was going to migrate most of my services to a collection of clustered Raspberry Pi 3B+
It shouldn't really be slow on a PI4, what storage are you using? The database will be practically empty with 9 devices. Is the slowdown all the time, or intermittent? If it's the latter, you can probably just reduce the number of poller threads. We assume a faster system so default to 2x as many threads as cores, so yours will be trying to poll 8 devices simultaneously. 1 or 2 threads would be better to spread the load over a longer period.
It's also quite i/o dependent, so make sure the SD is a good one.
@@dcvamps4594 When it starts polling and generating webpages it kicked usage up to 5.20+ and bogs it down...even had it self reboot a few times it got bogged down so hard...ended up moving it to an i5 system w/ 16 GB RAM to get it to run well, polling seems to take ALOT of resources, especially if you have multiple machines you are monitoring
It does calm down...but with it polling at approx 5min intervals it keeps it bogged way down
@@AdamArmstrong I have it in a DeskPi Pro case with a NVME M.2 to SATA adapter and it still bogged it down (even thru the USB3 connection)
@@haydenc2742 oof. That should easily have enough cpu and io to handle that small install. Do those devices have lots of ports? Try reducing the number of poller threads in the web config.
Would be nice to see how to setup service-admin-groups and automatic warning/alarm mails on failure.
Great tutorial, can you give an example of what the config should look like filled out for wmi, I'm getting access denied but if I test wmic on a windows device the credentials work.
Where did you get that shirt? I'd love to buy one like it as a gift for my brother.
I’d love a comparison to the TIG stack.
Can you use Obervium on Windows 10 devices? I have quite a few Desktop Windows devices and it'd be nice to monitor them like you do Windows Servers.
That T-shirt is winning, where can I grab one?
hey Jeff I just got a Tesla m40. I studied everything but always seemed to the "4g decoding" thing. do you know if those x79 Chinese boards have 4g docoding. I was trying it on my b85 board with e3 1245 v4. No luck. The only lga 1150 boards I've researched are a couple Asus z97 boards.
THANGS!!!!
Personally I prefer metricbeats and elastic for webapps and hosting monitoring. But this is interesting approach ...lower level data I think for network guys maybe appreciated
Observium and SNMP is only really common in Network gear. Usefulness is pretty limited tbh. Span ports with netflow and other stuff like that are way better. Combine with tools like Bro and elastic stacks (like SecurityOnion). For metrics Elastic+beats are pretty darn good. Personally prefer Prometheus for TSDB stuff but would not object to elastic it tuned right.
I have some rasperry pi's and NAS's that I would like to monitor. There is a comment that observian bogged down a pi4 but if a run observian on a more poweful machine, can I then monitor the rasperries (mostly rpi3, rpi4) without slowing them down?
Maybe I missed it, but once you have installed WMIC on the observium server, how do you add WMI client machines?
As far as I can tell in my digging (and replicating this setup) you can't. It looks like observium does SNMP or SNMP + WMI. Even according to Observium's documentation WMI is just for getting more granularity from windows services like exchange and SQL and I can't find anywhere where it says it natively support WMI only.
After several re-watches the video he only enables WMI on an already added machine.
I wasted 2 days setting , but learned a lot about getting WMI to report correctly from my servers, so I consider it a net win.
sir please can you make a video about How to configure 1 minute polling on Observium to send up/down alert fast thank you
11:45 DEPRECATED, not depreciated. On of my biggest pet peeves. (I still love you though)
I'd love to hear your thoughts and experiences with SIEM solutions, and whether any open source solution can be practical or provides a full toolset (asset detection, threat detection and mitigation)
Elasticsearch based solutions. Elastic has their own stuff. SecurityOnion. HELK. And so on.
Hi Jeff, I followed your guide and installed Ubuntu Server 20.04 and used the script like you did in your video. I am at the part where I am attempting to install the wmi plugin and fix the dependencies for it but once I do that systems stop polling. Looking into it when installing the two deb files it changes the python version from 3 to 2 and this causes the polling script to not be happy and say it is missing dependencies. Does this happen for you and if so how did you fix it?
Came here because of this..
Hey craft computing, Should i flash my gpu’s bios if my csm support cant stay disabled ?
@CraftComputing why you prefer full VM over containers?
Can this program scan a VMWare host and automatically add any VM it finds there?
Observium work with TrueNAS that I could use my Windows 10 machine to monitor what my TrueNAS server is doing?
I am trying to install this on a VM using your guide. I keep getting an "error: invalid option" after "Creating Observium dir"
Worried about the security of snmpv2 but accepts MD5 no questions asked. SMH.
Does Observium monitor the Windows application by using either SNMP or WMI?
I use observium and I agree it's very nice. However monitoring more in depth things like services/program/link to external machines doesn't seem to be user-friendly. Do you have examples?
I'm trying check-mk which is also very nice.
CheckMk / OMD can be dauthing to begin to use; but as a professional I must admit I love it :)
Use it a work monitoring a 849 hosts 27125 services; use it a home monitoring 19 hosts 164 services :-D
Is there a way to get all the data which observium is grabbing?
I don't know if API will work and do I need to configure Observium to allow API authentication.
this will work over VPN for remote PC and antenas? tnx
Nobody is going to talk about how that shirt isn't on the craft computing store???
Is there any reason to use md5 as auth algo? At a quick glance that sounds like a terrible idea
Is winrm the same as wmi? At work I use tools that require winrm config B4 it can collect windows metrics
And is dcom older winrm? Lol so many questions
What's the make and model number of that trackball you're using?
That is the Elecom HUGE. I've got a review of it on the channel.
Sadly configuring WMI to use a Local User Account breaks the whole WMI integration it has with Observium, insofar I haven't yet found a way to run it using a non-domain account without a WMI domain or WMI hostname.
It's SNMP secure ? I remember like 12 years ago we disabled all ok the office for safety reasons ...
in my observium, graph unable to generate.. please help
Oh my the t shirt
Hey,,, How to check OPTICAL POWER in observium
how about a video on Syslog server? What do you recommended, free is best lol
unrelated to the subject of the video but surprised Disney has not sued the pants off how ever sells that Daily Wiskey shirt.
Lenode, a nice idea. But do you trust your ISP that much....because once your ISP line goes down it takes all your services with it....and that's probably not what you want :P
What sets Observium apart from Zabbix?
Install script does not seem to support 21.10?
Why do just the update and not the upgrade?
Where are the chapters? I love chapters
For some reason my snmpd config file is totally different than yours. It only has 7 lines.
I figured it out. Observium wants to install snmpd for you. When doing this it overwrites the config file. Select N when it asks to install and manually install snmpd after.
Nvm this still doesn't work correctly.
@@ndb5348 I've run into the same issue. Looks totally different. are you to edit the snmp.conf on the Observium server? or the device being monitored?
So this only runs on Windows Sever or is that just what you wanted to use?
He installed it on Ubuntu
@@sccsuk Must have missed that... Thanks!
@@camerontgore no worries saw your post and though that's a bit pants if its windows only then he said Ubuntu vm lol 😆
@@sccsuk lol 😂 tbf my eyes started to glaze over when he started setting up his Windows Server so I absolutely missed it the first time. Can't wait to spin up a docker version in my home lab later today!
Is this better or Nagios?