I once lived in a house where the DSL would go out every day around 5 pm. I created a simple logger using ping to document this for the ISP, because they didn't believe me. When they finally sent a service person at the correct time, we discovered that one of their node boxes next to someone's lawn was being sprayed every day at 5 pm when the lawn sprinklers came on! No wonder it didn't turn up in their diagnostics!!
@@User_1795 If they put something dirt-cheap and simple (and they have to do this) on the "last mile" - it may go unnoticed for years. But how their equipment has survived regular showers? Miracle?
My Grandma has like 900 up and down. With a fiber going all the way to the router. In a distant village. In Ukraine. For ~15 USD. So you're actually having a first world problem, with an expensive slow Internet.
Well, it is about the same here, in Sweden. My mother and father 80+, which live in a rural area with 250+ inhabitants, do have faster Internet then what I have in the middle of a city. :-/
There are 4 available ISPs at my home in Siberia, 3 wired twisted pair etherned and one fiber (that's not counting mobile ones, which are 5 more or something). There's a promo option of 500Mbps symmetric for $8, but regular prices are more like $14 for the same speed. There are no 1gbps speeds (but you could connect to multiple ISPs and do ethernet bonding or something; i used to do that when unlimited internet was like 64kbps per ISP 13 years ago; pay-for-megabyte still was 100Mbps symmetric). One downside of siberian internet is that your latency to the nearest european datacenter is at least 100ms (50ms to Moscow). I was very surprised to see that here in Bay Area your internet is usually DOCSIS cable (a completely nonexistent thing in Russia; even ADSL is very rare, we jumped straight into 100Mbps TP at the very least with fiber coming a few years later) with 40Mbps up being the max that you could get at home for any amount money.
@@provod_ you are absolutely right. I'll tell you more: I've been to several places abroad, and the Bay Area was THE ONLY DAMN PLACE where staying in roaming and using 4G in roaming was actually cheaper than buying anything from a local carier.
US has some serious problem (and that's my guess) with legislation and taxes around infrastructure. Even Brazil, which is tax hell, we have 300Mb down, 150Mb up fiber everywhere, for about 40 USD. You would need to go into the mountains and where there is no paved roads to find places without decent internet.
In Australia with our National Broadband Network (NBN), a lot of ISPs were charging people for higher speeds than actually possible for their connection (e.g. charging 100mbs for a line that could only support 40mbs) this resulted in action by the Australian Consumer Commission. As a result, now, if you can prove you do not get at least greater than the next tier down in speed the ISP has to refund you the difference for all the months it has been charging you the higher rate. This tool could be quite useful for that.
Mine rarely doesn't give me full speed I get it's supposed to be 105mbps and like 12 or 15 up I don't remember but I have QOS set up but last time I turned it off and got like 135 I had not really heard of getting more before but was like cool
hahahahh as a turkish citizen, I just laughed. expensive internet: checked monitored internet: checked nigthmare of CGNAT: checked website blockage: checked
if you have the wan connections anyway then it might be worth using 'OneMarcFifty Virtual Network in Proxmox for MPTCP Test lab' as a starting point to increase the personal upload rate to a cheap remote diy vpn etc. "We build a Virtual Network in Proxmox for the MPTCP or OpenMPTCPRouter Test lab. In this video we do A LOT OF STUFF ! We create Proxmox network virtual networks, we use the tc netem QoS Filter of Linux for shaping of network lines, we add latency to the network in order to simulate a slow network, we install OpenWrt in Proxmox but also OpenMPTCPRouter. We will run a MATE desktop on a Linux machine in an LXC container and run the X2GO Server in the container plus the X2GO client Windows. Furthermore there is a nodejs client that shows ssh remote exexution integration. The three shaper machines are in fact linux LXCE containers which we turn into routers and dhcp servers. The full description with ALL commands, URLs and instructions etc is here (it is longer than 5000 characters)"
oc integrating Webtop in to the mix is also good, although i usually like vm's with 'nomachine' for no fuss single install better real time remote video streaming to a local android/linux/win gui terminal 'Techno Tim Linux desktop, inside of a container, inside of a browser??? Yes. A Webtop.'
I don't think they all do, I have Verizon Fios and I don't have a cap. However, Xfinity is available in my area also and I believe Xfinity has a data cap.
Yeah... i pay 40 euros for 1gb fiberglass a month. It's also giving me 950+/950+ mbps too, which is reasonable I think. But then again, I don't live in a third world country. It's a good thing that starlink comes to the rescue for those in need.
I have to disagree with this video. Isps can also speed up your traffic when connecting to sites like speedtest.com to make it seem like they're giving you what you asked for. We've seen similar things happen with mobile phones were they perform much better when being benchmarked than they would in the real world. There has to be another way to measure the data.
This is why they say "Speeds *up to* 1 Gbps". Speed isn't everything either - one time I was getting reasonable speeds but crippling packet loss due to a local fault.
Sooo considering speedtest-cli uses 1-1.3 GB every test, every 30 minutes would equal to around 50 GB a day or more. * 30, that's 1.5 TB a month, even with my gigabit internet through Comcast that would exceed the 1.2 TB/month quota and I would end up paying for extra bandwidth even if I never used my internet during those 30 days lol.
Good point-I didn't realize data caps were that low, OUCH! I might change the default to every 60 minutes at most, to help protect people from ISP stupidity. I also plan on making it easier to configure the interval so people could choose once a day or something more for long-term monitoring.
@@JeffGeerling I still really like this project, very impressive, I think I'm going to order a couple Shelly plugs! Thanks again for another awesome video! Long live the Pi generation. :)
I left Xfinity as soon as AT&T Fiber arrived. They actually threatened to BAN me for a year for using over 1 TB (back when 250MB was the limit). I had to move over to Comcast Business and sign a contract to get unlimited data. Then when they lifted the cap on their Xfinity (Residential) side they wouldn't let me move over. And then when the contract expired and AT&T Fiber arrived I said good bye and good riddance.
Thank you for the video Jeff. What about the impact to the other processes in your network while the test is happening? Have you noticed it? Also, the other way around, does the traffic in your network affects the result of the tests?
Verizon FIOS supposedly doesn't have a data cap, but I would not want a speedtest going off while trying to game online. Good video though. Maybe someone could make a much shorter version of speedtest.
Oh, believe me I have considered it often. But it's a lot lot LOT more than just the technical problems you have to deal with. I'm not sure if I'm up for the political and social maneuvering that's involved in the local community (which can often lead to a strong division and a lot of foul stares).
@@JeffGeerling well you don't really need to allow that stop you from starting the business. There are always others who can take care of the political and social maneuvering while you take care of the technological aspect of the business.
@@theundertaker5963 North America is literally the worst place to start up an internet service provider fresh from ground zero. You need millions in capital and political leverage to shoo out the 3rd party monopoly providers from your region. It's a financial risk that literally no one will give you money for, unless you have the cooperation of every single seat of power in local government.
@@JosephArata well, dont think so much "start up an internet service provider" more private diy Far bigger pipe than any single isp WAN... without the 3rd party isp hassle/continuous cost starting with how jeff Can make a long distance private diy big pipe to/from his sister ( how many miles line of site) with pi's, old cheap/free 5v-12v wireless routers + pv+ reclaimed batteries, several collectively routed with m.ruclips.net/video/S-Xmcig1ddA/видео.html 'OneMarcFifty Virtual Network in Proxmox for MPTCP Test lab' at both ends & potentially every pi relay in-between with cheap integrated m.ruclips.net/video/wEtlbNir8KE/видео.html LoRa LoRaWAN Range World Record Attempt (Re-published No. 120) Andreas Spiess for collective instrumentation & small quick txt, alerts etc ..collect up the sw tools and shell script it to make a working one off on demand clean usb Debian super light 'slax' slax org customize .php deployable from pi-KVM etc
In france, I pay 38€ for a sold _up to_ 8Gb down/1Gb up which is, in fact, a measured 5Gb down/1Gb up connection per month with no data cap. I don't complain with the difference, i was never able to use my connection at its full speed.
Grafana on itself does nothing, you need some data to display in Grafana. Typically this would be in a database (in this video, it's Prometheus, but it could also be MySQL, SQL Server, etc). But then you also need something to put in that database in the first place, which in this video are the speed test results and the website checks.
@@realityveil6151 Dude, you cannot put VM into active speaker to be working as LMS client and DAC. And this is only one example - actually 5 of my Pi are LMS clients and DACs. Everything has its purpose
Thanks Jeff! Xfinity/Comcast have a monopoly in our area. Feels like an abusive relationship. Being able to keep them honest, and know what our devices are doing is super useful. Cheers!
Of course they do. Time Warner won't operate in cities/states they are in. And they don't operate in states TW operates in... It's collusion. They need to be broken up. Wtf is congress even doing?
They're not lying. It says right on the ad "Up to...". What's more, the rates are for the last mile, ISPs make no guarantee of your throughput over the public internet, over which they have no control, and they don't even guarantee your throughput across their own backbone, because the internet is an "as-able" service. All you're really doing is generating pointless traffic for the express purpose of documenting what you already should know: the internet is not reliable. It's not designed to be reliable, in fact, the whole reason the ARPAnet was originally built was to try and create a communication network which could survive a nuclear war. If you want to buy a synchronous gigabit connection, you're welcome to, just be prepared to pay about a grand a month, because the ISP's cost model is built around the understanding that you'll be a client, not a host, if you're using residential internet. From a former ISP engineer: Stop doing what you're doing. You're making the problem worse by spamming out bogus internet traffic on a cron job. If you want to monitor the quality of your network connection, install ElasticSearch Beats or Cacti or something, and monitor your traffic sent and received, and set up Smokeping to monitor your latency. Don't be a douche-weasel and DDOS your upstream just to prove you're getting less than the marketing team printed on the box.
@@pgp-acc Most connection in HUN are asymetric dl/ul (even business packages other than VDSL), but for 7200HUF/mo (approx 20-22EUR) Telekom offers 2000/1000 on fiber here too (for residential consumers). Not anywhere of course, we have areas where ADSL on crap-old phone copper still cost like 9k HUF (30EUR) and barely reach 10M/500kbps up.
Nice video as usual 🙂 Well, I think that this problem is relatively extended in every country of the world I guess. Until 20 years ago in Italy there was a single telephone and internet operator but, at the end of this, nothing changed to much because the entire national infrastructure still was owned by that company, so the other "new companies" had to rent part of the network. To be short, this caused a lot of issues as there were more ISPs on the same infrastructure that wasn't upgraded as needed up during the years mainly for the high costs: here if you want to "put down" a new line/fiber you have to broke up streets as the old pipes are too small and full of old cables. Anyway, until 5 years ago I was paying 45€ monthly for a 6 Mbps ADSL, then the fiber (FTTC 100/20) finally came and, as I'm not so far to the cabinet I'm not losing too much signal. I still paying likely 50€ monthly but at least now all it's working as intended. Ah, phone calls are included 😆 Sorry for the poem and thanks for your work here.
Hey, you're sitting in the city with your 1 Gbps plan, out here in St. Charles County we get 25/5 and even then AT&T can't deliver it without some spottiness!
Hi Jeff, it would be great if u did a video about backing up the sd card on a pi to a nas. I am kinda worried it will fail and having to set everything up again.
It should be noted that if you have internet speed greater than 300 Mb/s, you MUST use a R Pi 4 as the 3B+ only has a 300 Mb/s NIC. Otherwise great video as always!
If you want the automated speed tests, then yes. If you're just using it as a pi-hole, then no. The traffic doesn't flow through the Pi; it just resolves DNS queries. But good point nonetheless.
So meaning from my home to ISP but not my home to ISP to Google for instance? So the advertised bandwidth means literally nothing if my ISP's connection to google is slow in that example?
So there are peering agreements for ISPs, they have to peer with someone else to get access to the internet. Literal ISPs for the ISPs. These are expensive contracts for multi gigabit connections. Not too mention BGP ASN’s and the constant battle to obtain more v4 addresses from ARIN or other addressing entities. They do have their own battles.
@@DeadlyDragon_ In a past job, I was the one to have to fight those battles - can 100% confirm they're tough. We were able to operate with only one ASN (as many nets are) by using the same routing policy everywhere, and we addressed (pun not intended) the IPv4 shortage problem by going the DS-Lite route (pun still not intended). But all of that was a walk in the park compared to the challenge of purchasing enough transit to meet customer demand. We kept having to readjust our oversell ratios as we updated our traffic engineering practices, and getting a good handle on what "peak demand" looks like is a surprisingly complicated question. We weren't yet big enough to get caching appliances from Akamai, Netflix, and friends... not that I wasn't checking constantly. But we always strove to do right by our customers - small ISPs have to or they don't survive. (So be a customer of a small ISP! Support local businesses that actually give a crap about you!) Glad I wasn't still working there for the COVID pandemic. I would hate to be the one to have to figure out how to handle a bunch of WFH traffic overnight! :)
i just wanna say, i set this up today as a linux noob. your tutorial was so easy to follow, but i did have pi knowledge. i had a big fight with errors, but fixed them by giving 'pi' docker perms [thanks issue tracker!] this tool is so nice. i have pi hole just to track queries, as i have two ad blockers already. the internet-tracker is super nice. im gonna gather data on this as i love seeing data. cheers!
Found this channel when looking for Raspberry pi. Had no Pi back then but still loved to watch this valuable content. Now, Thanks to God I got a Pi 4 4GB and I love watching to your videos. Keep up the good work.
In Thailand, many ISP can let's customer to set up Internet speed by their owners from the ISP website, for example, customer package is 500/500 Mbps, customer can set speed 500/500 or 600/400 or 800/200 or 900/100 Mbps.
Really great videos, and information on new and upcoming hardware, the only problem I see, is your github instructions on this are not the greatest. For someone like myself that is still learning about linux systems by utilizing a raspberry Pi, again for someone like myself hat knows enough to be dangerous, can really get frustrated with install instructions that are lacking, and would make someone want to just throw in the towel all together and stop learning more about raspberry pi's. This is by no means a dig on you, people like you are what made me want to get into buying my 1st raspberry Pi, but again with the github install instructions, its really hard to even tell what exactly it is that I am installing, or how specifically. Am I installing Ansible, or Pi Hole with docker, or do I have to have docker already installed, is it installing ansible? it's really hard to understand what all the instructions are having me do.
Spectrum isn't fleecing you. If you're not seeing over 700 Mbps, it's likely due to one of the following. 1. You're not using a Spectrum provided DOCSIS 3.1 2.5G modem. 2. Your router (and possible your switch) between your modem and Raspberry Pi is slowing you down due to its settings (packet inspection, prioritization, etc.) or hardware limitations.. 3. The Raspberry Pi, your switch, and possibly your router only have a 1 Gigabit ethernet port, which with ethernet and TCP/IP overhead means you're limited to about 900Mbps. 4. The software on the Rapsberry Pi is also slowing down your network speed. So do some more investigation on your home network and I'm sure you'll be able to obtain higher speeds.
You're lucky having 30. Here I can only get 1000/10. It's almost always saturated, and I have to host a lot of stuff on the cloud simply because my connection would get choked as soon as someone remotely starts a download.
Im thankful a smaller local ISP started up near me. I'm a few miles outside the ST Louis metro area and I just went from only having satellite to a fiber connection which is faster by 100 fold. Spectrum wanted 20k 6 months before Gateway fiber just ran the line down our street for free if we payed for a subscription. I pay for a gigabit connection. I still see what you get most of the time like 700-900 down depending on the time of day but my uploads are always right around 900+. Still very happy with them, I only spend about 90$ a month on it.
One missing feature. With everyone monitoring their ISP 7/24 there needs to be an opt in option for users to send anonymized data with ISP, Location down to the city level to a central site on the internet. Then this can be a public service that anyone can search their location/city and see the averages that users monitoring 7/24 in that city are getting from each ISP including outages. This will allow everyone to know before hand which ISP is better to go with and turn this project into the next level.
I'd like to also see a crowd-sourced ISP statistics! This way, no more pesky ISPs claiming "no our internet is not the problem, go restart your router."
This is a really cool idea, but I can't think of a way to stop ISPs from poisoning the data. As the service gets more popular, it's more likely that ISPs would send fake data to it
My pi server: fuguhub (which I can’t uninstall though I want to, the uninstall script doesn’t work and I can’t find it in Apache), temp monitoring with dht11 (integrated with Hubitat), pi hole, cloudflared DoH, and homebridge
Thank you Jeff! I have played around with a friends Raspberry Pi several years ago but didn't have any good use for it until now. Went out and bought my first Pi because of this video :)
Me: after promotion ends 220 mbps down and 25 up costs $150 a month with xfinity in suburbs of Atlanta. Oh, and only 1.2 tb of bandwidth. Canada: 1 gigabit fiber for $80 a month due to government-owned fiber cables that are subsidized and sold to various isps to encourage competition and drive down prices
ok KNOCK IT OFF!. You RUclips guys keep doing the same thing with the links. "It costs $35 bucks" Shows affiliated link to a Pi that's $64 bucks or a $99 kit.
getting 920-940 down and 500-550 up and i pay $55 a month for 1000/500 fiber with static IP. They promise at least 900 down and 400 up and at one point they wasn't able to deliver more than 700-800 down so i had free internet for 6 months as i told them i was holding them to their advertisement promises. We agreed that i kept paying until it was fixed, and then when it was fixed, verified by me, i would get as many months in return for free and on top of that a static IP for a year for free as i was paying for it at that point. I would have been fine with 50% off for the time frame but i kept my mouth shut and let them dig their own hole :D What i do instead of using pihole as my LAN DNS i use my UDM Pro, and then on the WAN interface tell the UDM that all WAN requests should go through the pihole. I don't have a great experience with PiHole as DNS on LAN to access my devices through their hostname, it was really hit and miss, so i moved the pihole to the edge of my network so it's not my LAN DNS. Also by doing it this way, you don't break any privacy laws there might be in your country, because your able to see whose device access what site, and you would be able to if the PiHole is your LANs DNS. By moving it to the WAN interface as DNS you only get two request sources and none of them is from any of the devices on the network, one is itself by localhost and the other is itself by IP on the device it's running, in my case as a docker on my NAS. On top of that if you run it on a pi and you only got a single one and it breaks down, you would have to go through updating your config on the router and then refresh all devices to get the new DNS, often by restarting if it's IoT devices, whereas if you do it WAN side, you would only need to do the router configuration and your up and running again with internet access :D
I'm likely doing the math wrong but 30~40MB every 30 minutes = ~43GB a month of wasted data for a dashboard, In a country that has data caps like we do this seems like a complete waste of resources.
Sadly I would need a 10 Gbps up and down on the Pi - Switzerland has good internet ;-) So far, my trouble was finding iPerf3 siting capable of delivering up to that speed. So far, fastest speeds I found was slightly above 4Gbps
Interesting-I've had some trouble with my HP desktop getting anything more than 4-5 Gbps, but I haven't had an issue with my two M1 macs-they get 9.4 Gbps or so on the local network. I don't have a faster Internet connection though so I'm not sure how it would do through the public Internet. I wonder if it's something related to the 10th-gen Intel i3 CPU I have in my Windows PC, something with the ConnectX driver (I tried an X3 and X2 card so far, same result on each), or something else entirely!
@@JeffGeerling Internally, yes 10 Gbs is no problem... it's getting that speed to the outside tested on iPerf3 servers that gets difficult: most servers do not offer that bandwidth.
It would be awesome to have a real time traffic monitoring in place. As the pi hole is already recording DNS requests call, it would be pretty nice to see what clients are currently doing on the Internet, since the pihole frontend it’s basically just a list of requests that were resolved. This data may probably be relevant to show such a real time traffic, but it itself doesn’t record the traffic volume for instance.
Running a speed test every 30 minutes is pretty abusive of the speedtest.net network bandwidth. Various organizations volunteer their bandwidth to the service. If you are going to run a speed test with that frequency, you should get a DigitalOcean droplet or similar for the purpose and run the Ookla server on that and point your client at it. For one thing, you'll know that you have the full 1GBps pipe on the droplet to yourself resulting in more consistent measurements and you also won't be abusing freely available resources.
None of the European comparisons are apples to apples. The large geography, fragmented regulation etc. of large land mass countries will always allow pockets of monopolies and more expensive operations than small densely populated land areas. You get gigabit or more for less than the price of a pizza in countries like South Korea et al. The better measure is the quality of service and uptime from any ISP.
@@pd8559 Well I pay $7 for 1 gigabit down and 500Mbps up. Real speeds are constant 940Mbps down, 510-520Mbps up. And as for the uptime, I am pretty much always online (me and my servers) and in 10 years I had 1 single time about 2 hours of Internet outage. SO I would say the uptime is good. I live in Romania.
Where I live, when you have high latency or slower than promised speeds, you call the ISP and it’s fixed in a few hours. ToS go both ways. Greetings from Estonia!
Symmetrical. You would prefer 150mbps of symmetrical. Asymmetrical means exactly what you already have with the 1gbps down and 40mbps up. Don't sweat it Jeff, I made this mistake too.
They beat me to it, but yeah...I much preferred my 75M synchronous/symmetric FiOS at my old address over the 1G asynchronous/asymmetric Spectrum Business at my current one. Unfortunately I have no other options besides cable here, the phone lines are too old to support more than 128k (per AT&T) and there is no fiber in the area.
@@MarkusOrt It's mostly TCP overhead which if you had a gig would come out to around 949Mbps max. There's just no way to get a 1000Mbps on a 1000Mbps link negotiation.
Wow, I had 40mbps upload around 2005, for $10 per month... That's what monopoly does... and we're talking here about the country that pretends to be a world leader :D
Thanks for your videos. I've learned a lot from you. I'm trying to build the ISP monitor, but I'm stuck on step 4 of the Setup (on GitHub). Look at that paragraph again -- as a novice like me. "Make copies of the following files and customize them to your liking: example.inventory.ini to inventory.ini (replace IP address with your Pi's IP, or comment that line and uncomment the connection=local line if you're running it on the Pi you're setting up). example.config.yml to config.yml" What IP address do you see that I have to replace? Maybe it doesn't show up because I'm setting it up on my Pi using VNC? I'm not great with code, so I'm stuck. Thanks in advance for you help.
4:50 - YES!! There is little to no reason ISPs couldn't offer a 150 symmetrical service. DOCSIS 3.0 supports 200Mbps upstream, 3.1 is over 1Gbps, and DOCSIS 4.0 is 6Gbps. They should make that the standard just to properly compete with fiber ISPs.
I run 2 VM's running debian 10 with PiHole installed on my home server. I have yet to set them up to sync together but Techno Tim has a walk through on how to do it! Though i'm guessing it's on Jeff's radar as well. I had to run 2, as 1: Router would not take a blank dns 2 and 0.0.0.0 didn't work lol 2: Setting up multiple interfaces in pihole was outside of my scope at the time (though i probably could do it now) and 3: redundancy! I can update one while the other takes care of all the dns requests.
Hi jeff, vishal here, am having a terrible issue with my pi or home network, when I'm trying to access my pi or virtual machine with ip there is no problem, but when i setup a local domain to it like homenet.net or something like that i can only access it on that particular computer(where site is hosted), and when i try to use that domain on different computer on same network it says domain unreachable, please tell me how to solve this issue... thank you..
Thank You Jeff.. I really like this project, looking to switch from High Cost Spectrum to T-Mobile and monitoring before/after would be perfect. Have everything installed however I am stuck at the "TASK [Ensure internet-monitoring environment is running.]" task. Throws an error "fatal: [localhost]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))"} ". Any ideas????
i just installed the internet monitor on my pi4 and it works great, just one thing - it took me like 10-15 minutes to figure out the password to the Grafana dashboard..and as this is the first Docker container I run , had no idea how to find its configuration files - until I realized the playbook (?) created another directory on my home folder with all the configs (internet-monitoring). maybe it's worth mentioning this somewhere in the Github readme? I originally followed the install instructions on the blog
Thanks for this. I already have a Pi running PiHole connected to the router, but I've been using a scheduled task on a windows machine to run an hourly speedtest to track my internet speeds. This looks like a much better setup.
Great video BUT... for people with metered Internet plans running Speedtest every half hour could be a very expensive mistake. It pulls and pushes a fair bit of data.
Right now it installs it by default, and there's no option to skip it-please open an issue here to get that feature added: github.com/geerlingguy/internet-pi/issues Though you can install it and just not use it too :)
When I tried to switch to an LTE ISP, I had to talk to the CenturyLink customer retention specialist. They pretty much gave up when I told them "Look, I'll pay you $200/mo if you will give me 100Mbps down." They act like they don't understand how 3Mbps is not sufficient for anything except for email/browsing nowadays. Without ad block, my wife's PC consumes the full 3Mbps down when she opens her forums due to all the auto starting videos/ads. I currently pay $90/mo for two 3Mbps dry DSL circuits. CenturyLink/Frontier/Hughes/Dishnet are going to be in a world of hurt when Starlink gets rolled out in earnest...
That would be a really nice improvement! For my own needs, the Pi is in a rack so I wouldn't see it, but it would be cool to have that data available. Maybe open an issue on the internet-pi repo and we'll see what we can do!
@@JeffGeerling maybe I can contribute as well.. I have some experience on these i2c displays ..I’ll try the repo and I’ll try to modify it so we can also print on the display
europe has already solved this problem for the most part and still working on it, US should learn a thing or two, we always learn things from each other, why not internet too
If this method can be used to measure an internet connection speed every X minutes, it could 100% utilize this internet channel when taking a measurements?
I'm really annoyed by Vodafone's stupid marketing. They promote their 1Gb/s with their cringe ads. The catch is the upload speed is 50Mb/s only. Also they promote(d) the 1Gb package is now with 50% discount. The catch? It does fricking work in my entire city and I live in decently large one. The best i can get is 100Mb and only 38Mb in some areas, so their claim "We bring the fastest Internet almost all households." O2 on the other hand have much better plans and prices. Their 1Gb/s Internet has 100Mb/s upload and its ≈8€/9,5$ cheaper than Vodafone.
We pay $50/month for AT&T internet and we're lucky if we get over 15 megabits per second. Absolute BS. We live in a major suburb, and they won't run fiber.
@@JeffGeerling I have recently begun getting ads in the mail from AT&T touting their "fiber." BUT ... I question whether this actually a fiber connection to the house, or is just a fiber backbone. I currently have Comcast 100Mbps/6 for $65 in a Chicago suburb , it is generally reliable and consistent on speed. So I am not terribly unhappy and don't see any compelling reason to go through the hassle of changing. At least for now. Higher upload would be nice, though.
Jeff, you should get or add an Ripe Atlas Probe! Contributing to the global data research as well as having a great additional monitoring for your ISP as well as connectivity and routing aka peering.
I feel bad for you, my ISP's Gigabit is really Gigabit, meaning 1000/1000, and it costs me 50$ a month ofc, it's not leased line experience, far from it, but I can safely say that I got about 90% of the expected speed on a 90% expected time of need.
Splunk is great too! Really nowadays we're spoiled for monitoring and visualization tools. I'm guessing in 10 years only a few will be left standing, just because I think there are *too many* good ones currently.
Wifi (and any wireless connection) is not just useless for checking speeds, its also useless for input, video, or data transfers. The only reason it even exists is for convenience. THATS IT the ONLY thing it does better then a wire is 'plug in', every single other thing you want to do with a connection is worse through wireless.
It's running speedtest through the pi's network port? We have 1gb fiber and we have a 2.5gb link to the pfsense box and a 10gb backbone. It woke the network right up!
Wouldn't running a speed test every 30 minutes throttle your speeds for about 30 seconds every 30 minutes? Seems like that would be annoying for gaming or streaming
I have the same plan same price with Spectrum, no other options. It goes out often and one time it took them 11 months to figure out a packet loss issue smh. I currently have a Pi 3b+ setup to email me ping results every 6 hours via a cron. Unfortunately the Pi 3b+ caps out at ~330Mbps in a speed test due to I believe the USB controller iirc so I am thinking of upgrading it to a 4. It’s also currently my Pihole. Great vid!
I once lived in a house where the DSL would go out every day around 5 pm. I created a simple logger using ping to document this for the ISP, because they didn't believe me. When they finally sent a service person at the correct time, we discovered that one of their node boxes next to someone's lawn was being sprayed every day at 5 pm when the lawn sprinklers came on! No wonder it didn't turn up in their diagnostics!!
Omg that had to suck. That's crazy they didn't even know they're shit was shorting.
@@User_1795 If they put something dirt-cheap and simple (and they have to do this) on the "last mile" - it may go unnoticed for years. But how their equipment has survived regular showers? Miracle?
Now that's real-world troubleshooting and diagnostics.
@@IDGinUkraine I'm sure these things are designed to mitigate the damage from water ingress. Probably some good drain holes and passive airflow.
Few years back i was on DSL and everytime somebody was using the phone the internet would stop 😂
My Grandma has like 900 up and down. With a fiber going all the way to the router. In a distant village. In Ukraine. For ~15 USD.
So you're actually having a first world problem, with an expensive slow Internet.
Well, it is about the same here, in Sweden. My mother and father 80+, which live in a rural area with 250+ inhabitants, do have faster Internet then what I have in the middle of a city. :-/
There are 4 available ISPs at my home in Siberia, 3 wired twisted pair etherned and one fiber (that's not counting mobile ones, which are 5 more or something). There's a promo option of 500Mbps symmetric for $8, but regular prices are more like $14 for the same speed. There are no 1gbps speeds (but you could connect to multiple ISPs and do ethernet bonding or something; i used to do that when unlimited internet was like 64kbps per ISP 13 years ago; pay-for-megabyte still was 100Mbps symmetric). One downside of siberian internet is that your latency to the nearest european datacenter is at least 100ms (50ms to Moscow).
I was very surprised to see that here in Bay Area your internet is usually DOCSIS cable (a completely nonexistent thing in Russia; even ADSL is very rare, we jumped straight into 100Mbps TP at the very least with fiber coming a few years later) with 40Mbps up being the max that you could get at home for any amount money.
@@provod_ you are absolutely right. I'll tell you more: I've been to several places abroad, and the Bay Area was THE ONLY DAMN PLACE where staying in roaming and using 4G in roaming was actually cheaper than buying anything from a local carier.
Same in svk
US has some serious problem (and that's my guess) with legislation and taxes around infrastructure. Even Brazil, which is tax hell, we have 300Mb down, 150Mb up fiber everywhere, for about 40 USD. You would need to go into the mountains and where there is no paved roads to find places without decent internet.
When your ISP promises "UP TO", and you get 3/4 of that promise, consider yourself lucky!
"Count your blessings"
@@JeffGeerling "Pray we don't alter the deal further" lol
@@mewtwo064 "HERE IS A UNICYCLE"
In Australia with our National Broadband Network (NBN), a lot of ISPs were charging people for higher speeds than actually possible for their connection (e.g. charging 100mbs for a line that could only support 40mbs) this resulted in action by the Australian Consumer Commission. As a result, now, if you can prove you do not get at least greater than the next tier down in speed the ISP has to refund you the difference for all the months it has been charging you the higher rate. This tool could be quite useful for that.
Mine rarely doesn't give me full speed I get it's supposed to be 105mbps and like 12 or 15 up I don't remember but I have QOS set up but last time I turned it off and got like 135 I had not really heard of getting more before but was like cool
hahahahh as a turkish citizen, I just laughed.
expensive internet: checked
monitored internet: checked
nigthmare of CGNAT: checked
website blockage: checked
At least I get a stable IP address!
Let's not forget "fair usage policy" and lack of ipv6 connection of any sort.
if you have the wan connections anyway then it might be worth using
'OneMarcFifty Virtual Network in Proxmox for MPTCP Test lab'
as a starting point to increase the personal upload rate to a cheap remote diy vpn etc.
"We build a Virtual Network in Proxmox for the MPTCP or OpenMPTCPRouter Test lab. In this video we do A LOT OF STUFF ! We create Proxmox network virtual networks, we use the tc netem QoS Filter of Linux for shaping of network lines, we add latency to the network in order to simulate a slow network, we install OpenWrt in Proxmox but also OpenMPTCPRouter. We will run a MATE desktop on a Linux machine in an LXC container and run the X2GO Server in the container plus the X2GO client Windows. Furthermore there is a nodejs client that shows ssh remote exexution integration. The three shaper machines are in fact linux LXCE containers which we turn into routers and dhcp servers.
The full description with ALL commands, URLs and instructions etc is here (it is longer than 5000 characters)"
oc integrating Webtop in to the mix is also good, although i usually like vm's with 'nomachine' for no fuss single install better real time remote video streaming to a local android/linux/win gui terminal
'Techno Tim Linux desktop, inside of a container, inside of a browser??? Yes. A Webtop.'
Website blockages when the government's involved for censorship reasons then that's great and is a reality for some countries
"My cable internet connection which is supposed to be 1Gbps, is only about 700Mbps on average..."
Me: **cries in 10Mbps ADSL**
ISPs in the US sell gigabit plans with a 1TB/mo data cap, which you could literally burn through in a few hours 😂
I don't think they all do, I have Verizon Fios and I don't have a cap. However, Xfinity is available in my area also and I believe Xfinity has a data cap.
I'm lucky to not have a data cap. I can't imagine living under those conditions.
@@abousono1 It’s mostly cable providers that do this, that’s true.
Yup comcast tried to pull this shit in the middle of the pandemic. They quickly pulled back and “graciously” decided to wait on fully implementing it.
I have Suddenlink with 400 down and 40 upload with unlimited data and for our 4-30-21 billing period my house used a whopping 3.2tb of data
hearing $150 a month for internet is giving me a heart attack
Yeah... i pay 40 euros for 1gb fiberglass a month. It's also giving me 950+/950+ mbps too, which is reasonable I think. But then again, I don't live in a third world country.
It's a good thing that starlink comes to the rescue for those in need.
@@MrMediator24 Very true in the USA, but i think/hope Starlink is going to change that.
go look at business internet, you have a stroke and heart attack at the same
Workers get overpaid there so it evens out, I guess.
for like $80 we get 50mbps in australia, and the average cost for a 1gbps is $321.50 all in aud
This channel is such a goldmine of useful content.
Agree, this is such a great channel.
Everything you could ever want to know about random electronics
I have to disagree with this video. Isps can also speed up your traffic when connecting to sites like speedtest.com to make it seem like they're giving you what you asked for.
We've seen similar things happen with mobile phones were they perform much better when being benchmarked than they would in the real world.
There has to be another way to measure the data.
This is why they say "Speeds *up to* 1 Gbps". Speed isn't everything either - one time I was getting reasonable speeds but crippling packet loss due to a local fault.
Sooo considering speedtest-cli uses 1-1.3 GB every test, every 30 minutes would equal to around 50 GB a day or more. * 30, that's 1.5 TB a month, even with my gigabit internet through Comcast that would exceed the 1.2 TB/month quota and I would end up paying for extra bandwidth even if I never used my internet during those 30 days lol.
Good point-I didn't realize data caps were that low, OUCH!
I might change the default to every 60 minutes at most, to help protect people from ISP stupidity. I also plan on making it easier to configure the interval so people could choose once a day or something more for long-term monitoring.
@@JeffGeerling I still really like this project, very impressive, I think I'm going to order a couple Shelly plugs! Thanks again for another awesome video! Long live the Pi generation. :)
I left Xfinity as soon as AT&T Fiber arrived. They actually threatened to BAN me for a year for using over 1 TB (back when 250MB was the limit). I had to move over to Comcast Business and sign a contract to get unlimited data. Then when they lifted the cap on their Xfinity (Residential) side they wouldn't let me move over. And then when the contract expired and AT&T Fiber arrived I said good bye and good riddance.
Thank you for the video Jeff. What about the impact to the other processes in your network while the test is happening? Have you noticed it? Also, the other way around, does the traffic in your network affects the result of the tests?
Verizon FIOS supposedly doesn't have a data cap, but I would not want a speedtest going off while trying to game online. Good video though. Maybe someone could make a much shorter version of speedtest.
I would like to see Jeff Gerling setting up his own ISP and show how to be an internet rebell.
Oh, believe me I have considered it often. But it's a lot lot LOT more than just the technical problems you have to deal with. I'm not sure if I'm up for the political and social maneuvering that's involved in the local community (which can often lead to a strong division and a lot of foul stares).
@@JeffGeerling well you don't really need to allow that stop you from starting the business. There are always others who can take care of the political and social maneuvering while you take care of the technological aspect of the business.
@@theundertaker5963 North America is literally the worst place to start up an internet service provider fresh from ground zero. You need millions in capital and political leverage to shoo out the 3rd party monopoly providers from your region. It's a financial risk that literally no one will give you money for, unless you have the cooperation of every single seat of power in local government.
@@JosephArata well, dont think so much "start up an internet service provider" more private diy Far bigger pipe than any single isp WAN... without the 3rd party isp hassle/continuous cost
starting with how jeff Can make a long distance private diy big pipe to/from his sister ( how many miles line of site)
with pi's, old cheap/free 5v-12v wireless routers + pv+ reclaimed batteries, several collectively routed with m.ruclips.net/video/S-Xmcig1ddA/видео.html
'OneMarcFifty Virtual Network in Proxmox for MPTCP Test lab' at both ends & potentially every pi relay in-between
with cheap integrated m.ruclips.net/video/wEtlbNir8KE/видео.html LoRa LoRaWAN Range World Record Attempt (Re-published No. 120) Andreas Spiess for collective instrumentation & small quick txt, alerts etc
..collect up the sw tools and shell script it to make a working one off on demand clean usb Debian super light 'slax' slax org customize .php deployable from pi-KVM etc
0:11 Ah, for 1 Gb/s down and 400 Mb/s up it's 20€/month in France.
In Mumbai we have FTTH , 1gb up and down with 3tb cap at around 56€/mo plus voip and a bunch of ott services .
I live in a rural area of Northern California, USA. $130/month for cellular based service… 10 down/3 up. Still wait for my Starlink service.
@@x5Deadmeat5x hope you get it soon👍
gbit down and 50 mbit up is 50€ in germany
In france, I pay 38€ for a sold _up to_ 8Gb down/1Gb up which is, in fact, a measured 5Gb down/1Gb up connection per month with no data cap.
I don't complain with the difference, i was never able to use my connection at its full speed.
Grafana is the only service that I wanted to have for a long time, but still very confused about how to start…. Video series Jeff ? ;)
Grafana on itself does nothing, you need some data to display in Grafana. Typically this would be in a database (in this video, it's Prometheus, but it could also be MySQL, SQL Server, etc). But then you also need something to put in that database in the first place, which in this video are the speed test results and the website checks.
so I started with 1 Pi as my DAC - now I am at 12 (doing many things - not music only), and now I need a dashboard to my Pis ;-)
A monitor to monitor all your Pis!
One monitor to monitor them all
Dude, get a hypervisor and run VMs, lol. 12 tiny physical boxes replaced with one big physical box running infinity tiny boxes
@@realityveil6151 Dude, you cannot put VM into active speaker to be working as LMS client and DAC. And this is only one example - actually 5 of my Pi are LMS clients and DACs. Everything has its purpose
Thanks Jeff! Xfinity/Comcast have a monopoly in our area. Feels like an abusive relationship. Being able to keep them honest, and know what our devices are doing is super useful. Cheers!
same.
Of course they do. Time Warner won't operate in cities/states they are in. And they don't operate in states TW operates in... It's collusion. They need to be broken up. Wtf is congress even doing?
They're not lying. It says right on the ad "Up to...". What's more, the rates are for the last mile, ISPs make no guarantee of your throughput over the public internet, over which they have no control, and they don't even guarantee your throughput across their own backbone, because the internet is an "as-able" service. All you're really doing is generating pointless traffic for the express purpose of documenting what you already should know: the internet is not reliable. It's not designed to be reliable, in fact, the whole reason the ARPAnet was originally built was to try and create a communication network which could survive a nuclear war.
If you want to buy a synchronous gigabit connection, you're welcome to, just be prepared to pay about a grand a month, because the ISP's cost model is built around the understanding that you'll be a client, not a host, if you're using residential internet.
From a former ISP engineer: Stop doing what you're doing. You're making the problem worse by spamming out bogus internet traffic on a cron job. If you want to monitor the quality of your network connection, install ElasticSearch Beats or Cacti or something, and monitor your traffic sent and received, and set up Smokeping to monitor your latency. Don't be a douche-weasel and DDOS your upstream just to prove you're getting less than the marketing team printed on the box.
Friend in Hungary pays €10 a month and gets 600Mb/s down and 200Mb/s up...
I'm not impressed!
Yup, DIGI FTTH is 1Gbps/300Mbps GPON for 3100HUF (approx 8-10EUR)
@@84GDi I'll have to tell my friend she was stitched up! 🤣
@@84GDi In Spain Digi offers 1Gbps/1Gbps for 20€. In Spain, there are only symmetric connections.
@@pgp-acc for 30€ you could get a 50/5 over cable here in germany. Or 100/10 for 35€
@@pgp-acc Most connection in HUN are asymetric dl/ul (even business packages other than VDSL), but for 7200HUF/mo (approx 20-22EUR) Telekom offers 2000/1000 on fiber here too (for residential consumers). Not anywhere of course, we have areas where ADSL on crap-old phone copper still cost like 9k HUF (30EUR) and barely reach 10M/500kbps up.
The open-source counterpart of Cisco Meraki Insight.
Nice video as usual 🙂
Well, I think that this problem is relatively extended in every country of the world I guess.
Until 20 years ago in Italy there was a single telephone and internet operator but, at the end of this, nothing changed to much because the entire national infrastructure still was owned by that company, so the other "new companies" had to rent part of the network.
To be short, this caused a lot of issues as there were more ISPs on the same infrastructure that wasn't upgraded as needed up during the years mainly for the high costs: here if you want to "put down" a new line/fiber you have to broke up streets as the old pipes are too small and full of old cables.
Anyway, until 5 years ago I was paying 45€ monthly for a 6 Mbps ADSL, then the fiber (FTTC 100/20) finally came and, as I'm not so far to the cabinet I'm not losing too much signal.
I still paying likely 50€ monthly but at least now all it's working as intended.
Ah, phone calls are included 😆
Sorry for the poem and thanks for your work here.
Hey, you're sitting in the city with your 1 Gbps plan, out here in St. Charles County we get 25/5 and even then AT&T can't deliver it without some spottiness!
Hi Jeff, it would be great if u did a video about backing up the sd card on a pi to a nas. I am kinda worried it will fail and having to set everything up again.
It should be noted that if you have internet speed greater than 300 Mb/s, you MUST use a R Pi 4 as the 3B+ only has a 300 Mb/s NIC. Otherwise great video as always!
If you want the automated speed tests, then yes. If you're just using it as a pi-hole, then no. The traffic doesn't flow through the Pi; it just resolves DNS queries. But good point nonetheless.
I found out years ago that the offered bandwidth is only guaranteed from the demarc point to your nearest ISP first hop.
So meaning from my home to ISP but not my home to ISP to Google for instance? So the advertised bandwidth means literally nothing if my ISP's connection to google is slow in that example?
@@spacescout87 exactly. But hen again, they have no control of the internet between their point and Google (or whatever you connect to).
So there are peering agreements for ISPs, they have to peer with someone else to get access to the internet. Literal ISPs for the ISPs. These are expensive contracts for multi gigabit connections. Not too mention BGP ASN’s and the constant battle to obtain more v4 addresses from ARIN or other addressing entities. They do have their own battles.
@@DeadlyDragon_ In a past job, I was the one to have to fight those battles - can 100% confirm they're tough. We were able to operate with only one ASN (as many nets are) by using the same routing policy everywhere, and we addressed (pun not intended) the IPv4 shortage problem by going the DS-Lite route (pun still not intended).
But all of that was a walk in the park compared to the challenge of purchasing enough transit to meet customer demand. We kept having to readjust our oversell ratios as we updated our traffic engineering practices, and getting a good handle on what "peak demand" looks like is a surprisingly complicated question. We weren't yet big enough to get caching appliances from Akamai, Netflix, and friends... not that I wasn't checking constantly. But we always strove to do right by our customers - small ISPs have to or they don't survive. (So be a customer of a small ISP! Support local businesses that actually give a crap about you!)
Glad I wasn't still working there for the COVID pandemic. I would hate to be the one to have to figure out how to handle a bunch of WFH traffic overnight! :)
@@CFSworks
Oh, if only I lived somewhere a small isp existed.
i just wanna say, i set this up today as a linux noob. your tutorial was so easy to follow, but i did have pi knowledge. i had a big fight with errors, but fixed them by giving 'pi' docker perms [thanks issue tracker!]
this tool is so nice. i have pi hole just to track queries, as i have two ad blockers already. the internet-tracker is super nice. im gonna gather data on this as i love seeing data.
cheers!
Found this channel when looking for Raspberry pi.
Had no Pi back then but still loved to watch this valuable content.
Now, Thanks to God I got a Pi 4 4GB and I love watching to your videos.
Keep up the good work.
In Thailand, many ISP can let's customer to set up Internet speed by their owners from the ISP website, for example, customer package is 500/500 Mbps, customer can set speed 500/500 or 600/400 or 800/200 or 900/100 Mbps.
Really great videos, and information on new and upcoming hardware, the only problem I see, is your github instructions on this are not the greatest. For someone like myself that is still learning about linux systems by utilizing a raspberry Pi, again for someone like myself hat knows enough to be dangerous, can really get frustrated with install instructions that are lacking, and would make someone want to just throw in the towel all together and stop learning more about raspberry pi's. This is by no means a dig on you, people like you are what made me want to get into buying my 1st raspberry Pi, but again with the github install instructions, its really hard to even tell what exactly it is that I am installing, or how specifically. Am I installing Ansible, or Pi Hole with docker, or do I have to have docker already installed, is it installing ansible? it's really hard to understand what all the instructions are having me do.
I still fail to see why you have less than 1m subscribers.
*Laughs in European*
Spectrum isn't fleecing you. If you're not seeing over 700 Mbps, it's likely due to one of the following. 1. You're not using a Spectrum provided DOCSIS 3.1 2.5G modem. 2. Your router (and possible your switch) between your modem and Raspberry Pi is slowing you down due to its settings (packet inspection, prioritization, etc.) or hardware limitations.. 3. The Raspberry Pi, your switch, and possibly your router only have a 1 Gigabit ethernet port, which with ethernet and TCP/IP overhead means you're limited to about 900Mbps. 4. The software on the Rapsberry Pi is also slowing down your network speed. So do some more investigation on your home network and I'm sure you'll be able to obtain higher speeds.
I feel you with cable upload speeds. Here in Ontario I can get 1000/30, which definitely hurts to pay for when all I really care about is upload!
You know the pain!!!
You're lucky having 30. Here I can only get 1000/10. It's almost always saturated, and I have to host a lot of stuff on the cloud simply because my connection would get choked as soon as someone remotely starts a download.
Cry’s in Australia internet getting 40/10 which is concerned good for my area
Im thankful a smaller local ISP started up near me. I'm a few miles outside the ST Louis metro area and I just went from only having satellite to a fiber connection which is faster by 100 fold. Spectrum wanted 20k 6 months before Gateway fiber just ran the line down our street for free if we payed for a subscription. I pay for a gigabit connection. I still see what you get most of the time like 700-900 down depending on the time of day but my uploads are always right around 900+. Still very happy with them, I only spend about 90$ a month on it.
One missing feature. With everyone monitoring their ISP 7/24 there needs to be an opt in option for users to send anonymized data with ISP, Location down to the city level to a central site on the internet. Then this can be a public service that anyone can search their location/city and see the averages that users monitoring 7/24 in that city are getting from each ISP including outages. This will allow everyone to know before hand which ISP is better to go with and turn this project into the next level.
I'd like to also see a crowd-sourced ISP statistics! This way, no more pesky ISPs claiming "no our internet is not the problem, go restart your router."
This is a really cool idea, but I can't think of a way to stop ISPs from poisoning the data. As the service gets more popular, it's more likely that ISPs would send fake data to it
Speedtest dot net already collects this kind of info, not 24/7 but it does save the speeds and providers for analytics.
Samknows
My pi server: fuguhub (which I can’t uninstall though I want to, the uninstall script doesn’t work and I can’t find it in Apache), temp monitoring with dht11 (integrated with Hubitat), pi hole, cloudflared DoH, and homebridge
Thank you Jeff! I have played around with a friends Raspberry Pi several years ago but didn't have any good use for it until now. Went out and bought my first Pi because of this video :)
Have fun with it!
Me: after promotion ends 220 mbps down and 25 up costs $150 a month with xfinity in suburbs of Atlanta. Oh, and only 1.2 tb of bandwidth.
Canada: 1 gigabit fiber for $80 a month due to government-owned fiber cables that are subsidized and sold to various isps to encourage competition and drive down prices
I wonder when you will move to Home Assistant ...slowly :D
Heh, it may happen, we'll see. I think there's already a drop-in plugin for the Shelly Plug there.
@@JeffGeerling Yea there is an integration for Shellies among many others :)
ok KNOCK IT OFF!. You RUclips guys keep doing the same thing with the links. "It costs $35 bucks" Shows affiliated link to a Pi that's $64 bucks or a $99 kit.
Thanks Jeff!
Once again you provide an excellent set of tools for us. I'll be using a Pi 4B in my cluster for a dedicated instance of this project.
getting 920-940 down and 500-550 up and i pay $55 a month for 1000/500 fiber with static IP. They promise at least 900 down and 400 up and at one point they wasn't able to deliver more than 700-800 down so i had free internet for 6 months as i told them i was holding them to their advertisement promises. We agreed that i kept paying until it was fixed, and then when it was fixed, verified by me, i would get as many months in return for free and on top of that a static IP for a year for free as i was paying for it at that point. I would have been fine with 50% off for the time frame but i kept my mouth shut and let them dig their own hole :D
What i do instead of using pihole as my LAN DNS i use my UDM Pro, and then on the WAN interface tell the UDM that all WAN requests should go through the pihole.
I don't have a great experience with PiHole as DNS on LAN to access my devices through their hostname, it was really hit and miss, so i moved the pihole to the edge of my network so it's not my LAN DNS. Also by doing it this way, you don't break any privacy laws there might be in your country, because your able to see whose device access what site, and you would be able to if the PiHole is your LANs DNS. By moving it to the WAN interface as DNS you only get two request sources and none of them is from any of the devices on the network, one is itself by localhost and the other is itself by IP on the device it's running, in my case as a docker on my NAS. On top of that if you run it on a pi and you only got a single one and it breaks down, you would have to go through updating your config on the router and then refresh all devices to get the new DNS, often by restarting if it's IoT devices, whereas if you do it WAN side, you would only need to do the router configuration and your up and running again with internet access :D
Cool Project !
looks like my pi 3b+ is getting some more work to do than just host a discord bot and a local chat server :)
@The Techie Scientist what kind of chat server do you host? I also want to try that out but I cant decide what application I should choose^^
@@paraniodify nodejs socket.io and express
@@TheTechieScientist thx mate :)
I'm likely doing the math wrong but 30~40MB every 30 minutes = ~43GB a month of wasted data for a dashboard, In a country that has data caps like we do this seems like a complete waste of resources.
In India internet is pretty cheap it's about 54 dollars for 1Gbps connection with no data cap and 10 dollars for 100Mbps connection with no data cap
In Italy we pay about 25/30 dollars for a stable gigabit connection. BUT, we pay the same for 100/50/20 or 7 Mbps ADSL!
Are you talking about the fiber broadband? There is a data cap of 3.3 TB. They just say it is unlimited.
@@UtkarshAmitabhSrivastava ya you are right but 3.3tb more than enough 1 tb is enough because generally we don't reach that limit
@@pavan13 yes that is enough for most people, but I think the gigabit plan should have higher cap. You could use all of it in just a few hours.
@@pavan13 If that's cheap, what about 1Gbps down and 500Mbps up, unlimited data, for $7?
Sadly I would need a 10 Gbps up and down on the Pi - Switzerland has good internet ;-)
So far, my trouble was finding iPerf3 siting capable of delivering up to that speed. So far, fastest speeds I found was slightly above 4Gbps
Interesting-I've had some trouble with my HP desktop getting anything more than 4-5 Gbps, but I haven't had an issue with my two M1 macs-they get 9.4 Gbps or so on the local network. I don't have a faster Internet connection though so I'm not sure how it would do through the public Internet.
I wonder if it's something related to the 10th-gen Intel i3 CPU I have in my Windows PC, something with the ConnectX driver (I tried an X3 and X2 card so far, same result on each), or something else entirely!
@@JeffGeerling Internally, yes 10 Gbs is no problem... it's getting that speed to the outside tested on iPerf3 servers that gets difficult: most servers do not offer that bandwidth.
CBI:- Lets track em
Jeff:- U know na I got y'all
One day this man will land a rover on Mars using only Raspberry Pi's
It would be awesome to have a real time traffic monitoring in place. As the pi hole is already recording DNS requests call, it would be pretty nice to see what clients are currently doing on the Internet, since the pihole frontend it’s basically just a list of requests that were resolved. This data may probably be relevant to show such a real time traffic, but it itself doesn’t record the traffic volume for instance.
Running a speed test every 30 minutes is pretty abusive of the speedtest.net network bandwidth. Various organizations volunteer their bandwidth to the service. If you are going to run a speed test with that frequency, you should get a DigitalOcean droplet or similar for the purpose and run the Ookla server on that and point your client at it. For one thing, you'll know that you have the full 1GBps pipe on the droplet to yourself resulting in more consistent measurements and you also won't be abusing freely available resources.
I get 990Mbps max min 950Mbps on my gigabit fiber ping under 5ms and 530Mbps upload for 31 euro
/me is jealous!
None of the European comparisons are apples to apples. The large geography, fragmented regulation etc. of large land mass countries will always allow pockets of monopolies and more expensive operations than small densely populated land areas. You get gigabit or more for less than the price of a pizza in countries like South Korea et al. The better measure is the quality of service and uptime from any ISP.
@@pd8559 Well I pay $7 for 1 gigabit down and 500Mbps up. Real speeds are constant 940Mbps down, 510-520Mbps up. And as for the uptime, I am pretty much always online (me and my servers) and in 10 years I had 1 single time about 2 hours of Internet outage. SO I would say the uptime is good. I live in Romania.
Where I live, when you have high latency or slower than promised speeds, you call the ISP and it’s fixed in a few hours. ToS go both ways.
Greetings from Estonia!
Symmetrical. You would prefer 150mbps of symmetrical.
Asymmetrical means exactly what you already have with the 1gbps down and 40mbps up.
Don't sweat it Jeff, I made this mistake too.
Yes, oops!
If you download at 1gbps your upload is almost maxed out sending ACKs which is probably part of the reason you „only“ get 930 Mbit/s
They beat me to it, but yeah...I much preferred my 75M synchronous/symmetric FiOS at my old address over the 1G asynchronous/asymmetric Spectrum Business at my current one. Unfortunately I have no other options besides cable here, the phone lines are too old to support more than 128k (per AT&T) and there is no fiber in the area.
@@MarkusOrt It's mostly TCP overhead which if you had a gig would come out to around 949Mbps max. There's just no way to get a 1000Mbps on a 1000Mbps link negotiation.
I came for the ansible playbooks... but i stayed for the hot takes...
Well they clearly state "up to" 1Gb per second.
This makes it right? No.
Interesting how in your video ruclips.net/video/spD4FLfi2a4/видео.html you're up to 891 mbps. What changed in-between this video and that one?
Man, this channel is a gem for the open source community. Love your content, keep it up 👍
Time Warner got that illuminati and FBI identified p3d0 symbol package deal logo
The ISPs in the USA are horrible. Subpar compared to ALOT of the world these days.
Damn i get 1Gbps/200mbps for 45$ in Portugal + other tv stuff
I pay $10 a month for 100 mpbs up and 100 mbps down unlimited internet in india
Wow, I had 40mbps upload around 2005, for $10 per month... That's what monopoly does... and we're talking here about the country that pretends to be a world leader :D
Thanks for your videos. I've learned a lot from you.
I'm trying to build the ISP monitor, but I'm stuck on step 4 of the Setup (on GitHub). Look at that paragraph again -- as a novice like me.
"Make copies of the following files and customize them to your liking:
example.inventory.ini to inventory.ini (replace IP address with your Pi's IP, or comment that line and uncomment the connection=local line if you're running it on the Pi you're setting up).
example.config.yml to config.yml"
What IP address do you see that I have to replace? Maybe it doesn't show up because I'm setting it up on my Pi using VNC?
I'm not great with code, so I'm stuck. Thanks in advance for you help.
Me too, at exactly the same point :( Did you ever work it out?
4:50 - YES!! There is little to no reason ISPs couldn't offer a 150 symmetrical service. DOCSIS 3.0 supports 200Mbps upstream, 3.1 is over 1Gbps, and DOCSIS 4.0 is 6Gbps. They should make that the standard just to properly compete with fiber ISPs.
~725mbps may be mostly the best the Pi can do with the onboard gigabit, to be honest. I'd probably try to do iperf tests locally just to confirm
I run 2 VM's running debian 10 with PiHole installed on my home server. I have yet to set them up to sync together but Techno Tim has a walk through on how to do it! Though i'm guessing it's on Jeff's radar as well. I had to run 2, as 1: Router would not take a blank dns 2 and 0.0.0.0 didn't work lol 2: Setting up multiple interfaces in pihole was outside of my scope at the time (though i probably could do it now) and 3: redundancy! I can update one while the other takes care of all the dns requests.
And here's me in the UK happy as shit because I manage to get 70 down 17 up. FML.
I have 400m from Spectrum and usually get ~430m
Hi jeff, vishal here, am having a terrible issue with my pi or home network, when I'm trying to access my pi or virtual machine with ip there is no problem, but when i setup a local domain to it like homenet.net or something like that i can only access it on that particular computer(where site is hosted), and when i try to use that domain on different computer on same network it says domain unreachable, please tell me how to solve this issue... thank you..
You have yo setup that domain on pihole and have the computers on your network use pihole as the dns server
Thank You Jeff..
I really like this project, looking to switch from High Cost Spectrum to T-Mobile and monitoring before/after would be perfect.
Have everything installed however I am stuck at the "TASK [Ensure internet-monitoring environment is running.]" task. Throws an error "fatal: [localhost]: FAILED! => {"changed": false, "msg": "Error connecting: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))"}
". Any ideas????
Please see this issue: github.com/geerlingguy/internet-pi/issues/22
i just installed the internet monitor on my pi4 and it works great, just one thing - it took me like 10-15 minutes to figure out the password to the Grafana dashboard..and as this is the first Docker container I run , had no idea how to find its configuration files - until I realized the playbook (?) created another directory on my home folder with all the configs (internet-monitoring). maybe it's worth mentioning this somewhere in the Github readme? I originally followed the install instructions on the blog
Yeah, I'm planning on revamping things slightly since more people are installing it now :)
Thanks for this. I already have a Pi running PiHole connected to the router, but I've been using a scheduled task on a windows machine to run an hourly speedtest to track my internet speeds. This looks like a much better setup.
Great video BUT... for people with metered Internet plans running Speedtest every half hour could be a very expensive mistake. It pulls and pushes a fair bit of data.
The default is now every hour, and you can easily override to a longer interval in the config too :)
Be aware that it could be that your ISP is prioritizing speed test requests ;) So they show a higher performance then you actually get.
lmao switzerland be like here take 25Gbit/s for 65$ a month, no restrictions and we'll help you set everything up
Fiber 7 FTW
in my eyes, gigabit internet that's not symmetrical is not true gigabit internet
I have pihole running on ubuntu already, how can I just install the internet monitoring tools? Thanks.
Right now it installs it by default, and there's no option to skip it-please open an issue here to get that feature added: github.com/geerlingguy/internet-pi/issues
Though you can install it and just not use it too :)
When I tried to switch to an LTE ISP, I had to talk to the CenturyLink customer retention specialist. They pretty much gave up when I told them "Look, I'll pay you $200/mo if you will give me 100Mbps down." They act like they don't understand how 3Mbps is not sufficient for anything except for email/browsing nowadays. Without ad block, my wife's PC consumes the full 3Mbps down when she opens her forums due to all the auto starting videos/ads. I currently pay $90/mo for two 3Mbps dry DSL circuits.
CenturyLink/Frontier/Hughes/Dishnet are going to be in a world of hurt when Starlink gets rolled out in earnest...
Hey jeff...is it possible to take a couple of the variables in the dashboard..like the speed and ping and print it on a OLED 128*64 i2c display?
That would be a really nice improvement! For my own needs, the Pi is in a rack so I wouldn't see it, but it would be cool to have that data available. Maybe open an issue on the internet-pi repo and we'll see what we can do!
@@JeffGeerling maybe I can contribute as well.. I have some experience on these i2c displays ..I’ll try the repo and I’ll try to modify it so we can also print on the display
Wow internet in the America is expensive. I pay 45 Euros for 130/30 on my vdsl.
Agreed :(
"the internet pi"
~Jeff Geerling (2021)
İ pay 25mbps 110₺( my income 2800₺)
missed opportunity to call it Wi-Pi
well this doesn't work anymore. Cannot install it
europe has already solved this problem for the most part and still working on it, US should learn a thing or two, we always learn things from each other, why not internet too
If this method can be used to measure an internet connection speed every X minutes, it could 100% utilize this internet channel when taking a measurements?
Pretty sure most of the world have way worse internet than you...I can get 50mbps download on a good day.
I'm really annoyed by Vodafone's stupid marketing.
They promote their 1Gb/s with their cringe ads. The catch is the upload speed is 50Mb/s only.
Also they promote(d) the 1Gb package is now with 50% discount. The catch? It does fricking work in my entire city and I live in decently large one. The best i can get is 100Mb and only 38Mb in some areas, so their claim "We bring the fastest Internet almost all households."
O2 on the other hand have much better plans and prices. Their 1Gb/s Internet has 100Mb/s upload and its ≈8€/9,5$ cheaper than Vodafone.
I'm on AT&T with the 1 gig plan at $60/m. Works fine for me, 1GBPS download and upload.
I did run your docker compose and within minutes it works flawlessly ! THX
We pay $50/month for AT&T internet and we're lucky if we get over 15 megabits per second. Absolute BS. We live in a major suburb, and they won't run fiber.
Same here, though at least we have cable. AT&T promised they'd have fiber in our neighborhood a few years back-still waiting for that to happen.
@@JeffGeerling I have recently begun getting ads in the mail from AT&T touting their "fiber." BUT ... I question whether this actually a fiber connection to the house, or is just a fiber backbone. I currently have Comcast 100Mbps/6 for $65 in a Chicago suburb , it is generally reliable and consistent on speed. So I am not terribly unhappy and don't see any compelling reason to go through the hassle of changing. At least for now. Higher upload would be nice, though.
Jeff, you should get or add an Ripe Atlas Probe! Contributing to the global data research as well as having a great additional monitoring for your ISP as well as connectivity and routing aka peering.
I feel bad for you, my ISP's Gigabit is really Gigabit, meaning 1000/1000, and it costs me 50$ a month
ofc, it's not leased line experience, far from it, but I can safely say that I got about 90% of the expected speed on a 90% expected time of need.
In Chattanooga we get municipal gigabit fiber (1000 up & 1000 down) for $70/month. Pretty sweet.
Thanks for the video! 👌
I'm a Splunker but thumbsup for your Monitoring ;-)
Splunk is great too! Really nowadays we're spoiled for monitoring and visualization tools. I'm guessing in 10 years only a few will be left standing, just because I think there are *too many* good ones currently.
Wifi (and any wireless connection) is not just useless for checking speeds, its also useless for input, video, or data transfers. The only reason it even exists is for convenience. THATS IT the ONLY thing it does better then a wire is 'plug in', every single other thing you want to do with a connection is worse through wireless.
It's running speedtest through the pi's network port? We have 1gb fiber and we have a 2.5gb link to the pfsense box and a 10gb backbone. It woke the network right up!
This is unacceptable for me. I have 500Mbps internet but noramllt I got 500~600Mbps in speedtesting.
Those prices are ridicilous! I thought i was paying too much (45€) for my 300/40mb DSL in Germany.
Wouldn't running a speed test every 30 minutes throttle your speeds for about 30 seconds every 30 minutes? Seems like that would be annoying for gaming or streaming
I have the same plan same price with Spectrum, no other options. It goes out often and one time it took them 11 months to figure out a packet loss issue smh. I currently have a Pi 3b+ setup to email me ping results every 6 hours via a cron. Unfortunately the Pi 3b+ caps out at ~330Mbps in a speed test due to I believe the USB controller iirc so I am thinking of upgrading it to a 4. It’s also currently my Pihole. Great vid!
You know the pain!
I’d like to see a phone system pi project