It would be great if you can revisit this video, since now traefik has an official plugin, the hub auto update itself (no cron needed), the dashboard looks cool, they have a centralized way to manage multiple instances, they added appsec WAF integration and probably more. There arent't many recent tutorials and you're always spot on with yours.
For a dude that seems to throw around containers like they are nothing, it's nice to see you admit to having to lookup something old-school like crontab. I've been watching your videos to get up to speed on containers.Thanks for the content. You are really good at it!
Hi mate, u can setup cloudflare bouncer to have crowdsec blacklist shared up to the cloudflare layer. Using it for enterprises u can ask cloudflare to have more than just 10000 ip addresses configured as a list (they raised me to 20k). After some months a bit of improvement is reached that way. Happy hardening u all 🎉
Great tutorial Tim! Anyone know of a suitable Crowdsec docker image for arm (Raspberry pi)? I had a look around and couldnt find one. I'd rather install on docker if at all possible.
I just set this up, be aware that if you're running behind a reverse proxy like cloudflare the traefik bouncer here doesn't use the correct IP address due using the incorrect header value.
I noticed that too and there is a PR out there for it. According the CrowdSec though, CloudFlare IPs are on their global allow list so they can’t be blocked. Hopefully the PR gets merged to look at the real ip in the header
@@TechnoTim The main problem isn't cloudflare IPs getting banned, it's that the bouncer doesn't block anything because it reads the headers and asks CrowdSec for information on the wrong IP. The PR is mine, I'm running it currently on my machine and it works well so hopefully the dev comes back soon!
excellent document, I installed it , but had an issue witth the bouncer, even if the manual added ip deciscion is properly added to the list, the bouncer does not blocks ,
I'm already using Traefik so now I'm definitely gonna check out CrowdSec. Looks cool and easy to configure. Thanks for another awesome and easy-to-follow tutorial! FYI, isn't the GID value in the docker-compose file supposed to have colon to specify the default value "${GID:-1000}" ?
Hey @TechnoTim, did you tried the Metabase Dashboard? It works fine, but after compose recreation the credentials are default again. Were can i find the credential information to persist?
14:45 - I have the folder but no logs. When I exec into traefik there are both the log files. I've gone through my yml files 5 times now a nd rewatched the video to this point a few more. My networks are the same. Maybe there is something different being I am trying a year after your post?
Thanks for the great tutorial and video. I've leared a lot from you over the last few months. After working through this and installing I have a question I'd like your input on. When proxying through Cloudflare, crowdsec is analyzing the Cloudfare IP, not the real IP of the client. Now I can imagine how this may be useful if someone decides to attack the IP directly or somehow gets around Cloudflare (I can't even begin to imagine how that's even possible), but I have my firewall to only accept connections from Cloudflare IPs on 80/443. So in that instance, can you still see any benefit to crowdsec? I know there are some complicated ways to have traefik be able to see the real IP from Cloudflare, but I haven't attempted that yet.
Hi Daniel, I appreciate the kind of higher level thinking of your comment. I have the exact same setup in regard to Cloudflare and only accepting Cloudflare IPs on 80/443, and had not thought of this yet. I wonder if you have done any more thinking about this. Are you still running Crowdsec or have you deemed it unnecessary? I see there is also a Cloudflare-Bouncer, which seems to take a different approach and updates your firewall settings in Cloudflare itself. Curious to hear what you think.
Hey Tim, great video appreciate all your hard work. I've been trying to install crowd sec for some time now in my environment. I have two raspberry pi's one 32 bit and one 64 bit. Crowd sec has given some instructions on how to install on arm but my Linux skills are lacking and well they don't show us like you do. Also I don't use traffic I use nginx reverse proxy. Should I even try or should I keep waiting for someone to make a video specifically for raspberry pi crowd sec with Nginx and docker.
I wouldnt run an ips firewall on a raspberry pi. Its like showing everyone you have a tank in the garden to defend your country but in reality that tank is just out of cardboard. The performance wont hold up.
Hi Tim ! How do you enroll your Crowdsec container in the cloud console ? I've done it with the cscli command but it needs to be done again after each re-creation ...
Hey, Tim. Really enjoying the channel and Discord. I have a question not crowdsec related but something I noticed in the video. Would you mind going into some detail on the ip whitelist(s) for Traefik? Couple of us trying to get it figured out and not having much luck. Definitely implementing crowdsec now!
Question: Won't access log grow to infinity? How big is your access.log file right now? What should be the cap? Request: Can you make an update to Grafana monitoring guide using influxdb and adding consolidating the alerts including crowdsec? I just want "BOGOOGA" sound alert on my phone if I am getting DDosed.
You should join our Discord and ask about the Grafana stuff if you want fast help. Also you can setup a number of notifications on CrowdSec when a scenario triggers. So basically your DDoS scenario would trigger an alert which would then be sent to your phone. No problem with CrowdSec.
How did you get Cloudflare to forward the real IP? In your case if you use Cloudflare which I think you do, the ban only worked because you have a local DNS. Banning your IP would otherwise do nothing because traefik and hence crowdsec would always see the Cloudflare IP assuming you have reverse proxy set up in CF
I did bind mount the only thing I want backed up, the config. The rest (like logs) is in a docker volume which I don't care too much about. Also, their docs say to do it like this and I had issues trying to bind locally.
I guess I would have started the bouncer before crowdsec, so that it's available when crowdsec starts up. Which means: crowdsec should depend on the bouncer. Am I wrong?
No since you can run everything distributed on different servers. You can have one agent receving logs from multiple other servers and controlling bouncers on remote firewalls, even across operating systems. So we can't depend the installation of the bouncer in the agent. And that is by design :-)
Hey Techno, amazing video! I was really exciting when I managed to config Crowd for read the traefik logs. I have a question unu Is it possible to configure Crowdsec to allow the connection from a specific origin, and ignore or prevent access to my service from other origins different from the domain I defined? The context is that I need to secure a backend that is exposed to the internet via traefik and an external frontend needs to consume it.
One thing, that you still dont follow: you dont capsule your services from another. You just use ONE network: "Proxy" if you want to protect your services even more, you should create seperate containers between the traefik and services like heimdall and co.
Thanks! I break up my compose files. Also, don't all services that are served through he reverse proxy need to be on the same docker network to proxy through?
We're based in EU so GDPR is obviously taken into consideration. The only data that's being collected is the ip of the offender, timestamp and metadata on the attack (=which scenario triggered). So nothing to worry about in terms of GDPR.
Hey, great vids, just started self hosting, you're giving me too many ideas... Anyway, I'd love a video covering how you might have setup services that use SMTP/email settings. Thinking WordPress, Vaultwarden etc. Thinking to have a single SMTP relay that everything points to, which then forwards out via Gmail/X service.
Thank for the video! I would really appreciate a tutorial for using this with nginx proxy manager as well. I'm trying to figure it out, but I haven't gotten it to work yet!
Sounds great, but this is not only open sourced but the database is managed by the community; what's to stop bad actors from listing valid sites as malicious? wouldn't that make this it's own kind of ddos attack if people can't access a site because someone fraudulently added it to a block list?
That's a good question. Very shortly described it's based on trust level but servers who report; the longer time they have sine so reliably, the higher trust ranking and the more do they count when determining wheter an IP is bad or not. Also, an ASN only gets one count. All this and more exists to make poisioning as expensive and hard as possible. If you have more questions, feel free to go to our Discord.
@@crowdsec Ok thanks, a few more things to consider would be oversight, I read that a university was banned despite making years of commits to Linux for posting some intentionally bad commits, and you said that an ASN only gets one count are their measures taking botnets into consideration? because I can't imagine it would be hard for someone with a large botnet spoofing an address to make it seem malicious.
@@festro1000 Where was the university banned? Was this in relation to CrowdSec? No, we're only taking their actual behaviour in terms of how realiably they send signals into consideration. Could you elaborate on the spoofing part?
Maybe I'm missing something. But didn't you configure a conditional forward in your UDM Pro so that only traffic from Cloudflare gets allowed? In other words, if the rest of the packets gets dropped, what's the advantage of this?
@@mormantu8561 Why would Cloudflare be the only thing that can reach Traefik? There's a lot to connect to out there... What if someone SSH tunnels to an unlucky internal device that's been compromised in order to pivot around inside the network? Better to have multiple walls to climb than just one you can walk around.
@@wyattarich True, but in another video he showed us that traffic on http(s) to his Traefik instance is only allowed from Cloudflare IP addresses. My comment was about why he would implement this if he has that rule, but come to think of it, maybe he means if Cloudflare fails to detect a threat. Whereas I thought that he meant what if someone or something bypasses Cloudflare entirely.
That's right. I don't mean that someone circumvented cloudflare, I mean that cloudflare's bot detection might not catch all bad actors, where this is yet another line of defense.
The Core Question for me is, can i make Trafik work behind an HaProxy. I have atm a haproxy running in my pfsense and i would like to keep that, but trafik with crowdsec would be a nice addition? What IP does Crowdsec ban ? For example can i tell it to ban cf-connecting-ip ?
well here you can use crowdsec with opnsense, ha proxy, nginx or as a container so quite some flexibility :) Ip are banned base on the sightings of all user of the community and curated by CrowdSec to avoid false positives and poisoning.
@@philippehumeau7972 Played around with it, atm still behind my haproxy and works well. I noticed some problems over the time with running software behind cloudflare. Some software is intelligent enough to recognize the real ip ( it can be seen through the CF-Connecting-IP Header), some just see the Cloudflare IP and the last thing I would want it to do is to block the Cloudflare ips =) the only thing really missing from traefik is brotli, but that's just personal preference
I always see these videos with crowd-sec, fail2ban, etc, and I want to add these to my setup, but what I always see left out is the explanation of what happens to self-hosted content that isn't accessed exclusively from a browser? Like emby/plex wallabag bitwarden, etc, that have a mobile app integration and even a possible chrome extension? Do they just break unless the app-code is specifically built to work with it? because it seems like crowdsec and f2b work by placing a sort of http "basic-auth" layer in front of it and forwarding the creds to the app and then logging the apps response and sifting through those logs with the bouncers etc, unless I misunderstood that, and if that's the case, what if the chrome extensions for bitwarden and the mobile apps for emby/plex aren't setup to expect that middleware layer between the emby server and the mobile app? For example, does the bitwarden mobile app need to be specifically developed to expect that middleware layer or is it a seemless interception of the creds the mobile app passes to what it thinks to be the bitwarden server and is in reality the traefik/crowdsec middleware? An alternative would be if the middleware just passes through traffic that has http-headers/user agent strings that identify it as a mobile app to maintain compatibility because it doesn't deal with mobile apps, but what stops bots from just using that user-agent string to bypass the middleware if that's how it works? Again If anyone has experience with this i'd love to hear any explanations or corrections of misunderstandings i might have. It's one of those things that i've searched the docs for but it seems like i won't know it it works or not until i attempt it unless someone else has already and can share their experience
what a brilliant video. i was thinking crowdsex not to long ago... but decided on a not yet... but maybe... hmm... i have a question though. What do you use for monitoring your network/home lab for failures/outages/etc etc ? I was looking at nagios but decided to stop looking there since core was note updated in 2 years... And the options are almost infinite... i'm a but lost at this point...
wait in the end it sounds like i have to manually add ips to the descsions list. I thought this is an automatic thing that bans any IP that appears SUS to my instance or is already known to be sus.
crowdsec and traefik seem to be seeing my docker bridge network gateway IP, not the client IP, so crowdsec doesn't seem to be working for me. Do you know what I would do to resolve it?
Your my boy blue! And I understand why you did this, and I am glad you made a video! Seriously But I don't know what you are talking about. You went from cards you colored with a crayon yourself to this... Quite a leap! Especially for me! Like I said, I'm glad you did it. Seriously. And in about 3 years when I have caught up with you, I will be thankful! Ha ha keep up the good work. Lots of your vids meant nothing at first and then a few months later, I was on bord.
Hi just wanted to ask u about your rack, on the bottom of it you have a 24 bay disk shelf what did you use to mount it in the rack? was it a Adjustable Rack Mount Server Shelf Rails 1U?
hey it's me again I'm have a question, with that configuration you will not have logs on the the stdthing (out/err/in) don't remember witch one docker logs use, that's OK for crowdsec that need that aparently but how to put those logs in loki for grafana ? did you try the traefik/grafana/crowdsec combo and how to make those those logs from file in the loki-driver too ? thanks :)
@@TechnoTim yeah except that for Traefik if you defined a file for the logs it will go to the file no more to stout... So you will have to set another job specific for Traefik, and may be some other container that will do the same: if log file is defined then pour in the file not stout anymore, and doing so I'm woundering how I can recognise that it's logs from container traefik. Using the same seentic in grafana... Or may be we should investigate the logs volume in crowdsec (or in the other way) ask crowdsec to look the logs from the grafana/loki logs directories...
I used your traefik 2 ingres guiide to set up traefik as reverse proxy in my k3s cluster (some Pi's, some x86 VM's and one Mac mini M1). I needed some time to figure out how to route on external endpoints in my network for services that are not in the cluster yet. Thanks to your new guides i will never be bored.
In case you see this, this is blocking every internal service but not the ones that are external. Guess it's due to the ip that we are blocking being internal.
That promotion has been going on for so long I'm really starting to wonder if they got an amazing deal on a container full of 240gb SATA SSDs or added a 0 to an order right before the price came down on 500's.
@@TechnoTim I'm a fan of Microcenter, just can't spend much money with them because they don't have a store near me and their web presence I so limited, and it is a great promotion. I'm just saying that after two years it's starting to feel like a creative solution to a massive overstock 😂
Hi Tim I'm having an issue that it only works for me when I block a local (docker) IP. If I block my public IP it still permits access. When I view the logs it only shows the local addresses. Any ideas?? Thank you
@@TechnoTim I got it working. I added a second interface on 'host' that seemed to fix it Nice videos BTW. I've been in the game 15 years and still learning 🇬🇧
I got hit with almost 8,000 requests on my Synology in three days. I watched the notification stream up into the notification box at a rapid enough pace I was legit afraid lol. Luckily I had protections in place because oh myyyy
@@TechnoTim no rush! Just excited to follow along! (Assuming the format of the vid). Have 1 Pi4 8GB, 2 Pi4 4GB, and 3 Jetson Nano 2GB that I wanna use :) Sorry just excited! Also thank you for getting me into homelabbing! Really appreciate it!
You can not protect your home lab against ddos. I suggest you not to waste time on this unless your home is inside a datacenter. Even if you set an ip whitelist to all ports and protocols it's not going to protect you. If your bandwidth can't handle it, there is nothing you can do. There is a possibility to use bgp flow to your advantage, but I don't know a single home internet provider who supports it and even then it's really limited
You can protect your homelab against DDOS by using Cloudflare which hides your public IP and points incoming traffic to their proxy which has DDOS protection.
Hi Tim, I tried crowdsec on traefik, but I think authelia is getting in the way ! I did many try to connect on my phone but no log in traefik yet when I want to see the log of the authelia application I can see the log : Unsuccessful 1FA authentication attempt by user '' and so far CS did not decide to block those try ! so it's great to block already known IPs I looking forward to an update so we can add authelia in the survey of CS :) I already found the collection and configuration now I need to put that togather and add a new aquisition in the list, but that part is a bit clouded for 1 folder it's clear, cristal clear, but can I add other foler with other labels... and what abount a bouncer for that app? may be it's not needed cause the app that will block is traefik; I'ld like to get the logs of the server hosting docker to be analysed too; to be sure no brute force will be attempted on my ssh even if I'm a no password guy I'd like to get those metrics in CS ;) So here you gave me way to criticaly upgrade my securiity :D again thanks dude :)
found part of the solution by putting the /etc/crowdsec/config.yaml file out of the container and changed the line acquisition_path to acquisition_dir and and create a folder in my mounted directory to put those acquisitions files instead of just having one file...
Hey I managed to get my phone blocked with multi testing wrong user and of course password \o/ So now that's done ! Extracting the configuration file and replace with no typo (I had a few so I had to precise) the path by a dir you can now put more than one file to the inquisition ;) And I just thought that I could just mount the file of my host to the CS pod so CS could do it's magic too for bad guys trying to ssh in even if the challange ssh key is stronger than password that does not mean you should put no security espacialy in those dark times ! xD So basicaly now the only thing is to do it now... yet for this one I think I should add a bouncer but how to give it access the system FW 🤔May be I will look at the bouncers and invest in a true FW it will not be lost xD 1 am here, need to sleep this over ;)
![I.N "HALLUCINATION" | [Stray Kids : SKZ-PLAYER]](http://i.ytimg.com/vi/n5B5q1Hwt_U/mqdefault.jpg)
New Customers Exclusive - Get a Free 240gb SSD at Micro Center: micro.center/1fbb85 (paid)
I'm def installing crowdsec on my homelab. Don't want any peepers on my NAS!
I'm the author of the traefik bouncer, thanks for showing my work! Great video, thank you for the content.
BTW, it's prononced F bo-na-lair. ;-)
:-) Good job, Fabien! We love it!
Thank you so much! Also, thank you for letting me know how to pronounce your name phonetically!
Next video: Tim takes jujitsu classes in case someone breaks in to steal his server.
Ninja vanish 🥷
Your content keeps getting better and better. Thanks so much for showing us how to geek out even more while keeping ourselves secure!
I appreciate that! Thank you so much! It gets harder and harder too :)
It would be great if you can revisit this video, since now traefik has an official plugin, the hub auto update itself (no cron needed), the dashboard looks cool, they have a centralized way to manage multiple instances, they added appsec WAF integration and probably more. There arent't many recent tutorials and you're always spot on with yours.
Yes! We need this!
For a dude that seems to throw around containers like they are nothing, it's nice to see you admit to having to lookup something old-school like crontab. I've been watching your videos to get up to speed on containers.Thanks for the content. You are really good at it!
Thank you! I always try to share what I know and what I don’t know!
crowdsec has massive potential and it's great to see that it's getting more love
Agreed! Huge potential!
Thanks for the nice words. Highly appreciated!
Hi mate, u can setup cloudflare bouncer to have crowdsec blacklist shared up to the cloudflare layer. Using it for enterprises u can ask cloudflare to have more than just 10000 ip addresses configured as a list (they raised me to 20k). After some months a bit of improvement is reached that way.
Happy hardening u all 🎉
For those curious if you lose your api key you can just do docker exec crowdsec cscli bouncers remove bouncer-traefik and just do the add again.
Great tutorial Tim!
Anyone know of a suitable Crowdsec docker image for arm (Raspberry pi)? I had a look around and couldnt find one. I'd rather install on docker if at all possible.
I know this vid is a year old, but good video, would you do a guide for the nginx proxy manager with crowdsec?
As usual, absolutly epic guide that made it easy for me to get this up and running. thanks heaps!
Very interesting Tim 👌
Happy to view more smiles 😜
More to come!
Tim the King!! as always - thanks for the video - keep them coming - learned a lot!
I just set this up, be aware that if you're running behind a reverse proxy like cloudflare the traefik bouncer here doesn't use the correct IP address due using the incorrect header value.
I noticed that too and there is a PR out there for it. According the CrowdSec though, CloudFlare IPs are on their global allow list so they can’t be blocked. Hopefully the PR gets merged to look at the real ip in the header
@@TechnoTim The main problem isn't cloudflare IPs getting banned, it's that the bouncer doesn't block anything because it reads the headers and asks CrowdSec for information on the wrong IP.
The PR is mine, I'm running it currently on my machine and it works well so hopefully the dev comes back soon!
Ah! Nice! Thank you! I have been watching that issue! Looks like it was merged!
You are now my master in networking.
excellent document, I installed it , but had an issue witth the bouncer, even if the manual added ip deciscion is properly added to the list, the bouncer does not blocks ,
I'm already using Traefik so now I'm definitely gonna check out CrowdSec. Looks cool and easy to configure. Thanks for another awesome and easy-to-follow tutorial!
FYI, isn't the GID value in the docker-compose file supposed to have colon to specify the default value "${GID:-1000}" ?
Hey @TechnoTim,
did you tried the Metabase Dashboard? It works fine, but after compose recreation the credentials are default again. Were can i find the credential information to persist?
14:45 - I have the folder but no logs. When I exec into traefik there are both the log files. I've gone through my yml files 5 times now a nd rewatched the video to this point a few more. My networks are the same. Maybe there is something different being I am trying a year after your post?
Thanks for the great tutorial and video. I've leared a lot from you over the last few months. After working through this and installing I have a question I'd like your input on. When proxying through Cloudflare, crowdsec is analyzing the Cloudfare IP, not the real IP of the client. Now I can imagine how this may be useful if someone decides to attack the IP directly or somehow gets around Cloudflare (I can't even begin to imagine how that's even possible), but I have my firewall to only accept connections from Cloudflare IPs on 80/443. So in that instance, can you still see any benefit to crowdsec? I know there are some complicated ways to have traefik be able to see the real IP from Cloudflare, but I haven't attempted that yet.
I think there is a PR out there to fix this!
Hi Daniel, I appreciate the kind of higher level thinking of your comment. I have the exact same setup in regard to Cloudflare and only accepting Cloudflare IPs on 80/443, and had not thought of this yet. I wonder if you have done any more thinking about this. Are you still running Crowdsec or have you deemed it unnecessary? I see there is also a Cloudflare-Bouncer, which seems to take a different approach and updates your firewall settings in Cloudflare itself. Curious to hear what you think.
Hey Tim, great video appreciate all your hard work. I've been trying to install crowd sec for some time now in my environment. I have two raspberry pi's one 32 bit and one 64 bit. Crowd sec has given some instructions on how to install on arm but my Linux skills are lacking and well they don't show us like you do. Also I don't use traffic I use nginx reverse proxy. Should I even try or should I keep waiting for someone to make a video specifically for raspberry pi crowd sec with Nginx and docker.
I wouldnt run an ips firewall on a raspberry pi. Its like showing everyone you have a tank in the garden to defend your country but in reality that tank is just out of cardboard. The performance wont hold up.
I’d give it a shot over waiting, but that’s just me!
The most likely place to find help on this topic is CrowdSec discourse server or the discord one (just google them)
Hi Tim ! How do you enroll your Crowdsec container in the cloud console ? I've done it with the cscli command but it needs to be done again after each re-creation ...
Hey, Tim. Really enjoying the channel and Discord. I have a question not crowdsec related but something I noticed in the video. Would you mind going into some detail on the ip whitelist(s) for Traefik? Couple of us trying to get it figured out and not having much luck. Definitely implementing crowdsec now!
Thanks! I might at some point or you can all join our discord! Some folks use it in there!
@@TechnoTim, cool. We’re there. Great place.
That sounds great!
How do you protect your services?
I watch your videos and use them in the home lab. (it is a work in progress).
I also use a vpn and an ip whitelist.
Thanks a lot ! Please show us how you configure a proxmox log parser, or iptables bouncer on an episode #2, would love it
Instead of crontab, you might want to get familiar with systemd timers. Much easier to manage in my opinion.
Good call!
@@TechnoTim Maybe systemd timers would make a good video?
Question: Won't access log grow to infinity? How big is your access.log file right now? What should be the cap?
Request: Can you make an update to Grafana monitoring guide using influxdb and adding consolidating the alerts including crowdsec? I just want "BOGOOGA" sound alert on my phone if I am getting DDosed.
You should join our Discord and ask about the Grafana stuff if you want fast help. Also you can setup a number of notifications on CrowdSec when a scenario triggers. So basically your DDoS scenario would trigger an alert which would then be sent to your phone. No problem with CrowdSec.
How did you get Cloudflare to forward the real IP? In your case if you use Cloudflare which I think you do, the ban only worked because you have a local DNS. Banning your IP would otherwise do nothing because traefik and hence crowdsec would always see the Cloudflare IP assuming you have reverse proxy set up in CF
Why dont you just use bind mounts instead of docker volumes? Aren’t binds easier to use and backup?
I did bind mount the only thing I want backed up, the config. The rest (like logs) is in a docker volume which I don't care too much about. Also, their docs say to do it like this and I had issues trying to bind locally.
How does it even work for you. All IPs are the network bridge gateway and it doesn't work for me. 😢
I guess I would have started the bouncer before crowdsec, so that it's available when crowdsec starts up. Which means: crowdsec should depend on the bouncer. Am I wrong?
No since you can run everything distributed on different servers. You can have one agent receving logs from multiple other servers and controlling bouncers on remote firewalls, even across operating systems. So we can't depend the installation of the bouncer in the agent. And that is by design :-)
I have been using crowdsec on my Debian server for the past week but I had no idea they Docker images and docker bouncers
You can ignore those ENV for k8s. They should have access to write to their PVC
Great to hear. CrowdSec is available on a lot of platforms :-)
Hey Techno, amazing video! I was really exciting when I managed to config Crowd for read the traefik logs.
I have a question unu
Is it possible to configure Crowdsec to allow the connection from a specific origin, and ignore or prevent access to my service from other origins different from the domain I defined? The context is that I need to secure a backend that is exposed to the internet via traefik and an external frontend needs to consume it.
What is nice? A great selfhosted solution. What is even better? A solution with awesome graphics!
Thank you!
can you show how to run crowdsec with nginx proxy manager ??
Good video, Unfortunatly I found crowdsec to be buggy when blocking ssh so I went back to fail2ban.
One thing, that you still dont follow: you dont capsule your services from another.
You just use ONE network: "Proxy" if you want to protect your services even more, you should create seperate containers between the traefik and services like heimdall and co.
Thanks! I break up my compose files. Also, don't all services that are served through he reverse proxy need to be on the same docker network to proxy through?
Hi, what extension do you use for highlighting arrays in the stack!?) it very useful
Rainbow indent!
never been happier to have a cleaned up Johnny Depp show me the way
In europe stuff like this is always a pain to check if it is in line with GDPR
We're based in EU so GDPR is obviously taken into consideration. The only data that's being collected is the ip of the offender, timestamp and metadata on the attack (=which scenario triggered). So nothing to worry about in terms of GDPR.
Hey, great vids, just started self hosting, you're giving me too many ideas...
Anyway, I'd love a video covering how you might have setup services that use SMTP/email settings. Thinking WordPress, Vaultwarden etc. Thinking to have a single SMTP relay that everything points to, which then forwards out via Gmail/X service.
If you want ideas, check out ruclips.net/video/IE5y2_S8S8U/видео.html 😀😀
@@TechnoTim nooooooo 😝 my wife has enough things breaking already
Man, I absolutely love your content and knowledge. Definitely appreciate ya'...
Thank you so much. That means a lot!
Thank for the video! I would really appreciate a tutorial for using this with nginx proxy manager as well. I'm trying to figure it out, but I haven't gotten it to work yet!
Sounds great, but this is not only open sourced but the database is managed by the community; what's to stop bad actors from listing valid sites as malicious? wouldn't that make this it's own kind of ddos attack if people can't access a site because someone fraudulently added it to a block list?
That's a good question. Very shortly described it's based on trust level but servers who report; the longer time they have sine so reliably, the higher trust ranking and the more do they count when determining wheter an IP is bad or not. Also, an ASN only gets one count. All this and more exists to make poisioning as expensive and hard as possible. If you have more questions, feel free to go to our Discord.
@@crowdsec Ok thanks, a few more things to consider would be oversight, I read that a university was banned despite making years of commits to Linux for posting some intentionally bad commits, and you said that an ASN only gets one count are their measures taking botnets into consideration? because I can't imagine it would be hard for someone with a large botnet spoofing an address to make it seem malicious.
@@festro1000 Where was the university banned? Was this in relation to CrowdSec? No, we're only taking their actual behaviour in terms of how realiably they send signals into consideration.
Could you elaborate on the spoofing part?
Maybe I'm missing something. But didn't you configure a conditional forward in your UDM Pro so that only traffic from Cloudflare gets allowed? In other words, if the rest of the packets gets dropped, what's the advantage of this?
Helps if someone or something makes it past Cloudflare. It adds IPS to my Traefik instance
@@TechnoTim But if someone makes it past Cloudflare the traffic gets dropped by your firewall right? So it doesn't reach your Traefik instance.
@@mormantu8561 Why would Cloudflare be the only thing that can reach Traefik? There's a lot to connect to out there... What if someone SSH tunnels to an unlucky internal device that's been compromised in order to pivot around inside the network? Better to have multiple walls to climb than just one you can walk around.
@@wyattarich True, but in another video he showed us that traffic on http(s) to his Traefik instance is only allowed from Cloudflare IP addresses. My comment was about why he would implement this if he has that rule, but come to think of it, maybe he means if Cloudflare fails to detect a threat. Whereas I thought that he meant what if someone or something bypasses Cloudflare entirely.
That's right. I don't mean that someone circumvented cloudflare, I mean that cloudflare's bot detection might not catch all bad actors, where this is yet another line of defense.
The Core Question for me is, can i make Trafik work behind an HaProxy. I have atm a haproxy running in my pfsense and i would like to keep that, but trafik with crowdsec would be a nice addition?
What IP does Crowdsec ban ? For example can i tell it to ban cf-connecting-ip ?
well here you can use crowdsec with opnsense, ha proxy, nginx or as a container so quite some flexibility :) Ip are banned base on the sightings of all user of the community and curated by CrowdSec to avoid false positives and poisoning.
@@philippehumeau7972 Played around with it, atm still behind my haproxy and works well. I noticed some problems over the time with running software behind cloudflare. Some software is intelligent enough to recognize the real ip ( it can be seen through the CF-Connecting-IP Header), some just see the Cloudflare IP and the last thing I would want it to do is to block the Cloudflare ips =) the only thing really missing from traefik is brotli, but that's just personal preference
@@manuelthallinger7297 No matter what there are ips that can't be blocked. Clouflare and other CDN provider's ips are among those.
Great video! I think I’ll deploy this at least on my Docker-Web server.
Thanks!
Brilliant idea :-)
@@crowdsec I'm sensing a bias... :D
I always see these videos with crowd-sec, fail2ban, etc, and I want to add these to my setup, but what I always see left out is the explanation of what happens to self-hosted content that isn't accessed exclusively from a browser? Like emby/plex wallabag bitwarden, etc, that have a mobile app integration and even a possible chrome extension?
Do they just break unless the app-code is specifically built to work with it? because it seems like crowdsec and f2b work by placing a sort of http "basic-auth" layer in front of it and forwarding the creds to the app and then logging the apps response and sifting through those logs with the bouncers etc, unless I misunderstood that, and if that's the case, what if the chrome extensions for bitwarden and the mobile apps for emby/plex aren't setup to expect that middleware layer between the emby server and the mobile app? For example, does the bitwarden mobile app need to be specifically developed to expect that middleware layer or is it a seemless interception of the creds the mobile app passes to what it thinks to be the bitwarden server and is in reality the traefik/crowdsec middleware? An alternative would be if the middleware just passes through traffic that has http-headers/user agent strings that identify it as a mobile app to maintain compatibility because it doesn't deal with mobile apps, but what stops bots from just using that user-agent string to bypass the middleware if that's how it works?
Again If anyone has experience with this i'd love to hear any explanations or corrections of misunderstandings i might have. It's one of those things that i've searched the docs for but it seems like i won't know it it works or not until i attempt it unless someone else has already and can share their experience
Will this still work through cloudflare, specifically does it know how to parse the cloudflare forwarded IP field?
It should be able to parse the header however I just noticed there is a PR to fix a bug with it, hopefully it gets merged! ☝️
Are you planning to do a crowdsec nginx proxy manager video tutorial? awesome video by the way but sadly i don't use treafik
what a brilliant video. i was thinking crowdsex not to long ago... but decided on a not yet... but maybe... hmm... i have a question though. What do you use for monitoring your network/home lab for failures/outages/etc etc ? I was looking at nagios but decided to stop looking there since core was note updated in 2 years... And the options are almost infinite... i'm a but lost at this point...
Thanks! Check out my video on Uptime Kuma!
wait in the end it sounds like i have to manually add ips to the descsions list. I thought this is an automatic thing that bans any IP that appears SUS to my instance or is already known to be sus.
How do you configure crowdsec to download their ban list and apply to your instance? Or is it automatic? This is kind of the whole point.
it's automatic (based on the scenario you run). you can list the content from the list with cscli
crowdsec and traefik seem to be seeing my docker bridge network gateway IP, not the client IP, so crowdsec doesn't seem to be working for me. Do you know what I would do to resolve it?
New Customers Exclusive - Get a Free 240gb SSD at Micro Center: micro.center/1fbb85 (paid)
Your my boy blue!
And I understand why you did this, and I am glad you made a video! Seriously
But I don't know what you are talking about. You went from cards you colored with a crayon yourself to this... Quite a leap! Especially for me!
Like I said, I'm glad you did it. Seriously. And in about 3 years when I have caught up with you, I will be thankful! Ha ha keep up the good work. Lots of your vids meant nothing at first and then a few months later, I was on bord.
Thank you!
Hi just wanted to ask u about your rack, on the bottom of it you have a 24 bay disk shelf what did you use to mount it in the rack? was it a Adjustable Rack Mount Server Shelf Rails 1U?
You can find all the gear I recommend here! kit.co/TechnoTim
2024 and I used this to learn about this. Still good to go
Well atleast you can say that the security information that you have been providing to us, works in with evidence :)
hey it's me again I'm have a question, with that configuration you will not have logs on the the stdthing (out/err/in) don't remember witch one docker logs use, that's OK for crowdsec that need that aparently but how to put those logs in loki for grafana ?
did you try the traefik/grafana/crowdsec combo and how to make those those logs from file in the loki-driver too ?
thanks :)
Haven’t tried it yet but anything that logs to stdout should be captured and sent using my method
@@TechnoTim yeah except that for Traefik if you defined a file for the logs it will go to the file no more to stout... So you will have to set another job specific for Traefik, and may be some other container that will do the same: if log file is defined then pour in the file not stout anymore, and doing so I'm woundering how I can recognise that it's logs from container traefik. Using the same seentic in grafana... Or may be we should investigate the logs volume in crowdsec (or in the other way) ask crowdsec to look the logs from the grafana/loki logs directories...
During you were DDOS attacked I tried to find your article about traeffik 2 and I was lost 😩
My Traefik guide is here! docs.technotim.live/posts/traefik-portainer-ssl/
I used your traefik 2 ingres guiide to set up traefik as reverse proxy in my k3s cluster (some Pi's, some x86 VM's and one Mac mini M1). I needed some time to figure out how to route on external endpoints in my network for services that are not in the cluster yet. Thanks to your new guides i will never be bored.
In case you see this, this is blocking every internal service but not the ones that are external. Guess it's due to the ip that we are blocking being internal.
The reason working in crown tabs is confusing is because you shouldn't. You better set it up via crony or similar. Btw same goes for the sudoers file
That promotion has been going on for so long I'm really starting to wonder if they got an amazing deal on a container full of 240gb SATA SSDs or added a 0 to an order right before the price came down on 500's.
It is generous! They have amazing deals on everything :)
@@TechnoTim I'm a fan of Microcenter, just can't spend much money with them because they don't have a store near me and their web presence I so limited, and it is a great promotion. I'm just saying that after two years it's starting to feel like a creative solution to a massive overstock 😂
Hi Tim
I'm having an issue that it only works for me when I block a local (docker) IP.
If I block my public IP it still permits access.
When I view the logs it only shows the local addresses. Any ideas??
Thank you
I thought there might have been a bug that was recently fixed
@@TechnoTim I got it working. I added a second interface on 'host' that seemed to fix it
Nice videos BTW. I've been in the game 15 years and still learning 🇬🇧
Tim what's the music during the crowd sec intro section.
It's in the description!
Micro center is like Disney land
Agreed! That's how I feel when I walk in!
Collaborative Open Source is the future
agreed!
We approve of this message!
Open Source is collaborative by definition, I would have said it's the past, the present, and must improve in the future ;)
This is really good. Thanks for putting this together.
Can we get a updated video
I got hit with almost 8,000 requests on my Synology in three days. I watched the notification stream up into the notification box at a rapid enough pace I was legit afraid lol. Luckily I had protections in place because oh myyyy
Calmly waiting on your Turing Pi cluster video... I.... Promise..... :)
I'm working on it!
@@TechnoTim no rush! Just excited to follow along! (Assuming the format of the vid). Have 1 Pi4 8GB, 2 Pi4 4GB, and 3 Jetson Nano 2GB that I wanna use :)
Sorry just excited!
Also thank you for getting me into homelabbing! Really appreciate it!
Windows is not yet supported
#soon
The container has crontab in it. Just mount a script with cscli hub update && cscli hub upgrade to /etc/periodic/hourly.
Would this be the same as the plugin that is available for traefik?
Yes!
Thanks for the demo and info, have a great day
Thanks, you too!
When I'm using CF proxy how to get real ip to crowdsec ?
seems still pretty complex but i feel following the steps it can be done
Love your stuff
Thank you!
Install starts at 5:15
Those 1.2m people should have clicked subscribe, those get through!
That's what I'm talking about!
Very well explained but you dont have the files on github :(
The link is in the description :(
Will this work with Nginx Proxy Manager?
Not sure, check their docs!
How do i reseve proxy outside doxker
I know he's going slow, which is helpful, but I just want to do a temperature check in the comment section. Does anyone fully get what he's saying?
I do
Late for the Party but thanks anyway. Works perfect!! 🙂🙂
Thanks!
Thank you so much!
hi, you have a github with this complete solution ?
In my docs, and in github
@@TechnoTim yes i think found :) thank you
You can not protect your home lab against ddos. I suggest you not to waste time on this unless your home is inside a datacenter. Even if you set an ip whitelist to all ports and protocols it's not going to protect you. If your bandwidth can't handle it, there is nothing you can do. There is a possibility to use bgp flow to your advantage, but I don't know a single home internet provider who supports it and even then it's really limited
Just keep using proxy man
And against brute force attacks and vulnerability scanning you should just use WAF for public services and VPN server for maintenence and stuff
You can protect your homelab against DDOS by using Cloudflare which hides your public IP and points incoming traffic to their proxy which has DDOS protection.
works a treat. cheers!
ur awesome tim
It works 🙂
congrats!
Informative video. It's not nice whoever did the ddos, did it in the first place. properly for internet cred if that a thing 😁😂
haha! I agree! It was all blocked but scary / awesome to see!
"ah-quiz" file, youre welcome
Acquis is pronounced "A Key"
Crowdsec? Seriously 😅
Hi Tim,
I tried crowdsec on traefik, but I think authelia is getting in the way ! I did many try to connect on my phone but no log in traefik yet when I want to see the log of the authelia application I can see the log : Unsuccessful 1FA authentication attempt by user '' and so far CS did not decide to block those try !
so it's great to block already known IPs I looking forward to an update so we can add authelia in the survey of CS :)
I already found the collection and configuration now I need to put that togather and add a new aquisition in the list, but that part is a bit clouded for 1 folder it's clear, cristal clear, but can I add other foler with other labels... and what abount a bouncer for that app? may be it's not needed cause the app that will block is traefik; I'ld like to get the logs of the server hosting docker to be analysed too; to be sure no brute force will be attempted on my ssh even if I'm a no password guy I'd like to get those metrics in CS ;)
So here you gave me way to criticaly upgrade my securiity :D
again thanks dude :)
found part of the solution by putting the /etc/crowdsec/config.yaml file out of the container and changed the line acquisition_path to acquisition_dir and and create a folder in my mounted directory to put those acquisitions files instead of just having one file...
Hey I managed to get my phone blocked with multi testing wrong user and of course password \o/
So now that's done ! Extracting the configuration file and replace with no typo (I had a few so I had to precise) the path by a dir you can now put more than one file to the inquisition ;)
And I just thought that I could just mount the file of my host to the CS pod so CS could do it's magic too for bad guys trying to ssh in even if the challange ssh key is stronger than password that does not mean you should put no security espacialy in those dark times ! xD
So basicaly now the only thing is to do it now... yet for this one I think I should add a bouncer but how to give it access the system FW 🤔May be I will look at the bouncers and invest in a true FW it will not be lost xD
1 am here, need to sleep this over ;)