Thanks for taking the time to both make this video and post it. It helped me out a ton (more so than the Elasticsearch 8.x book I am reading). My only disappointment was that it did not cover using nginx module (which is not your fault at all) as that is the module I was struggling with. Now with that said, using the information that you provided I was able to better understand not only what is going on inside of elasticsearch but also hot to ultimately fix the issue. Thanks a ton and I will be finishing out your series and keeping an eye out for any new posts
Actually thank you for your comment! A colleague and I were contemplating if we should do a series that goes through Beats with some of the popular services like nginx, mysql, postgres, kafka, etc... Based on your comment, now we're thinking we probably should add that to our task list of tutorials!
I had problems re-entering the API key so I had to revert back to username and password in the yml file, but everything else went as shown (using my data of course lol). Thank you!
Excellent Treasure of a Video. I have one question: I want to import a .log file which has old data from another system. When I mention the path, etc.. it still doesn't showup in discover. Any idea?
Greetings, First thank you, follow all your kibana and filebeat installation steps. And I have a server running collecting netflow data from my router. However I have a problem the graphics are only 30 minutes of data and they are already overwriting and I have 1.5tb of storage, how do I fix this? Do I need to make any other adjustments to use netflow? The router that sends has a traffic of 30~40Gbs
After setting up the user I am still not getting any data in the filebeat-* index. I checked for errors in journalctl - no errors. What could be the problem?
It's been a while since I recorded this so I apologize if errors. As if today the /use/share/filebeat/bin/filebeat setup [...options...] Will set up your data views, data streams, dashboards etc.... every time your run the command, it will "reset" all those settings. So you should run it only once unless you are intent in resetting things!
@@evermightsystems ok, thank you! Do you know if there is a way to not setup the dashboards though? I tried the setting setup.dashboards.enabled: false but dashboards still seem to be loading in
I have installed Elk with Kibana and Filebeat all logs are coming in some dashboard are working fine but I need [Filebeat System] Sudo Commands dashboard data not showing how to fix this I need sudo cmd data there Can you help me achieve this
Thanks for your message preet, I don't recall if i experienced this issue, sorry unabel to assist with this at this point. But if you discover the solution, let us all know!
First off all....Thank you for this video. it was very helpful. I have one question. I am planning to install filebeat on multiple instances, So all instances should we have setup.dashboards.enabled: true.. and if I set to false for other instances, will it insert data to the existing dashboard. I am bit confused...Can you please help me in this
My apologies for mistake in the video. You only have to run the "/use/share/file beat/bin/filebeat setup command" once on one machine. Don't need to run it again from any other machine after
@@evermightsystems Thank you Sir, I am from India, And would like to say that your videos' are very authentic, Thank you for making such useful video. Have you created any any video on monitoring Kubernetes [Kubeadm] , if yes can u share the link and if not can you please plan to make video for it...
No experience with kubernetes at the moment. I plan to do more research into kubernetes after my channel gets more views and subscribers. So if you think anyone else finds these videos helpful, please spread the word and share, thank you!
Hi again when I execute finally the next command "./filebeat -e" The terminal shows me the next error: {"log.level":"error","@timestamp":"2023-12-15T22:20:34.639+0100","log.origin":{"file.name":"instance/beat.go","file.line":1307},"message":"Exiting: error connecting to Kibana: fail to get the Kibana version: fail to parse kibana version (): passed version is not semver: ","service.name":"filebeat","ecs.version":"1.6.0"} Exiting: error connecting to Kibana: fail to get the Kibana version: fail to parse kibana version (): passed version is not semver: Can you help me? Best regars and thanks.
Done, I put in the filebeat.yml in kibana section http and not https and in the output.elasticsearch the section username and password and it works. How can I only put API_KEY and it works? Thanks.
@@evermightsystemsReally good job man, congratulations. The ELK manual or guide is not so many clear how it should be, then this video is fantastic to put it on work.
Hi sorry for late reply, hopefully you've resolved your issues by now! If not, just contact us through our website or by email. Then we can set up a zoom link to troubleshoot together.
![[ ElasticSearch 15 ] Elastic Stack | Running Filebeat in a container](/img/1.gif)
Written Summary Here: elasticsearch.evermight.com/filebeat-install-part-1
You are my hero man. I could start to work with ELK and Filebeat, very big video.
Thanks a lot.
Thanks for taking the time to both make this video and post it. It helped me out a ton (more so than the Elasticsearch 8.x book I am reading). My only disappointment was that it did not cover using nginx module (which is not your fault at all) as that is the module I was struggling with. Now with that said, using the information that you provided I was able to better understand not only what is going on inside of elasticsearch but also hot to ultimately fix the issue. Thanks a ton and I will be finishing out your series and keeping an eye out for any new posts
Actually thank you for your comment! A colleague and I were contemplating if we should do a series that goes through Beats with some of the popular services like nginx, mysql, postgres, kafka, etc... Based on your comment, now we're thinking we probably should add that to our task list of tutorials!
I had problems re-entering the API key so I had to revert back to username and password in the yml file, but everything else went as shown (using my data of course lol). Thank you!
best tutorial ever
Excellent Treasure of a Video.
I have one question: I want to import a .log file which has old data from another system. When I mention the path, etc.. it still doesn't showup in discover. Any idea?
Greetings,
First thank you, follow all your kibana and filebeat installation steps. And I have a server running collecting netflow data from my router. However I have a problem the graphics are only 30 minutes of data and they are already overwriting and I have 1.5tb of storage, how do I fix this? Do I need to make any other adjustments to use netflow? The router that sends has a traffic of 30~40Gbs
After setting up the user I am still not getting any data in the filebeat-* index. I checked for errors in journalctl - no errors. What could be the problem?
I am on version 8.13.2
Got your email, let's continue conversation over email because it is easier to share logs and troubleshooting steps
@@evermightsystems having the same problem here. Can you share how to troubleshooting?
Hi great video! Just wanted to confirm, Filebeat creates the dataview with the correct index pattern for you automatically upon setup right?
It's been a while since I recorded this so I apologize if errors. As if today the /use/share/filebeat/bin/filebeat setup [...options...] Will set up your data views, data streams, dashboards etc.... every time your run the command, it will "reset" all those settings. So you should run it only once unless you are intent in resetting things!
@@evermightsystems ok, thank you! Do you know if there is a way to not setup the dashboards though? I tried the setting setup.dashboards.enabled: false but dashboards still seem to be loading in
I have installed Elk with Kibana and Filebeat all logs are coming in some dashboard are working fine but I need [Filebeat System] Sudo Commands dashboard data not showing how to fix this I need sudo cmd data there
Can you help me achieve this
Thanks for your message preet, I don't recall if i experienced this issue, sorry unabel to assist with this at this point. But if you discover the solution, let us all know!
but also failed sudo command get logged towards auth.log, so amybe check there?
Thanks for the reply..
Your video is very useful.
Okay I will check the elk documents and find the right answer.
I have one more question. How do I check user login history ?
First off all....Thank you for this video. it was very helpful. I have one question. I am planning to install filebeat on multiple instances, So all instances should we have setup.dashboards.enabled: true.. and if I set to false for other instances, will it insert data to the existing dashboard. I am bit confused...Can you please help me in this
My apologies for mistake in the video. You only have to run the "/use/share/file beat/bin/filebeat setup command" once on one machine. Don't need to run it again from any other machine after
And you can leave the setup.dashboard.enabled commented out on in every machine once the dashboards have been loaded
@@evermightsystems Thank you Sir, I am from India, And would like to say that your videos' are very authentic, Thank you for making such useful video. Have you created any any video on monitoring Kubernetes [Kubeadm] , if yes can u share the link and if not can you please plan to make video for it...
No experience with kubernetes at the moment. I plan to do more research into kubernetes after my channel gets more views and subscribers. So if you think anyone else finds these videos helpful, please spread the word and share, thank you!
@@evermightsystems sure....will do
Hi again
when I execute finally the next command "./filebeat -e"
The terminal shows me the next error:
{"log.level":"error","@timestamp":"2023-12-15T22:20:34.639+0100","log.origin":{"file.name":"instance/beat.go","file.line":1307},"message":"Exiting: error connecting to Kibana: fail to get the Kibana version: fail to parse kibana version (): passed version is not semver: ","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: error connecting to Kibana: fail to get the Kibana version: fail to parse kibana version (): passed version is not semver:
Can you help me?
Best regars and thanks.
Done, I put in the filebeat.yml in kibana section http and not https
and in the output.elasticsearch the section username and password and it works.
How can I only put API_KEY and it works?
Thanks.
Glad it worked! I will release a video on how to use keystore and API keys with docker compose very soon! Just editing the video right now
@@evermightsystemsReally good job man, congratulations. The ELK manual or guide is not so many clear how it should be, then this video is fantastic to put it on work.
hello ,
security: server's certificate chain verification is enabled
handshake... ERROR x509: certificate signed by unknown authority
after:./ filebeat test output -c /etc/filebeat/filebeat.yml --path.data /var/lib/filebeat --path.home /usr/share/filebeat
Hey there I think your filebeat.yml will need to reference the CA file. If you have difficulty doing this, send us an email.
@@evermightsystems solved :)
Hello.
GET _ingest/geoip/stats
{
"stats": {
"successful_downloads": 0,
"failed_downloads": 0,
"total_download_time": 0,
"databases_count": 0,
"skipped_updates": 0,
"expired_databases": 0
},
"nodes": {}
}
your solution not running for me.
Hi sorry for late reply, hopefully you've resolved your issues by now! If not, just contact us through our website or by email. Then we can set up a zoom link to troubleshoot together.