New to HA ,and this channel has definitely been our #1 online resource: *** TIP *** not sure if this is known to all, but as a 'newbie' we found out the hard way: if you set up your 'super secure password' utilizing the character '@' in it (for example, to change the letter a,A with @, as in '@ppl3' instead of 'apple'), the database, logs, records, history will NOT START! Only after looking into the logs for an hour, we discovered the @ is used in the [>]recorder code as a designator for db/server location! So the only @ character in the [>]db_url line MUST BE the one between your 'password' and 'core-mariadb' FYI! :D
Just switched to MariaDB using this video after corruption with my SQLlite DB. Really appreciate the clear no-nonsense delivery of information. Surprised how easy it was to do. 5 minutes and all up and running.
I know this video is quite old and a lot has changed with home assistant so Im wondering if things like MariaDB are still the way to go or would it be a good thing to go back to default? as always thanks for the videos looking forward to the new sensor when it comes
Finally got round to adding an SSD to my install. These TTT videos are amazing. Every single one of them is so clear and concise. It's epic. Thanks again Lewis
@@EverythingSmartHome Hi! I have an update on the Maria DB and a possible issue. I've had to back out to my previous configuration. My hardware is an INTEL NUC Celeron 7CYJH with 8GB Ram and a 128GB SSD and I'm using NABU CASA. Basically Alexa Media TTS fails to work with Maria DB, at least it keeps requesting re-authentication and then the same again a few hours later. I backed out to the previous snapshot and all ok again. Very strange. I'll try installing again to double check, but has anyone else seen this issue?
Hello! That is strange, MariaDB doesn't have anything to do with the workings of HA, it only handles history and log book data, that is all. No integrations or anything is stored in the database. Did you check your logs before reverting? I've seen others complain that different things aren't working but it's because they've made an error in the mariadb setup and HA can't start properly, preventing it from loading components after the recorder. The HA logs should confirm this
@@EverythingSmartHome I will repeat the process to be 100% sure, but it was the only change I made. I have a policy of making one change at a time exactly for this reason, I'll feed back again. No doubt of the performance improvement on the NUC though. I was wondering if I could easily use a remote Maria DB as I run one on my Synology DiskStation for my Amateur Radio logbook.
Very important step missed - around 3:30 Restart HA, and wait and wait, make a cup of tea and wait ** Do not faff about with your config file think you have got your password wrong ....... patience is a virtue I don't have! Great Video again
Good info! I had recently set this up but now realized I’ve made some mistakes so will be going back and correcting them like going to an SSD card lol. One thing to note is that the history will have too much data, like all the entities so you can filter what you want in there by typing “include:” or “exclude:”. This is mentioned in the HA page
I've only recently installed Home Assistant as a VM on my Desktop running Windows 10 (cause it's always on). I've watched some of your videos and find them very useful and easy to understand. Now I've got all my smart blubs, smart plugs, printer, router info, person tracking working. I'm adding MariaDB and Visual Studio Code after watching this video. Just wanted to say "Thank you". Keep it going :-)
@@EverythingSmartHome makes sense, I did the MariaDB on a supervised Deb install and have let it run now a couple of days. I'll have a look into how the disk usage is now. Trying to read up on how I can controll how long data is kept and so on.
How about a video of a UPS for a Raspberry Pi? I'm living in an area with a lot of power outages and I am trying to find a solution to keep my Raspberry pi 4 running
Thanks! Tonight, I used this and your related video to setup both influxdb and mariadb with 372 day retention. I guess I'll have to use them to watch my free disk space, now ;) Tonight I also found the Plotly Graph Card (HACS) which I think I'll love. I've been using the apexcharts-card (HACS) and like it a lot.
Cheers ... sussed out my missing logbook and history. In MariaDB config I had changed the user name but hadn't updated the URL. Thanks for the kick in the arse. Been bugging me for months
HA newbie here. I'm on the 2024.2.1 version of HA and it looks like things have changed a bit with the UI since this tutorial was created. Would love an update!
Another great tutorial.. Thank you.. Like the Cards format you show at the end.. Could you go into more detail for that final monitoring dashboard look?
Thank you! Sure I can look at that, they are made using "Apex" cards from HACS, there are even some example code in their Github. Hope that helps a little!
Love your video's just wondering why you would put you db_url in the configuration.yml instead of only a reference to the secrets.yml seems the better place like that
I actually do this in my configuration. the entire parameter is in the secrets.yaml file. then I reference it in the configuration.yaml like: db_url: !secret mariadb_url .... while secrets.yaml is not any more secure than the configuration.yaml file since they are colocated and not enciphered, it DOES allow me to do things like backup everything to github and not push the secrets file to github. It also allows me to easily share configuration samples with people without having to scrub them.
I thoroughly appreciate your SD card warning. My strong preference is to not run Pi's from SD - I was so glad when the Pi 4's started shipping with built in support for booting from USB. I prefer to run them with m.2 SSDs. All that said, my HA now runs on a VM. I migrated a couple weeks ago and it's a LOT snappier than it was on the Pi4.
Waiting to order my blue when they come back in stock (fingers crossed) before i do the mariadb switch over.. my rpi3 is busy enough :-) Another good video.... nice one
I'm surprised to hear that SQLite isn't performing well (or reliably) considering Home Assistant only has 10 days of data and this should be way, way below any threshold where performance is suffering. Of course i've never analyzed the queries (the achillieas heel of all SQL database apps) but was wondering if you had any metrics on how MariaDB had improved things.
@@mrphyslaw i dont have nuc, they're somewhat expensive, i already have a rasp pi and a ssd is cheap as. i dont see a need for now with as little devices as i have.
@@EverythingSmartHome It can be done on a Raspberry Pi 3. Would have to be a USB to SATA Adapter with a SSD and would still need a SD Card. Just have to modify the System to use the SSD as the docker container storage. Could also be easily done with a existing system by using a directory link. Another option is a MicroSD to eMMC Adapter.
Is there a option to migrate the sensor history to mariaDB? I don't want to loose especially some of the history like the power usage! It is so helpful to see if I'm consuming more or less power and for that I need my history!
Thanks for another great tutorial. It goes straight to favorites folder. Here's another potential video for us - at 0:48 it shows disk usage as full as 91%. My one is bit lower but I'm sure I'll catch you up soon enough. So what's way to clean the disk and remove old/obsolete files that are not needed in HA any longer. (and how to to delete something essential for sure). Thanks.
Yeah it would be entirely dependent on what you have running etc but are you storing any CCTV images etc? And yes on a VM, I could do with increasing the size at some point 😅
Great video! Using Home Assistant over a year now. Just switched to MariaDB. One question,- in the roundup you showed a couple of graphs. Which card did you used for the network network_throughput_in and network_throughput_out. Awesome if you can share this :) Kudos for the video. Keep up the good work.
Lewis thanks again for a great video. I followed along and all went well except I am missing a history tab. Is there something I need to add to get it to appear? I also did not see the old db file to delete?
Great video, really easy to follow. Any chance on doing a tutorial on energy cost monitoring. Particularly daily/weekly/monthly running cost using day/night tariffs
It worked as you suggested, and I know you mentioned Sensor data will be lost, bu I was hoping I had heard you wrong. Of course, it is lost. Is there any way to get the sensor data back. Losing all the energy stats is a biggy. Is there a solution you are aware of?
Great video. I initially set up HA on an SD card and RPi 4 4G just to play around and see wat it could do. As I am planning on keeping it, is it possible to transfer HA to an SSD drive withing having to do a complete reinstall?
@EverythingSmartHome if you are running Home Assistant with Proxmox, would you continue to use the Add-On or move the database to a separate container or VM?
Any idea how one might swap history recording to a Docker based MariaDB instance? I’m already using MariaDB for a couple of databases and I not sure of the impact of having two MariaDB instances - one in HA and one in my Docker container.
@@EverythingSmartHome no problem, i've tried to watch a number of videos on the subject and they never quite seem to explain it clearly, you have a nice succint way of demonstrating it, definitely can tell you do prep and it shows
If you're seeing issues with recorder, logbook etc after switching over to MariaDB; try a shorter password and remove any symbols, restart after changes.
Big problem when the supersecure password has the @ character included. The recorder handshake was not happening, and no data was accumulating. No documentation that I could find. Changed the db name, changed the core-mariadb with the IP address, and eventually I noticed that there was a @#$@ sequence in the yaml file that didn't look good at all. Beginner's mistake, I guess...
I wouldn't say that was a mistake on your part. No reason you wouldn't think @ was OK. It was only chance that I didn't think to include that character in mine. It (the HA interface) should make it clear in the settings if the password is restricted to a set of special characters.
Have you tried using MariaDB with a socket connection? This would definitely be faster still. I have a feeling that socket may be obfuscated a bit because I think Maria DB is running inside a docker container and i'm not sure that HA and Maria have a shared volume where this socket can be reached.
Great video. Any suggestions on manually entering some of the historical sensor data ? Example: I have about 12 utility meters that keep count of my solar gen, electric usage, grid usage and battery usage. I have 3 years of history now so it would be a sad day to lost it. I just have 4 sets of sensors for daily, monthly, yearly and all time. :-) any advice on how to add if I convert are welcome
Thanks for this. But after following this tutorial and deleted the old database file I see the system monitor-stats didnt get removed. Are those stored somewhere else? Also my energy sensor didnt seem to get resetted!
Thanks for this video. I am also running HA virtualized using qemu/kvm and virtual manager however, I am unable to perform snapshots with the current HA qcow. I was wondering what were you using for snapshots? Thx.
very useful video. thanks for that. It did however leave me wondering if running the db in mardiab somehow will require me to take some steps to make sure that the database is also included in my auto backup to my nas. if anybody has any thoughts on this I would love to hear them!
adapters that begin with en are cause its in a VM. If you are on bare metal such as a raspberry pi and using the built in ethernet,. the adapter is likely going to be "eth0" or if using built in wifi "wlan0"
Great video, I've set up Maria DB a while back when I saw your video. Do you have any information/video on how to set up reporting or extract data from the events table? Keep up the great work!
Great video. Hoping for some helpfull tips for more stability problems of HA. Almost daily HA is in a hung status. How can I see why this is happening ?
Nice video, but this isnt working for me... meybe its because of newer verison of HA ? (2021.12.10 HA OS 7.2) but after MariaDB install whole HA is noticeably slower...
Despite the "speed increases" of the past 3 or 4 releases, I'm finding that with 2022.7 it takes much longer to access the HA web UI than it ever has. I'm running a VM with decent and dedicated resource allocation on a fast server and first access to HA feels like I'm on dialup.
I saw the same thing. Apparently Studio Code Server only shows files it can edit. Do you have Terminal installed? If so. you can do it by opening the terminal and entering the following commands: > cd /config > rm home-assistant_v2.db NOTE: instead of using the rm command (remove), I used the mv command (move - which is also used to rename) to rename it. I did this out of an abundance of caution (paranoia). That command is: > mv home-assistant_v2.db home-assistant_v2.db.bak After a few days I'll delete it with: > rm home-assistant_v2.db.bak
Hi, thanks for the video... planning to move to Maria DB... just a quick question. I use RPi 3 and a SSD as external data disk. Home assistant boots from an SD card , but all data is stored on SSD connected via USB port. I heard you mention not to use it with an SD card and to move to SSD. Tried a lot, but can't get to boot from SSD on RPI 3. So only did I start using the data disk feature. So is it ok to use Maria DB on my existing setup? Or will i have to boot from SSD (possible on RPI 4's but hardly available in the market right now)
Hi Lads, I understand Maria DB is sensitive to backups unless stopped prior to manual backups. Might I decide to use MariaDB in my forthcoming install , would this cause a problem with automatic routine backups with HA OS, generating corrupted files?
Hello, I'm planning to move my installation to an ODROID with eMMC. Is it ok to move to MariaDB with eMMC? Or is it not recommended like an SD card install?
Hey, Lewis! What do you thing about a video thats help us switching our Raspberry Pi SD card to a SSD? Some adaptors and SSDs recommendations might be a good options for the video too!
Im running home assistant on a VM in my Synology NAS. I also have a docker container with mariadb running for my kodi database. can i use this for home assistant too?
Top work sir, concise and bang on the money, I've been meaning to move off SQL-lite for a while now as I was getting table errors in the logs and the system started to feel a bit sluggish. Personally I use Netdata docker to monitor my systems though, as it's scalable and then centrally managed as I have a few low powered boxes running round the house that I need to keep an eye on. did I hear you say Node red ha ha ha ha
Hey, the main devs came out recently and said that most people should not be changing databases. any chances on updating the video so people know not to switch
Excellent...thanks so much. One more question. On my test machine running HA in Docker on Synology, the temperature resource worked fine (which I was a bit surprised) but when I tried the same on my "NUC" (intel laptop running the nuc HA software install....so no VM), it did not. Any idea why?
Thank you for explaining I had a Influxdb & Grafana Instalation running on my RP4/SSD before insatlling Mariadb - Mariadb is now running... but influx/Gafana not any longer :( - any idea?
Great video.... now how about a step further by covering how to keep your database from getting out of hand using recorder to set what to record and also how to cap size of database by setting number of days... is there a simple way to wipe the database or compress it... I don’t know.... I just think there has to be a way to keep the data beast under control and from growing tooooo laaaarrrrrge.
Hey Richard, thanks! I think the default amount of data to retain is 10 days and after that it's purged, but you can lower that value if you want, as Miguel mentioned have a check of the recorder documentation
I tried but never got the feeling it was working as my database seem to constantly grow in size. Is there an way to report that the purge has occurred or... track on a panel the size of the database?
Hi, again great channel! I have semi-related question, you are suggesting not to base the HA on SD card while using mariaDB and InfluxDB. The new "Home Assistant Blue" is based on eMMC, is it different? Is it reliable enough? Or the SSD suggestion still stands? Thank you in advance!
Would you do a video how to change the old InfluxDB/Grafana graphs over to MariaDB/Grafana? I looked in Grafana to change the data source, but MariaDB isn't one of them listed.
Hi. Great how to monitor lesson. All worked except the network monitor. Both In and Out return an unknown. I am running HA via a Hyper V VM. Is it possible that this will not work in a VM similar to your description on processor temperature? I have verified my network adapter name. Configuration check is "green" or good. Any thoughts are greatly appreciated.....
Hello and thanks! It should work in Hyper-V no problem, the only thing that won't work as mentioned is temperature. What was the error you got? If you need help feel free to hop into the discord server!
@@EverythingSmartHome I don't get an error. The state just shows "unknown" under developer tools and the history shows "0" for both in and out throughput lines. BTW, I love your channel. You really break things down into simple terms and procedures. I'l be sending you some coffee....
Having moved over to MariaDb i have run into all sorts of issues with snapshots - specifically the fact that the MariaDb does not appear to function if restored from a snapshot meaning that when i run into issues upon major core releases (which i always do), the restore does not work for history/maria etc and i have to nuke it and rebuild from scratch.. Would you have any recommendations as to how to backup MariaDb? Other users in the forums have recommended stopping the service before taking a snapshot which is fine however it would be great if this could be automated (i am using the GDrive backup solution which is amazing! - apart from with this Db)
Did you ever figure this out? I was looking into moving to MariaDB, but I use the "Home Assistant Google Drive Backup" addon and I'm concerned that my restores won't work, or won't include the MariaDB.
@@davidhill8151 hi David, not as yet no however I would say that HA seems to much better of late in terms of stability that I haven’t needed to restore from backup. Also made some tweaks to my settings so Maria us used got short term only with Influx being used for long term storage, with this setup loosing the Maria would not be too much of a problem.
Thank you for the guide! But the processor_use entity is not created with a normal installation (no docker) on my rpi2 with a ssd. What could be the problem?
@@EverythingSmartHome yes, it is home assistant OS. I run it on an SSD by configuring it so it uses the SD card to start, then switches to the SSD. Using a RPI2 is not recommended but i have to say it runs very smoothly in my case www.creationfactory.co/2020/10/configuring-home-assistant-to-run-off.html
Question, I use influxDB integration for my historical data and graphs through Grafana. Moving to mariaDB, should the influx data just pick up where it left off?
Great info as always. Quick question, i got mariaDB set up as per your video and all seemed fine, then i noticed a couple of esphome devices had gone offline, any ideas? I went back to a backup i made on proxmox and they came back. Any thoughts would be appreciated, Allan
Thanks Allan, that is strange, the database doesnt contain any information about your sensors and integrations, it only contains sensor history so it's not possible for it to make your device go offline. There was potentially another issue simaltaneously that caused it to go offline? Did you check your HA logs?
@@EverythingSmartHome Hi thanks for your reply, no not yet, I have recovered to a backup just to get things going again. I will try again soon and let you know, busy decorating as well today, thanks, Allan
Thanks for another great video! Quick question for you though... I am running on a Raspberry Pi 4 4GB. I do have a 256 GB M.2 NVME SSD in a usb 3.0 enclosure. Frenck had mentioned instead of changing the eeprom to boot from USB 3.0, you can just leave it booting up from microsd and write all additional data to the USB 3.0 drive. Any idea how to do that?
Hey thanks Anthony! You could do that, all you would need to do is mount the USB, however if your going to the hassle of plugging in an SSD, why not just boot from it instead? Was there another reason you don't want to?
Great video! What are the limitations on password complexity for the MariaDB instance? if using a password with only characters and numericals it works but a password ending with an exclamation mark seem to be mis-interpreted by the configuration.yaml. i.e. setting the password in the MariaDB config works but HA can't connect to it using the password in the db_url.
@@EverythingSmartHome I had to revert to a password with no exclamation mark at the end. The db-url didn't work with either single or double quotation marks surrounding the password...
@@EverythingSmartHome Thanks I'll try and test to but the "plain" password in the secrets.yaml file anyway just doing things step by step. Thanks again for the great video and channel!
Is it easy to migrate homeassistant from an SD card to SSD? And if I run from an SSD shouls I use the 64 bit version? And can you make a guide how to? I'm currently running on an pi4 4GB model. Thanks for the video's!
Check the linked video, the one before this (top 6 essential beginners tips). You don't need to use this add-on, you can do it however you normally access your files. Hope that helps
@@EverythingSmartHome I removed it via the terminal session in the end but just thought that utility looked good. I've already been through the 6 beginner tips :)
I don't understand, I did the mariadb update, pretty straight forward, but afterwards I get all kinds of errors. I get an error saying the following integratings and platforms could not start. recorder logbook history energy default config I dont undestand why, I saw a complex pw can cause an issue, mine is pretty straigh forward only 8 characters long. Any thoughts?
The errors are a knock on effect from the recorder not working - make sure add-on is started and check logs for errors. Also no @ symbol in the password
@@EverythingSmartHome did my comment get deleted or something? I left a reply with a link to imjur to see a screenshot to the log I have.... Does RUclips not like that?
@@EverythingSmartHome I just typed out the errors I was seeing. waiting on integrations to complete setup: spotify (warning) bootstrap.py error setting up entry home for smartthings (error) components/smarrthings/_init_.py setup failed for logbook: (dependencyError(...), 'Could not setup dependencies: recorder') (error) setup.py unable to setup dependencies for logbook. setup faild for dependencies:recorder (error) setup.py setup failed for recorder: Integration failed to initialize. (error) setup.py Thanks for the help!
New to HA ,and this channel has definitely been our #1 online resource: *** TIP *** not sure if this is known to all, but as a 'newbie' we found out the hard way: if you set up your 'super secure password' utilizing the character '@' in it (for example, to change the letter a,A with @, as in '@ppl3' instead of 'apple'), the database, logs, records, history will NOT START! Only after looking into the logs for an hour, we discovered the @ is used in the [>]recorder code as a designator for db/server location! So the only @ character in the [>]db_url line MUST BE the one between your 'password' and 'core-mariadb' FYI! :D
Thank you, this really helped me out Jose
Just switched to MariaDB using this video after corruption with my SQLlite DB. Really appreciate the clear no-nonsense delivery of information. Surprised how easy it was to do. 5 minutes and all up and running.
I know this video is quite old and a lot has changed with home assistant so Im wondering if things like MariaDB are still the way to go or would it be a good thing to go back to default? as always thanks for the videos looking forward to the new sensor when it comes
Simplicity is the keyword for that channel. Man you make things look so simple. What a great explanation as usual. Keep the good work 💪
Appreciate it Nelson!
Finally got round to adding an SSD to my install. These TTT videos are amazing. Every single one of them is so clear and concise. It's epic. Thanks again Lewis
Thanks for the tip on the Maria DB, even though I have been working with HA since 0.4, I still gain a lot of ideas from your channel.
Thank you, your a true veteran of Home Assistant! Appreciate the support!
@@EverythingSmartHome Hi! I have an update on the Maria DB and a possible issue. I've had to back out to my previous configuration. My hardware is an INTEL NUC Celeron 7CYJH with 8GB Ram and a 128GB SSD and I'm using NABU CASA.
Basically Alexa Media TTS fails to work with Maria DB, at least it keeps requesting re-authentication and then the same again a few hours later. I backed out to the previous snapshot and all ok again. Very strange. I'll try installing again to double check, but has anyone else seen this issue?
Hello! That is strange, MariaDB doesn't have anything to do with the workings of HA, it only handles history and log book data, that is all. No integrations or anything is stored in the database.
Did you check your logs before reverting? I've seen others complain that different things aren't working but it's because they've made an error in the mariadb setup and HA can't start properly, preventing it from loading components after the recorder. The HA logs should confirm this
@@EverythingSmartHome I will repeat the process to be 100% sure, but it was the only change I made. I have a policy of making one change at a time exactly for this reason, I'll feed back again. No doubt of the performance improvement on the NUC though. I was wondering if I could easily use a remote Maria DB as I run one on my Synology DiskStation for my Amateur Radio logbook.
Definitely check the logs after migrating to see if there any issues, find us on discord if you need some help and we can try to get you sorted
This has fixed my biggest gripe i had with my Setup: Web UI being sluggish!! You are my Hero!! ;)
Very important step missed - around 3:30 Restart HA, and wait and wait, make a cup of tea and wait ** Do not faff about with your config file think you have got your password wrong ....... patience is a virtue I don't have!
Great Video again
Haha glad you got through it in the end 😂
Good info! I had recently set this up but now realized I’ve made some mistakes so will be going back and correcting them like going to an SSD card lol. One thing to note is that the history will have too much data, like all the entities so you can filter what you want in there by typing “include:” or “exclude:”. This is mentioned in the HA page
Yes that's good information Miguel!
@@EverythingSmartHome what SSD do you recommend to buy?
I've only recently installed Home Assistant as a VM on my Desktop running Windows 10 (cause it's always on). I've watched some of your videos and find them very useful and easy to understand. Now I've got all my smart blubs, smart plugs, printer, router info, person tracking working. I'm adding MariaDB and Visual Studio Code after watching this video. Just wanted to say "Thank you". Keep it going :-)
That's awesome, thanks for the taking the time!
You are a hero sir! My home assistant owes its looks, functionality and general non broked ness to you! Thank you.
You could also just use
Recorder:
commit_interval: 30
So the data is stored and will only be written on the sd card every 30 seconds
My old db was 120MB when i deleted it. That installation of HA did run for less than a week! Scary numbers there!
Depends how many entities I guess!
@@EverythingSmartHome makes sense, I did the MariaDB on a supervised Deb install and have let it run now a couple of days. I'll have a look into how the disk usage is now. Trying to read up on how I can controll how long data is kept and so on.
Another great video, added to the todo list for the weekend. Thank you
Thanks John! Let me know how it goes!
@@EverythingSmartHome All working as expected, thanks Lewis
Will be adding the monitoring and will be in touch soon!
Let me know how it goes!
Woo hoo! MariaDB up and running with all sensors spewing data! 😀
Your on a roll! 😅
Hey, great video, but the adapter can also be named by "eth0" so it do not start with "en" :)
How about a video of a UPS for a Raspberry Pi? I'm living in an area with a lot of power outages and I am trying to find a solution to keep my Raspberry pi 4 running
Great video.... again!! You are awesome! Why? Because videos are high quality, quick, no-fluff, very informative. Keep up the good work!
Thanks my friend, really appreciate it! 🙏
Thanks! Tonight, I used this and your related video to setup both influxdb and mariadb with 372 day retention. I guess I'll have to use them to watch my free disk space, now ;) Tonight I also found the Plotly Graph Card (HACS) which I think I'll love. I've been using the apexcharts-card (HACS) and like it a lot.
Cheers ... sussed out my missing logbook and history. In MariaDB config I had changed the user name but hadn't updated the URL. Thanks for the kick in the arse. Been bugging me for months
Glad you got it working!
Thanks again!
Speeded up the reaction of devices and also restarting!
Love your videos! Keep it up!
Thanks my friend!
HA newbie here. I'm on the 2024.2.1 version of HA and it looks like things have changed a bit with the UI since this tutorial was created. Would love an update!
Another great tutorial.. Thank you.. Like the Cards format you show at the end.. Could you go into more detail for that final monitoring dashboard look?
Thank you! Sure I can look at that, they are made using "Apex" cards from HACS, there are even some example code in their Github. Hope that helps a little!
Love your video's just wondering why you would put you db_url in the configuration.yml instead of only a reference to the secrets.yml seems the better place like that
I actually do this in my configuration. the entire parameter is in the secrets.yaml file. then I reference it in the configuration.yaml like: db_url: !secret mariadb_url .... while secrets.yaml is not any more secure than the configuration.yaml file since they are colocated and not enciphered, it DOES allow me to do things like backup everything to github and not push the secrets file to github. It also allows me to easily share configuration samples with people without having to scrub them.
Couldn't wait for tomorrow...or the weekend.....Job done! Big Thumbs Up.....Spot on as usual, thanks!
Awesome Paul, glad it worked!
I thoroughly appreciate your SD card warning. My strong preference is to not run Pi's from SD - I was so glad when the Pi 4's started shipping with built in support for booting from USB. I prefer to run them with m.2 SSDs. All that said, my HA now runs on a VM. I migrated a couple weeks ago and it's a LOT snappier than it was on the Pi4.
Waiting to order my blue when they come back in stock (fingers crossed) before i do the mariadb switch over.. my rpi3 is busy enough :-) Another good video.... nice one
Nice, let me know how the blue goes! You've got lots of exciting projects now 😅
Good video, been meaning to setup MariaDB for a while watching this spurred me on. Now to see what difference it makes. Cheers!
Thanks Jon! Good luck!!
I'm surprised to hear that SQLite isn't performing well (or reliably) considering Home Assistant only has 10 days of data and this should be way, way below any threshold where performance is suffering. Of course i've never analyzed the queries (the achillieas heel of all SQL database apps) but was wondering if you had any metrics on how MariaDB had improved things.
Hey. Idea for a video: How to run home assistant on an SSD with a raspberry pi!
Just get a NUC or sff pc.
@@mrphyslaw i dont have nuc, they're somewhat expensive, i already have a rasp pi and a ssd is cheap as. i dont see a need for now with as little devices as i have.
Hey Finn, I don't have a Pi 4 to use but hopefully I can try and acquire one at some point!
@@EverythingSmartHome It can be done on a Raspberry Pi 3. Would have to be a USB to SATA Adapter with a SSD and would still need a SD Card. Just have to modify the System to use the SSD as the docker container storage. Could also be easily done with a existing system by using a directory link.
Another option is a MicroSD to eMMC Adapter.
try this video ruclips.net/video/Okf5WbJ_DAo/видео.html these steps worked for me
Thank you. This worked great. I love your videos. Please keep them coming.
Thanks for the support Andy!
Brilliant, thanks for this, will get it moved over tomorrow.
Let me know how it goes Justin!
Is there a option to migrate the sensor history to mariaDB? I don't want to loose especially some of the history like the power usage! It is so helpful to see if I'm consuming more or less power and for that I need my history!
Thanks for another great tutorial. It goes straight to favorites folder. Here's another potential video for us - at 0:48 it shows disk usage as full as 91%. My one is bit lower but I'm sure I'll catch you up soon enough. So what's way to clean the disk and remove old/obsolete files that are not needed in HA any longer. (and how to to delete something essential for sure). Thanks.
He's on a VM mate, chances are he only allocated a few gigs to the install
Yeah it would be entirely dependent on what you have running etc but are you storing any CCTV images etc?
And yes on a VM, I could do with increasing the size at some point 😅
What tool did you use to create the CPU, Memory, and Throughput graphs? The default Lovelace sensor graph doesn't provide that level of granularity.
great vid, everything went swimmingly.
was hoping for this, thanks so much as always
Got you covered 👊
Thanks for the great vids Lewis, all worked perfectly!
Excellent, glad it was useful!
Great video! Using Home Assistant over a year now. Just switched to MariaDB. One question,- in the roundup you showed a couple of graphs. Which card did you used for the network network_throughput_in and network_throughput_out.
Awesome if you can share this :)
Kudos for the video. Keep up the good work.
Lewis thanks again for a great video. I followed along and all went well except I am missing a history tab. Is there something I need to add to get it to appear? I also did not see the old db file to delete?
Great video, really easy to follow. Any chance on doing a tutorial on energy cost monitoring. Particularly daily/weekly/monthly running cost using day/night tariffs
Thanks David! I'd loved too but unfortunately don't have anything I could use to demonstrate it on at the moment
It worked as you suggested, and I know you mentioned Sensor data will be lost, bu I was hoping I had heard you wrong. Of course, it is lost. Is there any way to get the sensor data back. Losing all the energy stats is a biggy. Is there a solution you are aware of?
Great video. I initially set up HA on an SD card and RPi 4 4G just to play around and see wat it could do. As I am planning on keeping it, is it possible to transfer HA to an SSD drive withing having to do a complete reinstall?
Thanks! It would be possible yes, unfortunately I don't have a Pi 4 to be able to demonstrate it
@@EverythingSmartHome Have transferred HA from an SD card to SSD drive all went smoothly. Now going to install MariaDB as suggested.
@EverythingSmartHome if you are running Home Assistant with Proxmox, would you continue to use the Add-On or move the database to a separate container or VM?
Any idea how one might swap history recording to a Docker based MariaDB instance? I’m already using MariaDB for a couple of databases and I not sure of the impact of having two MariaDB instances - one in HA and one in my Docker container.
you can just create a database in your docker and use that one
Nice and concise, thanks. Idea for a video, MQQT, especially around naming conventions
Cheers Darren! Thanks for the suggestion!
@@EverythingSmartHome no problem, i've tried to watch a number of videos on the subject and they never quite seem to explain it clearly, you have a nice succint way of demonstrating it, definitely can tell you do prep and it shows
If you're seeing issues with recorder, logbook etc after switching over to MariaDB; try a shorter password and remove any symbols, restart after changes.
Thank you Kevin for that info. That fixed my issue with the recorder, logbook, etc. = )
Big problem when the supersecure password has the @ character included. The recorder handshake was not happening, and no data was accumulating. No documentation that I could find. Changed the db name, changed the core-mariadb with the IP address, and eventually I noticed that there was a @#$@ sequence in the yaml file that didn't look good at all. Beginner's mistake, I guess...
I wouldn't say that was a mistake on your part. No reason you wouldn't think @ was OK. It was only chance that I didn't think to include that character in mine. It (the HA interface) should make it clear in the settings if the password is restricted to a set of special characters.
Have you tried using MariaDB with a socket connection? This would definitely be faster still. I have a feeling that socket may be obfuscated a bit because I think Maria DB is running inside a docker container and i'm not sure that HA and Maria have a shared volume where this socket can be reached.
Great video. Any suggestions on manually entering some of the historical sensor data ? Example: I have about 12 utility meters that keep count of my solar gen, electric usage, grid usage and battery usage. I have 3 years of history now so it would be a sad day to lost it. I just have 4 sets of sensors for daily, monthly, yearly and all time. :-) any advice on how to add if I convert are welcome
+1: I actually came to this video for that exact answer
Thanks for this. But after following this tutorial and deleted the old database file I see the system monitor-stats didnt get removed. Are those stored somewhere else? Also my energy sensor didnt seem to get resetted!
Thanks for this video. I am also running HA virtualized using qemu/kvm and virtual manager however, I am unable to perform snapshots with the current HA qcow. I was wondering what were you using for snapshots? Thx.
very useful video. thanks for that.
It did however leave me wondering if running the db in mardiab somehow will require me to take some steps to make sure that the database is also included in my auto backup to my nas. if anybody has any thoughts on this I would love to hear them!
adapters that begin with en are cause its in a VM. If you are on bare metal such as a raspberry pi and using the built in ethernet,. the adapter is likely going to be "eth0" or if using built in wifi "wlan0"
Great video, I've set up Maria DB a while back when I saw your video. Do you have any information/video on how to set up reporting or extract data from the events table? Keep up the great work!
Great video. Hoping for some helpfull tips for more stability problems of HA. Almost daily HA is in a hung status. How can I see why this is happening ?
Thanks! Check your logs to see what's going on!
I cannot hit Like fast enough! Lewis you are too awesome!
Thanks Mark!! Glad you enjoyed it!
Hello, it's possible to use Grafana with MariaDB without installing InfluxDB?
Thanks for your inspire videos.
Nice video, but this isnt working for me... meybe its because of newer verison of HA ? (2021.12.10 HA OS 7.2) but after MariaDB install whole HA is noticeably slower...
Fantastic, keep up the good work!
Thanks buddy!
Despite the "speed increases" of the past 3 or 4 releases, I'm finding that with 2022.7 it takes much longer to access the HA web UI than it ever has. I'm running a VM with decent and dedicated resource allocation on a fast server and first access to HA feels like I'm on dialup.
So I did everything but nowhere in VSCode can I find "home-assistant_v2.db" to delete?
I saw the same thing. Apparently Studio Code Server only shows files it can edit. Do you have Terminal installed? If so. you can do it by opening the terminal and entering the following commands:
> cd /config
> rm home-assistant_v2.db
NOTE: instead of using the rm command (remove), I used the mv command (move - which is also used to rename) to rename it. I did this out of an abundance of caution (paranoia). That command is:
> mv home-assistant_v2.db home-assistant_v2.db.bak
After a few days I'll delete it with:
> rm home-assistant_v2.db.bak
Hi, thanks for the video... planning to move to Maria DB... just a quick question. I use RPi 3 and a SSD as external data disk. Home assistant boots from an SD card , but all data is stored on SSD connected via USB port. I heard you mention not to use it with an SD card and to move to SSD. Tried a lot, but can't get to boot from SSD on RPI 3. So only did I start using the data disk feature. So is it ok to use Maria DB on my existing setup? Or will i have to boot from SSD (possible on RPI 4's but hardly available in the market right now)
Great job as always
Appreciate it Alejandro
ever thought of adding a sensor to monitor the size of the mariadb database?
About the SD warning.... how about if I use a SD Card on a IPbox with HA ?
Anyway to do this while retaining all the sensor data?
Another great video - Thanks
Thanks Tony!
Where is the new database located? I want to create a sensor out of it!
Have these sensor settings changed now? I keep getting error messages when trying to type them in 🤔
Hi Lads, I understand Maria DB is sensitive to backups unless stopped prior to manual backups. Might I decide to use MariaDB in my forthcoming install , would this cause a problem with automatic routine backups with HA OS, generating corrupted files?
Hello, I'm planning to move my installation to an ODROID with eMMC. Is it ok to move to MariaDB with eMMC? Or is it not recommended like an SD card install?
Hey, Lewis! What do you thing about a video thats help us switching our Raspberry Pi SD card to a SSD? Some adaptors and SSDs recommendations might be a good options for the video too!
Hey! I'd love to but I don't have any Pi 4s I can use, hopefully I can acquire one soon!
Looking forward for this!
Im running home assistant on a VM in my Synology NAS. I also have a docker container with mariadb running for my kodi database. can i use this for home assistant too?
Great tutorial. Thanks.
Thanks for watching!
Top work sir, concise and bang on the money, I've been meaning to move off SQL-lite for a while now as I was getting table errors in the logs and the system started to feel a bit sluggish. Personally I use Netdata docker to monitor my systems though, as it's scalable and then centrally managed as I have a few low powered boxes running round the house that I need to keep an eye on. did I hear you say Node red ha ha ha ha
Thanks Mark! Hopefully you can migrate soon! That's a good solution also, wouldnt go changing!
and another one done :p ,although not the system monitoring since i don't see the point for myself :)
Nice! 🙌 Yeah for your setup probably doesn't make an awful lot of sense!
Hey, the main devs came out recently and said that most people should not be changing databases. any chances on updating the video so people know not to switch
Thank you, it was helpfull !
Thanks for the video. Perhaps it's already been answered, but how did you do that Bandwidth card?
Hey Brian, this a custom card from HACS called "Apex"
Excellent...thanks so much. One more question. On my test machine running HA in Docker on Synology, the temperature resource worked fine (which I was a bit surprised) but when I tried the same on my "NUC" (intel laptop running the nuc HA software install....so no VM), it did not. Any idea why?
Is there a need for a strong password at all when it's being stored in plain text in the HA config?
Thank you for explaining
I had a Influxdb & Grafana Instalation running on my RP4/SSD before insatlling Mariadb - Mariadb is now running... but influx/Gafana not any longer :( - any idea?
There shouldn't be an issue, can you check your logs?
@@EverythingSmartHome THank you I found the issue ... an old non-updated IP-Adress
Great video.... now how about a step further by covering how to keep your database from getting out of hand using recorder to set what to record and also how to cap size of database by setting number of days... is there a simple way to wipe the database or compress it... I don’t know.... I just think there has to be a way to keep the data beast under control and from growing tooooo laaaarrrrrge.
check out the recorder HA docs. They have what you need.
Hey Richard, thanks! I think the default amount of data to retain is 10 days and after that it's purged, but you can lower that value if you want, as Miguel mentioned have a check of the recorder documentation
I tried but never got the feeling it was working as my database seem to constantly grow in size. Is there an way to report that the purge has occurred or... track on a panel the size of the database?
Where is the new MariaDB file database located (in case you might want to back it up seperately)?
Hi, again great channel!
I have semi-related question, you are suggesting not to base the HA on SD card while using mariaDB and InfluxDB.
The new "Home Assistant Blue" is based on eMMC, is it different? Is it reliable enough? Or the SSD suggestion still stands? Thank you in advance!
Thanks! eMMC generally has much more endurance and write cycles so you should be fine, this is drive dependant but yeah you will be good to go!
@@EverythingSmartHome thank you! You are amazing!
And you are awesome!! Thanks for the support
Would you do a video how to change the old InfluxDB/Grafana graphs over to MariaDB/Grafana?
I looked in Grafana to change the data source, but MariaDB isn't one of them listed.
Yes that's correct you need influxdb installed also AFAIK
@@EverythingSmartHome OK - my mistake. I thought MariaDB took the place of InfluxDB and needed to be configured in Grafana.
Great videos!!
Thank you!
I am understanding that if you use MariaDB in place of InfluxDB then you cannot use Grafana?
Both can run at the same time, you don't have to chose mariaDB or influxdb - they do different things. Hope that helps!
Hi. Great how to monitor lesson. All worked except the network monitor. Both In and Out return an unknown. I am running HA via a Hyper V VM. Is it possible that this will not work in a VM similar to your description on processor temperature? I have verified my network adapter name. Configuration check is "green" or good. Any thoughts are greatly appreciated.....
Hello and thanks! It should work in Hyper-V no problem, the only thing that won't work as mentioned is temperature. What was the error you got? If you need help feel free to hop into the discord server!
@@EverythingSmartHome I don't get an error. The state just shows "unknown" under developer tools and the history shows "0" for both in and out throughput lines. BTW, I love your channel. You really break things down into simple terms and procedures. I'l be sending you some coffee....
Having moved over to MariaDb i have run into all sorts of issues with snapshots - specifically the fact that the MariaDb does not appear to function if restored from a snapshot meaning that when i run into issues upon major core releases (which i always do), the restore does not work for history/maria etc and i have to nuke it and rebuild from scratch.. Would you have any recommendations as to how to backup MariaDb? Other users in the forums have recommended stopping the service before taking a snapshot which is fine however it would be great if this could be automated (i am using the GDrive backup solution which is amazing! - apart from with this Db)
Did you ever figure this out? I was looking into moving to MariaDB, but I use the "Home Assistant Google Drive Backup" addon and I'm concerned that my restores won't work, or won't include the MariaDB.
@@davidhill8151 hi David, not as yet no however I would say that HA seems to much better of late in terms of stability that I haven’t needed to restore from backup. Also made some tweaks to my settings so Maria us used got short term only with Influx being used for long term storage, with this setup loosing the Maria would not be too much of a problem.
Hi Lewis: For some reason I lost the history icon after installing the MariaDB. I have a Home Assistant Blue.
That's strange, check the HA logs to make sure you didn't make a mistake and the database has actually loaded
@@EverythingSmartHome Fixed, re-installed, I think it was because I had a '@' in my password, Thank you!
Nice glad you got it!
@@luisabanto7563 Deff the @ in the password.. I just changed mine and it works perfect now.
Thank you for the guide! But the processor_use entity is not created with a normal installation (no docker) on my rpi2 with a ssd. What could be the problem?
Your welcome! Your using Home Assistant OS? (If you are btw then this runs docker) also you have an SSD on a Pi 2?!
@@EverythingSmartHome yes, it is home assistant OS. I run it on an SSD by configuring it so it uses the SD card to start, then switches to the SSD. Using a RPI2 is not recommended but i have to say it runs very smoothly in my case
www.creationfactory.co/2020/10/configuring-home-assistant-to-run-off.html
Question, I use influxDB integration for my historical data and graphs through Grafana. Moving to mariaDB, should the influx data just pick up where it left off?
There should be no issue since influxdb is a complete seperate standalone integration. Hope that helps!
Great info as always. Quick question, i got mariaDB set up as per your video and all seemed fine, then i noticed a couple of esphome devices had gone offline, any ideas? I went back to a backup i made on proxmox and they came back. Any thoughts would be appreciated, Allan
Thanks Allan, that is strange, the database doesnt contain any information about your sensors and integrations, it only contains sensor history so it's not possible for it to make your device go offline. There was potentially another issue simaltaneously that caused it to go offline? Did you check your HA logs?
@@EverythingSmartHome
Hi thanks for your reply, no not yet, I have recovered to a backup just to get things going again. I will try again soon and let you know, busy decorating as well today, thanks, Allan
No problem, if you need help please jump in the discord server and we can get you sorted!
@@EverythingSmartHome Cheers
Why sqlite less reliable ? Mariadb without high avail cluster is really more reliable ??
How can i add configuration validation in server controls?
Thanks for another great video! Quick question for you though... I am running on a Raspberry Pi 4 4GB. I do have a 256 GB M.2 NVME SSD in a usb 3.0 enclosure. Frenck had mentioned instead of changing the eeprom to boot from USB 3.0, you can just leave it booting up from microsd and write all additional data to the USB 3.0 drive. Any idea how to do that?
Hey thanks Anthony! You could do that, all you would need to do is mount the USB, however if your going to the hassle of plugging in an SSD, why not just boot from it instead? Was there another reason you don't want to?
@@EverythingSmartHome Not necessarily. Still learning Linux and the like , so working on getting more comfortable with it.
@@EverythingSmartHome I think this is a topic for the next one! What's the best way to get booted from the SSD?
I don't have a Pi 4 to be able to use at the moment but perhaps I can't ry get a good of one!
Great video! What are the limitations on password complexity for the MariaDB instance? if using a password with only characters and numericals it works but a password ending with an exclamation mark seem to be mis-interpreted by the configuration.yaml. i.e. setting the password in the MariaDB config works but HA can't connect to it using the password in the db_url.
Thanks! Yep it will most likely interpret it as an "include" try doing it in quotes
@@EverythingSmartHome thanks i will try both single quotes and double quotes as I don't know yaml syntax by heart
@@EverythingSmartHome I had to revert to a password with no exclamation mark at the end. The db-url didn't work with either single or double quotation marks surrounding the password...
That's annoying. You could try putting it into the secrets file but I'm guessing it would just do the same
@@EverythingSmartHome Thanks I'll try and test to but the "plain" password in the secrets.yaml file anyway just doing things step by step. Thanks again for the great video and channel!
Is it easy to migrate homeassistant from an SD card to SSD? And if I run from an SSD shouls I use the 64 bit version? And can you make a guide how to? I'm currently running on an pi4 4GB model. Thanks for the video's!
Thanks for watching! Yes it's easy, I would just use the snapshot restore feature and do a fresh install. Hope that helps!
To delete the old DB you use "File editor" from the left bar but I don't have this, what add-on is this?
Check the linked video, the one before this (top 6 essential beginners tips). You don't need to use this add-on, you can do it however you normally access your files. Hope that helps
@@EverythingSmartHome I removed it via the terminal session in the end but just thought that utility looked good. I've already been through the 6 beginner tips :)
I don't understand, I did the mariadb update, pretty straight forward, but afterwards I get all kinds of errors.
I get an error saying the following integratings and platforms could not start.
recorder
logbook
history
energy
default config
I dont undestand why, I saw a complex pw can cause an issue, mine is pretty straigh forward only 8 characters long.
Any thoughts?
The errors are a knock on effect from the recorder not working - make sure add-on is started and check logs for errors. Also no @ symbol in the password
@@EverythingSmartHome did my comment get deleted or something? I left a reply with a link to imjur to see a screenshot to the log I have.... Does RUclips not like that?
@@EverythingSmartHome I just typed out the errors I was seeing.
waiting on integrations to complete setup: spotify
(warning) bootstrap.py
error setting up entry home for smartthings
(error) components/smarrthings/_init_.py
setup failed for logbook: (dependencyError(...), 'Could not setup dependencies: recorder')
(error) setup.py
unable to setup dependencies for logbook. setup faild for dependencies:recorder
(error) setup.py
setup failed for recorder: Integration failed to initialize.
(error) setup.py
Thanks for the help!