The cause is that you need to stop the containers before backing up the volumes. If a DB transaction is occuring during backup, your DB might get corrupted. Either stop the containers, or carry out backup when no one will be using the DB, or use the DB specific cli to dump the db data into a file and then backup that file. You would need a bash script for this.
Looks like you might be a Tube Archivist user? Would you consider making a video/series on that? There are very few of them on YT and I don't really understand why. It's a great app!
7 месяцев назад
Thanks for this! I was thinking about using Rclone for this purpose, but it has no usable in-built webgui with scheduling at the moment. I could setup backing up my Home Assistant to Google Drive with Duplicati in a matter of minutes.
📹 **Video Overview**: Introduction to setting up Duplicati with Docker Compose for data backup. 🔍 **Duplicati Purpose**: - 🛠 Tool for backing up servers and data files. - ❗ Not a full systemwide backup tool, but ideal for specific backups. 🐋 **Docker Focus**: - 📂 Backing up Docker volumes and configs. - 🔄 Demonstrates setup for server backups. 💾 **Backup Importance**: - 🛡 Self-hosting requires personal backup management. - 🚫 Risks of data loss without backups. 🔧 **Setup Process**: - 📖 Provides documentation and Docker Compose files for guidance. - 🖥 Shows Duplicati running as a Docker container. 👀 **Duplicati Features**: - 🕒 Automatic backups scheduled (e.g., daily at 1:00 PM). - 🔄 Restoring files capability. - 🗂 Backup of Docker configs and volumes. ⚙ **Configuration Details**: - 📌 Setting up on various servers (e.g., home server, Raspberry Pi). - 🚀 Launching Duplicati with Docker Compose. 🚧 **Permissions and Security**: - 🛂 Discusses UID and GID for access control. - ⚠ Highlights potential security considerations. 📝 **Duplicati Setup Tutorial**: - 🖥 Demonstrates setting up on Raspberry Pi. - 📊 Explains Docker Compose configuration. - 🔄 Restarting server to run Duplicati. 💻 **First Run Setup**: - 🔐 Option to set a password for multi-user environments. - 🌐 Accessing Duplicati via web port 8200. - 🗃 Adding and configuring a new backup. Here are the key points from the text, each with a selected emoji: 1. 🗂 **Choose Backup Destination**: Options include NAS, cloud services like Azure, Google Drive, or mega.nz. 2. 🌐 **Cloud Backup with mega.nz**: Utilize mega.nz for cloud backup, creating a dedicated directory for your backups. 3. 🛠 **Setting Up Backup Folder**: Create and select a specific folder in your backup destination, like 'Alum backup' in mega.nz. 4. 🔑 **Enter Account Credentials**: Input username and password for your backup service account. 5. 🌍 **Test Connection**: Verify connection to your backup provider and create necessary folder paths. 6. ⏱ **Select Source Data Carefully**: Avoid backing up the entire system; choose specific directories or files, like the Docker directory. 7. 📁 **Exclude Unnecessary Data**: Omit large or irrelevant files like movies or downloads from your backup. 8. 🔄 **Schedule Regular Backups**: Set backups to run daily at a convenient time, such as 1:00 AM. 9. 📦 **Configure Backup File Size and Retention**: Set individual backup file sizes (e.g., 150 MB chunks) and choose a retention policy like smart backup retention. 10. 🔒 **Password Warning**: Acknowledge and accept if warned about a weak password. 11. 🚀 **Start the Backup Process**: Initiate the backup and verify that files are correctly uploaded to your chosen service. 12. 🔄 **Restoring Backups**: In case of server issues, restore files using the backup service, ensuring all necessary data like Docker containers are recovered. 13. 🤝 **Community Support**: Utilize Discord or RUclips comments for assistance and follow documentation for detailed setup instructions. 14. 🔗 **Additional Resources**: Check description for links to documentation and mega.nz sign-up if interested. 15. 👋 **Closing Remarks**: Thank you message and invitation to the next video.
Great video! How are you backing up the duplicati database that contains the information you need to restore in the event of a complete server rebuild? On a newly rebuilt server you will have an empty duplicati database.
Hey, reading the docs I was going based off this - “If you don't have an exported configuration file, you need to know the backend URL, credentials and the backup passphrase. Once entered all needed information, you can start restoring your files.” - duplicati.readthedocs.io/en/latest/03-using-the-graphical-user-interface/#restoring-files-if-your-duplicati-installation-is-lost
7 месяцев назад
@@Techdox maybe Duplicate can backup it's own config directory :)
what happens if the docker volumes or binds that you back up will have read / write ops when they are backed up? shouldn't the container be stoped before being backed up?
@@Techdox I've used it once to backup some stuff before a reinstall but considering some better backups for a bunch of stuff. When trying to decide between Duplicati and Restic there were a bunch of posts on reddit etc. talking about issues with Duplicati. Plan would be to backup to unraid server via Minio and rclone to MS Drive (6TB for $18 NZ/mth is the best deal I've found for cloud storage!) The restic UI (cli) is pretty nifty - havne't looked at any front end for it though.
Thanks for the help in Discord and here as well!
DUplicati never worked for me. I had always database problems and was not able to figure out the root cause.
The cause is that you need to stop the containers before backing up the volumes. If a DB transaction is occuring during backup, your DB might get corrupted. Either stop the containers, or carry out backup when no one will be using the DB, or use the DB specific cli to dump the db data into a file and then backup that file. You would need a bash script for this.
Looks like you might be a Tube Archivist user? Would you consider making a video/series on that? There are very few of them on YT and I don't really understand why. It's a great app!
Thanks for this! I was thinking about using Rclone for this purpose, but it has no usable in-built webgui with scheduling at the moment. I could setup backing up my Home Assistant to Google Drive with Duplicati in a matter of minutes.
📹 **Video Overview**: Introduction to setting up Duplicati with Docker Compose for data backup.
🔍 **Duplicati Purpose**:
- 🛠 Tool for backing up servers and data files.
- ❗ Not a full systemwide backup tool, but ideal for specific backups.
🐋 **Docker Focus**:
- 📂 Backing up Docker volumes and configs.
- 🔄 Demonstrates setup for server backups.
💾 **Backup Importance**:
- 🛡 Self-hosting requires personal backup management.
- 🚫 Risks of data loss without backups.
🔧 **Setup Process**:
- 📖 Provides documentation and Docker Compose files for guidance.
- 🖥 Shows Duplicati running as a Docker container.
👀 **Duplicati Features**:
- 🕒 Automatic backups scheduled (e.g., daily at 1:00 PM).
- 🔄 Restoring files capability.
- 🗂 Backup of Docker configs and volumes.
⚙ **Configuration Details**:
- 📌 Setting up on various servers (e.g., home server, Raspberry Pi).
- 🚀 Launching Duplicati with Docker Compose.
🚧 **Permissions and Security**:
- 🛂 Discusses UID and GID for access control.
- ⚠ Highlights potential security considerations.
📝 **Duplicati Setup Tutorial**:
- 🖥 Demonstrates setting up on Raspberry Pi.
- 📊 Explains Docker Compose configuration.
- 🔄 Restarting server to run Duplicati.
💻 **First Run Setup**:
- 🔐 Option to set a password for multi-user environments.
- 🌐 Accessing Duplicati via web port 8200.
- 🗃 Adding and configuring a new backup.
Here are the key points from the text, each with a selected emoji:
1. 🗂 **Choose Backup Destination**: Options include NAS, cloud services like Azure, Google Drive, or mega.nz.
2. 🌐 **Cloud Backup with mega.nz**: Utilize mega.nz for cloud backup, creating a dedicated directory for your backups.
3. 🛠 **Setting Up Backup Folder**: Create and select a specific folder in your backup destination, like 'Alum backup' in mega.nz.
4. 🔑 **Enter Account Credentials**: Input username and password for your backup service account.
5. 🌍 **Test Connection**: Verify connection to your backup provider and create necessary folder paths.
6. ⏱ **Select Source Data Carefully**: Avoid backing up the entire system; choose specific directories or files, like the Docker directory.
7. 📁 **Exclude Unnecessary Data**: Omit large or irrelevant files like movies or downloads from your backup.
8. 🔄 **Schedule Regular Backups**: Set backups to run daily at a convenient time, such as 1:00 AM.
9. 📦 **Configure Backup File Size and Retention**: Set individual backup file sizes (e.g., 150 MB chunks) and choose a retention policy like smart backup retention.
10. 🔒 **Password Warning**: Acknowledge and accept if warned about a weak password.
11. 🚀 **Start the Backup Process**: Initiate the backup and verify that files are correctly uploaded to your chosen service.
12. 🔄 **Restoring Backups**: In case of server issues, restore files using the backup service, ensuring all necessary data like Docker containers are recovered.
13. 🤝 **Community Support**: Utilize Discord or RUclips comments for assistance and follow documentation for detailed setup instructions.
14. 🔗 **Additional Resources**: Check description for links to documentation and mega.nz sign-up if interested.
15. 👋 **Closing Remarks**: Thank you message and invitation to the next video.
thank you very mcuh
Well done
Great video! How are you backing up the duplicati database that contains the information you need to restore in the event of a complete server rebuild? On a newly rebuilt server you will have an empty duplicati database.
Hey, reading the docs I was going based off this - “If you don't have an exported configuration file, you need to know the backend URL, credentials and the backup passphrase. Once entered all needed information, you can start restoring your files.” - duplicati.readthedocs.io/en/latest/03-using-the-graphical-user-interface/#restoring-files-if-your-duplicati-installation-is-lost
@@Techdox maybe Duplicate can backup it's own config directory :)
what happens if the docker volumes or binds that you back up will have read / write ops when they are backed up? shouldn't the container be stoped before being backed up?
Yeah if it’s active data then you could have a cron job that stops the containers before the backup etc
Interested to know if you looked at restic?
I've looked into it but have yet to actually use it. Might have a play later today. Do you use it?
@@Techdox I've used it once to backup some stuff before a reinstall but considering some better backups for a bunch of stuff. When trying to decide between Duplicati and Restic there were a bunch of posts on reddit etc. talking about issues with Duplicati.
Plan would be to backup to unraid server via Minio and rclone to MS Drive (6TB for $18 NZ/mth is the best deal I've found for cloud storage!)
The restic UI (cli) is pretty nifty - havne't looked at any front end for it though.