• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

duplicati

Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

June 6, 2019 by Simon

This guide will show how you can automatically backup files to a Backblaze B2 clouds bucket with the Duplicati software on Windows and Linux.

I have blogged about how you can use the Backblaze personal backup program ( here, Use my link and get your first month of Backblaze backups free). The personal application from Backblaze is cool but what if you wanted to store 10GB for free (the first 10GB is free) or setup different buckets for different computers or applications.

Welcome to Backblaze B2 Cloud Storage.

In Australia B2 cloud costs (+GST).

  • First 10GB is free.
  • $0.005c per GB stored per month.
  • To download, 1x GB costs $0.01c.

Check our my other related posts

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways

This works alongside the private Backblaze backup mentioned here.

Screenshot: Duplicati Downalods page https://www.duplicati.com/download

Backblaze B2 Pricing

How does Backblaze B2 Compare that to other cloud data storage provides.

Screenshot of https://www.backblaze.com/b2/cloud-storage.html

Comparison chart.

ProviderStorage ($/GB/Month)Download ($/GB)
Backblaze$0.005$0.01
Amazon S3$0.021$0.05+
Microsoft Azure$0.018+$0.05+
Google cloud$0.020$0.08

Vendor cost breakdown (source).

Read here to view the official page on pricing.

Snip:

Storage:

The first 10 GB of total storage is free, and is $0.005 per GB per month for additional storage beyond 10 GB.   

Download:

The first 1 GB of downloads per day are free.  The price to download stored data costs $0.01 per GB beyond the free daily 1 GB. 

Transactions:

A 'transaction' refers to any time an api call is made with your account, such as listing the contents of your bucket or downloading a file.   B2 will charge for some transactions however, for the average user, these charges are largely negligible unless you are making an extremely high volume of api calls.

Class A transactions are free.

The first 2,500 Class B transactions are free each day. Additional Class B API calls are charged at $0.004 per 10,000 calls.  

The first 2,500 Class C transactions are free each day. Additional Class C API calls are charged is $0.004 per 1,000 calls.

Vendor Relability

https://downdetector.com is  a great site for viewing is  a vendor is down or has issues, lets compare, Backblaze with Amazon, Azure and Google Cloud

  • Downdetector.com: Backblaze
  • Downdetector.com: Azure
  • Downdetector.com: Amazon
  • Downdetector.com: Google Cloud

Backblaze have has far fewer downtime issues during the great outage of May/June 2019 compared to its competitors. What I like is Backblaze tell us how they do it and tell us what hard drives are good or bad.

Creating a Backblaze B2 Account

Login/SignUp to backblaze at https://secure.backblaze.com/user_signin.htm

You can have a sneak peek at the personal Backblaze backup signup steps in this guide (why not take up a free 1 month back free by clicking this link when you create an account).

From your account page click billing and add your account details

Screenshot of https://secure.backblaze.com/billing.htm?billing_page=b2

Read the B2 Strater Guide: https://www.backblaze.com/b2/docs/

Also review your review the following under ‘My Settings‘

  • Phone Numer (mobile) for usage alerts.
  • Two Factor Authorization for extra security (don’t forget to çopy your backup codes)

Setup Billing Alerts (just in case)

In the Backblaze portal, under ‘Caps and Alerts‘ you should set..

  • Alerts to your mobile phone (for Daily storage caps,bandwidth caps and class B/C caps)
  • Under ‘Daily Storage Caps’ click ‘edit’ and set monetary alerts (you can increase alert limits later)
Added $5 and $10 daily storage alerts

Click ‘Edit Caps’ and set daily limits on all categories

Edit Caps button

I set alert limits of 10c (I can increase this later)

Daily 10c alerts should see is costs creep above $3 a month.

Screenshot of cap alerts, I set 10c warnings on everyitng

Create a cloud bucket to store files

Go to https://secure.backblaze.com/b2_buckets.htm and click ‘Create Bucket’

Screenshot of https://secure.backblaze.com/b2_buckets.htm (create bucket button)

You can create upto 100 buckets per account.

I created a descriptive bucket name that reminded me what server, drive and folder this was used to back up.

"#######-drive-backup-folder" (####### was the name of my computer)
Screenshot of bucket being created an named.

I created a number of buckets (all private).

The Backblaze portal lists all of my new storage buckets.

Screenshot of my 4 new storage buckets

Bucket Storage (Lifestyle Settings)

You can click the ‘Lifecycle Settings‘ button and set the desired buckets file retention policy (overwrite or keep all versions of a file etc).

Read the official Lifestyle settings faq here.

The default is to keep all versions of a file. Choose a setting that is required for each bucket(for me ‘Keep only the latest version of the file’ is best and lowers potential cost blowouts. Having all versions of a file may be handy for a Documents folder.

Screenshot of  Lifestyle settings at https://secure.backblaze.com/b2_buckets.htm

You could opt to keep all versions of a file (forever) but that would be expensive.

You tell Backblaze to delete older versions of a file after xx days (this is a good option)

You can also setup custom lifecycle rules to hide a file with a certain prefix after xx days then delete after xx days.

Create a Master Application Key for all buckets

You will need to create a Master Application Key that will grant you full access (Permissions: listKeys, writeKeys, deleteKeys, listBuckets, writeBuckets, deleteBuckets, listFiles, readFiles, shareFiles, writeFiles, deleteFiles ) to each bucket.

Snip from: https://secure.backblaze.com/app_keys.htm

Application keys are used as a pair: Key ID and Application Key. This allows B2 to communicate securely with different devices or apps. Once you generate your Master Application Key, this key has full capabilities. Create your own Application Keys to limit features like read/write. Learn more.

Do write down this “KeyID” and “applicationKey”

Screenshot of https://secure.backblaze.com/app_keys.htm (generated "KeyID" and "applicationKey")

Don’t lose the Master Application Key.

Create an App Key for each bucket

A master key is like an administrator account, its great for complete access but you should create a separate key per bucket for better security.

Back under the App Keys you can click “Add a New Application Key” for each bucket.

Screenshot of Add a new application key button

Read the official guide on Application Keys here

Screenshot of add application key (add name, option to choose a bucket and assign read/write acess

Do Set..

  • “Name of Key” (similar to your bucket (e.g “my-bucket-name-key”))
  • “Allow access to Bucket(s)” (choose a bucket to grant access to).
  • “Type of Access” (Read and Write, Read Only, Write Only)

Optional..

  • Set a “File name prefix” (snip from here)
"The file name prefix of files the download authorization token will allow b2_download_file_by_name to access. For example, if you have a private bucket named "photos" and generate a download authorization token for the fileNamePrefix "pets/" you will be able to use the download authorization token to access: https://f345.backblazeb2.com/file/photos/pets/kitten.jpg but not: https://f345.backblazeb2.com/file/photos/vacation.jpg."
  • Set a “Duration” (snip from here)
"The number of seconds before the authorization token will expire. The minimum value is 1 second. The maximum value is 604800 which is one week in seconds."

I now has 1x Master Application key and 4x Application Keys for my 4x Buckets

Screenshot of https://secure.backblaze.com/app_keys.htm (1 master key and 4x app keys)

TIP: Make sure you save the keys, if you did not grab the keys delete the keys and buckets (if there is not data and try again). If you don;t have the keys you are locked out of the buckets

OK, Let’s fill those buckets with data with Duplicati

Go to https://www.duplicati.com/ and familiarise yourself with the site

Setup Duplicati on Windows (10)

Go to https://www.duplicati.com/download and download the latest Windows client. At the time of writing Duplicati 2.0.4.5 beta was the latest version. Thew instal was 1MB (the download serve was not that fast)

Warning: Beta software may have bugs, feel free to check out the Github page for Issues etc.

Duplicati 2.0.4.5 download button

Open the install and Click ‘Next‘

Duplicati install wizard start, click Next

Agree to the ‘Licence Agreement‘ by clicking ‘Next‘

Duplicati licence agreement screenshot

Select Duplicati components to install and click ‘Next‘

Install an components screenshot.

Tick ‘Launch Duplicati‘ an click ‘Finish‘

Duplicati has finished installing screenshot.

Using Duplicati Software

Upon first open you will be prompted to set a password (do this even if you are the only user of thew computer)

First run screenshot

Set a strong and unique password

Screenshot of setting a duplicati password

Click ‘Ok‘

Ok Button

When the page reloads it will report it is not logged in.

Screenshot of user is not logged in error.

Enter your password

Enter password screenshot

The main screen of Duplicati reports that there are no scheduled set.

Screenshot no scheduled tasks screenshot.

OK, lets create a backup schedule.

Creating a Backup on Windows

If your screen is small you will need to click the menu at the top right and click ‘Add Backup’, if your screen is large there will be an ‘Add Backup’ menu on the left hand side of the screen.

Duplicati menu screenshot

Select ‘Configure a new backup‘ and click ‘Next‘

Configure a new backup screenshot

You will be able to name and describe the backup schedule name and be able to set up an encryption password.

Then click ‘Next‘

Screenshot of create backup schedule with name, description and encryption password

Under ‘Storage Type‘ choose ‘B2 Cloud Storage‘

When you choosing ‘B2 Cloud Storage‘ you will be directed to the screen below where you stecify..

  • B2 Cloud bucket name
  • Path in the bucket to store the files (leave blank)
  • Enter your Backblaze Account ID (e.g. ##########################)
  • Enter the Buckey ApplicationKey (e.g. ###############################)

Click ‘Next‘

Screenshot of enter B2 credentials

I entered my credentials

I entered my B2 bucket credentials

Click ‘Test Connection‘, if you entered everything OK you should see ‘Connection Worked‘

Connection worked Screenshot

Click ‘Next‘

Next button screenshot

Now you can select the folders to backup on your local computer

Duplicati, select folders to backup.

Click ‘Next‘

Next button

Now you can choose a schedule to backup

Screenshot choose time and days to run the backup.

Click ‘Next‘

Click Next button screenshot

Now you can set the remote backup size and backup retention mode.

Read the official guide on setting the volume size here

  • Increase the ‘Remote Volume Size’ if you are worried about consuming your daily API quota in Backblaze (this may happen if your upload speed is fast and your backup is very large).
  • Lower the ‘Remote Volume Size’ size if backups are small or your upload speed is low.
Screenshot of remote volume size and backup retention

Choose a desirable backup retention mode. Duplicati only offers full backups (not incremental or differential backups). This will be an issue if your backups are large.

  • Keep all backups (“Nothing will be deleted. The backup size will grow with each change.“)
  • Delete backups that are older than (“If at least one newer backup is found, all backups older than this date are deleted.“)
  • Keep a specific number of backups (“Once there are more backups than the specified number, the oldest backups are deleted.“)
  • Smart Backup Retention (“Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup.“)
  • Custom backup retention (“Enter a retention strategy manually. Placeholders are D/W/Y for days/weeks/years and U for unlimited. The syntax is: 7D:1D,4W:1W,36M:1M. This example keeps one backup for each of the next 7 days, one for each of the next 4 weeks, and one for each of the next 36 months. This can also be written as 1W:1D,1M:1W,3Y:1M.“)

Each storage option has pro’s and cons with potential cost implications. I would not set 300GB to “Keep all backups” and backup daily. That would be expensive.

Screenshot of the 5 backup retention modes

Click ‘Save‘

Save button screenshot

TIP: Don”t forget to save your encryption key if you set one

Screenshot of a duplicati warning saying have we saved our encryption ley

Now you will see the backup schedule on your Duplicati screen.

You can run, edit or delete the backup status here.

Screenshot of the backup schedule on the home screen.

I clicked ‘Run‘ and the backup was under way.

A progress bar updates as the backup happens.

Screenshot of backup underway

A slight delay happens at the end of the backup.

Screenshot of backup waiting to finish screen.

I logged into the Backblaze portal to see how many files and usage credits were used.

Note it seems?

Screenshot of backblaze portal

When the files are backed up a verification processes happens.

screenshot of verify files screen

Now the backup has completed.

Screenshot of the backup completed screen

I waited 2 minutes and finally Backblaze updated it’s status, Duplicati used 3 files and 87MB

Screenshot of B2 Cloud uploads stats

I logged into the view the Duplicati files that were uploaded and I could see the three files were there and they were encrypted.

Screenshot of B2 cloud encrypted Duplicati files.

Awesome

Restoring a Backup on Windows

Having a backup is useless you can restore the files (especially when they are encrypted).

Lets restore a few files and see if it works.

I opened Duplicati and clicked ‘Restore‘

Screenshot: Duplicati restore menu item

I clicked the bottom option that listed my bucket name/backup name and clicked ‘Next‘

Screenshot restore from the backup

Before I ran the restore I deleted some location files that were just backed up.

Screenshot of me deleting 2 files.

Duplicati now asked me to choose files to restore.

Screenshot restore files treeview with selections

I chose the original location to restore to and clicked ‘Restore‘

Screenshot restore to.

Duplicati took a few minutes to restore the files.

Screenshot: Duplicati downloading files

The restore was a success.

Screenshot restore success

Nice

Backblaze Costs?

Beware, make sure you calculate the costs to upload and download files.

Do set alarms on cap’s limits and review them often.

Screenshot of data usage and caps.

Second Backup Schedule on Windows

I quickly set a second backup schedule of a different folder to a different bucket.

This backup was 300GB so I will only backup once a month

Schedule once a month backup schedule.

The backup is listed on the front screen of Duplicati.

Do create multiple buckets and set multiple backup schedules to break up the backups into smaller chunks.

Screenshot of multiple backup schedules.

Setup Duplicati on Linux (Debian or Ubuntu)

Lets setup B2 Cloud Storage backup on Linux.

I visited the Duplicati forums here. Duplicati informed me that the Linux client needs mono installed

Duplicati depends on other software. For Windows, Microsoft .NET Framework 4.5 or higher needs to be installed. Linux and Mac OS X require Mono to be installed.

Mono is a Cross Platform open source .NET that runs on Linux (read here)

On Ubuntu 1.08 I installed mono with these commands

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu stable-bionic main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

On Debian 9.9 I installed Mono with these commands (from here)

sudo apt install apt-transport-https dirmngr gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/debian stable-stretch main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

I rebooted the serves and the CPU and memory went crazy after I installed Mono. My Nixstats Service reports were going off telling me my servers CPU was busy.

If you don’s have reports on Linux server CPU/MEM/Disk reports coming to you via Email and Telegram then read this.

Screensot of Nixstats Reports

I felt sorry for my web and database server server(s) so I rebooted them, wow, My website came back up in about 10 seconds thanks to UpClouds fast Max IOPS disks.

Installing Duplicati on Linux

I visited the download Duplicati page here and copied the Ubuntu and Debian download URL.

At the time of writing it was..

https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I downloaded the deb package

cd /utils/duplicati
wget https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I then ran this command to install the package

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

I received this error even thpogh libmono-2.0-1 was installed

dpkg: dependency problems prevent configuration of duplicati:
duplicati depends on libmono-2.0-1; however:
Package libmono-2.0-1 is not installed.

This forum post helped me, I ran

sudo apt install -f

Output

sudo apt install -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
Suggested packages:
  indicator-application
The following NEW packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
0 upgraded, 6 newly installed, 0 to remove and 7 not upgraded.
1 not fully installed or removed.
Need to get 143 kB of archives.
After this operation, 541 kB of additional disk space will be used.
Do you want to continue? [Y/n]

And I ran the following

sudo apt dist-upgrade --fix-missing

I now ran the following to install the duplicati package.

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

It worked on Ubuntu 18.08 bt not Debian 9.9

Debian produced the following error

sudo dpkg -i duplicati_2.0.4.5-1_all.deb                (Reading database ... 70082 files and directories currently installed.)
Preparing to unpack duplicati_2.0.4.5-1_all.deb ...
Unpacking duplicati (2.0.4.5-1) over (2.0.4.5-1) ...
dpkg: dependency problems prevent configuration of duplicati:
 duplicati depends on libappindicator0.1-cil | libappindicator3-0.1-cil; however:
  Package libappindicator0.1-cil is not installed.
  Package libappindicator3-0.1-cil is not installed.

dpkg: error processing package duplicati (--install):
 dependency problems - leaving unconfigured
Processing triggers for mime-support (3.60) ...
Errors were encountered while processing:
 duplicati

On Debian I installed the stretch version of the package

sudo apt-get install libappindicator1

Then in Debian I fixed broken packages

sudo apt --fix-broken install

This I was able to install Duplicati on Debian

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

Running Duplicati CLI

I can now Run Duplicati CLI (it works)

duplicati-cli

See duplicati.commandline.exe help <topic> for more information.
  General: example, changelog
  Commands: backup, find, restore, delete, compact, test, compare, purge, vacuum
  Repair: repair, affected, list-broken-files, purge-broken-files
  Debug: debug, logging, create-report, test-filters, system-info, send-mail
  Targets: aftp, amzcd, azure, b2, box, cloudfiles, dropbox, ftp, file,
  googledrive, gcs, hubic, jottacloud, mega, msgroup, onedrive, onedrivev2,
  sharepoint, openstack, rclone, s3, ssh, od4b, mssp, sia, tahoe, webdav
  Modules: aes, gpg, zip, 7z, console-password-input, mssql-options,
  hyperv-options, http-options, sendhttp, sendmail, runscript, sendxmpp,
  check-mono-ssl
  Formats: date, time, size, encryption, compression
  Advanced: mail, advanced, returncodes, filter, filter-groups, <option>

http://www.duplicati.com/              Version:  - 2.0.4.5_beta_2018-11-28

TIP: Don’t run “duplicati” that’s the desktop verison.

Creating a Duplicati Backup on Linux

I read this page and this page to get an understanding of how to back up from the command line. I was not sure if I needed to run a service or run the CLI from a bach script so I asked on the forums here.

Within minutes I had this reply

kenkendk
1h
Hi @feartec, welcome to the forum!

The CLI and Web-UI are not the same, and operate independently.

If you use the CLI, you need to use cron or something similar to run the backups periodically. You can use duplicati-cli help to get started. You do not need the server running for this.

For the web-UI we do not currently have any tools for manipulating the backups other than the main WebUI itself. You need to somehow forward/tunnel access to the server, and then access the WebUI from a local browser.

Backing up a folder with Duplicati to a Backblaze B2 Bucket

I created this folder structure

/test/
/test/testfile.txt < hello
/test/subfolder/
/test/subfolder/file.txt < world

I ran this command to back up the folder structure above to backblaze (official guide). See advanced switches.

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=**********

TIP: read more about “dblock” size here

Output

Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  4 files need to be examined (8 bytes)
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1021 bytes) ...
  Downloading file (1.06 KB) ...
  Downloading file (2.00 KB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:06
  Remote files: 3
  Remote size: 4.05 KB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 0
  Files deleted: 0
  Files changed: 0
  Data uploaded: 0 bytes
  Data downloaded: 4.05 KB
Backup completed successfully!

I logged into Backblaze to check the backups

Duplicati backup success

I created a second test file and added 2 test files

mkdir /test2
cd /test2
wget http://ipv4.download.thinkbroadband.com/10MB.zip
wget http://ipv4.download.thinkbroadband.com/5MB.zip

ls -al
total 15368
...
-rw-r--r--  1 user user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

I ran the backup with the second folder in the command line

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=********** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/07/2019 23:05:58
Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  7 files need to be examined (15.00 MB)
  Uploading file (15.03 MB) ...
  Uploading file (11.43 KB) ...
  Uploading file (1.29 KB) ...
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1.29 KB) ...
  2 files need to be examined (8 bytes)
  Downloading file (11.43 KB) ...
  Downloading file (15.03 MB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:10
  Remote files: 6
  Remote size: 15.05 MB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 2
  Files deleted: 0
  Files changed: 0
  Data uploaded: 15.05 MB
  Data downloaded: 15.05 MB

Success

Restoring a Duplicati Backup on Linux

O’k lets delete the local “/test2/” folder then restore it

rm -R /test2

Now lets restore it (official guide)

/usr/bin/duplicati-cli restore "b2://bucket-name-goes-here/test" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --passphrase=************* --overwrite=true --restore-path="/test2"

Output

Restore started at 06/07/2019 23:21:51
Checking remote backup ...
  Listing remote folder ...
Checking existing target files ...
  2 files need to be restored (15.00 MB)
Scanning local files for needed data ...
  Downloading file (15.03 MB) ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 0 (0 bytes) files to /test2
Duration of restore: 00:00:05

Did it work

cd /test2
ls -al
total 15368
...
-rw-r--r--  1 use user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

Yes it worked

Backup 2 folders an emailing the results

I use Google GSuite for email and have an app password setup just for duplicati. Here is the official guide for sending an email after a backup operation

I downloaded a 100MB file

cd /test2
wget http://ipv4.download.thinkbroadband.com/100MB.zip

I ran this command to backup an emailthe progress

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=***************** [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/08/2019 00:02:28
Checking remote backup …
Listing remote folder …
Scanning local files …
8 files need to be examined (115.00 MB)
7 files need to be examined (15.00 MB)
Uploading file (49.99 MB) …
Uploading file (18.01 KB) …
Uploading file (49.99 MB) …
Uploading file (17.97 KB) …
Uploading file (233.33 KB) …
Uploading file (33.17 KB) …
Uploading file (1.42 KB) …
Compacting remote backup …
Checking remote backup …
Listing remote folder …
Verifying remote backup …
Remote backup verification completed
Downloading file (1.42 KB) …
4 files need to be examined (15.00 MB)
Downloading file (18.01 KB) …
Downloading file (49.99 MB) …
0 files need to be examined (0 bytes)
Duration of backup: 00:00:15
Remote files: 13
Remote size: 115.33 MB
Total remote quota: 0 bytes
Available remote quota: 0 bytes
Files added: 1
Files deleted: 0
Files changed: 0
Data uploaded: 100.28 MB
Data downloaded: 50.01 MB
Backup completed successfully!

Email

Screenshot of and email from Duplicati software

Backup Types and Versions

Read up here on backup retention policies here

TIP: Use the desktop Duplicati software to generate a backup then use the export to command line option to get the command line parameters to use in the command line version.

Screenshot of the desktop version of Duplicati and the export to command line link

After you click Commandline you can view each separate command liene option and parameter.

This was handy for finding this backup retention parameter

--retention-policy="1W:1D,4W:1W,12M:1M

“This will give me “one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup. “

Setup Auto Backup Cron Scripts

Login to your linux machines and edit a bash file (e.g in “/scripts/backupnow.sh” folder).

Make the script executable

chmod +x /scripts/backupnow.sh

Add the following

#!/bin/bash

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=123 [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Now you can add the script to your crontab by running the following.

crontab -e

Make sure you have the following in your crontab

SHELL=/bin/bash

Add a cron entry to run the bash script

#Backup Database
0 22 * * * /bin/bash /scripts/backupnow.sh

Use https://crontab.guru/ to check the time you want to run thew script

https://crontab.guru/ screenshot

Links

  • B2 Starter Guides
  • Duplicati Manual

 

 

 

Version

v1.1 Fixed typo (”downalod” thanks to @NicolasWolf for reporting.)

v1.0 Initial Post (added retention polity and Linux crontabs

v0.95 Backup, Restore and Send Email

v0.91 Added Duplicati service info

v0.9 Working Copy

Filed Under: Uncategorized Tagged With: B2, backblaze, Backup, cli, cloud, duplicati, Linux, windows

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) IoT (9) LetsEncrypt (7) Linux (21) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) Performance (6) PHP (13) Scalability (12) Scalable (14) Security (45) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (45) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT