• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

Backup

Using Acronis True Image 2020 to backup and restore your Windows computer

May 24, 2020 by Simon

This is a lengthy (but simple) guide on how I get files ready for a clean install of Windows, backup and restore Windows (C Drive and associated hidden Windows partitions) using Acronis True Image 2020).

Backup and Restore Windows Prerequisites

  • You will need to buy a licence of Acronis True Image 2020 to..
    • Backup Windows
    • Restore Windows
  • You will need a second hard disk larger than your Windows drive (to backup to).

Backup Rules

  1. Backup to Three Places.
  2. Two of the media need to be different
  3. One of the backups need to be offsite

Even though I use Back blaze software to automate backup of all files on my PC for $6.60 (AUD) a month. I pay an extra $2 (AUD) a month Backblaze will retain all of my files for up to 1 year. I also back up data to Backblaze Bucket(s) with Duplicati.

With all the horror stories of Windows updates killing machine’s it’s time to return to using Acronis True Image (not a paid endorsement) to start backing up the drive my Windows is on (all partitions)

I remember using Acronis True Image 8 in 2004 and loved it. Time to buy the new version. I loaded https://acronis.com/ (Australia Region) and noticed a cloud backup in the marketing information. I hope they still do local backups line in 2004.

Acronis True Image 2020 Cloud

I loaded the Acronis True Image 2020 link from the downtown menu (Australian link).

Acronis True Image 2020 Download Menu at acronis dot com

$69.99 seems fair,

Acronis True Image 2020 buy now button at acronis dot com

Darn, $69.99 is a yearly subscription. I don’t like subscriptions if I can get away with it.

Acronis True Image 2020 subscription and standalone options  at acronis dot com

I clicked Buy under standard licence.

Acronis True Image 2020 was $83.99 AUD at acronis dot com

$83.99 is a bit high

Saving a dollar

TIP: Make sure you install the Honey Extension (from joinhoney.com) and use the Google Chrome Web Browser before purchasing to get a discount on Acronis True Image 2020.

Honey applied a discount price

The Honey extension applied the coupon so fast I could not get a screenshot  it(so here is a screenshot with American Currency from a different web browser)

Honey applying a coupon in seconds

Now back to my purchase (and country), $33.60 is a nice saving.  

I am paying for this because 16+ years ago I used an older version of Acronis True Image and the free version backed up OK but when I tried to restore from a boot CD I needed to pay?

Honey saved me $33.80

Honey browser plugin saved me $33.6 AUD

I am happy to pay $50.33 for Acronis True Image 2020.

After I purchased Acronis True Image 2020 I was presented with links to download the related files to use the software and to restore.

Download options after I purchased

I downloaded the main Windows Acronis True Image Application for Windows and the Universal Restore program.

The files were about 900MB in total.

I had to download 900MB

While the downloads were downloading I created an Acronis account 

I added my purchased licence to my Account.

I created an account at acronis dot com

I was shown a welcome to Acronis page after creating an account.

Welcome to Acronis page

The downloads finished and I started the AcronisTrueImage2020.exe setup file

Acronis Setup

I clicked Install

Installing Acronis

After the setup completed I was directed to a Quick Start Guide

Quick Start Guide after setup

Starting Acronis True Image 2020

When I loaded Acronis True Image 2020 for the fits time I had to agree to the Licence Agreement

Licence Agreement

A pasted in my just purchased Licence key

Start a trial or activate with a serial number

I already had an account (created minutes earlier in a web browser)  so I clicked Sign In.

Create an account or sign in

Sign In Screen

Sign In Screen

I was presented with a help wizard on startup.

Wizard Help

Now I am ready to create my first backup.

Creating a Backup

Acronis True Image 2020 wanted me to back up all of my PC but it is 8TB, I don’t have an 8TB drive handy to back up to. For the life of Me I could not see how much 8TB would cost per month to backup.

By default Acronis wants to backup your whole PC

I clicked on my computer name (“ZENIGMA”) and selected “Disks and partitions“

I un-ticked all system drives and partitions that were not my C Drive disk. I have installed Windows 10 onto a Corsair MP600 M.2 PCIE 1TB SSD and I want to back it up. The M.2 SSD connects to my main board.

Corsair MP600 SSD, Credit Corsair.

I selected this drive to backup.

View of all disks and partitions, I ticked the MP600 drive.

Oh dang, I don’t have enough space free on any drive to backup the C Drive to., Time to move some files.

Backup Destination

I have 8 partitions (drive letters) on 5 SATA hard rives and 2 M.2 SSD’s. I think I can move some data and free some space for this backup.

I had to shuffle data across drives to create enough space to backup my c drive

File Musical Chair’s

I used WinDirStat from https://windirstat.net/ to scan my U drive (my newest hard drive) for files to move.

I used WInDIRStat to find files I can move

WinDirStat shows a nice graph of files on my U Drive. WinDirStat colour codes file types by colour and also sizes squared by actual file size.

Windirstats scan results, Loads o Music I can ,move.

I could see 240GB of music that could be moved. My wife and I have converted all CD’s that we have purchased since the 1990’s to live alongside out Apple iTunes collection.

250GB of Music needs moving

I used the Windows feature to right click on the Music folder and move it  the folder to another drive.

Move the Music folder in Windows

Windows confirmed hat I wanted to move this folder.

Move Confirm Dialog

It took 2 hours to move my Music folder to a smaller slower drive.

btw Pink Floyd are Awesome

Now that I have 800GB free I can backup up my C:\ Disk

I re opened Acronis True image and resumed from Backup point where I left off and made sure I was backing up my C Drive (Corsair MP600 SSD)

My C drive is  500gb and the destination has 888GB free

Acronis showed the Windows drive to web backed up.

I clicked “Select destination“.

Backup Dialog

I clicked “Browse“

Backup Destination

I selected “U:\AcronisBackup” as the target to backup C drive to

Choose a backup destination folder

I was prompted for a encryption password (nice)

Enter an Backup Encryption Password

Now the backup is ready to start.

Backup Now

I clicked Back up now

Backup in Progress

The Backup was going to take about 40 minutes.   I could see My C and U drive were busy in Task Manager

Backup in Progress

After the backup is complete Acronis verifies the Backup

Verify Backup Image

After the backup finished I could see that Acronis managed to backup C drive (570GB) to a 329GB file on a different drive

Backup complete

Now I need to restore the backup to confirm it works.

Only a working backup is of value.

Creating Rescue (Recovery) DVD Media?

Now that I have a backup of Windows I need to create a boot media that will allow me to restore the backup image in case Windows dies. 

I clicked in the 4th icon from the top in Acronis True Image 2020 then Rescue Media Builder

Rescue Media Builder

I clicked Simple

Simple or Advanced

First I created a DVD boot disc.

I selected my DVD Dive (it had a blank DVD in it)

The Rescue Media needs a 800MB CD or DVD.

Ready to burn the DVD

I clicked Proceed to burn the rescue media to the blank DVD

DVD Burning.

The Rescue Media DVD was burnt to a blank DVD, a few tips on using the Media were presented

Rescue Media complete, 3 steps to use.

I labelled the DVD Rescue Media and put it in a safe place.

Creating Rescue (Recovery) USB Media?

Now I will create a USB rescue Media Key.

I opened Acronis, Click the Tools Icon (same as the DVD method above), then Rescue Media Builder, click Simple and select your Empty USB key.

TIP: Make sure it is the correct Key as they key will be deleted first.

USB Key Selected

As with the DVD method above the Rescue Media files to be copied were about 800MB.

Write to USB Key.

When the UBS key is prepared you will receive the same success screen as the DVD method above.

Restoring a backup from the DVD Rescue Media

I inserted the Rescue DVD into my DVD drive, I rebooted my PC and pressed F2 to enter my BIOS (your key may be different).

I navigated to the section in my BIOS where I could choose my Boot device and made the DVD drive boot first, this will allow my Rescue DVD to boot before Windows.

BIOS Screen DVD Boos before C drive

I rebooted my PC and I pressed a key when prompted to boot from the Rescue DVD

The DVD buzzed away for a minute and then I was prompted with Back Up or Recover menu. I clicked U:\Blog\Acronis True Image\DVD Restore.

Back Up or Recover screenshot

I navigated to the U Drive (SFEARABY) Drive and selected the Backup I had taken earlier.

FYI: The Drive letters did not match in Windows and the Rescue Media. Take note of tour drive names. My U:\ Was listed as E:\.

I entered the password that I used to encrypt the backup.

I selected the backup of my C Drive to restore. I assume you can choose multiple, drives if you had backed up multiple drives.

I selected Recover whole disk and partitions.

Recover whole disk or single partition.

I was asked to select the date to backup from (I assume this is for when you have multiple backup events over many days).

I was again asked to select partitions to recover. 

Select Partitions.

A summary of the restore was presented and I clicked Proceed.

The Recovery was underway, It took about 8 minutes to restore over 500GB.

Recovery in progress.

It worked.

Recovery worked.

I restarted my PC and removed the Rescue DVD.

Windows was all restored to it’s earlier state.

Restoring a backup from the USB Rescue Media

Restoring is exactly the same as the DVD method.

  • Reboot and Enter your BIOS
  • Set the Boos Order to allow the USB key to boot first
  • Reboot (no need to press a key to boot from USB)
  • Backup and restore (as needed)

I was able to restore the backup no problems,

FYI: I had the same slowness in the UI in the same spots as the DVD, nothing that breaks the process.

The USB and DVD Recovery offers a backup and restore options.

USB Recovery

The Recovery media also allows you to clone a disk and setup a secure hidden partition for backups.

Other OPtions

Backup Protection

Acronis True Image 2020 also protects backup locations and prevents access to them.  I had to Allow Windows to be able to delete a backup.

Once I allowed Windows to be able to access the backup I was able to delete it.

Acronis allowing windows to manage backups files on Windows

Backup Schedules

Acronis True Image 2020 allows you to set Backup Schedules.

Backup Schedules

I do not have enough disk space to setup anything other than one full backup at  a time 

Single file backup

Backblaze

I still have Backblaze automatically backing up my files from within windows and this is my 2nd later of defence.

Backblaze will backup the 300+ GB backup.

300GB backup file/

Other Features

Active Protection

Acronis True Image 2020 has built in Active Protection to watch processes for ransomware or cryptographic. Read more here: https://www.acronis.com/en-au/active-protection/

I have had one false positive upgrading Java but ill leave it enabled.

Active e protection status screen

A nice log is available of all events and whitelisted apps.

Active protection log/

I have had to exclude apps but this has settles down.

Whitelist app

A notification appears on blocked or whitelisted events 🙂

possible ransomware detected.

I can simply block or allow to deal with each exception.

possible rancomware history

Thus is a nice feature.

Cloud Backup

I am now using any Cloud backup features in Acronis, ill let Back blaze handle that.

Archive large or old files

The Archive large files or old files is a bit basic for me to use.

Archive old or large files screen.

I will keep using CCCleaner and WinDirStats for this job.

Alternative disk clean up tools

Even the Windows default Disk Cleanup is good.

Windows disk cleanup

Cloud Sync

I do not use Cloud Sync, I would but I can’t find a price on the Acronis site at all.

I noticed that on the DVD and USB recovery boot I can restore a windows from a Acronis cloud backup (nice).

Other Tools

  • Clone disk (like EASE US?)
  • Universal restore?
  • Try and Decide
  • System Clean Up (backup First)
  • Acronis Secure Zone
  • Drive Cleanser
  • Third Party Tools

Acronis True Image (Conclusion)

I am being picky here as I loved using Acronis 10 years ago and I have rose coloured glasses.

Pros

  • It works
  • You can backup Windows while Windows is open
  • DVD and USB Rescue Media creation is easy
  • Backup Schedules
  • Notifications of disk space and operational state
  • Exclusions of non essential files
  • Backup Validation (at time of creation and schedule)
  • Ability to call pre and post backup commands
  • Choice of Incremental, Full or differential backups schemes
  • Cleanup of older incremental versions (and full version frequency)
  • Simple single version backup (single file, overwrites each time)
  • It works.

Cons

  • Some Buttons in the GUI are slow to respond
  • The DVD an USB Rescue Media is little unresponsive at times during the restore (but it works)
  • The DVD and USB Rescue Media GUI can use a bit of design love (I am sure it was better 10 years ago (basic and clean))
  • When I restored I had no Idea what C:\ Drive was listed as H:\ (loads of confusion on the internet)

Disclaimer

I am not paid by Acronis to say this, this is just my guide for my friends.  I paid with my own cash.  I take no responsibility if this does not work for you.  Good luck though.

Troubleshooting

DVD Rescue Media

The DVD/USB Rescue Media is a bit laggy. At one point in the recovery I has a white screen for over a minute. I waited and the screen came good

 

Laggy Screen

Also the Menu in the DVD/USB toolbar does seem a bit squashed.

Squashed Screen

Bonus: Windows System Restore

There may be a time when you have created a full Operating system backup but the restore does not work (e.g hardware has failed (and been changed)). Acronis True Image 2020 will allow you to restore files from a backup image (and not the whole partition or disk). 

Create System Restore

Make sure System Protection is enabled for your C Drive

System Protection On C Drive

Create a System Restore Image

If you have Windows 10 it is a good idea to create a System Restore point ever few months just in-case Windows goes bad.

Click Your Start Button then type “Create a restore point“

Windows 10 has a System Restore feature

Click your C Drive and click “Create“

Create restore point

The restore point will be created

Creating restore point.

Restoring a restore point

At any time you feel that Windows is sick (and still working) you can restore and old system restore.

Restore a restore point screenshot

Click the the desired restore point then click Next

All restore points list

You will be able to restore the old system snapshot.

Bonus: Clean Windows 10 Install

TBA: Blog post coming soon

Bonus: How I prepare all files needed for a clean install

Before you do a clean install of Windows 10 you may need to document what software you have installed so you have a change to reinstall them after you reinstall Windows.

This is possibly the most boring job but the one with the most reward. The hardest party is knowing what software you have an need.

Finding Installed Software with SUMo

I use a free program called SUMo from KC Software (not a plug) to list all installed programs and to see what software is out of date.

SUMo App Screenshot

SUMo informed me that I have 256 software products installed.

  • 7 were really out of date
  • 62 products had recent updates

I took this list and downloaded and installed new copies of the apps I had installed (as newer version’s of apps can be more secure) and I also saved the installer files to a folder away from C Drive (e.g B:\Installs).

I am a bit OSD and I have created 10 folders under the B:\Installs folder that contain installs from Essential Drivers to non essential apps that I can use in the event of a clean install of Windows 

In the Installs folder I have these sub folders

\100 Drivers\
\200 Essential Apps\
\300 Productivity Apps\
\400 Development Apps\
\500 Utils\
\600 Games\
\700 Virtual Machine OS Installs\
\800 Virtual Machine Apps\
\900 Learning\

When reinstalling Windows I start by installing all apps in each group (starting with “100 Drivers”), The larger the parent folders’ number is the less important the files are int hat folder.

I have a Legend.txt with a summary of the contents of each folder

Install Folders Legend

100+ ~ 199 Drivers
     - Mainboard ASUS RUF X570 (WIFI)
     - AMD Chipset Drivers
     - Nvidia Video Card
     - Etc
201+ ~ 299 Essential Apps
     - Printer Drives
     - 1Pasword
     - Antivrus
     - Backblaze
     - VPN
     - Acronis
     - Etc
300+ ~ 399  Productivity Apps
     - Microsoft Office 
     - Microsoft Visio
     - Etc
400+ ~ 499  Development Apps
     - Visual Studio
     - Arduino IDE
     - Fritrzing
     - Microsoft SQL Developer
     - MySQL
     - Python
     - PHP Storm
     - Etc
500+ ~ 599 Utils
     - CPU-Z
     - Core Temp
     - Acrbat Reader
     - Etc
600+ ~ 699 Games
     - Steam Installer
     - Origin Installer
     - Etc
700+ ~ 799 Virtual Machine OS Installs
     - Windows 3.11
     - Windows 95
	 - Etc
800+ ~ 899 Virtual Machine Apps
     - Office 95
     - Office XP
     - Etc
900+ ~ 909 Learning
     - Python Books
     - Etc

You can create whatever numbering scheme you want.

I also have a cached folder of my Development Software (Visual Studio) Install cache at “B:\Installs\vscache“

All of my install folders.

Under my drivers folder I have all the drivers I need to reinstall windows

All the drivers for my system.

An under my “500 Utils” folder is all the utilities I install (many).

Hundreds of sub folders with utilities in them

The main thing is I have all the files and installers needed to do a clean setup of Windows 10 if need be.

Links

  • Acronis Personal Backup
  • Welcome to Acronis True Image 2020

 

Version History

v1.1 Added “Backup and Restore Windows Prerequisites”

v1.0 Initial

Filed Under: Backblaze, Backup Tagged With: acronis, Backup, crash, restore, windows

Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

June 6, 2019 by Simon

This guide will show how you can automatically backup files to a Backblaze B2 clouds bucket with the Duplicati software on Windows and Linux.

I have blogged about how you can use the Backblaze personal backup program ( here, Use my link and get your first month of Backblaze backups free). The personal application from Backblaze is cool but what if you wanted to store 10GB for free (the first 10GB is free) or setup different buckets for different computers or applications.

Welcome to Backblaze B2 Cloud Storage.

In Australia B2 cloud costs (+GST).

  • First 10GB is free.
  • $0.005c per GB stored per month.
  • To download, 1x GB costs $0.01c.

Check our my other related posts

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways

This works alongside the private Backblaze backup mentioned here.

Screenshot: Duplicati Downalods page https://www.duplicati.com/download

Backblaze B2 Pricing

How does Backblaze B2 Compare that to other cloud data storage provides.

Screenshot of https://www.backblaze.com/b2/cloud-storage.html

Comparison chart.

ProviderStorage ($/GB/Month)Download ($/GB)
Backblaze$0.005$0.01
Amazon S3$0.021$0.05+
Microsoft Azure$0.018+$0.05+
Google cloud$0.020$0.08

Vendor cost breakdown (source).

Read here to view the official page on pricing.

Snip:

Storage:

The first 10 GB of total storage is free, and is $0.005 per GB per month for additional storage beyond 10 GB.   

Download:

The first 1 GB of downloads per day are free.  The price to download stored data costs $0.01 per GB beyond the free daily 1 GB. 

Transactions:

A 'transaction' refers to any time an api call is made with your account, such as listing the contents of your bucket or downloading a file.   B2 will charge for some transactions however, for the average user, these charges are largely negligible unless you are making an extremely high volume of api calls.

Class A transactions are free.

The first 2,500 Class B transactions are free each day. Additional Class B API calls are charged at $0.004 per 10,000 calls.  

The first 2,500 Class C transactions are free each day. Additional Class C API calls are charged is $0.004 per 1,000 calls.

Vendor Relability

https://downdetector.com is  a great site for viewing is  a vendor is down or has issues, lets compare, Backblaze with Amazon, Azure and Google Cloud

  • Downdetector.com: Backblaze
  • Downdetector.com: Azure
  • Downdetector.com: Amazon
  • Downdetector.com: Google Cloud

Backblaze have has far fewer downtime issues during the great outage of May/June 2019 compared to its competitors. What I like is Backblaze tell us how they do it and tell us what hard drives are good or bad.

Creating a Backblaze B2 Account

Login/SignUp to backblaze at https://secure.backblaze.com/user_signin.htm

You can have a sneak peek at the personal Backblaze backup signup steps in this guide (why not take up a free 1 month back free by clicking this link when you create an account).

From your account page click billing and add your account details

Screenshot of https://secure.backblaze.com/billing.htm?billing_page=b2

Read the B2 Strater Guide: https://www.backblaze.com/b2/docs/

Also review your review the following under ‘My Settings‘

  • Phone Numer (mobile) for usage alerts.
  • Two Factor Authorization for extra security (don’t forget to çopy your backup codes)

Setup Billing Alerts (just in case)

In the Backblaze portal, under ‘Caps and Alerts‘ you should set..

  • Alerts to your mobile phone (for Daily storage caps,bandwidth caps and class B/C caps)
  • Under ‘Daily Storage Caps’ click ‘edit’ and set monetary alerts (you can increase alert limits later)
Added $5 and $10 daily storage alerts

Click ‘Edit Caps’ and set daily limits on all categories

Edit Caps button

I set alert limits of 10c (I can increase this later)

Daily 10c alerts should see is costs creep above $3 a month.

Screenshot of cap alerts, I set 10c warnings on everyitng

Create a cloud bucket to store files

Go to https://secure.backblaze.com/b2_buckets.htm and click ‘Create Bucket’

Screenshot of https://secure.backblaze.com/b2_buckets.htm (create bucket button)

You can create upto 100 buckets per account.

I created a descriptive bucket name that reminded me what server, drive and folder this was used to back up.

"#######-drive-backup-folder" (####### was the name of my computer)
Screenshot of bucket being created an named.

I created a number of buckets (all private).

The Backblaze portal lists all of my new storage buckets.

Screenshot of my 4 new storage buckets

Bucket Storage (Lifestyle Settings)

You can click the ‘Lifecycle Settings‘ button and set the desired buckets file retention policy (overwrite or keep all versions of a file etc).

Read the official Lifestyle settings faq here.

The default is to keep all versions of a file. Choose a setting that is required for each bucket(for me ‘Keep only the latest version of the file’ is best and lowers potential cost blowouts. Having all versions of a file may be handy for a Documents folder.

Screenshot of  Lifestyle settings at https://secure.backblaze.com/b2_buckets.htm

You could opt to keep all versions of a file (forever) but that would be expensive.

You tell Backblaze to delete older versions of a file after xx days (this is a good option)

You can also setup custom lifecycle rules to hide a file with a certain prefix after xx days then delete after xx days.

Create a Master Application Key for all buckets

You will need to create a Master Application Key that will grant you full access (Permissions: listKeys, writeKeys, deleteKeys, listBuckets, writeBuckets, deleteBuckets, listFiles, readFiles, shareFiles, writeFiles, deleteFiles ) to each bucket.

Snip from: https://secure.backblaze.com/app_keys.htm

Application keys are used as a pair: Key ID and Application Key. This allows B2 to communicate securely with different devices or apps. Once you generate your Master Application Key, this key has full capabilities. Create your own Application Keys to limit features like read/write. Learn more.

Do write down this “KeyID” and “applicationKey”

Screenshot of https://secure.backblaze.com/app_keys.htm (generated "KeyID" and "applicationKey")

Don’t lose the Master Application Key.

Create an App Key for each bucket

A master key is like an administrator account, its great for complete access but you should create a separate key per bucket for better security.

Back under the App Keys you can click “Add a New Application Key” for each bucket.

Screenshot of Add a new application key button

Read the official guide on Application Keys here

Screenshot of add application key (add name, option to choose a bucket and assign read/write acess

Do Set..

  • “Name of Key” (similar to your bucket (e.g “my-bucket-name-key”))
  • “Allow access to Bucket(s)” (choose a bucket to grant access to).
  • “Type of Access” (Read and Write, Read Only, Write Only)

Optional..

  • Set a “File name prefix” (snip from here)
"The file name prefix of files the download authorization token will allow b2_download_file_by_name to access. For example, if you have a private bucket named "photos" and generate a download authorization token for the fileNamePrefix "pets/" you will be able to use the download authorization token to access: https://f345.backblazeb2.com/file/photos/pets/kitten.jpg but not: https://f345.backblazeb2.com/file/photos/vacation.jpg."
  • Set a “Duration” (snip from here)
"The number of seconds before the authorization token will expire. The minimum value is 1 second. The maximum value is 604800 which is one week in seconds."

I now has 1x Master Application key and 4x Application Keys for my 4x Buckets

Screenshot of https://secure.backblaze.com/app_keys.htm (1 master key and 4x app keys)

TIP: Make sure you save the keys, if you did not grab the keys delete the keys and buckets (if there is not data and try again). If you don;t have the keys you are locked out of the buckets

OK, Let’s fill those buckets with data with Duplicati

Go to https://www.duplicati.com/ and familiarise yourself with the site

Setup Duplicati on Windows (10)

Go to https://www.duplicati.com/download and download the latest Windows client. At the time of writing Duplicati 2.0.4.5 beta was the latest version. Thew instal was 1MB (the download serve was not that fast)

Warning: Beta software may have bugs, feel free to check out the Github page for Issues etc.

Duplicati 2.0.4.5 download button

Open the install and Click ‘Next‘

Duplicati install wizard start, click Next

Agree to the ‘Licence Agreement‘ by clicking ‘Next‘

Duplicati licence agreement screenshot

Select Duplicati components to install and click ‘Next‘

Install an components screenshot.

Tick ‘Launch Duplicati‘ an click ‘Finish‘

Duplicati has finished installing screenshot.

Using Duplicati Software

Upon first open you will be prompted to set a password (do this even if you are the only user of thew computer)

First run screenshot

Set a strong and unique password

Screenshot of setting a duplicati password

Click ‘Ok‘

Ok Button

When the page reloads it will report it is not logged in.

Screenshot of user is not logged in error.

Enter your password

Enter password screenshot

The main screen of Duplicati reports that there are no scheduled set.

Screenshot no scheduled tasks screenshot.

OK, lets create a backup schedule.

Creating a Backup on Windows

If your screen is small you will need to click the menu at the top right and click ‘Add Backup’, if your screen is large there will be an ‘Add Backup’ menu on the left hand side of the screen.

Duplicati menu screenshot

Select ‘Configure a new backup‘ and click ‘Next‘

Configure a new backup screenshot

You will be able to name and describe the backup schedule name and be able to set up an encryption password.

Then click ‘Next‘

Screenshot of create backup schedule with name, description and encryption password

Under ‘Storage Type‘ choose ‘B2 Cloud Storage‘

When you choosing ‘B2 Cloud Storage‘ you will be directed to the screen below where you stecify..

  • B2 Cloud bucket name
  • Path in the bucket to store the files (leave blank)
  • Enter your Backblaze Account ID (e.g. ##########################)
  • Enter the Buckey ApplicationKey (e.g. ###############################)

Click ‘Next‘

Screenshot of enter B2 credentials

I entered my credentials

I entered my B2 bucket credentials

Click ‘Test Connection‘, if you entered everything OK you should see ‘Connection Worked‘

Connection worked Screenshot

Click ‘Next‘

Next button screenshot

Now you can select the folders to backup on your local computer

Duplicati, select folders to backup.

Click ‘Next‘

Next button

Now you can choose a schedule to backup

Screenshot choose time and days to run the backup.

Click ‘Next‘

Click Next button screenshot

Now you can set the remote backup size and backup retention mode.

Read the official guide on setting the volume size here

  • Increase the ‘Remote Volume Size’ if you are worried about consuming your daily API quota in Backblaze (this may happen if your upload speed is fast and your backup is very large).
  • Lower the ‘Remote Volume Size’ size if backups are small or your upload speed is low.
Screenshot of remote volume size and backup retention

Choose a desirable backup retention mode. Duplicati only offers full backups (not incremental or differential backups). This will be an issue if your backups are large.

  • Keep all backups (“Nothing will be deleted. The backup size will grow with each change.“)
  • Delete backups that are older than (“If at least one newer backup is found, all backups older than this date are deleted.“)
  • Keep a specific number of backups (“Once there are more backups than the specified number, the oldest backups are deleted.“)
  • Smart Backup Retention (“Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup.“)
  • Custom backup retention (“Enter a retention strategy manually. Placeholders are D/W/Y for days/weeks/years and U for unlimited. The syntax is: 7D:1D,4W:1W,36M:1M. This example keeps one backup for each of the next 7 days, one for each of the next 4 weeks, and one for each of the next 36 months. This can also be written as 1W:1D,1M:1W,3Y:1M.“)

Each storage option has pro’s and cons with potential cost implications. I would not set 300GB to “Keep all backups” and backup daily. That would be expensive.

Screenshot of the 5 backup retention modes

Click ‘Save‘

Save button screenshot

TIP: Don”t forget to save your encryption key if you set one

Screenshot of a duplicati warning saying have we saved our encryption ley

Now you will see the backup schedule on your Duplicati screen.

You can run, edit or delete the backup status here.

Screenshot of the backup schedule on the home screen.

I clicked ‘Run‘ and the backup was under way.

A progress bar updates as the backup happens.

Screenshot of backup underway

A slight delay happens at the end of the backup.

Screenshot of backup waiting to finish screen.

I logged into the Backblaze portal to see how many files and usage credits were used.

Note it seems?

Screenshot of backblaze portal

When the files are backed up a verification processes happens.

screenshot of verify files screen

Now the backup has completed.

Screenshot of the backup completed screen

I waited 2 minutes and finally Backblaze updated it’s status, Duplicati used 3 files and 87MB

Screenshot of B2 Cloud uploads stats

I logged into the view the Duplicati files that were uploaded and I could see the three files were there and they were encrypted.

Screenshot of B2 cloud encrypted Duplicati files.

Awesome

Restoring a Backup on Windows

Having a backup is useless you can restore the files (especially when they are encrypted).

Lets restore a few files and see if it works.

I opened Duplicati and clicked ‘Restore‘

Screenshot: Duplicati restore menu item

I clicked the bottom option that listed my bucket name/backup name and clicked ‘Next‘

Screenshot restore from the backup

Before I ran the restore I deleted some location files that were just backed up.

Screenshot of me deleting 2 files.

Duplicati now asked me to choose files to restore.

Screenshot restore files treeview with selections

I chose the original location to restore to and clicked ‘Restore‘

Screenshot restore to.

Duplicati took a few minutes to restore the files.

Screenshot: Duplicati downloading files

The restore was a success.

Screenshot restore success

Nice

Backblaze Costs?

Beware, make sure you calculate the costs to upload and download files.

Do set alarms on cap’s limits and review them often.

Screenshot of data usage and caps.

Second Backup Schedule on Windows

I quickly set a second backup schedule of a different folder to a different bucket.

This backup was 300GB so I will only backup once a month

Schedule once a month backup schedule.

The backup is listed on the front screen of Duplicati.

Do create multiple buckets and set multiple backup schedules to break up the backups into smaller chunks.

Screenshot of multiple backup schedules.

Setup Duplicati on Linux (Debian or Ubuntu)

Lets setup B2 Cloud Storage backup on Linux.

I visited the Duplicati forums here. Duplicati informed me that the Linux client needs mono installed

Duplicati depends on other software. For Windows, Microsoft .NET Framework 4.5 or higher needs to be installed. Linux and Mac OS X require Mono to be installed.

Mono is a Cross Platform open source .NET that runs on Linux (read here)

On Ubuntu 1.08 I installed mono with these commands

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu stable-bionic main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

On Debian 9.9 I installed Mono with these commands (from here)

sudo apt install apt-transport-https dirmngr gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/debian stable-stretch main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

I rebooted the serves and the CPU and memory went crazy after I installed Mono. My Nixstats Service reports were going off telling me my servers CPU was busy.

If you don’s have reports on Linux server CPU/MEM/Disk reports coming to you via Email and Telegram then read this.

Screensot of Nixstats Reports

I felt sorry for my web and database server server(s) so I rebooted them, wow, My website came back up in about 10 seconds thanks to UpClouds fast Max IOPS disks.

Installing Duplicati on Linux

I visited the download Duplicati page here and copied the Ubuntu and Debian download URL.

At the time of writing it was..

https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I downloaded the deb package

cd /utils/duplicati
wget https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I then ran this command to install the package

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

I received this error even thpogh libmono-2.0-1 was installed

dpkg: dependency problems prevent configuration of duplicati:
duplicati depends on libmono-2.0-1; however:
Package libmono-2.0-1 is not installed.

This forum post helped me, I ran

sudo apt install -f

Output

sudo apt install -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
Suggested packages:
  indicator-application
The following NEW packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
0 upgraded, 6 newly installed, 0 to remove and 7 not upgraded.
1 not fully installed or removed.
Need to get 143 kB of archives.
After this operation, 541 kB of additional disk space will be used.
Do you want to continue? [Y/n]

And I ran the following

sudo apt dist-upgrade --fix-missing

I now ran the following to install the duplicati package.

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

It worked on Ubuntu 18.08 bt not Debian 9.9

Debian produced the following error

sudo dpkg -i duplicati_2.0.4.5-1_all.deb                (Reading database ... 70082 files and directories currently installed.)
Preparing to unpack duplicati_2.0.4.5-1_all.deb ...
Unpacking duplicati (2.0.4.5-1) over (2.0.4.5-1) ...
dpkg: dependency problems prevent configuration of duplicati:
 duplicati depends on libappindicator0.1-cil | libappindicator3-0.1-cil; however:
  Package libappindicator0.1-cil is not installed.
  Package libappindicator3-0.1-cil is not installed.

dpkg: error processing package duplicati (--install):
 dependency problems - leaving unconfigured
Processing triggers for mime-support (3.60) ...
Errors were encountered while processing:
 duplicati

On Debian I installed the stretch version of the package

sudo apt-get install libappindicator1

Then in Debian I fixed broken packages

sudo apt --fix-broken install

This I was able to install Duplicati on Debian

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

Running Duplicati CLI

I can now Run Duplicati CLI (it works)

duplicati-cli

See duplicati.commandline.exe help <topic> for more information.
  General: example, changelog
  Commands: backup, find, restore, delete, compact, test, compare, purge, vacuum
  Repair: repair, affected, list-broken-files, purge-broken-files
  Debug: debug, logging, create-report, test-filters, system-info, send-mail
  Targets: aftp, amzcd, azure, b2, box, cloudfiles, dropbox, ftp, file,
  googledrive, gcs, hubic, jottacloud, mega, msgroup, onedrive, onedrivev2,
  sharepoint, openstack, rclone, s3, ssh, od4b, mssp, sia, tahoe, webdav
  Modules: aes, gpg, zip, 7z, console-password-input, mssql-options,
  hyperv-options, http-options, sendhttp, sendmail, runscript, sendxmpp,
  check-mono-ssl
  Formats: date, time, size, encryption, compression
  Advanced: mail, advanced, returncodes, filter, filter-groups, <option>

http://www.duplicati.com/              Version:  - 2.0.4.5_beta_2018-11-28

TIP: Don’t run “duplicati” that’s the desktop verison.

Creating a Duplicati Backup on Linux

I read this page and this page to get an understanding of how to back up from the command line. I was not sure if I needed to run a service or run the CLI from a bach script so I asked on the forums here.

Within minutes I had this reply

kenkendk
1h
Hi @feartec, welcome to the forum!

The CLI and Web-UI are not the same, and operate independently.

If you use the CLI, you need to use cron or something similar to run the backups periodically. You can use duplicati-cli help to get started. You do not need the server running for this.

For the web-UI we do not currently have any tools for manipulating the backups other than the main WebUI itself. You need to somehow forward/tunnel access to the server, and then access the WebUI from a local browser.

Backing up a folder with Duplicati to a Backblaze B2 Bucket

I created this folder structure

/test/
/test/testfile.txt < hello
/test/subfolder/
/test/subfolder/file.txt < world

I ran this command to back up the folder structure above to backblaze (official guide). See advanced switches.

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=**********

TIP: read more about “dblock” size here

Output

Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  4 files need to be examined (8 bytes)
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1021 bytes) ...
  Downloading file (1.06 KB) ...
  Downloading file (2.00 KB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:06
  Remote files: 3
  Remote size: 4.05 KB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 0
  Files deleted: 0
  Files changed: 0
  Data uploaded: 0 bytes
  Data downloaded: 4.05 KB
Backup completed successfully!

I logged into Backblaze to check the backups

Duplicati backup success

I created a second test file and added 2 test files

mkdir /test2
cd /test2
wget http://ipv4.download.thinkbroadband.com/10MB.zip
wget http://ipv4.download.thinkbroadband.com/5MB.zip

ls -al
total 15368
...
-rw-r--r--  1 user user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

I ran the backup with the second folder in the command line

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=********** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/07/2019 23:05:58
Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  7 files need to be examined (15.00 MB)
  Uploading file (15.03 MB) ...
  Uploading file (11.43 KB) ...
  Uploading file (1.29 KB) ...
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1.29 KB) ...
  2 files need to be examined (8 bytes)
  Downloading file (11.43 KB) ...
  Downloading file (15.03 MB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:10
  Remote files: 6
  Remote size: 15.05 MB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 2
  Files deleted: 0
  Files changed: 0
  Data uploaded: 15.05 MB
  Data downloaded: 15.05 MB

Success

Restoring a Duplicati Backup on Linux

O’k lets delete the local “/test2/” folder then restore it

rm -R /test2

Now lets restore it (official guide)

/usr/bin/duplicati-cli restore "b2://bucket-name-goes-here/test" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --passphrase=************* --overwrite=true --restore-path="/test2"

Output

Restore started at 06/07/2019 23:21:51
Checking remote backup ...
  Listing remote folder ...
Checking existing target files ...
  2 files need to be restored (15.00 MB)
Scanning local files for needed data ...
  Downloading file (15.03 MB) ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 0 (0 bytes) files to /test2
Duration of restore: 00:00:05

Did it work

cd /test2
ls -al
total 15368
...
-rw-r--r--  1 use user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

Yes it worked

Backup 2 folders an emailing the results

I use Google GSuite for email and have an app password setup just for duplicati. Here is the official guide for sending an email after a backup operation

I downloaded a 100MB file

cd /test2
wget http://ipv4.download.thinkbroadband.com/100MB.zip

I ran this command to backup an emailthe progress

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=***************** [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/08/2019 00:02:28
Checking remote backup …
Listing remote folder …
Scanning local files …
8 files need to be examined (115.00 MB)
7 files need to be examined (15.00 MB)
Uploading file (49.99 MB) …
Uploading file (18.01 KB) …
Uploading file (49.99 MB) …
Uploading file (17.97 KB) …
Uploading file (233.33 KB) …
Uploading file (33.17 KB) …
Uploading file (1.42 KB) …
Compacting remote backup …
Checking remote backup …
Listing remote folder …
Verifying remote backup …
Remote backup verification completed
Downloading file (1.42 KB) …
4 files need to be examined (15.00 MB)
Downloading file (18.01 KB) …
Downloading file (49.99 MB) …
0 files need to be examined (0 bytes)
Duration of backup: 00:00:15
Remote files: 13
Remote size: 115.33 MB
Total remote quota: 0 bytes
Available remote quota: 0 bytes
Files added: 1
Files deleted: 0
Files changed: 0
Data uploaded: 100.28 MB
Data downloaded: 50.01 MB
Backup completed successfully!

Email

Screenshot of and email from Duplicati software

Backup Types and Versions

Read up here on backup retention policies here

TIP: Use the desktop Duplicati software to generate a backup then use the export to command line option to get the command line parameters to use in the command line version.

Screenshot of the desktop version of Duplicati and the export to command line link

After you click Commandline you can view each separate command liene option and parameter.

This was handy for finding this backup retention parameter

--retention-policy="1W:1D,4W:1W,12M:1M

“This will give me “one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup. “

Setup Auto Backup Cron Scripts

Login to your linux machines and edit a bash file (e.g in “/scripts/backupnow.sh” folder).

Make the script executable

chmod +x /scripts/backupnow.sh

Add the following

#!/bin/bash

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=123 [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Now you can add the script to your crontab by running the following.

crontab -e

Make sure you have the following in your crontab

SHELL=/bin/bash

Add a cron entry to run the bash script

#Backup Database
0 22 * * * /bin/bash /scripts/backupnow.sh

Use https://crontab.guru/ to check the time you want to run thew script

https://crontab.guru/ screenshot

Links

  • B2 Starter Guides
  • Duplicati Manual

 

 

 

Version

v1.1 Fixed typo (”downalod” thanks to @NicolasWolf for reporting.)

v1.0 Initial Post (added retention polity and Linux crontabs

v0.95 Backup, Restore and Send Email

v0.91 Added Duplicati service info

v0.9 Working Copy

Filed Under: Uncategorized Tagged With: B2, backblaze, Backup, cli, cloud, duplicati, Linux, windows

Backing up your computer automatically with BackBlaze software (no data limit)

June 2, 2019 by Simon

Backblaze ( https://www.backblaze.com/ ) is an awesome company who not only create a mega reliable backup infrastructure but they also tell us how they do it and tell us what hard drives are good or bad.

Use my link and get your first month of Backblaze backups free: https://secure.backblaze.com/r/00e1wj

They are so confident and experienced that they offer unlimited backups for $6.60 a month here in Australia. They state they have 750 Petabytes stored on their infrastructure & have restored over 40 billion files to customers.

Check our my other related posts

  • Backing up files to a Backblaze B2 Cloud Bucket with Duplicati
  • How to back up an iPhone (including photos and videos) multiple ways

Main features of Backblaze.

  • No limit to files that you back up (number or size or files)
  • Web-based file recovery (if needed)
  • They will mail you a hard drive if you need to restore large amounts of files.
  • Ability to locate lost or stolen computers.
  • Mobile app access your backups.
  • Business options for large computer fleets.

Creating a Back Blaze Account

Go to backblaze and create an account.

Login to backblaze at https://secure.backblaze.com/user_signin.htm

Screenshot Login.

Choose a Backblaze Plan

Go to: https://secure.backblaze.com/buy_plan.htm to choose a backup and payment plan. Use my link and get your first month free.

Australian Prices below (add +10% GST).

  • $6 AUD a month
  • or $60 for 12 Months
  • or $110 for 24 Months
Screenshot $6 AUD a month or $60 for 12 Months or $110 for 24 Months

Choose a payment plan

Pay by the month, year or bi year

Enter payment details

Screenshot, add CC or paypal

Click “Buy Backblaze”

Use my link and get your first month of Backblaze backups free: https://secure.backblaze.com/r/00e1wj

Done, now I can download the client.

Installing the Backblaze Software

  • Windows Client
  • Mac Client

When the download is down you can start the install.

You will need to login to the install to streamline the setup.

Backblaze install screen asking me to login

After you login the installation will begin.

Backblaze is installing and checking what files need backing up.

Backblaze now reports that the install has complete. Backblaze reports that I have 379GB to backup.

Backblaze reports that I have 379GB to backup.

Pressing OK shows Backblaze is already uploading my files.

Also, I have 15 days of free trial before being billed.

Clicking on the backblaze system tray icon reveals a few options

Backblaze system tray icon menu

Backblaze Preferences

Backblaze preferences can be opened by clicking Settings from the main screen.

I..

  • Renamed the computer.
  • Disabled backup on battery power
  • Set upload threads to 5 (then lowering if the network gets busy or when the backup is almost complete)
  • Disabled Automatic throttling and set faster backups
  • I set a private encryption key
I entered and encryption key

There is a 50 char limit on encryption keys

50 char limit on encryption keys message

Warning about not forgetting the encryption key

Message don;t loose the key

All other preferences look ok.

Backblaze preferences

That’s it the backup is uploading automatically and there is nothing else I need to do.

Backup is happily uploading.

Backblaze Portal

If you login to the backblaze portal ( here ) you can view your files (you will need to enter the private encryption keys if you opted to encrypt your files).

Backblaze portal

Initial Backup time

Your initial backup may have a while to upload depending on your backup size and internet connection speed.

I was able to upload 400GB in 3 days on my Internet Plan. Smaller files are uploaded first then larger ones. I have 2 files left to upload totalling 7GB.

Read the official guide on speeding up uploads here.

Backblaze files uploading

One way to make the initial backups faster is to see what files are queues to upload in the Backblaze preferences and then excluding unwanted files and folders.

Backblaze files ro backup queue

I can see my Internet Explorer cache and Google Chrome Temporary Internet File(s) folders are queued to be back up.

Time to exclude these folders in the Backblaze Settings under exclusions

C:\Users\Simon Fearby\AppData\Local\Microsoft\Windows\INetCache\IE
C:\Users\Simon Fearby\AppData\Local\Google\Chrome\User Data\Profile 1\Code Cache\
C:\ProgramData\MySQL\MySQL Server 8.0\Data\#innodb_temp\
C:\ProgramData\USOShared\Logs\
C:\Users\Simon Fearby\AppData\Roaming\Adobe\SLData\SLCache\

Time to exclude these folders.

Backblaze exclude folders

That knocked off a few thousand files to backup 🙂

Restore Files from Backblaze

If you restore files you have a few options

  • Download the files (free)
  • Save files to B2 Cloud (Backblaze)
  • USB Flash Drive
  • USB Hard Drive
restore options. 1) download, 2) save to BS cloud, 3) USB Key, 4) USB Hard Drive

Choosing to download files you are given a treeview to restore files from.

Treeview folder to restore files

I selected a test file to restore (restores are not instant) and was informed I would be emailed when the files are ready to download. Thats cool

Screenshot, restore pending.

I waited 3 minutes and received an email that my files were ready to download.

Email: download ready

I hope this helps someone.

Other Links

Check out my guide on How to back up an iPhone photos and videos blog post here.

Use my link and get your first month of Backblaze backups free: https://secure.backblaze.com/r/00e1wj

View Official Backblaze guides here.

https://help.backblaze.com/hc/en-us/sections/203997408-B2-Guides

Version

v1.3 Initial Backup section

v1.1 Added free month link

v1.0 Initial Draft

Filed Under: Backblaze, Backup Tagged With: Automatic, backblaze, Backup

How to back up an iPhone (including photos and videos) multiple ways

June 2, 2019 by Simon

This guide is for and anyone who needs to back up their iPhone (complete device along with separate backups of photos and videos).

This is not a paid promo, I don’t get a kickback for mentioning the awesome app below.

Check our my other related posts

  • Backing up files to a Backblaze B2 Cloud Bucket with Duplicati
  • Backing up your computer automatically with BackBlaze software (no data limit)

At the time of writing iTunes v12.9.5.7 was the latest version of iTunes.

iTunes is Apple official software for getting files too and from an iPhone, iPod or iPad. If rumours are right Apple will kill off iTunes software at the end of 2019. If Apple kills off the iTunes software I will update this guide in the future.

Disclaimer: I provide no warranty or support for this app, this is what happened to work for me.

Why write a guide on Backing up a mobile phone?

  • It’s not that simple an I get asked how to do this almost every week by someone.
  • No one thinks to backup up their photos until they fear their phone is lost or stolen.
  • I am about to publish a few posts on backing up (development machines, servers etc) with automated software and with free open source software so this post will be handy to link in what data you can back up in that post.
  • The last time I wrote an article on backups was in March 2016 and its not that great these days.

Golden rules of backing up.

  1. Backup to Three different locations.
  2. Two of the media need to be different.
  3. One of the locations needs to be offsite
  4. Test your backups (a backup is only good if you can restore). (from @Daniel15)
  5. If the backup is encrypted make sure you have the decryption keys. (from @Daniel15)

Backing up the official way (with iTunes)

Apple prefers you just pay them to extend your free 5GB iCloud storage and not worry about backups and not to worry yourself with manual steps.

My wife’s iPhone is always popping up messages saying that her iCloud storage is nearly full and she should upgrade the free space limit.

Screenshot of iCloud setting on iPhone saying your iCloud is nearly full but you have used 5GB or 5GB?

I know she has 100GB+ images and movies on her iPhone but iCloud has only backed up 4GB of photos and videos. Do not trust any backup statuses unless you can verify all files from a restore.

For the record I use an Android phone (Android backup guide coming soon).

iCloud is a good idea for automatic backup but I prefer to see my photos and backup them up myself. Also, Apple iCloud is not free from troubles. If you want Apple to handle backups then I do suggest you upgrade your iCloud storage from the free 5GB to a more sensible 200GB or more.

Screenshot of https://support.apple.com/en-au/HT201318

Here in Australia Apple charge the following for extra storage.

  • 50GB = $1.49 a month
  • 200GB = $4.49 a month
  • 2TB (2,0000GB) = $14.99 a month

A rough estimate: 50GB is enough to store about 12,000 files (9,000 photos + 3,000 videos) *

  • * = based on stats from the backup below (depending on the size an length of your videos).
Screenshot of apple iCloud pricing 50GB = $1.49 a month
200GB = $4.49 a month
2TB (2,0000GB) = $14.99 a month

You can upgrade your paid Apple iCloud backup limit from your iCloud Storage Settings from your iPhone’s settings screen.

Screenshot of iPhone iCloud upgrade button.

IMHO: Apple does make it clear what you are getting before you purchase (but they do not push it enough and people assume their data is safe).

iPhone picture of 50GB = $1.49 a month
200GB = $4.49 a month
2TB (2,0000GB) = $14.99 a month

TIP: Please review your existing iPhone data usage and iCloud usage before it’s too late

I don’t want ot pay Apple for more iCloud storage

I like you, I am tight too but I do pay Google $2.49 a month for 100GB backup storage on my Android phone. Google Drive (Google storage is a bit cheaper than Apple).

Screenshot of Google Drive https://www.google.com/drive/

It is nice knowing your phone is auto backed up.

Install iTunes on Windows (10)

On your Windows 10 computer click the start button. If you don’t have a Microsoft Account go here and create one. Click the start button and type “itunes” (you don’t need to type into a box, just start typing when you click the start button)’.

Then click “iTunes, install app”

Screenshot of a Windows start menu and Get iTunes icon

If you are logged into your Windows 10 store with your Microsoft ID click “Get” (if not you will need to login to the store).

Screenshot of me logged into the Windows Microsoft Store.

After you click ‘Get’, iTunes will start downloading.

Screenshot of the Windows Store downloading iTunes

When iTunes is downloaded Windows will install it. When it’s installed click ‘Launch’

iTunes is installed, Click launch screenshot

You will need to agree to Apple’s terms of service. Click ‘Agree’

Apple terms of service screenshot.

When iTunes opens click “Agree”

Screenshot of Apple iTunes asking if we can agree to share analytics data

Now login to iTunes with your Apple ID (if you don’t have one create one here)’.

Screenshot of iTunes open with an arrow pointing to the Account menu

Click the ‘Account’ then ‘Sign In’ menu.

Screenshot of the account then sign in menu

Login to iTunes with your Apple ID

Screenshot of the apple sign in box.

Optional: If you have Two Factor Authorisation (2FA) of Apple ID’s turned on (you should) you will need to enter a 6 digit code.

Screenshot of an apple 6 digit 2FA code

Apple Two Factor Authorisation (2FA) will send one of your other devices a login code that you will need to use to login (confirm you own the account)

Screenshot of apples 2FA web page at https://support.apple.com/en-au/HT204915

TIP: Check out https://twofactorauth.org/ to see what other sites use Two Factor Authorisation (2FA). I use hardware Yubico YubiKeys to protect logins to WordPress, Linux and websites.

Now back to the article, iTunes should be ready to allow us to backup our iPhone.

Screenshot of me logged into iTunes

Before I continue I will click ‘Edit‘ then ‘Preferences‘ menu to view where iTunes will download media too (different ot backup data), not important but I just want ot see if it is not pointed ot my smaller C drive before I backup my phone

I changed the location to a larger S:\Drive’

Screenshot of Apple setting screen (tab Advanced)

I was greeted with a message asking me to confirm that I wanted to sync the iPhone that I just plugged in called ‘EllieRose’, I clicked ‘Continue’

Confirm access to the iPhone screenshot

On the iPhone I also clicked ‘Trust’ to allow iTunes to talk to it.

iPhone screenshot to allow iTunes to talk to it.

I was then prompted to download a software update to the iPhone, It appears this phone is not running the latest software

Screenshot of a question to update to iOS v 12.3.1

I was prompted to sync the iPhone to the computer

screenshot of a prompt to sync purchases from the iPhone to the computer

A backup of the phone was underway.

Screenshot of iTunes backup progress with a progress bar

If the screen above does not appear click the icon below to view the backup and restore iPhone menu.

Screenshot showing the iPhone Device button in iTunes

The iPhone was backing up to this folder on my computer:

C:\Users\Simon Fearby\Apple\MobileSync\Backup

I was not prompted for a location to back up so and I will move this backup folder after the backup completes (so my C: drive does not fill up).

My drives.

  • C:\ = 500GB drive (faster SSD)
  • S:\ = 2,000GB Drive (slower Magnetic)
Screenshot showing  iTunes backup and  the C:\Users\Simon Fearby\Apple\MobileSync\Backup folder

The iTunes iPhone backup is now complete (it took about 2 hours).

Screenshot of the complete backup

Now that the iPhone was backed up iTunes the Operating System update started to run.

Screenshot of iTunes verifying the iPhone update

Update verified.

Screenshot of iTunes updating the iPhone OS.

During the update the iPhone was unavailable.

Screenshot of iPhone installing firmware

Done, the iPhone had updated it’s operating system in about 30 minutes.

Screenshot of iTunes showing the newer OS version.

Now lets see how much space I have available on my C Drive.

Screenshot of disk usage on my c and S drive.

It looks like the iPhone backup iTunes made was 60GB in size.

Screenshot of Windows reporting the backup was 60GB in size.

Using Windows Explorer I moved the backup from..

C:\Users\Simon Fearby\Apple\MobileSync\Backup

to..

S:\Backup\AlisonsiPhone\iTunes\31May2019

(right click drag an drop action in explorer)

Screenshot of right click drag and drop move folder.

It took me about 20 minutes to move the 60GB folder.

Screenshot of thew Windows copy dialog progress.

TIP: You can (should) also copy this backup folder to..

  • A removable hard drive
  • Removable USB Flash Drive
  • A NAS or SAN Drive
  • etc

Now this backup is available for me to restore in the future if I need it.

In the case of a restore, I just need to move the backup into the iTunes expected location.

Copy from..

S:\Backup\AlisonsiPhone\iTunes\31May2019

..to..

C:\Users\Simon Fearby\Apple\MobileSync\Backup

The Catch

The catch with back-ups made with iTunes is they are useless if you wanted to restore individual files (say to find a photograph or video). iTunes backups are usually tens of thousands of files with random folders and filenames

Screenshot of iTunes nasty backup, i obfuscated all files to random guid filenames

If you want to JUST backup photos and videos read on.

How to backup just photos and videos from your iPhone.

This part of the guide needs a paid version of the iOS App Photo Transfer App. In Australia, the app is free but to turn it into a full version needed to restore everything (via in-app purchase) it cost $10.99.

This app is well worth $10.99 AUD (it may be cheaper in other countries) to have it push photos an videos from your iPhone to a free companion Windows program (a Mac versions exist too).

Screenshot of http://phototransferapp.com/

Buy the iOS version and install it on your iPhone.

Then download the free Windows version: http://phototransferapp.com/win/

Extract the files from the zip file.

I extracted PhotoTransferapp.exe from PhotoTransferapp.zip

When I run PhotoTransferApp.exe I get an error saying I need to install Adobe AIR run times.

Screenshot of me installing Adobe Air run times from https://get.adobe.com/air/

Go to https://get.adobe.com/air/ and download and install the Air run time.

Screenshot of https://get.adobe.com/air/

Install the Adobe Air run time.

Install Adobe Air From https://get.adobe.com/air/

I installed Adobe AIR and again reopened the PhotoTansferApp.exe and was prompted to allow access to the Windows Firewall.

I was on a home network so I enabled firewall access.

Screenshot of Photo Transfer App asking fore firewall access.

The Photo Transfer App prompted me to ‘Discover Devices‘ and to make sure the Transfer app is running on the iPhone.

The Photo Transfer App prompted me to 'Discover Devices' and to make sure the Transfer app is running on the iPhone.

Before I clicked ‘Discover Devices” on the Photo Transfer App on Windows I opened the Transfer app (mentioned above) on the iPhone.

Photo of the Transfer app on the iPhone

On the iPhone I clicked ‘Send‘

Click Send

The transfer app will ask for access to your photos, you will need to press OK.

Screenshot of the transfer app asking for permissions to photos

On the iPhone again I clicked ”Windows‘

On the iPhone again I clicked  ''Windows'

The iOS Transfer app now said I should run the ‘Photo Transfer App‘ on Windows.

The iOS Transfer app now said I should run the 'Photo Transfer App' on Windows.

TIP: You can transfer over WiFi (if your iPhone and Windows device is on the same WiFi Network) or you can transfer over a USB cable.

Screenshot: transfer via WiFi or USB?

Before I started the ‘Detect Device‘ or ‘transfer‘, I set the Backup path location to my S: drive by clicking settings in the bottom right and choosing a folder (as my C Drive is a bit small).

Screenshot of Photo Transfer App choosing a folder to backup to.

I noticed the port of 57777, I temporarily disabled the whole Windows Firewall just in case it prevents the photo backup.

In hindsight, it was not a good idea to disable the whole firewall, but because I was at home on a safe network I felt safe to do so. Next time I will not disable the firewall and see if this still works.

If you are on an internet cafe, school or university network do not disable your firewall.

I clicked ”Start” then typed ‘firewall‘ and clicked ‘Windows Defender Firewall‘

Screenshot Windows Defender firewall icon

I turned off my firewall.

Screenshot Firewall disabled.

I clicked ‘Detect Devices‘ and the iPhone ‘EllieRose‘ appeared on the left. I doubled clicked on the iPhone name and was prompted with an Authorization required message.

Screenshot: Authorisation required message.

I looked at the ‘Transfer‘ app on iOS and clicked ‘Yes, always‘ to allow access.

Screenshot: Authorisation yes/no on the iPhone

After 20 seconds I can see photos on my iPhone on Windows.

Screenshot, all of my iPhone images were appearing on Window.

I selected all camera albums to backup and clicked backup. I prompt to upgrade to the full version will appear, you will need to buy the upgrade.

Full backup happens only if you upgrade to a full version via an in app purchase

After the full version if purchased the backup will be allowed.

You can upgrade from the free to paid full version from the main screen of the Transfer app.

Screenshot get full access by upgrading the free app to a paid version

The upgrade In App Purchase is $10.99 AUD

Screenshot of a $10.99 In App Purchase on the iPhone

When you purchase the In App Purchase you can run the backup again.

Screenshot of all camera categories on the iPhone on windows

Now I could see photos being copied from the iPhone to my defined backup folder.

Screenshot underway.

I went to bed as I knew there were about 100GB of files on the iPhone and this was going to take a while.

In the morning the backup was done.

Screenshot backup done.

26,000 files were backed up (over 100GB).

I now had a full iPhone backup made by iTunes and a copy of all photos and videos.

Screenshot showing the 2 backups (a) iTunes made and b) Photo Transfer App made)

I turned on the Firewall again.

Screenshot: I re enabled the firewall.

I now had 160GB of backed up photos, videos and phone backup.

160GB of files  from both backups.

Backup your iPhone backups to the cloud

The steps to do this two ways will be added soon.

I will add a section on how you can back up the iTunes and manually synced photo and video backups to the cloud automatically and a more complex but cheaper was for 0.005c per GB.

Watch this post.

How to backup and Android Phone

Article coming soon.

Other Links

Read the official iTunes/iCloud backup guide from Apple here: https://support.apple.com/en-au/HT203977

Backing up your computer automatically with BackBlaze software (no data limit)

Versions

v1.3 Added back blaze article link

v1.2 Added more images (from a phone that does not have the iOS app already)

v1.1 Added an Android heading.

v1.0 Initial Post

Filed Under: Apple, Backup, Cloud, Google Tagged With: Apple, Backup, iCloud, iPhone

How to backup and restore a MySQL database on Windows and Linux

April 21, 2019 by Simon

Why backup and restore

This is a quick guide demonstrating how you can backup and restore a MySQL database on Windows and Linux using Adminer.

You may need to know how to backup a restore a database for a number of reasons..

e.g

  • Send the database to someone to debug or give feedback while learning.
  • Move the database from a local machine to the cloud
  • Move the database from cloud vendor A to cloud vendor B
  • etc.

Having a backup of the VM is good but having a backup of the database too is better. I use UpCloud for hosting my VM’s and setting backups is easy. But I cannot download those backups.

UpCloud Backup Screen

Murphy’s Law

“If anything can go wrong, it will”

The most important reason for taking a backup and knowing how to restore it is for disaster recovery reasons.

Backup (the easiest way) with Adminer

Adminer is a free PHP based IDE for MySQL and other databases. Simply install Adminer and save the file on your local computer or remote web server directory.

FYI: The Adminer author Jakub Vrana has a patron page, I am a patron of this awesome software.

Snip from Adminers website. “Adminer (formerly phpMinAdmin) is a full-featured database management tool written in PHP. Conversely to phpMyAdmin, it consist of a single file ready to deploy to the target server. Adminer is available for MySQL, MariaDB, PostgreSQL, SQLite, MS SQL, Oracle, Firebird, SimpleDB, Elasticsearch andMongoDB.”

adminer.php file icon screenshot

TIP: The file would be publicly accessible to anyone so don’t save it to a common area, obfuscate the file, protect it of delete the file when you are done using it.

Once Adminer is installed load it in a web browser, login with your MySQL credentials. Once you login you will see all databases and an Import and Export menu.

Adminer main screen, all databases and import and export menu.

tbtest is a simple database with one table and 4 fields (ID, Key, Value and Modified)

.Click Export to open the export screen.

Export screen showing a list of databases and export options

Click Export, a SQL file will be generated (this is the export of the database).

Here is a save of the file:
https://fearby.com/wp-content/uploads/export.txt

Exported view of https://dev.mysql.com/doc/workbench/en/wb-admin-export-import-management.html

Its that simple.

If I add a binary blob file to the table and upload a PNG file lets see how the export looks.

Screenshot o the new table with a blog field in Adminer UI

Let export the database again in Adminer and check out the output. I used Sublime Text editor to view the export file.

New Export shows the binary file in the Backup SQL file

Restore (the easiest way) with Adminer

OK lets delete the tbtest database and then restore it with Adminer. I used Adminer to delete (DROP) the database.

Database dropped with Adminer

Database “dbtest” deleted.

Now lets create a blank database to restore to (same name).

Create database screen.

Database created.

dbtest created.

Now lets import the database backup using Adminer.

Click Import, select the backup file and un-tick Stop on errors.

Import screenshot, dxtest selectded, Restore file selected, stop on errors disabled

TIP: The 2MB next the the choose file button is defined by your web server and PHP configuration. If you are trying to import a larger database (e.g 80MB) first increase the limits in your web server and PHP (via php.ini).

The Import (restore should take seconds)

Import Success

The database was imported from a backup, all tables and records imported just fine.

The database was imported from a backup

Bonus methods.

On Ubuntu use this guide to backup from the command line. If you use the Oracle MySQL Workbench read this.

I hope this helps someone.

Filed Under: Adminer, Backup, Database, MySQL, Restore Tagged With: and, Backup, How, Linux, MySQL, on, restore, to, windows

How to backup WordPress on a host that has CPanel

April 29, 2018 by Simon

This is a simple guide that demonstrates how you can log in to a host that offers the CPanel tools to backup all of your website files (and databases). Backing up your website should be done often and especially before you migrate to any another website host. I used to change hosts every few years (they don’t own your site, you do).

I have a number of guides on moving away from CPanel, setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line copying files to a server via command line editing remote files locally etc but how do you manage a website with CPanel?

You can normal login to CPanel tools on a shared host by loading www.yourdomainnam.com/cpanel (failing that login to your domain hosts web GUI and find your CPanel interface there).

Step 1: Login to your Host

Login to your web host

Login to Host

Step 2: Find your CPanel Interface

Hosts are a bit different but in this case, I just click my domain to find the CPanel link.

Domain

I found it, I clicked the CPanel login.

Cpanel link

Step 3: CPanel Applications

CPanel does offer good tools to manage your websites like web-based File Manager and Database tool called phpMyAdmin.

CPanel Options

Aside: CPanel/Hosts Downsides

The thing I don’t like about hosts that offer CPanel is they usually limit delivery of your website to extract more money. Nothing worse than receiving Resource Limit Is Reached errors.

cpenal_usage_exceeded

Also shared hosts usually lag way behind in newer software versions like PHP and MySQL (this is a security concern).

TIP: You can scan your site for vulnerabilities using Qualsys Freescan, Zap or Kali Linux.

Here is a security scan of a shared host (with CPanel) that I was using in 1999. Note the high vulnerabilities and old version of Linux.

Static Website

Also, a shared host will often overcharge you (e.g $150 a year) for a poorly configured SSL certificate.

This was an SSL cert I paid $150 a year for (evaluated with SSL Labs SSL Test) on a shared host with CPanel.

Aside: Self Managed Upsides

After I moved my domain to a self-managed virtual machine I migrated WordPress, set up a free SSL certificate, sped up my site with a CDN, setup Cloudflare, setup better TLS security etc

When you manage your own server you can install a free SSL certificate in under 1 minute.

Below is my SSL certificate. A strong SSL certificate will increase search engine traffic

SSL Cert

Aside: Compare Shared host speed v Self Managed

FYI: https://gtmetrix.com/ is a great site for measuring the speed of a website (shared of self-managed). I found great speed improvements after moving away from a host offering CPanel, tweaking the server and setting up cloudflare. A self-managed server will allow you to tweak anything you want.

GTMetrix results:

Shared Host Speed

I like how self-managed servers allow you to scale the server’s resources yourself, move servers or add storage etc.

Aside: SSL Certificate

If you have an SSL cert you should test it often as vulnerabilities pop up from time to time.

FYI: All sites will soon require an SSL certificate to be sent traffic from search engines (no SSL = lower traffic).

SSL Test my site: https://dev.ssllabs.com/ssltest/analyze.html?d=fearby.com&s=104.27.154.69

Now enough with the self-managed serve asides and back to how to backup your website with CPanel tools.

Step 4: Backup your web files in CPanel

Use the File Explorer app in CPanel

File Manager

Highlight all files that you want to backup (highlight everything but not past backup files).

backup-cpanel-006

View the files to compress summary

Compress Summary

Click Compress Files(s) and view the backup progress

Backup Progress

You can now download the backup zip file in your browser (click the file and click Download).

Downaload ZIp

Download Progress.

Download File

Step 5: Backup your database in CPanel

Now we need to backup any MySQL database(s) that may be used by WordPress

Open the phpMyAdmin app in CPanel.

phpMyAdmin

FYI: Alternatively, you can use a free tool called Adminer to backup and restore our database.

Click your WordPress database (on the left). You can identify your current WordPress database by opening the wp-config.php file.

Database Options

The first step is to perform an online cold backup of the WordPress database.

  1. Click the database you want to backup (on the left)
  2. Click the Options tab
  3. Under the Copy To frame enter a new database name to copy the database too
  4. Click Go

Online Backup

Now you have an online cold spare that you can use just in case the original database corrupts itself. You can rename the database or configure WordPress to point to this new database if need be.

backup-cpanel-013

Now let’s download a copy of the database (Repeat for multiple databases).

  1. Click the database you want to backup
  2. Click the Export tab
  3. Click Go

backup-cpanel-014

You should now have a backup of your website in a zip file and an export of your database in a .sql text file, SQL files can be re-imported to databases later.

TIP: Backup often.

Export Done

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial post

Filed Under: CPanel Tagged With: a, Adminer, Backup, cpanel, has, host, How, MySQL, on, that, to, wordpress, zip

How to backup an Ubuntu VM in the cloud via crontab entries that trigger Bash Scripts, SSH, rsync and email backup alerts

August 20, 2017 by Simon

Here is how I backup a number of Ubuntu servers with crontab entries, bash scripts and rsync and send backup email.

Read more on useful terminal commands here for as low as $2.5 a month. Read on setting up a Digital Ocean Ubuntu server here for as low as $5 a month here ($10 free credit). Read more on setting up an AWS Ubuntu server here.

I have  6 numbered scripts in my scripts folder that handle backups, I call these scripts at set times via the crontab list.

fyi: Paths below have been changed for the purpose of this post (security).

1 1 * * * /bin/bash /scripts-folder/0.backupfiles.sh >> /backup-folder/0.backupfiles.log
3 1 * * * /bin/bash /scripts-folder/1.backupdbs.sh >> /backup-folder/1.backupdbs.log
5 1 * * * /bin/bash /scripts-folder/2.shrinkmysql.sh >> /backup-folder/2.shrinkmysql.log
10 1 * * * /bin/bash /scripts-folder/3.addtobackuplog.sh >> /backup-folder/3.addtobackuplog.log
11 1 * * * /bin/bash /scripts-folder/4.syncfiles.sh >> /backup-folder/4.syncfiles.log
15 1 * * * /bin/bash /scripts-folder/5.sendbackupemail.sh > /dev/null 2>&1

https://crontab.guru/ is great for specifying times to run jobs on each server (I backup one server at 1 AM,, another at 2 AM etc (never at the same time))

Bring up your crontab list

crontab -e

Check out the Crontab schedule generator here.

Below is the contents of my /scripts/0.backupfiles.sh (sensitive information removed).

I use this script to backup folders and configuration data

cat /scripts-folder/0.backupfiles.sh
#!/bin/bash

echo "Deleting old NGINX config..";
rm /backup-folder/config-nginx.zip

echo "Backing Up NGNIX..";
zip -r -9 /backup-folder/config-nginx.zip /etc/nginx/ -x "*.tmp" -x "*.temp" -x"./backup-folder/*.bak" -x "./backup-folder/*.zip"

echo "Deleting old www backup(s) ..";
#rm /backup-folder/www.zip
echo "Removing old www backup folder";
rm -R /backup-folder/www
echo "Making new backup folder at /backup-folder/www/";
mkdir /backup-folder/www

echo "Copying /www/ to /backup-folder/www/";
cp -rTv /www/ /backup-folder/www/
echo "Done copying /www/ to /backup-folder/www/";

Below is the contents of my /scripts-folder/1.backupdbs.sh (sensitive information removed).

I use this script to dump my MySQL database.

cat /scripts-folder/1.backupdbs.sh
#!/bin/bash

echo "$(date) 1.backupdbs.sh ...." >> /backup-folder/backup.log

echo "Removing old SQL backup..":
rm /backup-folder/mysql/database-dump.sql

echo "Backing up SQL";
/usr/bin/mysqldump --all-databases > /backup-folder/mysql/database-dump.sql -u 'mysqluser' -p'[email protected]$word'

echo "Done backing up the database";

Below is the contents of my /scripts-folder/2.shrinkmysql.sh (sensitive information removed).

I use this script to tar my SQL dumps as these files can be quite big

cat /scripts-folder/2.shrinkmysql.sh
#!/bin/bash

echo "$(date) 2.shrinkmysql.sh ...." >> /backup-folder/backup.log

echo "Backing up MySQL dump..";
tar -zcf /backup-folder/mysql.tgz /backup-folder/mysql/

echo "Removing old MySQL dump..";
rm /backup-folder/mysql/*.sql

Below is the contents of my /scripts-folder/3.addtobackuplog.sh (sensitive information removed).

This script is handy for dumping extra information.

cat /scripts-folder/3.addtobackuplog.sh
#!/bin/bash

echo "$(date) 3.addtobackuplog.sh ...." >> /backup-folder/backup.log

echo "Server Name.." >> /backup-folder/backup.log
grep "server_name" /etc/nginx/sites-available/default

echo "$(date) Timec" >> /backup-folder/backup.log
sudo hwclock --show  >> /backup-folder/backup.log

echo "$(date) Uptime, Load etc" >> /backup-folder/backup.log
w -i >> /backup-folder/backup.log

echo "$(date) Memory" >> /backup-folder/backup.log
free  >> /backup-folder/backup.log

echo "$(date) Disk Space" >> /backup-folder/backup.log
pydf >> /backup-folder/backup.log

echo "Firewall" >> /backup-folder/backup.log
ufw status >> /backup-folder/backup.log

echo "Adding to Backup Log file..";
echo "$(date) Nightly MySQL Backup Successful....." >> /backup-folder/backup.log

Below is the contents of my /scripts-folder/4.syncfiles.sh (sensitive information removed).

This script is the workhorse routine that rsyncs files to the source to the backup server (a dedicated Vulr server with an A Name record attaching the server to my domain).

I installed sshpass to pass in the ssh user password (after ssh is connected (authorized_keys set), I tried to setup a rsync daemon but had no luck).  I ensured appropriate ports were opened on the source (OUT 22, 873) and backup server (IN 22 873).

cat /scripts-folder/4.syncfiles.sh
#!/bin/bash

echo "$(date) 4.syncfiles.sh ...." >> /backup-folder/backup.log
echo "Syncing Files.";

sudo sshpass -p 'Y0urW0rkingSSHR00tPa$0ord' rsync -a -e  'ssh -p 22 ' --progress -P /backup-folder backup-server.yourdomain.com:/backup-folder/1.www.server01.com/

ufw firewall has great rules for allowing certain IP’s to talk on ports.

Set Outbound firewall rules (to certain IP’s)

sudo ufw allow from 123.123.123.123 to any port 22

Change 123.123.123.123 to your backup server.

Set Inbound firewall rules (to certain IP’s)

sudo ufw allow out from 123.123.123.123 to any port 22

Change 123.123.123.123 to your sending server.

You can and should setup rate limits on IP’s hitting certain ports.

udo ufw limit 22 comment 'Rate limit for this port has been reached'

Install Fail2Ban to automatically ban certain users. Fail2Ban reads log file that contains password failure report
and bans the corresponding IP addresses using firewall rules.  Read more on securing Ubuntu in the cloud here.

Below is the contents of my /scripts-folder/5.sendbackupemail.sh (sensitive information removed).

This script sends an email and attaches a zip file of all log files generated through the backup process.

cat /scripts/5.sendbackupemail.sh
#!/bin/bash

echo "$(date) 5.sendbackupemail.sh ...." >> /backup-folder/backup.log

echo "Zipping up log Files.";

zip -r -9 /backup-folder/backup-log.zip /backup-folder/*.log

echo "Sending Email";
sendemail -f [email protected] -t [email protected] -u "Backup Alert" -m "server01 has been backed up" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp Y0urGSu1tePasswordG0e$Here123 -a /backup-folder/backup-log.zip

Read my guide on setting up sendmail here.

Security Considerations

You should never store passwords in scripts that talk to SSH connections, create MySQL dumps or when talking to email servers, I will update this guide when I solving all of these cases.  Also, create the least access required for user accounts where possible.

Target Server Configuration

Alos you can see in /scripts-folder/4.syncfiles.sh that I am saving to the ‘/backup-folder/1.www.server01.com/’ folder, you can make as many folders as you want to make the most of the backup server.  I would advise you not use the server for anything else like web servers and apps as this server is holding important stuff.

backup-server.yourdomain.com:/backup-folder/1.www.server01.com/

I have a handy script to delete all backups (handy during testing).

#!/bin/bash

echo "Deleting Backup Folders..........................................";

echo " Deleting /backup-folder/1.www.server01.com";
rm -R /backup-folder/1.www.server01.com

echo " Deleting /backup-folder/2.www.server02.com";
rm -R /backup-folder/2.www.server02.com

echo " Deleting /backup-folder/3.www.server03.com";
rm -R /backup-folder/3.www.server03.com

echo " Deleting /backup-folder/4.www.server04.com";
rm -R /backup-folder/4.www.server04.com

echo " Deleting /backup-folder/5.www.server05.com";
rm -R /backup-folder/5.www.server05.com

echo " Deleting /backup-folder/6.www.server06.com";
rm -R /backup-folder/6.www.server06.com

echo " Deleting /backup-folder/7.www.server07.com";
rm -R /backup-folder/7.www.server07.com

echo " Deleting /backup-folder/8.www.server08.com";
rm -R /backup-folder/8.www.server08.com

echo "
";

echo "Creating Backup Folders.........................................";

echo " Making folder /backup-folder/1.www.server01.com";
mkdir /backup-folder/1.www.server01.com

echo " Making folder /backup-folder/2.www.server02.com";
mkdir /backup-folder/2.www.server02.com

echo " Making folder /backup-folder/3.www.server03.com";
mkdir /backup-folder/3.www.server03.com";

echo " Making folder /backup-folder/4.www.server04.com";
mkdir /backup-folder/4.www.server04.com

echo " Making folder /backup-folder/5.www.server04.com";
mkdir /backup-folder/5.www.server04.com

echo " Making folder /backup-folder/6.www.server05.com";
mkdir /backup-folder/6.www.server04.com

echo " Making folder /backup-folder/7.www.server06.com";
mkdir /backup-folder/7.www.server04.com

echo " Making folder /backup-folder/8.www.server07.com";
mkdir /backup-folder/8.www.server08.com

echo "
";

echo "Backup Folder Contents.........................................";
ls /backup-folder -al
echo "
";

echo "Folder Strcuture...............................................";
cd /backup-folder
pwd
tree -a -f -p -h  -l -R

echo "
";

echo "How big is the backup folder...................................";
du -hs /backup-folder

echo "
";

echo "Done...........................................................";

Ensure your backup server is just for backups and only allows traffic from known IP’s

ufw status
Status: active

To                         Action      From
--                         ------      ----
22                         ALLOW       123.123.123.123
22                         ALLOW       123.123.123.124
22                         ALLOW       123.123.123.125
22                         ALLOW       123.123.123.126
22                         ALLOW       123.123.123.127
22                         ALLOW       123.123.123.128
22                         ALLOW       123.123.123.129
22                         ALLOW       123.123.123.130
53                         ALLOW       Anywhere

22                         ALLOW OUT   123.123.123.123
22                         ALLOW OUT   123.123.123.124
22                         ALLOW OUT   123.123.123.125
22                         ALLOW OUT   123.123.123.126
22                         ALLOW OUT   123.123.123.127
22                         ALLOW OUT   123.123.123.128
22                         ALLOW OUT   123.123.123.129
22                         ALLOW OUT   123.123.123.130

Change the 123.x.x.x servers to your servers IP’s

Tip: Keep an eye on the backups with tools like ncdu

sudo ncdu /backup-folder
ncdu 1.11 ~ Use the arrow keys to navigate, press ? for help
--- /backup ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    1.0 GiB [##########] /6.www.server01.com
  462.1 MiB [####      ] /1.www.server02.com
  450.1 MiB [####      ] /5.www.server03.com
   60.1 MiB [          ] /2.www.server04.com
  276.0 KiB [          ] /3.www.server05.com
  276.0 KiB [          ] /4.www.server06.com
e   4.0 KiB [          ] /8.www.server07.com
e   4.0 KiB [          ] /7.www.server08.com

Installing SSH on OSX

If you want to backup to this server with OSyouou will need to install sshpass

curl -O -L http://downloads.sourceforge.net/project/sshpass/sshpass/1.06/sshpass-1.06.tar.gz && tar xvzf sshpass-1.06.tar.gz
cd sshpass-1.06
./configure
sudo make install

sshpass should be installed

sshpass -V
sshpass 1.06
(C) 2006-2011 Lingnu Open Source Consulting Ltd.
(C) 2015-2016 Shachar Shemesh
This program is free software, and can be distributed under the terms of the GPL
See the COPYING file for more information.

Using "assword" as the default password prompt indicator.

I have not got sshp[ass working yet error “Host key verification failed.”  I had to remove the back known host from ~/.ssh/known_hosts” on OSX

But this worked on OSX

rsync -a -e 'ssh -p 22 ' --progress -P ~/Desktop [email protected]:/backup/8.Mac/

Note: Enter the servers [email protected] before the hostname or rsync will use the logged in OSX username

Don’t forget to check the backup serves disk usage often.

disk usage screenshot

Output from backing up an incremental update (1x new folder)

localhost:~ local-account$ rsync -a -e  'ssh -p 22 ' --progress -P /Users/local-account/folder-to-backup [email protected]:/backup/the-computer/
[email protected]'s password: 
building file list ... 
51354 files to consider
folder-to-backup/
folder-to-backup/TestProject/
folder-to-backup/TestProject/.git/
folder-to-backup/TestProject/.git/COMMIT_EDITMSG
          15 100%    0.00kB/s    0:00:00 (xfer#1, to-check=16600/51354)
folder-to-backup/TestProject/.git/HEAD
          23 100%   22.46kB/s    0:00:00 (xfer#2, to-check=16599/51354)
folder-to-backup/TestProject/.git/config
         137 100%  133.79kB/s    0:00:00 (xfer#3, to-check=16598/51354)
folder-to-backup/TestProject/.git/description
          73 100%   10.18kB/s    0:00:00 (xfer#4, to-check=16597/51354)
folder-to-backup/TestProject/.git/index
        1581 100%  220.56kB/s    0:00:00 (xfer#5, to-check=16596/51354)
folder-to-backup/TestProject/.git/hooks/
folder-to-backup/TestProject/.git/hooks/README.sample
         177 100%   21.61kB/s    0:00:00 (xfer#6, to-check=16594/51354)
folder-to-backup/TestProject/.git/info/
folder-to-backup/TestProject/.git/info/exclude
          40 100%    4.88kB/s    0:00:00 (xfer#7, to-check=16592/51354)
folder-to-backup/TestProject/.git/logs/
folder-to-backup/TestProject/.git/logs/HEAD
         164 100%   20.02kB/s    0:00:00 (xfer#8, to-check=16590/51354)
folder-to-backup/TestProject/.git/logs/refs/
folder-to-backup/TestProject/.git/logs/refs/heads/
folder-to-backup/TestProject/.git/logs/refs/heads/master
         164 100%   20.02kB/s    0:00:00 (xfer#9, to-check=16587/51354)
folder-to-backup/TestProject/.git/objects/
folder-to-backup/TestProject/.git/objects/05/
folder-to-backup/TestProject/.git/objects/05/0853a802dd40cad0e15afa19516e9ad94f5801
        2714 100%  294.49kB/s    0:00:00 (xfer#10, to-check=16584/51354)
folder-to-backup/TestProject/.git/objects/11/
folder-to-backup/TestProject/.git/objects/11/729e81fc116908809fc17d60c8604aa43ec095
         105 100%   11.39kB/s    0:00:00 (xfer#11, to-check=16582/51354)
folder-to-backup/TestProject/.git/objects/23/
folder-to-backup/TestProject/.git/objects/23/768a20baaf8aa0c31b0e485612a5e245bb570d
         131 100%   12.79kB/s    0:00:00 (xfer#12, to-check=16580/51354)
folder-to-backup/TestProject/.git/objects/27/
folder-to-backup/TestProject/.git/objects/27/3375fc70381bd2608e05c03e00ee09c42bdc58
         783 100%   76.46kB/s    0:00:00 (xfer#13, to-check=16578/51354)
folder-to-backup/TestProject/.git/objects/2a/
folder-to-backup/TestProject/.git/objects/2a/507ef5ea3b1d68c2d92bb4aece950ef601543e
         303 100%   26.90kB/s    0:00:00 (xfer#14, to-check=16576/51354)
folder-to-backup/TestProject/.git/objects/2b/
folder-to-backup/TestProject/.git/objects/2b/f8bd93d56787a7548c7f8960a94f05c269b486
         136 100%   12.07kB/s    0:00:00 (xfer#15, to-check=16574/51354)
folder-to-backup/TestProject/.git/objects/2f/
folder-to-backup/TestProject/.git/objects/2f/900764e9d12d8da7e5e01ba34d2b7b2d95ffd4
         209 100%   17.01kB/s    0:00:00 (xfer#16, to-check=16572/51354)
folder-to-backup/TestProject/.git/objects/36/
folder-to-backup/TestProject/.git/objects/36/d2c80d8893178d7e1f2964085b273959bfdc28
         201 100%   16.36kB/s    0:00:00 (xfer#17, to-check=16570/51354)
folder-to-backup/TestProject/.git/objects/3d/
folder-to-backup/TestProject/.git/objects/3d/e5a02083dbe9c23731a38901dca9e913c04dd0
         130 100%   10.58kB/s    0:00:00 (xfer#18, to-check=16568/51354)
folder-to-backup/TestProject/.git/objects/40/
folder-to-backup/TestProject/.git/objects/40/40592d8d4d886a5c81e1369ddcde71dd3b66b5
         841 100%   63.18kB/s    0:00:00 (xfer#19, to-check=16566/51354)
folder-to-backup/TestProject/.git/objects/87/
folder-to-backup/TestProject/.git/objects/87/60f48ddbc9ed0863e3fdcfce5e4536d08f9b8d
          86 100%    6.46kB/s    0:00:00 (xfer#20, to-check=16564/51354)
folder-to-backup/TestProject/.git/objects/a9/
folder-to-backup/TestProject/.git/objects/a9/e6a23fa34a5de4cd36250dc0d797439d85f2ea
         306 100%   22.99kB/s    0:00:00 (xfer#21, to-check=16562/51354)
folder-to-backup/TestProject/.git/objects/b0/
folder-to-backup/TestProject/.git/objects/b0/4364089fdc64fe3b81bcd41462dd55edb7a001
          57 100%    4.28kB/s    0:00:00 (xfer#22, to-check=16560/51354)
folder-to-backup/TestProject/.git/objects/be/
folder-to-backup/TestProject/.git/objects/be/3b93d6d8896d69670f1a8e26d1f51f9743d07e
          60 100%    4.19kB/s    0:00:00 (xfer#23, to-check=16558/51354)
folder-to-backup/TestProject/.git/objects/d0/
folder-to-backup/TestProject/.git/objects/d0/524738680109d9f0ca001dad7c9bbf563e898e
         523 100%   36.48kB/s    0:00:00 (xfer#24, to-check=16556/51354)
folder-to-backup/TestProject/.git/objects/d5/
folder-to-backup/TestProject/.git/objects/d5/4e024fe16b73e5602934ef83e0b32a16243a5e
          69 100%    4.49kB/s    0:00:00 (xfer#25, to-check=16554/51354)
folder-to-backup/TestProject/.git/objects/db/
folder-to-backup/TestProject/.git/objects/db/3f0ce163c8033a175d27de6a4e96aadc115625
          59 100%    3.84kB/s    0:00:00 (xfer#26, to-check=16552/51354)
folder-to-backup/TestProject/.git/objects/df/
folder-to-backup/TestProject/.git/objects/df/cad4828b338206f0a7f18732c086c4ef959a7b
          51 100%    3.32kB/s    0:00:00 (xfer#27, to-check=16550/51354)
folder-to-backup/TestProject/.git/objects/ef/
folder-to-backup/TestProject/.git/objects/ef/e6d036f817624654f77c4a91ae6f20b5ecbe9d
          94 100%    5.74kB/s    0:00:00 (xfer#28, to-check=16548/51354)
folder-to-backup/TestProject/.git/objects/f2/
folder-to-backup/TestProject/.git/objects/f2/b43571ec42bad7ac43f19cf851045b04b6eb29
         936 100%   57.13kB/s    0:00:00 (xfer#29, to-check=16546/51354)
folder-to-backup/TestProject/.git/objects/fd/
folder-to-backup/TestProject/.git/objects/fd/f3f97d1b6e9d8d29bb69a88c4d89ca752bd937
         807 100%   49.26kB/s    0:00:00 (xfer#30, to-check=16544/51354)
folder-to-backup/TestProject/.git/objects/info/
folder-to-backup/TestProject/.git/objects/pack/
folder-to-backup/TestProject/.git/refs/
folder-to-backup/TestProject/.git/refs/heads/
folder-to-backup/TestProject/.git/refs/heads/master
          41 100%    2.50kB/s    0:00:00 (xfer#31, to-check=16539/51354)
folder-to-backup/TestProject/.git/refs/tags/
folder-to-backup/TestProject/TestProject.xcodeproj/
folder-to-backup/TestProject/TestProject.xcodeproj/project.pbxproj
       11476 100%  659.24kB/s    0:00:00 (xfer#32, to-check=16536/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/contents.xcworkspacedata
         156 100%    8.96kB/s    0:00:00 (xfer#33, to-check=16534/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/xcuserdata/
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/xcuserdata/simon.xcuserdatad/
folder-to-backup/TestProject/TestProject.xcodeproj/project.xcworkspace/xcuserdata/simon.xcuserdatad/UserInterfaceState.xcuserstate
        8190 100%  470.47kB/s    0:00:00 (xfer#34, to-check=16531/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/xcschemes/
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/xcschemes/TestProject.xcscheme
        3351 100%  192.50kB/s    0:00:00 (xfer#35, to-check=16527/51354)
folder-to-backup/TestProject/TestProject.xcodeproj/xcuserdata/simon.xcuserdatad/xcschemes/xcschememanagement.plist
         483 100%   27.75kB/s    0:00:00 (xfer#36, to-check=16526/51354)
folder-to-backup/TestProject/TestProject/
folder-to-backup/TestProject/TestProject/AppDelegate.swift
        2172 100%  117.84kB/s    0:00:00 (xfer#37, to-check=16524/51354)
folder-to-backup/TestProject/TestProject/Info.plist
        1442 100%   78.23kB/s    0:00:00 (xfer#38, to-check=16523/51354)
folder-to-backup/TestProject/TestProject/ViewController.swift
         505 100%   27.40kB/s    0:00:00 (xfer#39, to-check=16522/51354)
folder-to-backup/TestProject/TestProject/Assets.xcassets/
folder-to-backup/TestProject/TestProject/Assets.xcassets/AppIcon.appiconset/
folder-to-backup/TestProject/TestProject/Assets.xcassets/AppIcon.appiconset/Contents.json
        1077 100%   58.43kB/s    0:00:00 (xfer#40, to-check=16519/51354)
folder-to-backup/TestProject/TestProject/Base.lproj/
folder-to-backup/TestProject/TestProject/Base.lproj/LaunchScreen.storyboard
        1740 100%   94.40kB/s    0:00:00 (xfer#41, to-check=16517/51354)
folder-to-backup/TestProject/TestProject/Base.lproj/Main.storyboard
        1695 100%   91.96kB/s    0:00:00 (xfer#42, to-check=16516/51354)

sent 1243970 bytes  received 1220 bytes  75466.06 bytes/sec
total size is 10693902652  speedup is 8588.17

Update with no files to upload

localhost:~ local-account$ rsync -a -e  'ssh -p 22 ' --progress -P /Users/local-account/folder-to-backup [email protected]:/backup/the-computer/
[email protected]'s password: 
building file list ... 
51354 files to consider

sent 1198459 bytes  received 20 bytes  82653.72 bytes/sec
total size is 10693902652  speedup is 8922.90

Backup is easy..

rsync -a -e  'ssh -p 22 ' --progress -P /Users/local-account/folder-to-backup [email protected]:/backup/the-computer/

If you want incremental and full backups try Duplicity.

Hope this helps.

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.7 Duplicity

Filed Under: Advice, AWS, Backup, Cloud, Development, Digital Ocean, Domain, Firewall, MySQL, Networking, Security, Share, Transfer, Ubuntu, VM, Vultr Tagged With: Backup, bash script, rsync, send email, server

What is you backup plan

March 3, 2016 by Simon Fearby

What would you do right now if the computer or device you were reading this on died? Do you have adequate backups and do you keep your most important files safe?

I wrote an article back in 2013 about good backup plans and I used to spend 3 hours every month burning backup DVD’s and adding them to an indexed catalogue. I did not trust Apple iCloud to back up my photos and have had my fair share of lost photos. I came up with a plan for managing and sorting photos from multiple digital devices that put me in charge of copying photos from devices, backing up high res photos and shrinking photos for local use.

Fast forward 3 years and I am now capturing insane amounts of digital photos and videos and I am no longer backing up every month due to the massive amounts of data to now what seems like the small capacity of DVD’s. I am instead throwing all of my new files into an “Incoming” folder to be processed at a later date. This was fine as I assumed the data would be safe.

Murphy’s Law: Anything that can go wrong will go wrong.

On a recent hot summers day, I was performing maintenance on my primary computer (removing dust from the processor fan) and one of the hard drives dropped 30 centimetres to the ground. I thought that all would be fine and the data would be safe as the hard drive was a good quality one. Nope, the drive is dead, the drive was corrupted and fails to show files. The drive spins up ok and does not sound like anything is broken inside. That drive had my last 3 years of photos on it.

How can I prevent this in future?

This happened because I failed to allocate time to sort and backup files and I assumed that Murphy’s Law would not apply to me.

My traditional golden rules of backup:

  1. Backup to Three different locations.
  2. Two of the media need to be different.
  3. One of the locations needs to be offsite.

Simon’s new rules of backup:

  1. Backup to Four different locations (g DropBox, Time Machine, Backblaze, Google Drive, NAS).
  2. Three of the backup media need to be different (g Cloud, DVD, NAS).
  3. Two of the locations need to be offsite (g Dropbox, Backblaze).
  4. One of the backups needs to be automated (e.g Dropbox, Backblaze, Time Capsule).
  5. Zero worries

Why so many backups you may ask? Each backup method is best for different scenarios. Dropbox is good for recently changing files, Cloud backups are good for everything and local backups are good for when cloud providers stuff up or to cover major infrastructure damage/failure overseas.

The ultimate local backup solution is a Drobo 5N 5 NAS from https://www.pccasegear.com/ . Simply insert hard drives (up to 5 3.5″/64TB) plug it into a router or switch and you are good to go. Read more here. Even my new NBN router has a slot for a UBS stick to be inserted (see on the right below) allowing me to create a local WiFi file share.

Router

Simple Cloud Backup Options:

Dropbox is the simplest and safest method of syncing and storing files offsite. You can store 2GB of files for free or 1TB (10,000 GB) for $11.58 AUD a month). Dropbox also works on mobile and tablet device too so it is a no-brainer. Encryption options are not baked in so it is not as secure. Install Dropbox at home and work and share files between locations via Dropbox.

Use this link ( https://db.tt/iZmBvErC ) to signup at Dropbox and get a bonus 500MB (2,500MB total) for free.

Backblaze.com was my favourite cloud storage provider for Windows or Mac that offers encrypted backups. Use this link to get a month free https://secure.backblaze.com/r/017nsm. It costs $5 a month and there are virtually no limits on the data you can store. My laptop has 90GB of data stored in Backblaze and it is all encrypted. Back blaze care about backup so much they have open-sourced their backup enclosure designs: https://www.backblaze.com/blog/backblaze-storage-pod-4/

Use this link to get a month’s free cloud backup to give it a try: https://secure.backblaze.com/r/017nsm

Advanced Cloud Backup:

If you do not trust cloud providers to store your data and you want to have offsite backups on the internet you can buy your own server online and manage your own files encryption and sync methods.

Digital Ocean allows you to create and store your own files in the cloud from $5 a month (but you seed to set it up). Use this link and get a free server for 1 x month with 20GB storage. You can also create a 50GB cloud repository using a Vultr server. I even back up my servers to another server automatically.

Manual Backup

Performing a manual backup by plugging in a removable hard drive feels good but chances are you will not do it often enough. Most of the time simply highlighting all files and copy-paste it to the new drive fails to be effective because files are often in use or access is denied to them by the operating system. Manually copying all files every month talks time and copying many smaller files over and over again will increase the wear on your drive and fragment your drive.

If you are working on important documents then do save as to a new filename frequently (and copy those to offsite destinations), never trust computers.

Manual backups are ok but they are only as good as the data you copy and the frequency you do it.

Network Attached Storage (NAS)

The main downside of cloud backup is it can take weeks if not months to get a full backup into the cloud. That’s where NAS (Network Attached Storage) comes into save the day.

A NAS is a mini-computer that sits on your network (wired or wireless) and allows you to store files quickly with or without redundancy.

The ultimate local backup solution is a Drobo 5N 5 NAS from https://www.pccasegear.com/ . Simply insert hard drives (up to 5 3.5″/64TB) plug it into a router or switch and you are good to go. Read more here.

You can choose a NAS with different drive capacities and you can allocate more space to redundancy (referred to as RAID read about raid here https://www.cloudwards.net/what-is-raid/ ). You would want to at least mirror (duplicate) data (RAID 1) so if one drive fails you have a copy of files on the other disk.

Windows Advice:

Windows 7 had a nice Backup feature built-in but Microsoft has changed that in Windows 8.1 and 10

https://support.microsoft.com/en-au/help/17143/windows-10-back-up-your-files

I feel Windows approach to automatically create backups of files on the C drive is a bad idea and copies of hundreds of thousands of files only increased the space ( http://www.apple.com/au/shop/product/ME182X/A/airport-time-capsule-3tb?fnode=5f ) used and wears out the drive.

Apple Mac Advice:

Apple OSX comes with a handy feature called Time Machine, if you plug in a large capacity drive OSX will ask if it can use the drive to store backups of your computer. Time Machine does a great job of keeping multiple versions of files and automatically phase out older versions when space on the backup drive gets low. The only catch is you may need to plug in your backup drive every few days/weeks to let Time Capsule do its thing.

Apple sells a wireless Time Capsule for those that do not want to plug in a drive to backup. Time capsules can be a bit tricky to set up if your home or office has a wireless network already but once it is set up it is rock solid.
http://www.apple.com/au/shop/product/ME182X/A/airport-time-capsule-3tb?fnode=5f

Technical Bonus:

Don’t assume new storage devices are more reliable if anything the cut-throat nature of the PC industry is driving down reliability and quality. If someone is telling you otherwise they are lying or are trying to sell you something.

Most larger capacity drives (over 1TB) store data on spinning metal platters and the quality and reliability varies between brand and models. Backblaze has a great write-up on hard drive reliability (based on the experience of tens of thousands of drives): https://www.backblaze.com/blog/hard-drive-failure-rates-q3-2017/

Ever-increasing “data capacity” of devices is delivering devices that are more fragile and ones that dedicate less physical space to actually store every bit of data on your computer. Most Solid State drives store data in shared cells ( MLC ) and data is more prone to being corrupted due to manufacturers increasing consumers risk to stay profitable.

My recovery tool of choice for any dead or deleted SD card, USB key or physical drive is http://www.diydatarecovery.nl as it ignores the file application table and looks at the sectors and tried to build files back from there. The only catch is recovery scans take days and you need a new/empty drive that is bigger than the drive you are scanning to restore files to. Here is my mini-review os DIY DataRecovery.

So by now, you have:

  • Signed up for 2GB free Dropbox using this link to get a bonus 500MB
  • Signed up for a free Backblaze account to get a free months trial using this link
  • Reviewed your local backup options?

Don’t forget to use good filenames

The World’s Best File Naming System (YouTube)

Good luck and happy backing up.

Restoring Files

If you have backup often up then all will be well when disaster strikes, if not you may need to use Restore Software.

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.3 File naming…

Filed Under: Backup Tagged With: Backup, nas

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) HTTPS (6) IoT (9) LetsEncrypt (7) Linux (20) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) PHP (13) Scalability (12) Scalable (14) Security (44) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (44) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT