• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

Backup

Goodbye Dropbox, One Drive, iCloud and Hello Nextcloud private cloud on UpCloud

June 14, 2020 by Simon

I recently came across NextCloud Hub server (free on self-hosted servers) and I wanted to set up my own private cloud server to store my own files.

I wan’t to be able to access my files on Windows, Mac, Android and iOS.

Most of all I want a place in the cloud (that I own) that I can upload my Acronis backup of C Drive as the Backblaze client (read my review of Backblaze here) is a bit slow at uploading a 150GB backup file to the USA.

To create my own Nextcloud server I will need to login to these services.

  • I logged into my Domain Name provider porkbun.com (to ensure I had a domain name)
  • I logged into Cloudflare.com (to manage my DNS for a subdomain (redirected from PorkBun)).
  • I logged into my UpCloud.com account. (to deploy a new virtual machine)

Fyi: If you don’t have a favourite virtual machine provider you can use my referral link to obtain $25 free credit (only if you are new to UpCloud). Every new user who signs up with my referral link will receive a $25 bonus to get started. That’s 5 months free server (1 CPU and 1GB memory Linux server) 

Post Index

  1. NextCloud System Requirements
  2. Creating a new Virtual Machine at UpCloud
  3. Updating Ubuntu
  4. Installing Common Software Packages
  5. Securing SSH with the Google Authentication PAM module
  6. Installing a Firewall
  7. Installing NGINX and DNS
  8. Installing PHP/PHP-FPM
  9. Installing MySQL
  10. Nixstats
  11. CronTab Updates
  12. Misc Security Stuff

1. NextCloud System Requirements

I checked the NextCloud version 18  system requirements and it needs the following to deploy.

  • Ubuntu 18.04 LTS (recommended)
  • MySQL 5.7+ or MariaDB 10.2+ (recommended)
  • Nginx with php-fpm
  • PHP 4 (recommended)

Minimum Memory Requirements 

Nextcloud needs a minimum of 128MB RAM, and they recommend a minimum of 512MB.

I can deploy a server with at least 512MB memory free. The minimum UpCloud server I can deploy comes with 1GB of memory for $5 a month.

Time to create a new server.

2. Creating a new Virtual Machine at UpCloud

I logged into UpCloud and  clicked “Deploy Server“

Deploy Server Button

I selected Singapore as the place to deploy my new server (as it was closest to me here in Australia). UpCloud does not have servers here in Australia yet.

I chose to deploy a server in Singapore

I checked https://wondernetwork.com/pings/ to ensure Singapore is the fastest location near.  My server https://fearby.com is located in Chicago as it’s closer to my average readers and search engines.

Ping Speeds

I would like my NextCloud server to be as fast as possible to me. Singapore is the faster UpCloud datacenter near me.

I selected a server with 1 CPU Core, 1GB of Memory, 25GB of storage and 1TB of network traffic. I will add a 500GB drive to this server for additional storage.

If the server needs more resources I will upgrade it later.

Server tier's $5 month to $640 a month

The only downside of a $5/m server is the 1TB network quota. If I overuse the network (downloads) I will get an extra charge. 

I reached out to the support to verify the costs if I go over my quota.

Long answered the question.

My question to UpCloud chat support.

Q1) With a $5/m server with 1TB quota what is the over charge costs if I go over 1TB
Q2) Is 1TB quota up and down or just down?

Prompt Answer

Hi Simon,

Good to speak to you again.

A1) Only Simple plans include monthly allowance of outgoing network traffic. After the allowance, the cost is $0.01/GB. It was a lot higher, but we reduced it to make it more competitive.

A2) The quota is for outgoing network traffic from your servers, all incoming and private traffic between your UpCloud servers is free of charge.
Regards,
~Long Lam

I hope this is helpful, let us know if you have any further questions. 

Based on this information if I use all of my 1TB Monthly quota downloading files and I download and extra 150GB (e.g A 150GB Acronis backup image) it will cost $1.5 extra. That’s not bad.

UpCloud Chat Support

Before I selected a server type (Simple or Flexible) or storage type  (MAX IOPS or HDD) I jumped onto the UpCloud chat and asked a few questions.

Q1) Hello, When deploying a server is there a cost difference between MAX IOPS and HDD storage? I am looking at a 500GB drive

A1) Storage (MaxIOPS), per GB $0.00031/ hourly $0.22/ monthly, Storage (HDD), per GB $0.000078/hourly  $0.06 / monthly 

Q2) What’s the difference between Simple and Flexible?

A2) Flexible will/turn out more expensive depending on your use case, generally, it is more suited for short term deployments.

> With our flexible plans, you decide yourself how much CPU, memory and block storage your cloud server is allocated. This gives you incredible flexibility and allows you to fully customise your cloud server according to your specific needs.
 
>Do also note when flexible plans are shutdown we only charge you for allocated storages and IPv4. Whereas in simple plans, it will be charged fully even when shutdown.
 
> Our simple plans are billed by the hour, up to a limit of 672 hours per month. Should you decide to use your fewer hours, you will only be billed for the hours you actually used.
Question 1 to UpCloud chat

UpCloud has very responsive and helpful chat staff.  I never had this level of help with Vultr, Digital Ocean or AWS.

Question 2 to UpCloud

After I chatted with UpCloud support I decided to deploy a simple (Ubuntu 18.04) Server with 1 CPU Core, 1TB network traffic, 1GB of memory, 25GB system drive and an extra 500GB storage device.

When you create a server you can add an extra storage device. Nice.

Add a new device to the main storage device.

When adding an extra storage device you can choose faster MaxIOPS storage or slower HDD based storage. 

I will choose HDD storage as it will be cheaper for a 500GB device.

Second storage MaxIOPS or HDD storage

I created a 500GB storage device for a Nextcloud data drive.

You can create up to 2TB storage devices with UpCloud.

Name of the second storage device

I selected Ubuntu 18.04 LTS as the operating system.

I chose Ubuntu as the operating system

I configured a login method as “Only SSH Keys” as I have already added my SSH key with a passphrase.

Login method SSH Keys only

I selected my SSH key.

If you have not previously added an SSH Key to UpCloud then click Add new. Read more here.

I selected an Initialisation script I previously created (that just outputs a “Hello World” to a text file). One day I will create an Ansible or Terraform script to set up a server.

Select SSH Key and choose an init script

I clicked Deploy

Fyi: If you don’t have a favourite virtual machine provider you can use my referral link to obtain $25 free credit (only if you are new to UpCloud). Every new user who signs up with my referral link will receive a $25 bonus to get started. That’s 5 months free server (1 CPU and 1GB memory Linux server).

I entered my desired hostname 

Deploying a server at UpCloud

I had a notification that the UpCloud Deploy is being deployed..

Deploy Underway

I could see in my UpCloud dashboard that the server was being deployed.

List of all my servers at UpCloud

Server deploy is underway

Wow that took a whole minute to deploy a 525GB Server.

Deploy log said it took 1 minute to deploy

Wow UpCloud are fast

Configuring the server with Putty

Now it is time to connect to the Ubuntu Servers CLI and configure the server.  I grabbed the IP address that was listed at UpCloud.

I opened Putty  and added the IP address for the server.

New Putty connection

Under the Auth section in Putty I added the path to my SSH Private Key (the same one that configured in the new server)

Putty add ppk file

I saved the connection and clicked Open. I clicked Yes to the SSH fingerprint when I verified it was correct.

SSH Connect Verity

I now had root access to my new server.

Default login

Time to update Ubuntu.

3. Updating Ubuntu

I ran this command to update Ubuntu.

sudo apt-get update && sudo apt-get upgrade

Confirming the 2x storage disks

I ran this command to verify I had the 2 storage devices I selected at server deploy.

sudo lsblk |grep disk
vda    252:0    0   25G  0 disk
vdb    252:16   0  500G  0 disk

Yes, I have a 25GB disk and a 500GB disk

4. Installing Common Software Packages

I installed these packages

sudo apt-get install htop
sudo apt-get install lshw
sudo apt-get install ufw
sudo apt-get install ncdu
sudo apt-get install nmap
sudo apt-get install iozone3
sudo apt install pydf
sudo apt install mc
sudo apt install nnn

5. Securing SSH with the Google Authentication PAM module

Before I carry on any further I need to enable hardware 2FA login protections to all SSH logins. I will follow the guide I created here (Setup two factor authenticator protection at login on Ubuntu or Debian).

Warning: Take a backup of your server first. If you set this up wrong say bye-bye to your server. If I lose my YubiCo YubiKey and forget my backup codes I will have a hard time getting back in.

I will force all SSH logins to require my Hardware YubiCo YubiKey to be inserted (to generate a temporary One Time Password (OTP)).

You don’t need a YubiCo YubiKey, a generic software authentication app is OK but I prefer hardware devices as they are more secure.

YubiKey In USB Port Photo

I set the timezone to match Australia/Sydney. If I enabled a 2FA (OTP) at login with a different timezone than my connecting machine I would never be able to login to my server as my server and local PC need to be in the same timezone.

I ran this command to set the time in Ubuntu.

pkg-reconfigure tzdata

I then checked the time

sudo hwclock --show
2020-05-31 23:17:02.873751+1000

I installed the Google Authentication PAM Module (read more)

sudo apt install libpam-google-authenticator

I ran this command to configure the Google PAM Module

google-authenticator

I was presented with these questions

Do you want authentication tokens to be time-based (y/n) y

I was presented with a secret key, verification code and backup codes (I saved these somewhere safe)

Do you want me to update your “/root/.google_authenticator” file? (y/n) y

Do you want to disallow multiple uses of the same authentication
token? This restricts you to one login about every 30s, but it increases
your chances to notice or even prevent man-in-the-middle attacks (y/n) y

By default, a new token is generated every 30 seconds by the mobile app.
In order to compensate for possible time-skew between the client and the server, we allow an extra token before and after the current time. This allows for a time skew of up to 30 seconds between authentication server and client. If you experience problems with poor time synchronization, you can increase the window from its default size of 3 permitted codes (one previous code, the current code, the next code) to 17 permitted codes (the 8 previous codes, the current
code, and the 8 next codes). This will permit for a time skew of up to 4 minutes between client and server.

Do you want to do so? (y/n) y

If the computer that you are logging into isn’t hardened against brute-force
login attempts, you can enable rate-limiting for the authentication module.
By default, this limits attackers to no more than 3 login attempts every 30s.

Do you want to enable rate-limiting? (y/n) y

I can review all config values later with this command

sudo nano ~/.google_authenticator

Now I will enable 2FA at login by editing this file

sudo nano /etc/pam.d/sshd

I searched for “@include common-auth” then added this line after it.

auth required pam_google_authenticator.so

I then comment out the following line (this is the most important step, this forces 2FA)

#@include common-auth

Picture of my /etc/pam.d/sshd changes

pam chnages

I saved the file /etc/pam.d/sshd 

Now I can enable the PAM Module by editing this file

sudo nano /etc/ssh/sshd_config

I searched for

ChallengeResponseAuthentication

And change the value to “yes”

I ensured the following line exists

UsePAM yes

I added this line then saved /etc/ssh/sshd_config

AuthenticationMethods publickey,password publickey,keyboard-interactive

Now I edited /etc/pam.d/common-auth

sudo nano /etc/pam.d/common-auth

I added the following line before the line that says “auth [success=1 default=ignore] pam_unix.so nullok_secure”

auth required pam_google_authenticator.so

Now I can restart the SSH Service and test the 

/etc/init.d/ssh restart
[ ok ] Restarting ssh (via systemctl): ssh.service.

I restarted my putty session and reconnected to my server and I was prompted for the password for my private key and the randomly generated one-time password that was linked to my YubiCo YubiKey. Nice

Now I need to whitelist my SSH port to select IP’s.

6. Installing a Firewall

I installed the UFW firewall by typing this command

sudo apt-get install ufw

I configured UFW to rate limit SSH logins by typing this command

sudo ufw limit ssh comment 'Rate limit hit for openssh server'
Rules updated
Rules updated (v6)

I configured some common ports

sudo ufw allow ssh/tcp
sudo ufw logging on
sudo ufw allow http
sudo ufw allow https
sudo ufw allow 22
sudo ufw allow 53
sudo ufw allow 80
sudo ufw allow 443
sudo ufw allow 873

I added Cloudflare firewall rules (as my domain is behind their firewall and I will remove all direct IP access to my server later)

sudo ufw allow from 173.245.48.0/20
sudo ufw allow from 103.21.244.0/22
sudo ufw allow from 103.22.200.0/22
sudo ufw allow from 103.31.4.0/22
sudo ufw allow from 141.101.64.0/18
sudo ufw allow from 108.162.192.0/18
sudo ufw allow from 190.93.240.0/20
sudo ufw allow from 188.114.96.0/20
sudo ufw allow from 197.234.240.0/22
sudo ufw allow from 198.41.128.0/17
sudo ufw allow from 162.158.0.0/15
sudo ufw allow from 104.16.0.0/12
sudo ufw allow from 172.64.0.0/13
sudo ufw allow from 2400:cb00::/32
sudo ufw allow from 2405:8100::/32
sudo ufw allow from 2405:b500::/32
sudo ufw allow from 2606:4700::/32
sudo ufw allow from 2803:f800::/32
sudo ufw allow from 2c0f:f248::/32
sudo ufw allow from 2a06:98c0::/29

I added appropriate Whitelisted IP’s that can connect to Port 22 (SSH), removed blanket port 22 access and I configured my firewall to allow 91 incoming and outgoing rules (this is a secret)

I reloaded and enabled the firewall.

sudo ufw reload
sudo ufw disable
sudo ufw enable

7. Installing NGINX and DNS

I update Ubuntu again

sudo apt-get update && sudo apt-get upgrade

I installed Nginx

sudo apt-get install nginx

I edited my NGINX config and I change the default www folder location. 

I also configured the log file location, mime types, max body size, gzip, default ports, ssl cert paths, security headers, default page, server name, sensitive file block rules, dns server, cache headers etc.

Read more to here to configure Nginx etc.

Fyi: Nginx config file locations

sudo nano /etc/nginx/nginx.conf
sudo nano /etc/nginx/sites-available/default

I typed my servers IP address into a web browser

Nginx installed

I created an index.html file in the www folder and added “Hello World” to the file.

If I type my server’s IP address into a browser I can see this file.

My DNS is with Cloud flare so I logged in and added 2 DNS entries (IPv4 and IPv6) that direct traffic my new server IP(s) for this subdomain. To obtain the IP addresses I logged into UpCloud and clicked my server then clicked Network and noted my IPv4 and IPv6 addresses.

I then went to Cloudflare and added a DNS record for IPv4 and IPv6 pointing to my servers IP(s). I enabled Cloudflare Proxying to allow Cloud flare to try and hide the IP of the server.  I then configured my firewall to block access to the IP except via Cloudflare and my whitelist.

I then checked for worldwide DNS propagation with https://www.whatsmydns.net/. After 3 minutes my DNS changes were all around the world. Thanks, Cloudflare.

I tried loading my site but CLiudflare said it was down.

Site wont load.

I created a new HTTPS certificate at Cloud flare just to be sure and added it to my sites.

Generated  new SSL cert

After investigating further I found this was because my primary website has a “Strict-Transport-Security header and I had enabled Full (Strict) SSL/TLS Encryption. I changed this to Full at Cloudflare.

Cloudflare HTTPS section

My site was now working.

SIte works

8. Installing PHP/PHP-FPM

To Install PHP 7.4 I ran this command to be able to get the latest version of PHP

sudo apt-get update
sudo apt -y install software-properties-common
sudo add-apt-repository ppa:ondrej/php
sudo apt-get update

I installed PHP 7.4 with this command

sudo apt -y install php7.4

I checked that PHP is installed by running 

php -v
PHP 7.4.6 (cli) (built: May 14 2020 10:02:44) ( NTS )
Copyright (c) The PHP Group
Zend Engine v3.4.0, Copyright (c) Zend Technologies
    with Zend OPcache v7.4.6, Copyright (c), by Zend Technologies

I setup some PHP Modules

sudo apt install php7.4-common php7.4-mysql php7.4-xml php7.4-xmlrpc php7.4-curl php7.4-gd php7.4-imagick php7.4-cli php7.4-dev php7.4-imap php7.4-mbstring php7.4-soap php7.4-zip php7.4-bcmath php7.4-tidy 

I noticed apache2 installed (and broke my Nginx)  so I uninstalled it.

 sudo apt-get remove apache2

I also blocked apache from installing again

apt-mark hold apache2
apache2 set on hold.

I checked to make sure Apache was blocked from installing

apt-mark hold apache*

apache2 was already set on hold.
apache2-bin set on hold.
apache2-utils set on hold.
apache2-data set on hold.
apache2-doc set on hold.
apache2-suexec-pristine set on hold.
apache2-suexec-custom set on hold.
apache2-dbg set on hold.
apache2-dev set on hold.
apache2-ssl-dev set on hold.
apachedex set on hold.
apacheds set on hold.
apachetop set on hold.

Now I will install PHP-FPM.

FPM is a process manager to manage FastCGI in PHP

sudo apt-get install php7.4-fpm

I checked the status of the PHP FPM service with

sudo service php7.4-fpm status

Output

php7.4-fpm.service - The PHP 7.4 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php7.4-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2020-06-06 21:34:31 AEST; 1min 54s ago
     Docs: man:php-fpm7.4(8)
  Process: 7767 ExecStopPost=/usr/lib/php/php-fpm-socket-helper remove /run/php/php-fpm.sock /etc/php/7.4/fpm/pool.d/www.conf 74 (code=exited, status=0/SUCCESS)
  Process: 7772 ExecStartPost=/usr/lib/php/php-fpm-socket-helper install /run/php/php-fpm.sock /etc/php/7.4/fpm/pool.d/www.conf 74 (code=exited, status=0/SUCCESS)
 Main PID: 7769 (php-fpm7.4)
   Status: "Processes active: 0, idle: 2, Requests: 0, slow: 0, Traffic: 0req/sec"
    Tasks: 3 (limit: 1147)
   CGroup: /system.slice/php7.4-fpm.service
           |-7769 php-fpm: master process (/etc/php/7.4/fpm/php-fpm.conf)
           |-7770 php-fpm: pool www
           `-7771 php-fpm: pool www

I might add some PHP child workers if I add more CPU’s to this server later

I edited my php.ini

sudo nano /etc/php/7.4/fpm/php.ini

I made these changes to php.ini

file_uploads = On
allow_url_fopen = On
memory_limit = 512M
post_max_size = 50M
upload_max_filesize = 50M
cgi.fix_pathinfo = 0
max_execution_time = 360
date.timezone = Australia/Sydney

I added read this page (Nginx Configuration) and edited my /etc/nginx/sites-enabled/default

I tested and reloaded the Nginx config and restarted NGINX and PHP

nginx -t
nginx -s reload

sudo systemctl restart nginx.service
sudo systemctl restart php7.4-fpm

sudo systemctl status nginx.service
sudo systemctl status php7.4-fpm

To test PHP FPM I created a php file in my website root and added the following text

<?php phpinfo( ); ?>

I loaded this file in a browser and I confirmed that PHP-FPM was installed.

The test was ok (I deleted this test file), I deleted the index.html and created an index.php file

PHP-FPM test ok

9. Installing MySQL

To install MySQL I ran the following command

fyi: All usernames and database names are for example only.

sudo apt install mysql-server

I configured MySQL With this command

sudo mysql_secure_installation
Securing the MySQL server deployment.

Connecting to MySQL using a blank password.

...
Would you like to setup VALIDATE PASSWORD plugin?
y


There are three levels of password validation policy:
STRONG

Please set the password for root here.
New password:
**************************************************

Re-enter new password:
**************************************************

Estimated strength of the password: 100

Do you wish to continue with the password provided?
y


Remove anonymous users?
y

Disallow root login remotely?
y

Remove test database and access to it?
y

Reload privilege tables now?
y

Now to test MySQL I will login to it

sudo mysql -u root -p
************************************************************

Now I ran the following to create a database for Nextcloud

mysql> CREATE DATABASE databasename CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci;
Query OK, 1 row affected (0.00 sec)

I verified the database was created

mysql> SHOW DATABASES;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| databasename       |
| performance_schema |
| sys                |
+--------------------+
5 rows in set (0.00 sec)

I created a database user 

mysql> CREATE USER 'username'@'localhost' IDENTIFIED BY '************************************';
Query OK, 0 rows affected (0.00 sec)

I verified the use was created with this command

mysql> SELECT User,Host FROM mysql.user;
+------------------+-----------+
| User             | Host      |
+------------------+-----------+
| **************** | localhost |
| **************** | localhost |
| **************** | localhost |
| username         | localhost |
| **************** | localhost |
+------------------+-----------+
5 rows in set (0.00 sec)

I set permissions to add the user to the database

mysql> GRANT ALL PRIVILEGES ON `databasename`.* TO 'username'@'localhost';
Query OK, 0 rows affected (0.00 sec)

I verified the permissions with this command

mysql> SHOW GRANTS FOR 'username'@'localhost';
+--------------------------------------------------------------------------+
| Grants for [email protected]                                      |
+--------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'username'@'localhost'                       |
| GRANT ALL PRIVILEGES ON `databasename`.* TO 'username'@'localhost' |
+--------------------------------------------------------------------------+
2 rows in set (0.00 sec)

Finally I flushed permissions

mysql> FLUSH PRIVILEGES;

Now the databases is ready for Nextcloud

10. Nixstats

If you do not know what Nixstat’s is check out my post here Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc

I logged into Nixstats and click Add Server. I ran the provided install command.

wget -q -N --no-check-certificate https://nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ################## ##########################

Todo: Configure Nixstats PHP-FPM and NGINX Reporting (work in progress). My firewall rules are too tight for this install.

Handy Links

  • Monitoring Nginx with Nixstats
  • https://help.nixstats.com/en/article/monitoring-php-fpm-1tlyur6/

11. CronTab Updates

I created a update.sh file that I can call from a crontab entry to update Ubuntu and other software every xx hours.

I added this to my crontab.

12. Misc Security Stuff

I made sure my firewall only allowed traffic to my server was from Cloudflare IP’s and Whitelisted IP’s

Cloud flare IP’s can be found here.

https://www.cloudflare.com/ips-v4/
https://www.cloudflare.com/ips-v6/

At the time of writing the IP’s are 

173.245.48.0/20
103.21.244.0/22
103.22.200.0/22
103.31.4.0/22
141.101.64.0/18
108.162.192.0/18
190.93.240.0/20
188.114.96.0/20
197.234.240.0/22
198.41.128.0/17
162.158.0.0/15
104.16.0.0/12
172.64.0.0/13
131.0.72.0/22
2400:cb00::/32
2606:4700::/32
2803:f800::/32
2405:b500::/32
2405:8100::/32
2a06:98c0::/29
2c0f:f248::/32

I blocked access to my webserver (port 80 and 443) to anyone but Cloudflare.

I whitelisted DNS traffic to only Up Cloud. Thanks, Lon.

Up Cloud support is awesome.

UpCloud Support

Installing NextCloud

Finally I can Install Nextcloud, I navigated to https://nextcloud.com/install/ and clicked Download for Server

Download Nextcloud

I will use the Web installer to Install Nextcloud.

Web Installer Tab

Nextcloud web installer instructions

Setup Instructions

Snip about the Nextcloud Installer from the download page

The Web Installer is the easiest way to install Nextcloud on a web space. It checks the dependencies, downloads Nextcloud from the official server, unpacks it with the right permissions and the right user account. Finally, you will be redirected to the Nextcloud installer.

1) Right-click here and save the file to your computer
2) Upload setup-nextcloud.php to your web space
3) Point your web browser to setup-nextcloud.php on your webspace
4) Follow the instructions and configure Nextcloud
5) Login to your newly created Nextcloud instance!

You can find further instructions in the Nextcloud Admin Manual.

Note that the installer uses the same Nextcloud version as available for the built in updater in Nextcloud. After a major release it can take up to a month before it becomes available through the web installer and the updater. This is done to spread the deployment of new major releases out over time.

I used WinSCP to upload the setup-nextcloud.php to my Nginx web root  folder

WinSCP uploading

I loaded the setup-nextcloud.php file from, my web browser.

Loading setup-nextcloud.php

I entered “.” to install Nextcloud to the website root.

Install Next cloud to .

There is no way Nextcloud installed in 2 seconds, I checked the size of the disk usage in my website root.

sudo du -hs /web-root
313M

Nextcloud took about 10 seconds to download 313MB onto my UpCloud Server.

Fyi: I installed the SpeedTest CLI app and ran a benchmark and UpCloud Chicago can download as 937Mbps/sec and UpCloud Singapore can download at 717Mbps/sec. 

Nextcloud is installed.

Now I need to enter the data root folder for Nextcloud . I installed lswh to be able to see my 500GB disk.

sudo apt-get install lshw

I ran the following to see my disks

sudo lshw -class disk -short
H/W path        Device     Class      Description
=================================================
**********      /dev/vda   disk       26GB Virtual I/O device
**********      /dev/vdb   disk       536GB Virtual I/O device

I formatted my disk

sudo mkfs.ext4 /dev/vdb

I created a new folder under mount to connect to the partition. The folder name is a made-up sample

sudo mkdir -p /mnt/foldername

I mounted the partition to the folder

sudo mount /dev/vdb /mnt/foldername

I made sure Nginx can access the folder

sudo chown -R www-data:www-data /mnt/foldername

I changed to the partition mount

cd /mnt/foldername

I created a test 490GB file

fallocate -l 490G test.file

I checked the file

ls -al
-rw-r--r-- 1 username useername     526133493760 Jun  9 19:38 test.file

I deleted this test file and set this mount point as the data file in Nextcloud setup.

I added a new Nextcloud admin username and password,  mount folder for Nextcloud data folder, the SQL database user/password/database name and host and clicked Finish Setup

Nextcloud details

Nextcloud was setup.

Misc Setup

I ran the /settings/admin/overview report to see if I needed to perform andy final setup steps. I have a few missing php modules and a few optimisation tasks that need resolving.

Links to resolve.

  • Path Fixes
  • PHP Memory Limit
  • PHP Server Tuning

Nextcloud External Security Scan

I loaded https://scan.nextcloud.com/ to perform a external security scan.

Security Scan

Scan Results

All good so far.

Adding Two-Factor Authentication (YubiKeys)

I noticed in the Nextcloud security setting page I can setup a YubiKey as a pass-wordless  login device.

Web AuthN device

This would allow me to insert my YubiKey to login automatically

Auto login.

I added my YubiKey and gave it a name.

Name a YubiKey

The password-less login method is a bit insecure as anyone that has my YubiKey can access my site.

I think I will set up a Two-Factor Authentication/OTP login method and link that to my YubiKey.  I visited the /settings/apps/security page and installed the Two-Factor TOTP Provider app.

Install the OTP App
Install; the OTL app

I clicked the checkbox next to TOTP

Enable TOPT

The app generated a QR code that my YubiCo Authentication App can use to link to Nectcloud

I verified the QR scan and entered the 6 number verification code from my YubiCo Authenticator app

Scan the QR Code

Two Factor logins are now enabled.

2FA Enabled/

Now after I log in I have to enter a temporary 6 digit number that is only valid for 30 seconds (and only after entering my YubiCo YubiKey into my PC and entering its password)

2FA enabled at login/

Nice

Nextcloud Overview

I logged into Nextcloud and was greeted with a wizard.

Welcome screen

The sample images in the welcome screen are a bit small.

welcome screen summary

I can add native apps to Windows, Mac, iOS and Android or I can log in via the web page.

App downalod options

Pointers to the manual, community help and forums.

Help options

Main screen is clean.

Main Screen

A user context Menu is linked in the top right.

Drop down menu.

I setup email alerts (I allowed outgoing ports in my firewall)

sudo ufw allow out 465/tcp
sudo ufw allow out 465/udb

I used my GSuite account to send emails.

email settings

Syncing Files from my PC to Nextcloud

I tried uploading my 150GB Acronis Backup image file to Nextcloud by the web interface but this will fail for sure, this will take many hours.

Acronis image uploading.

I decided to configure Acronis True Image to split backups into 100MB chunks.

100GB file sizes

I created another Acronis image of my Windows Drive.

Nextcloud Windows App

I visited https://nextcloud.com/install/ and installed the Nextcloud Windows app to sync files.

Download windows app

I clicked Windows

Windows Download

Click Next

Click Next

Click Next

Click Next

Click Install

Click Install

Nextcloud sync app is now installing

Installing Wizard

Next cloud sync is now installed.

Run Nextcloud

Click Log in

Login Screen

Enter your Nextcloud server https address and click Next

Enter https server

A web browser login screen appeared and I logged in 

Login to the web app.

After I logged in Nextcloud sync was connected

Sync Connected

I was prompted to sync everything online to my local PC or choose folders to Sync .

Sync File dialog.

All files that were in Nextcloud synced down (that I selected)

Nextcloud sync

I set Nextcloud to start at Windows start.

Start at startup.

I reviewed Download and Upload limits

I decided to add my U:\AcronisBackup folder to my Nextcloud server.

U:\AcronisBackup added rto sync

I was asked to add this to a remote Nextcloud folder.

add to destination folder dialog

Files were backing up.

I has 150GB of Acronis backup files backing up.

I could see each 100MB section of the Acronis Backup appearing in the Nextcloud web app.

Nextcloud Web site

I noticed that the raw file system list of files was about 30 seconds ahead of the web list.

ls -al list of the file system

I had an Alert from my Acronis Backup software that new backup files were downloading.

The Acronis backup folder started backing up but I noticed it was redownloading to a new folder.  I don’t want this.

I allowed Nextcloud to access backup files

I paused the Nextloud Sync and my 150GB Backup was re-downloading to a new folder.

pause backup

It looks like U:\AcronisBackup was backing up then downloading to U:\Nextcloud\Simon\AcronisBackup.

File dialog

I moved my Acronis backup from U:\AcronisBackup to U:\Nextcloud\Simon\ZENigma (ZENnigma is the name of my PC)

I moved my 150GB backup files into Nextcloud folder/

I deleted the old sync of U:\AcronisBackup and started the Nextcloud Sync again

Sync restarted

Now my Acronis backup (150GB) was backing up to Nextcloud.

Backup working

It took 24 hours to backup 150GB from my PC to my server in Singapore.

I can see a handy summary of synced files and disk space used/free.

Done

I can control the sync with a System Tray App.

Sys Tray APp

Nextcloud Conclusion

Pros

  • Free
  • Works well.
  • I have an offsite location for backups and an area for file sharing with my family
  • Faster than Backblaze and Dropbox

Cons

  • Needs better Hardware 2FA support
  • Some Nextcloud web pages are not mobile-friendly (e.g add new user)
  • Needs better post install security checks
  • Web view of files could be updated more often, there is as 30-second delay between the web list of files and a CLI list in Putty of /mnt/foldername/username/files/

Troubleshooting

NGINX website is not loading

Check to see if a package has downloaded apache (this will take out Nginx).

Also, make sure you have set permissions on the folder that holds your SSL Certificates and allow your Nginx www-data user read access.

sudo chown -R www-data:www-data /etc/nginx/https-cert/

Deleting a MySQL Database

I had an issue where Nextcloud did not like the database I created so I ran the following to revoke the database users permissions, remove the user and I deleted the database.

Command to revoke the users MySQL permissions

sudo mysql -u root -p
*************************************
mysql> REVOKE ALL PRIVILEGES, GRANT OPTION FROM 'databaseusername'@'localhost';

Delete the MySQL user

sudo mysql -u root -p
Enter password: *************************************
mysql> DROP USER 'databaseusername'@'localhost';

I reset flushed permissions

sudo mysql -u root -p
Enter password: *************************************
mysql> 
FLUSH PRIVILEGES;

To delete the database run the following.

mysqladmin -u root -p drop databasename
Enter password: *************************************
Dropping the database is potentially a very bad thing to do.
Any data stored in the database will be destroyed.

Do you really want to drop the 'databasename' database [y/N] y
Database "databasename" dropped

Thanks for Reading

Fyi: If you don’t have a favourite virtual machine provider you can use my referral link to obtain $25 free credit (only if you are new to UpCloud). Every new user who signs up with my referral link will receive a $25 bonus to get started. That’s 5 months free server (1 CPU and 1GB memory Linux server) 

v1.1

Filed Under: 2nd Factor, Backblaze, Backup, Database, Domain, Google, Nextcloud, Putty, SSH, UpCloud Tagged With: backblaze, Dropbox, Google One, Nextcloud

Finding software that needs updating on Windows using SUMo (Review)

May 30, 2020 by Simon

Recently I blogged about using Acronis True Image 2020 to backup and restore your Windows computer after a friend had to manually reinstall her Windows PC.

Her Windows PC died and she could not recover Windows with a System Restore snapshot or via the retailers systems image recovery process.

I use Back Blaze software to automatically backup my computer but I do not have an up to date collection of needed installation files in case I need to perform a Clean Installation of Windows 10.

I found SUMo in the process of assisting my friend.

SUMo to the Rescue

I found SUMo (Software Update Monitor) from KC Softwares that allows me to analyse my existing installed software and their versions, find and download new versions (installers or setup files) for my software.

I am using the Pro version of SUMo (buy here).

Tip: The trial version will inform you of an update but it will not show you links to obtain the updates easily.

SUMo Trial Limitations

The best feature of SUMo is I can find apps on my PC that need updating.

Also in one click (Right click) you will be directed to the applications official site (official site (or App page on SUMo)) and be able to download the app installer to the new version of the app.

Rigth click on an outdated app in SUMo to get the new aopo.

The Trial version is limited to informing you that an update is available.

SUMo telling your there is an update available

Also updating SUMo to the PRO version streamlines it’s own updates too. You can update SUMo in the PRO version with one click. 🙂

SUMo Pro Scan

After I opening SUMo PRO (I waited for it to find installed apps) then I clicked “Check” to scan all my apps for updates.

SUMo happily went through the list of my installed apps and gathered the applications versions (and checked online if newer versions existed).

SUMO scanning my installed software.

SUMo found 239 installed applications on my PC, 188 were up to date, 53 needed minor (recent updates) and 2 needed major (old) updates.

SUMo report of installed software

Its awesome that in one click and I can see what software needs updating. In the process I was able to see what applications I don’t use (they were old versions) and I uninstalled them

SUMo Settings

I will update my software and to ensure my software does not get out of date I enabled “Load at startup” and “Check for new software at startup” (SUMo).

I did not enable “Scan for new software at startup” but seeing SUMo Start and update will remind me to check for updates if I have time.

SUMo settings.

I also enabled “Show file path” in the SUMo Application list so I can see where the programs are.  I found multiple instances of Python installed all over my PC.

SUMo FIle Path

Getting Application Updates

I right clicked on each of my applications (that needed updating) and clicked “Get Update from…” then “.. from developer’s Site (Direct)“.

If you don’t want to right click you can click the “Get Update” button in the tool bar to get selected applications update. 

This will open a web browser page to the software vendors website.

Get Uopdate

SUMo also has a page available with stats on the application’s known statistics (nice table of versions, beta or stable status and numbers of users).

Updating Software

I was able to right click “Get Update from…” – “.. from developer’s Site (Direct)” for each application in the list that needed an update. I was directed to the software vendors page for each software where I could download the app updates.

Within minutes I had dozens of vendor pages open with links to new versions of my outdated software.

Dozens of site pages with fresh app installs.

Within a short period I had amasses gigabytes of new application installs.

4.5GB of data of new application installs

I installed each of the new updates and then moved the new installs to my B:\Installs folder (in case I have to manually reinstall Windows in the future).

Install Locations

Checking Application Update Status

After I updated my out of date applications I then right clicked on the apps in SUMo and clicked Check to re check the status of the applications 

Check the uodate status

SUMo Right Click Actions

Fyi: SUMo has a number of actions in the right click menu (on an installed Application).

  • Delete: Removes item from list. Will appear again on next scan or manually addition
  • Ignore: Blacklist : this Software/company pair will never appear again, whatever the version, file path, …
  • Skip: Items remains listed but a SPECIFIC version gets no longer notified as update for a given time (1 day / week / month) or forever.

(snip from here)

I had a few items I Deleted from the list or Ignored.

Rigth click menu

My Visual Studio (development) software was reporting an older version of Python (2.7). I updated Visual Studio to the latest version but Python 2.7 remained. I deleted Visual Studio from the list and was prompted  (this is not deleting the application from the hard disk just from the generated list.

Delete confirmation/ box.

After a few hours updating apps

I whittled down the list of my applications that needed updating 

I could sort applications in the list by OK status or apps that needed Minor or Major updates applied.

SUMO app list

All Apps Updated

I can now say that all of my Applications are all up to date.

All apps upto date

Export

I used the export feature in SUMo to build an excel file listing all my applications.

Fyi: SUMo exported this list as a “;” separated file, I opened the file in Sublime Text Editor and replaced “;” with TAB and then pasted this list back into  Excel.

Excel Export of installed apps.

Conclusion

SUMo allowed me to update my installed applications and update my aged cache of application installs I was keeping in case I needed to manually rebuild Windows.

SUMo made this an absolute pleasure to do.

Pros

  • Price: 19.99 Euros (about $33.27 AUD) is a bargain.
  • Simplicity (it just works so well, why does Windows not do this)
  • Saves me time finding out of date apps and update files for an emergency and inevitable manual Windows reinstall.
  • My apps are more secure after updating them

Cons

  • I needed this in my life 10 years ago
  • Would be nice to have an inbuilt help file (instead of email support)

Try the free version here

v 1.0 Initial Post

Filed Under: Backup, Security, Uncategorized, Update, Windows Tagged With: find, software, SUMo, updates

Using Acronis True Image 2020 to backup and restore your Windows computer

May 24, 2020 by Simon

This is a lengthy (but simple) guide on how I get files ready for a clean install of Windows, backup and restore Windows (C Drive and associated hidden Windows partitions) using Acronis True Image 2020).

Backup and Restore Windows Prerequisites

  • You will need to buy a licence of Acronis True Image 2020 to..
    • Backup Windows
    • Restore Windows
  • You will need a second hard disk larger than your Windows drive (to backup to).

Backup Rules

  1. Backup to Three Places.
  2. Two of the media need to be different
  3. One of the backups need to be offsite

Even though I use Back blaze software to automate backup of all files on my PC for $6.60 (AUD) a month. I pay an extra $2 (AUD) a month Backblaze will retain all of my files for up to 1 year. I also back up data to Backblaze Bucket(s) with Duplicati.

With all the horror stories of Windows updates killing machine’s it’s time to return to using Acronis True Image (not a paid endorsement) to start backing up the drive my Windows is on (all partitions)

I remember using Acronis True Image 8 in 2004 and loved it. Time to buy the new version. I loaded https://acronis.com/ (Australia Region) and noticed a cloud backup in the marketing information. I hope they still do local backups line in 2004.

Acronis True Image 2020 Cloud

I loaded the Acronis True Image 2020 link from the downtown menu (Australian link).

Acronis True Image 2020 Download Menu at acronis dot com

$69.99 seems fair,

Acronis True Image 2020 buy now button at acronis dot com

Darn, $69.99 is a yearly subscription. I don’t like subscriptions if I can get away with it.

Acronis True Image 2020 subscription and standalone options  at acronis dot com

I clicked Buy under standard licence.

Acronis True Image 2020 was $83.99 AUD at acronis dot com

$83.99 is a bit high

Saving a dollar

TIP: Make sure you install the Honey Extension (from joinhoney.com) and use the Google Chrome Web Browser before purchasing to get a discount on Acronis True Image 2020.

Honey applied a discount price

The Honey extension applied the coupon so fast I could not get a screenshot  it(so here is a screenshot with American Currency from a different web browser)

Honey applying a coupon in seconds

Now back to my purchase (and country), $33.60 is a nice saving.  

I am paying for this because 16+ years ago I used an older version of Acronis True Image and the free version backed up OK but when I tried to restore from a boot CD I needed to pay?

Honey saved me $33.80

Honey browser plugin saved me $33.6 AUD

I am happy to pay $50.33 for Acronis True Image 2020.

After I purchased Acronis True Image 2020 I was presented with links to download the related files to use the software and to restore.

Download options after I purchased

I downloaded the main Windows Acronis True Image Application for Windows and the Universal Restore program.

The files were about 900MB in total.

I had to download 900MB

While the downloads were downloading I created an Acronis account 

I added my purchased licence to my Account.

I created an account at acronis dot com

I was shown a welcome to Acronis page after creating an account.

Welcome to Acronis page

The downloads finished and I started the AcronisTrueImage2020.exe setup file

Acronis Setup

I clicked Install

Installing Acronis

After the setup completed I was directed to a Quick Start Guide

Quick Start Guide after setup

Starting Acronis True Image 2020

When I loaded Acronis True Image 2020 for the fits time I had to agree to the Licence Agreement

Licence Agreement

A pasted in my just purchased Licence key

Start a trial or activate with a serial number

I already had an account (created minutes earlier in a web browser)  so I clicked Sign In.

Create an account or sign in

Sign In Screen

Sign In Screen

I was presented with a help wizard on startup.

Wizard Help

Now I am ready to create my first backup.

Creating a Backup

Acronis True Image 2020 wanted me to back up all of my PC but it is 8TB, I don’t have an 8TB drive handy to back up to. For the life of Me I could not see how much 8TB would cost per month to backup.

By default Acronis wants to backup your whole PC

I clicked on my computer name (“ZENIGMA”) and selected “Disks and partitions“

I un-ticked all system drives and partitions that were not my C Drive disk. I have installed Windows 10 onto a Corsair MP600 M.2 PCIE 1TB SSD and I want to back it up. The M.2 SSD connects to my main board.

Corsair MP600 SSD, Credit Corsair.

I selected this drive to backup.

View of all disks and partitions, I ticked the MP600 drive.

Oh dang, I don’t have enough space free on any drive to backup the C Drive to., Time to move some files.

Backup Destination

I have 8 partitions (drive letters) on 5 SATA hard rives and 2 M.2 SSD’s. I think I can move some data and free some space for this backup.

I had to shuffle data across drives to create enough space to backup my c drive

File Musical Chair’s

I used WinDirStat from https://windirstat.net/ to scan my U drive (my newest hard drive) for files to move.

I used WInDIRStat to find files I can move

WinDirStat shows a nice graph of files on my U Drive. WinDirStat colour codes file types by colour and also sizes squared by actual file size.

Windirstats scan results, Loads o Music I can ,move.

I could see 240GB of music that could be moved. My wife and I have converted all CD’s that we have purchased since the 1990’s to live alongside out Apple iTunes collection.

250GB of Music needs moving

I used the Windows feature to right click on the Music folder and move it  the folder to another drive.

Move the Music folder in Windows

Windows confirmed hat I wanted to move this folder.

Move Confirm Dialog

It took 2 hours to move my Music folder to a smaller slower drive.

btw Pink Floyd are Awesome

Now that I have 800GB free I can backup up my C:\ Disk

I re opened Acronis True image and resumed from Backup point where I left off and made sure I was backing up my C Drive (Corsair MP600 SSD)

My C drive is  500gb and the destination has 888GB free

Acronis showed the Windows drive to web backed up.

I clicked “Select destination“.

Backup Dialog

I clicked “Browse“

Backup Destination

I selected “U:\AcronisBackup” as the target to backup C drive to

Choose a backup destination folder

I was prompted for a encryption password (nice)

Enter an Backup Encryption Password

Now the backup is ready to start.

Backup Now

I clicked Back up now

Backup in Progress

The Backup was going to take about 40 minutes.   I could see My C and U drive were busy in Task Manager

Backup in Progress

After the backup is complete Acronis verifies the Backup

Verify Backup Image

After the backup finished I could see that Acronis managed to backup C drive (570GB) to a 329GB file on a different drive

Backup complete

Now I need to restore the backup to confirm it works.

Only a working backup is of value.

Creating Rescue (Recovery) DVD Media?

Now that I have a backup of Windows I need to create a boot media that will allow me to restore the backup image in case Windows dies. 

I clicked in the 4th icon from the top in Acronis True Image 2020 then Rescue Media Builder

Rescue Media Builder

I clicked Simple

Simple or Advanced

First I created a DVD boot disc.

I selected my DVD Dive (it had a blank DVD in it)

The Rescue Media needs a 800MB CD or DVD.

Ready to burn the DVD

I clicked Proceed to burn the rescue media to the blank DVD

DVD Burning.

The Rescue Media DVD was burnt to a blank DVD, a few tips on using the Media were presented

Rescue Media complete, 3 steps to use.

I labelled the DVD Rescue Media and put it in a safe place.

Creating Rescue (Recovery) USB Media?

Now I will create a USB rescue Media Key.

I opened Acronis, Click the Tools Icon (same as the DVD method above), then Rescue Media Builder, click Simple and select your Empty USB key.

TIP: Make sure it is the correct Key as they key will be deleted first.

USB Key Selected

As with the DVD method above the Rescue Media files to be copied were about 800MB.

Write to USB Key.

When the UBS key is prepared you will receive the same success screen as the DVD method above.

Restoring a backup from the DVD Rescue Media

I inserted the Rescue DVD into my DVD drive, I rebooted my PC and pressed F2 to enter my BIOS (your key may be different).

I navigated to the section in my BIOS where I could choose my Boot device and made the DVD drive boot first, this will allow my Rescue DVD to boot before Windows.

BIOS Screen DVD Boos before C drive

I rebooted my PC and I pressed a key when prompted to boot from the Rescue DVD

The DVD buzzed away for a minute and then I was prompted with Back Up or Recover menu. I clicked U:\Blog\Acronis True Image\DVD Restore.

Back Up or Recover screenshot

I navigated to the U Drive (SFEARABY) Drive and selected the Backup I had taken earlier.

FYI: The Drive letters did not match in Windows and the Rescue Media. Take note of tour drive names. My U:\ Was listed as E:\.

I entered the password that I used to encrypt the backup.

I selected the backup of my C Drive to restore. I assume you can choose multiple, drives if you had backed up multiple drives.

I selected Recover whole disk and partitions.

Recover whole disk or single partition.

I was asked to select the date to backup from (I assume this is for when you have multiple backup events over many days).

I was again asked to select partitions to recover. 

Select Partitions.

A summary of the restore was presented and I clicked Proceed.

The Recovery was underway, It took about 8 minutes to restore over 500GB.

Recovery in progress.

It worked.

Recovery worked.

I restarted my PC and removed the Rescue DVD.

Windows was all restored to it’s earlier state.

Restoring a backup from the USB Rescue Media

Restoring is exactly the same as the DVD method.

  • Reboot and Enter your BIOS
  • Set the Boos Order to allow the USB key to boot first
  • Reboot (no need to press a key to boot from USB)
  • Backup and restore (as needed)

I was able to restore the backup no problems,

FYI: I had the same slowness in the UI in the same spots as the DVD, nothing that breaks the process.

The USB and DVD Recovery offers a backup and restore options.

USB Recovery

The Recovery media also allows you to clone a disk and setup a secure hidden partition for backups.

Other OPtions

Backup Protection

Acronis True Image 2020 also protects backup locations and prevents access to them.  I had to Allow Windows to be able to delete a backup.

Once I allowed Windows to be able to access the backup I was able to delete it.

Acronis allowing windows to manage backups files on Windows

Backup Schedules

Acronis True Image 2020 allows you to set Backup Schedules.

Backup Schedules

I do not have enough disk space to setup anything other than one full backup at  a time 

Single file backup

Backblaze

I still have Backblaze automatically backing up my files from within windows and this is my 2nd later of defence.

Backblaze will backup the 300+ GB backup.

300GB backup file/

Other Features

Active Protection

Acronis True Image 2020 has built in Active Protection to watch processes for ransomware or cryptographic. Read more here: https://www.acronis.com/en-au/active-protection/

I have had one false positive upgrading Java but ill leave it enabled.

Active e protection status screen

A nice log is available of all events and whitelisted apps.

Active protection log/

I have had to exclude apps but this has settles down.

Whitelist app

A notification appears on blocked or whitelisted events 🙂

possible ransomware detected.

I can simply block or allow to deal with each exception.

possible rancomware history

Thus is a nice feature.

Cloud Backup

I am now using any Cloud backup features in Acronis, ill let Back blaze handle that.

Archive large or old files

The Archive large files or old files is a bit basic for me to use.

Archive old or large files screen.

I will keep using CCCleaner and WinDirStats for this job.

Alternative disk clean up tools

Even the Windows default Disk Cleanup is good.

Windows disk cleanup

Cloud Sync

I do not use Cloud Sync, I would but I can’t find a price on the Acronis site at all.

I noticed that on the DVD and USB recovery boot I can restore a windows from a Acronis cloud backup (nice).

Other Tools

  • Clone disk (like EASE US?)
  • Universal restore?
  • Try and Decide
  • System Clean Up (backup First)
  • Acronis Secure Zone
  • Drive Cleanser
  • Third Party Tools

Acronis True Image (Conclusion)

I am being picky here as I loved using Acronis 10 years ago and I have rose coloured glasses.

Pros

  • It works
  • You can backup Windows while Windows is open
  • DVD and USB Rescue Media creation is easy
  • Backup Schedules
  • Notifications of disk space and operational state
  • Exclusions of non essential files
  • Backup Validation (at time of creation and schedule)
  • Ability to call pre and post backup commands
  • Choice of Incremental, Full or differential backups schemes
  • Cleanup of older incremental versions (and full version frequency)
  • Simple single version backup (single file, overwrites each time)
  • It works.

Cons

  • Some Buttons in the GUI are slow to respond
  • The DVD an USB Rescue Media is little unresponsive at times during the restore (but it works)
  • The DVD and USB Rescue Media GUI can use a bit of design love (I am sure it was better 10 years ago (basic and clean))
  • When I restored I had no Idea what C:\ Drive was listed as H:\ (loads of confusion on the internet)

Disclaimer

I am not paid by Acronis to say this, this is just my guide for my friends.  I paid with my own cash.  I take no responsibility if this does not work for you.  Good luck though.

Troubleshooting

DVD Rescue Media

The DVD/USB Rescue Media is a bit laggy. At one point in the recovery I has a white screen for over a minute. I waited and the screen came good

 

Laggy Screen

Also the Menu in the DVD/USB toolbar does seem a bit squashed.

Squashed Screen

Bonus: Windows System Restore

There may be a time when you have created a full Operating system backup but the restore does not work (e.g hardware has failed (and been changed)). Acronis True Image 2020 will allow you to restore files from a backup image (and not the whole partition or disk). 

Create System Restore

Make sure System Protection is enabled for your C Drive

System Protection On C Drive

Create a System Restore Image

If you have Windows 10 it is a good idea to create a System Restore point ever few months just in-case Windows goes bad.

Click Your Start Button then type “Create a restore point“

Windows 10 has a System Restore feature

Click your C Drive and click “Create“

Create restore point

The restore point will be created

Creating restore point.

Restoring a restore point

At any time you feel that Windows is sick (and still working) you can restore and old system restore.

Restore a restore point screenshot

Click the the desired restore point then click Next

All restore points list

You will be able to restore the old system snapshot.

Bonus: Clean Windows 10 Install

TBA: Blog post coming soon

Bonus: How I prepare all files needed for a clean install

Before you do a clean install of Windows 10 you may need to document what software you have installed so you have a change to reinstall them after you reinstall Windows.

This is possibly the most boring job but the one with the most reward. The hardest party is knowing what software you have an need.

Finding Installed Software with SUMo

I use a free program called SUMo from KC Software (not a plug) to list all installed programs and to see what software is out of date.

SUMo App Screenshot

SUMo informed me that I have 256 software products installed.

  • 7 were really out of date
  • 62 products had recent updates

I took this list and downloaded and installed new copies of the apps I had installed (as newer version’s of apps can be more secure) and I also saved the installer files to a folder away from C Drive (e.g B:\Installs).

I am a bit OSD and I have created 10 folders under the B:\Installs folder that contain installs from Essential Drivers to non essential apps that I can use in the event of a clean install of Windows 

In the Installs folder I have these sub folders

\100 Drivers\
\200 Essential Apps\
\300 Productivity Apps\
\400 Development Apps\
\500 Utils\
\600 Games\
\700 Virtual Machine OS Installs\
\800 Virtual Machine Apps\
\900 Learning\

When reinstalling Windows I start by installing all apps in each group (starting with “100 Drivers”), The larger the parent folders’ number is the less important the files are int hat folder.

I have a Legend.txt with a summary of the contents of each folder

Install Folders Legend

100+ ~ 199 Drivers
     - Mainboard ASUS RUF X570 (WIFI)
     - AMD Chipset Drivers
     - Nvidia Video Card
     - Etc
201+ ~ 299 Essential Apps
     - Printer Drives
     - 1Pasword
     - Antivrus
     - Backblaze
     - VPN
     - Acronis
     - Etc
300+ ~ 399  Productivity Apps
     - Microsoft Office 
     - Microsoft Visio
     - Etc
400+ ~ 499  Development Apps
     - Visual Studio
     - Arduino IDE
     - Fritrzing
     - Microsoft SQL Developer
     - MySQL
     - Python
     - PHP Storm
     - Etc
500+ ~ 599 Utils
     - CPU-Z
     - Core Temp
     - Acrbat Reader
     - Etc
600+ ~ 699 Games
     - Steam Installer
     - Origin Installer
     - Etc
700+ ~ 799 Virtual Machine OS Installs
     - Windows 3.11
     - Windows 95
	 - Etc
800+ ~ 899 Virtual Machine Apps
     - Office 95
     - Office XP
     - Etc
900+ ~ 909 Learning
     - Python Books
     - Etc

You can create whatever numbering scheme you want.

I also have a cached folder of my Development Software (Visual Studio) Install cache at “B:\Installs\vscache“

All of my install folders.

Under my drivers folder I have all the drivers I need to reinstall windows

All the drivers for my system.

An under my “500 Utils” folder is all the utilities I install (many).

Hundreds of sub folders with utilities in them

The main thing is I have all the files and installers needed to do a clean setup of Windows 10 if need be.

Links

  • Acronis Personal Backup
  • Welcome to Acronis True Image 2020

 

Version History

v1.1 Added “Backup and Restore Windows Prerequisites”

v1.0 Initial

Filed Under: Backblaze, Backup Tagged With: acronis, Backup, crash, restore, windows

Backing up your computer automatically with BackBlaze software (no data limit)

June 2, 2019 by Simon

Backblaze ( https://www.backblaze.com/ ) is an awesome company who not only create a mega reliable backup infrastructure but they also tell us how they do it and tell us what hard drives are good or bad.

Use my link and get your first month of Backblaze backups free: https://secure.backblaze.com/r/00e1wj

They are so confident and experienced that they offer unlimited backups for $6.60 a month here in Australia. They state they have 750 Petabytes stored on their infrastructure & have restored over 40 billion files to customers.

Check our my other related posts

  • Backing up files to a Backblaze B2 Cloud Bucket with Duplicati
  • How to back up an iPhone (including photos and videos) multiple ways

Main features of Backblaze.

  • No limit to files that you back up (number or size or files)
  • Web-based file recovery (if needed)
  • They will mail you a hard drive if you need to restore large amounts of files.
  • Ability to locate lost or stolen computers.
  • Mobile app access your backups.
  • Business options for large computer fleets.

Creating a Back Blaze Account

Go to backblaze and create an account.

Login to backblaze at https://secure.backblaze.com/user_signin.htm

Screenshot Login.

Choose a Backblaze Plan

Go to: https://secure.backblaze.com/buy_plan.htm to choose a backup and payment plan. Use my link and get your first month free.

Australian Prices below (add +10% GST).

  • $6 AUD a month
  • or $60 for 12 Months
  • or $110 for 24 Months
Screenshot $6 AUD a month or $60 for 12 Months or $110 for 24 Months

Choose a payment plan

Pay by the month, year or bi year

Enter payment details

Screenshot, add CC or paypal

Click “Buy Backblaze”

Use my link and get your first month of Backblaze backups free: https://secure.backblaze.com/r/00e1wj

Done, now I can download the client.

Installing the Backblaze Software

  • Windows Client
  • Mac Client

When the download is down you can start the install.

You will need to login to the install to streamline the setup.

Backblaze install screen asking me to login

After you login the installation will begin.

Backblaze is installing and checking what files need backing up.

Backblaze now reports that the install has complete. Backblaze reports that I have 379GB to backup.

Backblaze reports that I have 379GB to backup.

Pressing OK shows Backblaze is already uploading my files.

Also, I have 15 days of free trial before being billed.

Clicking on the backblaze system tray icon reveals a few options

Backblaze system tray icon menu

Backblaze Preferences

Backblaze preferences can be opened by clicking Settings from the main screen.

I..

  • Renamed the computer.
  • Disabled backup on battery power
  • Set upload threads to 5 (then lowering if the network gets busy or when the backup is almost complete)
  • Disabled Automatic throttling and set faster backups
  • I set a private encryption key
I entered and encryption key

There is a 50 char limit on encryption keys

50 char limit on encryption keys message

Warning about not forgetting the encryption key

Message don;t loose the key

All other preferences look ok.

Backblaze preferences

That’s it the backup is uploading automatically and there is nothing else I need to do.

Backup is happily uploading.

Backblaze Portal

If you login to the backblaze portal ( here ) you can view your files (you will need to enter the private encryption keys if you opted to encrypt your files).

Backblaze portal

Initial Backup time

Your initial backup may have a while to upload depending on your backup size and internet connection speed.

I was able to upload 400GB in 3 days on my Internet Plan. Smaller files are uploaded first then larger ones. I have 2 files left to upload totalling 7GB.

Read the official guide on speeding up uploads here.

Backblaze files uploading

One way to make the initial backups faster is to see what files are queues to upload in the Backblaze preferences and then excluding unwanted files and folders.

Backblaze files ro backup queue

I can see my Internet Explorer cache and Google Chrome Temporary Internet File(s) folders are queued to be back up.

Time to exclude these folders in the Backblaze Settings under exclusions

C:\Users\Simon Fearby\AppData\Local\Microsoft\Windows\INetCache\IE
C:\Users\Simon Fearby\AppData\Local\Google\Chrome\User Data\Profile 1\Code Cache\
C:\ProgramData\MySQL\MySQL Server 8.0\Data\#innodb_temp\
C:\ProgramData\USOShared\Logs\
C:\Users\Simon Fearby\AppData\Roaming\Adobe\SLData\SLCache\

Time to exclude these folders.

Backblaze exclude folders

That knocked off a few thousand files to backup 🙂

Restore Files from Backblaze

If you restore files you have a few options

  • Download the files (free)
  • Save files to B2 Cloud (Backblaze)
  • USB Flash Drive
  • USB Hard Drive
restore options. 1) download, 2) save to BS cloud, 3) USB Key, 4) USB Hard Drive

Choosing to download files you are given a treeview to restore files from.

Treeview folder to restore files

I selected a test file to restore (restores are not instant) and was informed I would be emailed when the files are ready to download. Thats cool

Screenshot, restore pending.

I waited 3 minutes and received an email that my files were ready to download.

Email: download ready

I hope this helps someone.

Other Links

Check out my guide on How to back up an iPhone photos and videos blog post here.

Use my link and get your first month of Backblaze backups free: https://secure.backblaze.com/r/00e1wj

View Official Backblaze guides here.

https://help.backblaze.com/hc/en-us/sections/203997408-B2-Guides

Version

v1.3 Initial Backup section

v1.1 Added free month link

v1.0 Initial Draft

Filed Under: Backblaze, Backup Tagged With: Automatic, backblaze, Backup

How to back up an iPhone (including photos and videos) multiple ways

June 2, 2019 by Simon

This guide is for and anyone who needs to back up their iPhone (complete device along with separate backups of photos and videos).

This is not a paid promo, I don’t get a kickback for mentioning the awesome app below.

Check our my other related posts

  • Backing up files to a Backblaze B2 Cloud Bucket with Duplicati
  • Backing up your computer automatically with BackBlaze software (no data limit)

At the time of writing iTunes v12.9.5.7 was the latest version of iTunes.

iTunes is Apple official software for getting files too and from an iPhone, iPod or iPad. If rumours are right Apple will kill off iTunes software at the end of 2019. If Apple kills off the iTunes software I will update this guide in the future.

Disclaimer: I provide no warranty or support for this app, this is what happened to work for me.

Why write a guide on Backing up a mobile phone?

  • It’s not that simple an I get asked how to do this almost every week by someone.
  • No one thinks to backup up their photos until they fear their phone is lost or stolen.
  • I am about to publish a few posts on backing up (development machines, servers etc) with automated software and with free open source software so this post will be handy to link in what data you can back up in that post.
  • The last time I wrote an article on backups was in March 2016 and its not that great these days.

Golden rules of backing up.

  1. Backup to Three different locations.
  2. Two of the media need to be different.
  3. One of the locations needs to be offsite
  4. Test your backups (a backup is only good if you can restore). (from @Daniel15)
  5. If the backup is encrypted make sure you have the decryption keys. (from @Daniel15)

Backing up the official way (with iTunes)

Apple prefers you just pay them to extend your free 5GB iCloud storage and not worry about backups and not to worry yourself with manual steps.

My wife’s iPhone is always popping up messages saying that her iCloud storage is nearly full and she should upgrade the free space limit.

Screenshot of iCloud setting on iPhone saying your iCloud is nearly full but you have used 5GB or 5GB?

I know she has 100GB+ images and movies on her iPhone but iCloud has only backed up 4GB of photos and videos. Do not trust any backup statuses unless you can verify all files from a restore.

For the record I use an Android phone (Android backup guide coming soon).

iCloud is a good idea for automatic backup but I prefer to see my photos and backup them up myself. Also, Apple iCloud is not free from troubles. If you want Apple to handle backups then I do suggest you upgrade your iCloud storage from the free 5GB to a more sensible 200GB or more.

Screenshot of https://support.apple.com/en-au/HT201318

Here in Australia Apple charge the following for extra storage.

  • 50GB = $1.49 a month
  • 200GB = $4.49 a month
  • 2TB (2,0000GB) = $14.99 a month

A rough estimate: 50GB is enough to store about 12,000 files (9,000 photos + 3,000 videos) *

  • * = based on stats from the backup below (depending on the size an length of your videos).
Screenshot of apple iCloud pricing 50GB = $1.49 a month
200GB = $4.49 a month
2TB (2,0000GB) = $14.99 a month

You can upgrade your paid Apple iCloud backup limit from your iCloud Storage Settings from your iPhone’s settings screen.

Screenshot of iPhone iCloud upgrade button.

IMHO: Apple does make it clear what you are getting before you purchase (but they do not push it enough and people assume their data is safe).

iPhone picture of 50GB = $1.49 a month
200GB = $4.49 a month
2TB (2,0000GB) = $14.99 a month

TIP: Please review your existing iPhone data usage and iCloud usage before it’s too late

I don’t want ot pay Apple for more iCloud storage

I like you, I am tight too but I do pay Google $2.49 a month for 100GB backup storage on my Android phone. Google Drive (Google storage is a bit cheaper than Apple).

Screenshot of Google Drive https://www.google.com/drive/

It is nice knowing your phone is auto backed up.

Install iTunes on Windows (10)

On your Windows 10 computer click the start button. If you don’t have a Microsoft Account go here and create one. Click the start button and type “itunes” (you don’t need to type into a box, just start typing when you click the start button)’.

Then click “iTunes, install app”

Screenshot of a Windows start menu and Get iTunes icon

If you are logged into your Windows 10 store with your Microsoft ID click “Get” (if not you will need to login to the store).

Screenshot of me logged into the Windows Microsoft Store.

After you click ‘Get’, iTunes will start downloading.

Screenshot of the Windows Store downloading iTunes

When iTunes is downloaded Windows will install it. When it’s installed click ‘Launch’

iTunes is installed, Click launch screenshot

You will need to agree to Apple’s terms of service. Click ‘Agree’

Apple terms of service screenshot.

When iTunes opens click “Agree”

Screenshot of Apple iTunes asking if we can agree to share analytics data

Now login to iTunes with your Apple ID (if you don’t have one create one here)’.

Screenshot of iTunes open with an arrow pointing to the Account menu

Click the ‘Account’ then ‘Sign In’ menu.

Screenshot of the account then sign in menu

Login to iTunes with your Apple ID

Screenshot of the apple sign in box.

Optional: If you have Two Factor Authorisation (2FA) of Apple ID’s turned on (you should) you will need to enter a 6 digit code.

Screenshot of an apple 6 digit 2FA code

Apple Two Factor Authorisation (2FA) will send one of your other devices a login code that you will need to use to login (confirm you own the account)

Screenshot of apples 2FA web page at https://support.apple.com/en-au/HT204915

TIP: Check out https://twofactorauth.org/ to see what other sites use Two Factor Authorisation (2FA). I use hardware Yubico YubiKeys to protect logins to WordPress, Linux and websites.

Now back to the article, iTunes should be ready to allow us to backup our iPhone.

Screenshot of me logged into iTunes

Before I continue I will click ‘Edit‘ then ‘Preferences‘ menu to view where iTunes will download media too (different ot backup data), not important but I just want ot see if it is not pointed ot my smaller C drive before I backup my phone

I changed the location to a larger S:\Drive’

Screenshot of Apple setting screen (tab Advanced)

I was greeted with a message asking me to confirm that I wanted to sync the iPhone that I just plugged in called ‘EllieRose’, I clicked ‘Continue’

Confirm access to the iPhone screenshot

On the iPhone I also clicked ‘Trust’ to allow iTunes to talk to it.

iPhone screenshot to allow iTunes to talk to it.

I was then prompted to download a software update to the iPhone, It appears this phone is not running the latest software

Screenshot of a question to update to iOS v 12.3.1

I was prompted to sync the iPhone to the computer

screenshot of a prompt to sync purchases from the iPhone to the computer

A backup of the phone was underway.

Screenshot of iTunes backup progress with a progress bar

If the screen above does not appear click the icon below to view the backup and restore iPhone menu.

Screenshot showing the iPhone Device button in iTunes

The iPhone was backing up to this folder on my computer:

C:\Users\Simon Fearby\Apple\MobileSync\Backup

I was not prompted for a location to back up so and I will move this backup folder after the backup completes (so my C: drive does not fill up).

My drives.

  • C:\ = 500GB drive (faster SSD)
  • S:\ = 2,000GB Drive (slower Magnetic)
Screenshot showing  iTunes backup and  the C:\Users\Simon Fearby\Apple\MobileSync\Backup folder

The iTunes iPhone backup is now complete (it took about 2 hours).

Screenshot of the complete backup

Now that the iPhone was backed up iTunes the Operating System update started to run.

Screenshot of iTunes verifying the iPhone update

Update verified.

Screenshot of iTunes updating the iPhone OS.

During the update the iPhone was unavailable.

Screenshot of iPhone installing firmware

Done, the iPhone had updated it’s operating system in about 30 minutes.

Screenshot of iTunes showing the newer OS version.

Now lets see how much space I have available on my C Drive.

Screenshot of disk usage on my c and S drive.

It looks like the iPhone backup iTunes made was 60GB in size.

Screenshot of Windows reporting the backup was 60GB in size.

Using Windows Explorer I moved the backup from..

C:\Users\Simon Fearby\Apple\MobileSync\Backup

to..

S:\Backup\AlisonsiPhone\iTunes\31May2019

(right click drag an drop action in explorer)

Screenshot of right click drag and drop move folder.

It took me about 20 minutes to move the 60GB folder.

Screenshot of thew Windows copy dialog progress.

TIP: You can (should) also copy this backup folder to..

  • A removable hard drive
  • Removable USB Flash Drive
  • A NAS or SAN Drive
  • etc

Now this backup is available for me to restore in the future if I need it.

In the case of a restore, I just need to move the backup into the iTunes expected location.

Copy from..

S:\Backup\AlisonsiPhone\iTunes\31May2019

..to..

C:\Users\Simon Fearby\Apple\MobileSync\Backup

The Catch

The catch with back-ups made with iTunes is they are useless if you wanted to restore individual files (say to find a photograph or video). iTunes backups are usually tens of thousands of files with random folders and filenames

Screenshot of iTunes nasty backup, i obfuscated all files to random guid filenames

If you want to JUST backup photos and videos read on.

How to backup just photos and videos from your iPhone.

This part of the guide needs a paid version of the iOS App Photo Transfer App. In Australia, the app is free but to turn it into a full version needed to restore everything (via in-app purchase) it cost $10.99.

This app is well worth $10.99 AUD (it may be cheaper in other countries) to have it push photos an videos from your iPhone to a free companion Windows program (a Mac versions exist too).

Screenshot of http://phototransferapp.com/

Buy the iOS version and install it on your iPhone.

Then download the free Windows version: http://phototransferapp.com/win/

Extract the files from the zip file.

I extracted PhotoTransferapp.exe from PhotoTransferapp.zip

When I run PhotoTransferApp.exe I get an error saying I need to install Adobe AIR run times.

Screenshot of me installing Adobe Air run times from https://get.adobe.com/air/

Go to https://get.adobe.com/air/ and download and install the Air run time.

Screenshot of https://get.adobe.com/air/

Install the Adobe Air run time.

Install Adobe Air From https://get.adobe.com/air/

I installed Adobe AIR and again reopened the PhotoTansferApp.exe and was prompted to allow access to the Windows Firewall.

I was on a home network so I enabled firewall access.

Screenshot of Photo Transfer App asking fore firewall access.

The Photo Transfer App prompted me to ‘Discover Devices‘ and to make sure the Transfer app is running on the iPhone.

The Photo Transfer App prompted me to 'Discover Devices' and to make sure the Transfer app is running on the iPhone.

Before I clicked ‘Discover Devices” on the Photo Transfer App on Windows I opened the Transfer app (mentioned above) on the iPhone.

Photo of the Transfer app on the iPhone

On the iPhone I clicked ‘Send‘

Click Send

The transfer app will ask for access to your photos, you will need to press OK.

Screenshot of the transfer app asking for permissions to photos

On the iPhone again I clicked ”Windows‘

On the iPhone again I clicked  ''Windows'

The iOS Transfer app now said I should run the ‘Photo Transfer App‘ on Windows.

The iOS Transfer app now said I should run the 'Photo Transfer App' on Windows.

TIP: You can transfer over WiFi (if your iPhone and Windows device is on the same WiFi Network) or you can transfer over a USB cable.

Screenshot: transfer via WiFi or USB?

Before I started the ‘Detect Device‘ or ‘transfer‘, I set the Backup path location to my S: drive by clicking settings in the bottom right and choosing a folder (as my C Drive is a bit small).

Screenshot of Photo Transfer App choosing a folder to backup to.

I noticed the port of 57777, I temporarily disabled the whole Windows Firewall just in case it prevents the photo backup.

In hindsight, it was not a good idea to disable the whole firewall, but because I was at home on a safe network I felt safe to do so. Next time I will not disable the firewall and see if this still works.

If you are on an internet cafe, school or university network do not disable your firewall.

I clicked ”Start” then typed ‘firewall‘ and clicked ‘Windows Defender Firewall‘

Screenshot Windows Defender firewall icon

I turned off my firewall.

Screenshot Firewall disabled.

I clicked ‘Detect Devices‘ and the iPhone ‘EllieRose‘ appeared on the left. I doubled clicked on the iPhone name and was prompted with an Authorization required message.

Screenshot: Authorisation required message.

I looked at the ‘Transfer‘ app on iOS and clicked ‘Yes, always‘ to allow access.

Screenshot: Authorisation yes/no on the iPhone

After 20 seconds I can see photos on my iPhone on Windows.

Screenshot, all of my iPhone images were appearing on Window.

I selected all camera albums to backup and clicked backup. I prompt to upgrade to the full version will appear, you will need to buy the upgrade.

Full backup happens only if you upgrade to a full version via an in app purchase

After the full version if purchased the backup will be allowed.

You can upgrade from the free to paid full version from the main screen of the Transfer app.

Screenshot get full access by upgrading the free app to a paid version

The upgrade In App Purchase is $10.99 AUD

Screenshot of a $10.99 In App Purchase on the iPhone

When you purchase the In App Purchase you can run the backup again.

Screenshot of all camera categories on the iPhone on windows

Now I could see photos being copied from the iPhone to my defined backup folder.

Screenshot underway.

I went to bed as I knew there were about 100GB of files on the iPhone and this was going to take a while.

In the morning the backup was done.

Screenshot backup done.

26,000 files were backed up (over 100GB).

I now had a full iPhone backup made by iTunes and a copy of all photos and videos.

Screenshot showing the 2 backups (a) iTunes made and b) Photo Transfer App made)

I turned on the Firewall again.

Screenshot: I re enabled the firewall.

I now had 160GB of backed up photos, videos and phone backup.

160GB of files  from both backups.

Backup your iPhone backups to the cloud

The steps to do this two ways will be added soon.

I will add a section on how you can back up the iTunes and manually synced photo and video backups to the cloud automatically and a more complex but cheaper was for 0.005c per GB.

Watch this post.

How to backup and Android Phone

Article coming soon.

Other Links

Read the official iTunes/iCloud backup guide from Apple here: https://support.apple.com/en-au/HT203977

Backing up your computer automatically with BackBlaze software (no data limit)

Versions

v1.3 Added back blaze article link

v1.2 Added more images (from a phone that does not have the iOS app already)

v1.1 Added an Android heading.

v1.0 Initial Post

Filed Under: Apple, Backup, Cloud, Google Tagged With: Apple, Backup, iCloud, iPhone

How to backup and restore a MySQL database on Windows and Linux

April 21, 2019 by Simon

Why backup and restore

This is a quick guide demonstrating how you can backup and restore a MySQL database on Windows and Linux using Adminer.

You may need to know how to backup a restore a database for a number of reasons..

e.g

  • Send the database to someone to debug or give feedback while learning.
  • Move the database from a local machine to the cloud
  • Move the database from cloud vendor A to cloud vendor B
  • etc.

Having a backup of the VM is good but having a backup of the database too is better. I use UpCloud for hosting my VM’s and setting backups is easy. But I cannot download those backups.

UpCloud Backup Screen

Murphy’s Law

“If anything can go wrong, it will”

The most important reason for taking a backup and knowing how to restore it is for disaster recovery reasons.

Backup (the easiest way) with Adminer

Adminer is a free PHP based IDE for MySQL and other databases. Simply install Adminer and save the file on your local computer or remote web server directory.

FYI: The Adminer author Jakub Vrana has a patron page, I am a patron of this awesome software.

Snip from Adminers website. “Adminer (formerly phpMinAdmin) is a full-featured database management tool written in PHP. Conversely to phpMyAdmin, it consist of a single file ready to deploy to the target server. Adminer is available for MySQL, MariaDB, PostgreSQL, SQLite, MS SQL, Oracle, Firebird, SimpleDB, Elasticsearch andMongoDB.”

adminer.php file icon screenshot

TIP: The file would be publicly accessible to anyone so don’t save it to a common area, obfuscate the file, protect it of delete the file when you are done using it.

Once Adminer is installed load it in a web browser, login with your MySQL credentials. Once you login you will see all databases and an Import and Export menu.

Adminer main screen, all databases and import and export menu.

tbtest is a simple database with one table and 4 fields (ID, Key, Value and Modified)

.Click Export to open the export screen.

Export screen showing a list of databases and export options

Click Export, a SQL file will be generated (this is the export of the database).

Here is a save of the file:
https://fearby.com/wp-content/uploads/export.txt

Exported view of https://dev.mysql.com/doc/workbench/en/wb-admin-export-import-management.html

Its that simple.

If I add a binary blob file to the table and upload a PNG file lets see how the export looks.

Screenshot o the new table with a blog field in Adminer UI

Let export the database again in Adminer and check out the output. I used Sublime Text editor to view the export file.

New Export shows the binary file in the Backup SQL file

Restore (the easiest way) with Adminer

OK lets delete the tbtest database and then restore it with Adminer. I used Adminer to delete (DROP) the database.

Database dropped with Adminer

Database “dbtest” deleted.

Now lets create a blank database to restore to (same name).

Create database screen.

Database created.

dbtest created.

Now lets import the database backup using Adminer.

Click Import, select the backup file and un-tick Stop on errors.

Import screenshot, dxtest selectded, Restore file selected, stop on errors disabled

TIP: The 2MB next the the choose file button is defined by your web server and PHP configuration. If you are trying to import a larger database (e.g 80MB) first increase the limits in your web server and PHP (via php.ini).

The Import (restore should take seconds)

Import Success

The database was imported from a backup, all tables and records imported just fine.

The database was imported from a backup

Bonus methods.

On Ubuntu use this guide to backup from the command line. If you use the Oracle MySQL Workbench read this.

I hope this helps someone.

Filed Under: Adminer, Backup, Database, MySQL, Restore Tagged With: and, Backup, How, Linux, MySQL, on, restore, to, windows

How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains

March 31, 2019 by Simon

This is a quick guide that will show you how you can connect to a cloud server via SFTP with the PHPStorm IDE from Jet Brians and deploy files from your localhost to the cloud. This is my opinion, I am not paid to promote PHPStorm or UpCloud.

Pre-Requisites/Assumptions

This guide will assume you already have (or know how to)..

  • Buy a domain name and point it’s DNS to a server (I use Namecheap.com for buying domains)
  • Buy and deploy a server in the cloud. I have used AWS, Digital Ocean, Vultr but now use UpCloud for deploying fast self-managed servers. Read this guide here to see how I create a server from scratch on Up Cloud.
  • Setup SSH access to your server and configure a firewall.
  • You have or know how to set up PHP and Web Servers and configure them on your localhost and remote server (guides here, here, here, here, here and here).
  • etc ( check out all my guides here https://fearby.com/all )

I am using Windows 10 Home (with IIS Web Server (document root redirected to S:\Code\) and a pre built Ubuntu Cloud servers.

IIS pointing to S:\Code

Why no FTP? I do not create FTP servers on my serves to increase security and I only access servers via SSH via white-listed IP’s and then authenticate with hardware 2FA keys from YubiCo (read me 2FA guide here and also how to secure *nix servers and WordPress with 2FA).

Background

2 years ago I used to use the Cloud 9 IDE to connect to, and code files on cloud servers and life was good. I could configure and connect to servers, drag and drop files, run bash scripts from a web page and close the Cloud 9 browsers tabs, travel hundreds of kilometres and log back into C9 and all code and bash scripts would reappear.

Here is the Cloud 9 IDE showing code on the left and a Browser on the right.

C9.IDE showing code on the left and web page on the right

With Cloud 9 code could be easily accessed, edited and run.

See screenshot of code running from a Cloud 9 hosted server with properties windows on the right.

C9 IDE showing  a bash terminal windows and code

Regrettably, I cancelled my $9/m subscription to Cloud 9 after a minor stroke and since then I have gone back to a terminal screen to code and transfer files. In the last 2 years.

Screenshot of the putty program connected to a Ubuntu box editing a file with nano

Uploading and downloading files on mass is painful via pure SSH.

AWS has since purchased Cloud 9 and I am not sure if it will ditch support of non-AWS servers in the future. AWS is good but servers are very expensive for what you get IMHO. I have found Disk IO on UpCloud is awesome (also UpCloud support is great (I am not paid to say that)).

PHPStorm IDE?

A quick Google of IDE’s like Cloud 9 mentioned PHPStorm from Jet Brains. I have used IntelliJ IDEA from Jet Brains before and PHPStorm seems to be very popular.

Go to
https://www.jetbrains.com/phpstorm/ and see the features of PHPStorm

Watch PHPStorm in action

Whats new in PHPStorm 2019.1

Install PHPStorm

Visit https://www.jetbrains.com/phpstorm/download/ and download and install PHPStorm (free trial 30 days)

System requirements

  • Microsoft 10/8/7/Vista/2003/XP (incl. 64-bit)
  • 2 GB RAM minimum
  • 4 GB RAM recommended
  • 1024×768 minimum screen resolution

Pricing

PHPStorm is $8.90/m for individual use (or $19.90/m commercial). For the first 12 months of uninterrupted subscription payments qualify you for receiving a perpetual fallback license (20% discount for an uninterrupted subscription for a 2nd year, 40% discount for an uninterrupted subscription for 3rd year onwards).

PHPStorm work on Windows, OSX or Linux. This great an I use Windows locally and Linux remotely but I’m keen to use Linux locally to match local and remote dev environments.

Official PHPStorm Pricing Page:
https://www.jetbrains.com/phpstorm/buy/#edition=commercial

fyi: Jet Brains has free licencing for individual use for Students and faculty members.

Creating a Project

Open PHPStorm and select “Create New Project”.

Create New Project screen

Choose a project type on the left (e.g “PHP Empty Project“) and choose a location to save too (I chose “S:\code\php001) on my local machine. I chose “S:\code\php001” on my local machine.

New Project screenshot asking for a name and save location

Choose a folder to save to.

Choose a project and location to save to locally

Click “OK” to create the project.

PHPStorm will have created a project for you. You will notice a “.idea“folder under the location you saved with these files.

  • misc.xml
  • modules.xml
  • php001.xml
  • wordspace.xml

Do not delete these files.

Creating your first PHP file

You can right click on the project root and select New then PHP File

Right click on the root in the tree view then new PHP File

Or clicking the File then New menu choosing PHP File.

File new PHP File dialog

Name the file (e.g index.php)

Naming a file index.php

The file has been created and its available in my localhost web server.

Screenshot showing PHPStorm with index.php, S:\Code showing index.php and http://localhpst/php001/index.php loading

Creating a Deploy Target

Now we need to specify a deploy target in PHPStorm to push the file changes to the cloud. Backup your server (yes backup your server just in case).

Open your PHPStorm project and click Tools, Deployment and then Configuration.

Click Tools, Deployment and then Configuration.

Click the plus icon near the top left and choose SFTP

Screenshot showing add new deployment server (SFTP)

Name the deployment target (e.g “server (project)”)

Screenshot of an input box showing a server name
  • Enter your “server name” or IP and port
  • Enter your “ssh username” (ensure the SSH user had write access to the wwwroot folder and the web server can read the files written by this user)
  • Under password I chose “Key pair OpenSSH or Putty (as I had SSH details already setup in Putty details
  • You can add your ppk private key from Putty (use the puttygen program to conbvert ssh public and private kets to ppk format)
  • If you have a passphrase on your SSH key add it now
  • Enter your web servers remote path (for the project)
  • Enter your web server URL
Screenshot showing a server name, port, username, password, ssh file passphrase, root path and web server url.

I did SSH to my remote server and created the destination folder. This will ensure I can deploy code here (PHPStorm does not create the remote path fpor you ).

mkdir /wwwroot/php001
chown -R www-data:www-data /wwwroot/php001/

Click Test Connection

Test Successful screenshot

No we need to click the Mappings tab and add a mapping.

  • Local path is your local path
  • Deployment path is / (the web root path is carried forward from the previous tab)
  • Web path is the web path that is entered in the browser
Screenshot showing a manual file mapping of local and remote file locations

Click Add New Mapping. Now we are ready to deploy

Deploying code to the cloud

I right clicked on the root note in PHPStorm and created an index.php file.

Creating an index.php file by file new

I edited the index.php on my local machine and then click the Tools then Deployment and choose “Upload to fearby.com (php001)” menu.

Manual upload available in Tools menu then Deployment menu

The File Transfer output window showed the transfer progress.

Screenshot showing the file transfer window output saying the file uploaded.

I loaded https://fearby.com/php001/index.php in Google chrome. It worked.

Screenshot showing https://fearby.com/php001/index.php loaded in a bowser

Don’t forget to turn off Automatic uploads under Tools, Deployment menu.

'Screenshot showing Automatic updated turned on

Now when I create new files or change existing files they will auto upload.

Sourc Control

I will add this soon.

Shell Command

You can also open an SSH console to the server and run commands

e.g zip files

zip -r backup.zip .

I can also open a folder window in PHPStorm and show all remote files by clicking Tools then Deployment then Show Remote Files, Zip files can be easily downlaoded or other files uploaded. Nice.

Screenshot showing remote files

Linux Client

I will review the Linux PHPStorm client soon.

Troubleshooting

Watch the Official guide on Deployment and Remote Hosts in PhpStorm – PhpStorm Video Tutorial

Good Luck. I hope this guide helps someone.

Version

1.2 Removed advertisements

1.1 Minor Updates

1.0 Initial Version

Filed Under: 2FA, Backup, Cloud, Code, Git, GUI, IDE, Linux, SSH

I am moving away from Apple hardware

February 3, 2019 by Simon

My Late 2012 Mac Book Pro Retina laptop is all but dead, it has many dead pixels and because of the poor cooling and is NOT a joy to use anymore. It does not “JUST WORK” and personally, I do not think “thinner” laptops can handle Australian summers as its hardware cooling it inadequate above 40c air temperatures.

My laptop processor would spend more time thermal throttling (at 104c)  in Web Browsers and text editors that at normal speeds. Opening up productivity apps like Photoshop or Premiere Pro would send the laptop into meltdown.

Image of temperate monitoring showing an overheating macbook when the apple is idle

Frequent high temps were common.

Temp monitoring showing 100+c temps

Attempted Fixes

Warning Disclaimer: My laptop is out of warranty and I know my way around the inside of computer hardware without zapping it. Do not attempt to open your laptop unless you know what you are doing, have backed up your data and are prepared to brick your computer.

  • I removed dust from inside the laptop.
  • I tried to only use the laptop refrigerative air conditioning
  • I replaced the thermal paste on the CPU and GPU (3 times)
  • I reinstalled OSX Mojave and reset the SMC and PRAM multiple times.
  • I ran the fans at 100% (see post here), The fans were operating at full capacity and were not broken.

The stock thermal paste was crusty after 5 years. The plastic CPU/GPU cover was visibly cooked.

Picture of dry stock thermal paste

I ordered some new Thermal grizzly thermal paste, I had some older silicone paste on hand just in case.

Picture of thermal paste options

After many reapplications of the Thermal Grizzly, the older silicone paste seemed to work the better???

Picture of thermal paste applied on a processor

After a few months, all of the fixes above did not seem to work. OSX Mojave would spin up the CPU and GPU into a frenzy overloading the single heat pipe within minutes.

Time to try some more drastic cooling modifications?

I tried improving the efficiency of the single (copper) heat pipe that is shared between an Intel i7 2.6 GHz and an Nvidia Video Card by removing the black paint by stripping the paint with acetone.

Picture of the apple heat pipe in a jar of acetone

I manually removed paint from in between the heat sink fins with a LED to reveal the metal.

Picture of paint being removed from the apple heat pipe fins

I reinstalled the heat pipe with high hopes? That looks nice 🙂Picture of the heat pipe minus paint reinstalled

I removed the old thermal paste and added new paste. First I tried Thermal Grizzly Cryonaut. I re-applied the paste three separate times as each application was not that much better than the old crusty stock paste from Apple. Did I have a bad batch of Thermal Grizzly?, It seemed thick and not very viscous. I ended up using an old tube of silicone paste (the white stuff) as my Arctic Silver was too old to try and I did not want to order more.

More heatpipe post re installation pictures

With the silicone paste applied and the paint removed temperatures were about 15c lower at max, I still had frequent thermal throttling but at least I had a reserve buffer.

This was all before the Aussie Heatwaves and high temperatures soon returned.

Is there still room for improvement?

How heat pipes work

Picture of how heatpipes work

Heat pipes have an evaporating (hot part) and condensing zone (cool part) on the heat pipe. I noticed Apple’s “stock” condensing fins were small, would improving this zone help?. Time to improve the condensers zones by adding larger copper heat sinks to the bare side of the heat pipe.

I purchased a few copper Xeon/Sun server sized heat sinks and thermal epoxied them to the condensing end of the heat pipe. Yes, they would protrude out the bottom of the case but #Meh.  I can fix that by extending the base of the laptop down and making it thicker (old school style).

The server heat sinks arrived

Side on picture of server heatpipes

I cut the heat sinks in half.

Picture of a hacksaw cutting heatsinks

I packed the fins with paper before cutting to ensure the cut did not damage the fins.

Picture of a cut heatsink

After cutting, I wiped the copper heat sinks with vinegar to restore the surface to a nice copper shine.

I tested the heat sink idea with silicone paste first

picture of silicone tested on the heatsink

Temps were 25c lower, Now it’s time to use Arctic Silver Thermal Epoxy 

Picture of two part thermal epoxy

I applied the Thermal Epoxy to the heat pipe (I temporary had foil strips above the fans so I did not block them while the epoxy dried.

Picture of epoxy applied

I then stuck the heat sink’s to the heat pipe (with Arctic Silver Thermal Epoxy).

Heatsinks Thermal Expoxied on

I toyed with a clear case but decided against it for static electricity and stability reasons.

Clear Case pon the botom of the laptop picture

I purchased a second Mac Book base for so I could cut holes for the heat sinks to protrude and use the original base to hide the modification.

Cut holes in the base of the laptop base and purchased a second case bottom

I made a 30 mm base wall so I could use it as a wall between the laptop base and the new 30mm lower base.

picture of the base side wall i made

I added some 5-volt and 12 -volt fans inside the new extended 30mm base.

Collage of base assembly, screws and wires

Finished Product

A normal looking Mac Book except for the 30mm lower base and internal 5V or 12V fans.

Picture of the final mod with 50c lower temps and a 30mm bottom slab under the laptop

External power plugs on the left side, I will add lights at a later stage.

picture of the external power plugs flush with the case on the side of the laptop

Are the temps lower?

50 lower temps screenshot

Videos

Video: Mac Book Pro cooling mod, I can now watch 1080p videos without maxing the CPU

Video: Mac Book Pro cooling mod with external powered 5v or 12v fans

Conclusion

50c lower temps are nicer at idle but in Premiere Pro (exporting video) the laptop was still thermal throttling like mad and temps were terrible (100+). Lets not get started when I start some development VM’s

Conclusion 2 weeks later

This is still not a joy to use. I don’t think I have the right to expect a 5-year-old laptop to keep up running a CPU/GPU intensive OS and applications.

Time to buy a new computer, Apple still makes thin and overheating laptops by the looks of it? 

Maybe I need to buy a fridge to stick a computer in a fridge to use these days?

YouTube users indicate Apple has a problem with heat.

What computer do I get next?

Not an Apple made one. I will be moving back to Windows for local development and Linux on servers

Dell Alienware has many heat pipes.

Picture inside a dell alienware laptop with more heatpies

Acer Predator 500

I read a few reviews (e.g this one from Ultra book reviews) and Acer have good cooling.

picture of Acer Predator cooling and heatpipes

MSI GT Series laptops look the best if cooling is important.

Picture of a MSI GT laptop with 9 heatpipes

Or should I build a custom desktop with way more cores

CPU: Threadripper 2950X 16C 32T 

SSD: M.2 SSD: Samsung 970 PRO 512GB
MOBO: Asus Zenith Extreme
Power: Corsair RM1000x 1000W
MEM: Quad 3600 Mhz 
GPU: AMD Radeon VII Navi 3980

Thanks for reading.

 

v1.3 Added videos

v1.2 Updated alt tag descriptions

v1.1 Added “I will be moving back to Windows for local development and Linux on servers”

1.0 Initial Draft

 

Filed Under: Advice, Apple, Backup, Computer, Copper, Disaster Recovery, Heat, Maintenance, Uncategorized Tagged With: Apple, Heat, Macbook

No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

August 5, 2018 by Simon

No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Aside

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name here

Domain names for just 88 cents!

Now on with the post.

Murphy’s Law

I recently had an issue where I set up a website for a friend. I invested 6 hours into setting up..

I setup…

  • Debian OS
  • NGINX Web Server
  • MySQL Database
  • PHP 7.2
  • PHP-FPM Child Workers
  • HTTPS Certificate
  • Security (Firewall/Headers/SSH, WordPress, Plugins etc).
  • Installed WordPress and Plugins
  • Setup DNSSEC
  • Etc

I had tested GTMetrix scores = less than 1 second.  Security headers were tested and I was happy with the site.

The server and backups were automatically deleted after 7 days while I was away from my keyboard because I assumed the account was valid and had credits.

Lesson Learned

  • Always have a backup (of the server, setup/ www, MySQL etc).
  • Script setups (Ansible, Puppet or Scripts) to sate time redeploying if need be.
  • Backups are not always available.
  • Do have setup documented (Check)
  • Do have a disaster plan

I have guides on setting up a server on UpCloud, AWS, Vultr, Digital Ocean but setting up can be rather repetitive so how can you prevent resetting up servers?

Why Plan for the Worst

  • Companies disappear.
  • Some hosts go down.
  • Some hosts have weird trial modes and internal process that could take your site down.
  • Human error?
  • Murphy’s Law

How I will prevent this in future

  1. I am building a Java desktop app for Windows/OSX/Linux app that will deploy and set up on UpCloud/Vultr/Digital Ocean providers and allow for 1 click deploy and backup and restore.
  2. I am going to re-establish replication between servers with RSync etc.
  3. I am going to start to automate installs and environments.
  4. I am going to set up a hot (ready to go) Green/Blue mirrored environments (www and DB server) on different providers in case of emergency. Then I can set the active live servers with DNS (blog posts soon).
  5. Consider a server farm (same provider or different providers)

I hope this guide helps someone.

Please consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial Post

Filed Under: Backup, Disaster Recovery, Restore, Security, Server, VM Tagged With: a, are, different, have, hot, I, matter, No, on a, provider, ready, recommend, server-provider, spare, strongly, Using, what, you

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

June 25, 2018 by Simon

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).

Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Spoiler: UpCloud performance is great.

Upcloud Site Speed in GTMetrix

Why Upgrade

I have 1 CPU, 1 GB memory and 50GB storage and it is running well?  I have PHP child workers set up and have set up the preferred use of memory over swap file usage.

View of htop querying processes on Ubuntu

Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here).  To further increase the speed of the WordPress I have configured the OS to use memory over the disk.  About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).

Graph of memory falling below 100MB every day

I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.

How to Upgrade

This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.

UpCloud Pricing: https://www.upcloud.com/pricing/

Pricing table form https://www.upcloud.com/pricing/

Upgrade an UpCloud VM

I shut down my existing VM. Read this guide to setup a VM.

shutdown -P

Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.

Upgrade the Server, shut it down the server and choose upgrade

I confirmed the upgrade options (2x CPU, 4096 MB Memory).

Confirm Upgrade Options

Click Update

Upgrade Applied

After 10 seconds you can start your server from the UpCloud server.

I confirmed the CPU upgrade was visible in the VM

cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.978
cache size      : 16384 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 6199.95
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.978
cache size      : 16384 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 6199.95
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Software Tweaks Post Upgrade.

I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.

worker_processes auto;
worker_cpu_affinity auto;

I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase memory usage and child workers. I doubled child workers and max memory limit.

memory_limit = 3072M
pm.max_children = 80
pm.start_servers = 30
pm.min_spare_servers = 10
pm.max_spare_servers = 30

I restarted NGINX and PHP

nginx -t
nginx -s reload
/etc/init.d/nginx restart
service php7.2-fpm restart

I tweaked WordPress max memory limits

define( 'WP_MEMORY_LIMIT','3072M');
define( 'WP_MAX_MEMORY_LIMIT','3072M');

MySQL Tweaks: I logged into MySQL

mysql -u root -p

I ran “SHOW GLOBAL STATUS” to view stats

mysql> SHOW GLOBAL STATUS;
+-----------------------------------------------+--------------------------------------------------+
| Variable_name                                 | Value                                            |
+-----------------------------------------------+--------------------------------------------------+
| Aborted_clients                               | 0                                                |
| Aborted_connects                              | 0                                                |
| Binlog_cache_disk_use                         | 0                                                |
| Binlog_cache_use                              | 0                                                |
| Binlog_stmt_cache_disk_use                    | 0                                                |
| Binlog_stmt_cache_use                         | 0                                                |
| Bytes_received                                | 3179986                                          |
| Bytes_sent                                    | 223872114                                        |
| Com_admin_commands                            | 0                                                |
| Com_assign_to_keycache                        | 0                                                |
| Com_alter_db                                  | 0                                                |
| Com_alter_db_upgrade                          | 0                                                |
| Com_alter_event                               | 0                                                |
| Com_alter_function                            | 0                                                |
| Com_alter_instance                            | 0                                                |
| Com_alter_procedure                           | 0                                                |
| Com_alter_server                              | 0                                                |
| Com_alter_table                               | 0                                                |
| Com_alter_tablespace                          | 0                                                |
| Com_alter_user                                | 0                                                |
| Com_analyze                                   | 0                                                |
| Com_begin                                     | 0                                                |
| Com_binlog                                    | 0                                                |
| Com_call_procedure                            | 0                                                |
| Com_change_db                                 | 284                                              |
| Com_change_master                             | 0                                                |
| Com_change_repl_filter                        | 0                                                |
| Com_check                                     | 0                                                |
| Com_checksum                                  | 0                                                |
| Com_commit                                    | 0                                                |
| Com_create_db                                 | 0                                                |
| Com_create_event                              | 0                                                |
| Com_create_function                           | 0                                                |
| Com_create_index                              | 0                                                |
| Com_create_procedure                          | 0                                                |
| Com_create_server                             | 0                                                |
| Com_create_table                              | 0                                                |
| Com_create_trigger                            | 0                                                |
| Com_create_udf                                | 0                                                |
| Com_create_user                               | 0                                                |
| Com_create_view                               | 0                                                |
| Com_dealloc_sql                               | 0                                                |
| Com_delete                                    | 18                                               |
| Com_delete_multi                              | 0                                                |
| Com_do                                        | 0                                                |
| Com_drop_db                                   | 0                                                |
| Com_drop_event                                | 0                                                |
| Com_drop_function                             | 0                                                |
| Com_drop_index                                | 0                                                |
| Com_drop_procedure                            | 0                                                |
| Com_drop_server                               | 0                                                |
| Com_drop_table                                | 0                                                |
| Com_drop_trigger                              | 0                                                |
| Com_drop_user                                 | 0                                                |
| Com_drop_view                                 | 0                                                |
| Com_empty_query                               | 0                                                |
| Com_execute_sql                               | 0                                                |
| Com_explain_other                             | 0                                                |
| Com_flush                                     | 0                                                |
| Com_get_diagnostics                           | 0                                                |
| Com_grant                                     | 0                                                |
| Com_ha_close                                  | 0                                                |
| Com_ha_open                                   | 0                                                |
| Com_ha_read                                   | 0                                                |
| Com_help                                      | 0                                                |
| Com_insert                                    | 342                                              |
| Com_insert_select                             | 0                                                |
| Com_install_plugin                            | 0                                                |
| Com_kill                                      | 0                                                |
| Com_load                                      | 0                                                |
| Com_lock_tables                               | 0                                                |
| Com_optimize                                  | 0                                                |
| Com_preload_keys                              | 0                                                |
| Com_prepare_sql                               | 0                                                |
| Com_purge                                     | 0                                                |
| Com_purge_before_date                         | 0                                                |
| Com_release_savepoint                         | 0                                                |
| Com_rename_table                              | 0                                                |
| Com_rename_user                               | 0                                                |
| Com_repair                                    | 0                                                |
| Com_replace                                   | 0                                                |
| Com_replace_select                            | 0                                                |
| Com_reset                                     | 0                                                |
| Com_resignal                                  | 0                                                |
| Com_revoke                                    | 0                                                |
| Com_revoke_all                                | 0                                                |
| Com_rollback                                  | 0                                                |
| Com_rollback_to_savepoint                     | 0                                                |
| Com_savepoint                                 | 0                                                |
| Com_select                                    | 16358                                            |
| Com_set_option                                | 849                                              |
| Com_signal                                    | 0                                                |
| Com_show_binlog_events                        | 0                                                |
| Com_show_binlogs                              | 0                                                |
| Com_show_charsets                             | 0                                                |
| Com_show_collations                           | 0                                                |
| Com_show_create_db                            | 0                                                |
| Com_show_create_event                         | 0                                                |
| Com_show_create_func                          | 0                                                |
| Com_show_create_proc                          | 0                                                |
| Com_show_create_table                         | 0                                                |
| Com_show_create_trigger                       | 0                                                |
| Com_show_databases                            | 3                                                |
| Com_show_engine_logs                          | 0                                                |
| Com_show_engine_mutex                         | 0                                                |
| Com_show_engine_status                        | 0                                                |
| Com_show_events                               | 0                                                |
| Com_show_errors                               | 0                                                |
| Com_show_fields                               | 921                                              |
| Com_show_function_code                        | 0                                                |
| Com_show_function_status                      | 0                                                |
| Com_show_grants                               | 0                                                |
| Com_show_keys                                 | 1                                                |
| Com_show_master_status                        | 0                                                |
| Com_show_open_tables                          | 0                                                |
| Com_show_plugins                              | 0                                                |
| Com_show_privileges                           | 0                                                |
| Com_show_procedure_code                       | 0                                                |
| Com_show_procedure_status                     | 0                                                |
| Com_show_processlist                          | 0                                                |
| Com_show_profile                              | 0                                                |
| Com_show_profiles                             | 0                                                |
| Com_show_relaylog_events                      | 0                                                |
| Com_show_slave_hosts                          | 0                                                |
| Com_show_slave_status                         | 0                                                |
| Com_show_status                               | 6                                                |
| Com_show_storage_engines                      | 0                                                |
| Com_show_table_status                         | 0                                                |
| Com_show_tables                               | 2                                                |
| Com_show_triggers                             | 0                                                |
| Com_show_variables                            | 6                                                |
| Com_show_warnings                             | 1                                                |
| Com_show_create_user                          | 0                                                |
| Com_shutdown                                  | 0                                                |
| Com_slave_start                               | 0                                                |
| Com_slave_stop                                | 0                                                |
| Com_group_replication_start                   | 0                                                |
| Com_group_replication_stop                    | 0                                                |
| Com_stmt_execute                              | 4                                                |
| Com_stmt_close                                | 4                                                |
| Com_stmt_fetch                                | 0                                                |
| Com_stmt_prepare                              | 4                                                |
| Com_stmt_reset                                | 0                                                |
| Com_stmt_send_long_data                       | 4                                                |
| Com_truncate                                  | 2                                                |
| Com_uninstall_plugin                          | 0                                                |
| Com_unlock_tables                             | 0                                                |
| Com_update                                    | 70                                               |
| Com_update_multi                              | 0                                                |
| Com_xa_commit                                 | 0                                                |
| Com_xa_end                                    | 0                                                |
| Com_xa_prepare                                | 0                                                |
| Com_xa_recover                                | 0                                                |
| Com_xa_rollback                               | 0                                                |
| Com_xa_start                                  | 0                                                |
| Com_stmt_reprepare                            | 0                                                |
| Connection_errors_accept                      | 0                                                |
| Connection_errors_internal                    | 0                                                |
| Connection_errors_max_connections             | 0                                                |
| Connection_errors_peer_address                | 0                                                |
| Connection_errors_select                      | 0                                                |
| Connection_errors_tcpwrap                     | 0                                                |
| Connections                                   | 292                                              |
| Created_tmp_disk_tables                       | 1124                                             |
| Created_tmp_files                             | 30                                               |
| Created_tmp_tables                            | 1369                                             |
| Delayed_errors                                | 0                                                |
| Delayed_insert_threads                        | 0                                                |
| Delayed_writes                                | 0                                                |
| Flush_commands                                | 1                                                |
| Handler_commit                                | 6094                                             |
| Handler_delete                                | 33                                               |
| Handler_discover                              | 0                                                |
| Handler_external_lock                         | 38571                                            |
| Handler_mrr_init                              | 0                                                |
| Handler_prepare                               | 0                                                |
| Handler_read_first                            | 2299                                             |
| Handler_read_key                              | 134761                                           |
| Handler_read_last                             | 237                                              |
| Handler_read_next                             | 310119                                           |
| Handler_read_prev                             | 2733                                             |
| Handler_read_rnd                              | 222350                                           |
| Handler_read_rnd_next                         | 472820                                           |
| Handler_rollback                              | 0                                                |
| Handler_savepoint                             | 0                                                |
| Handler_savepoint_rollback                    | 0                                                |
| Handler_update                                | 15605                                            |
| Handler_write                                 | 17310                                            |
| Innodb_buffer_pool_dump_status                | Dumping of buffer pool not started               |
| Innodb_buffer_pool_load_status                | Buffer pool(s) load completed at 180624 23:38:01 |
| Innodb_buffer_pool_resize_status              |                                                  |
| Innodb_buffer_pool_pages_data                 | 1035                                             |
| Innodb_buffer_pool_bytes_data                 | 16957440                                         |
| Innodb_buffer_pool_pages_dirty                | 0                                                |
| Innodb_buffer_pool_bytes_dirty                | 0                                                |
| Innodb_buffer_pool_pages_flushed              | 1936                                             |
| Innodb_buffer_pool_pages_free                 | 7144                                             |
| Innodb_buffer_pool_pages_misc                 | 13                                               |
| Innodb_buffer_pool_pages_total                | 8192                                             |
| Innodb_buffer_pool_read_ahead_rnd             | 0                                                |
| Innodb_buffer_pool_read_ahead                 | 0                                                |
| Innodb_buffer_pool_read_ahead_evicted         | 0                                                |
| Innodb_buffer_pool_read_requests              | 306665                                           |
| Innodb_buffer_pool_reads                      | 950                                              |
| Innodb_buffer_pool_wait_free                  | 0                                                |
| Innodb_buffer_pool_write_requests             | 26509                                            |
| Innodb_data_fsyncs                            | 1229                                             |
| Innodb_data_pending_fsyncs                    | 0                                                |
| Innodb_data_pending_reads                     | 0                                                |
| Innodb_data_pending_writes                    | 0                                                |
| Innodb_data_read                              | 16273920                                         |
| Innodb_data_reads                             | 1078                                             |
| Innodb_data_writes                            | 2857                                             |
| Innodb_data_written                           | 53379584                                         |
| Innodb_dblwr_pages_written                    | 1275                                             |
| Innodb_dblwr_writes                           | 109                                              |
| Innodb_log_waits                              | 0                                                |
| Innodb_log_write_requests                     | 450                                              |
| Innodb_log_writes                             | 585                                              |
| Innodb_os_log_fsyncs                          | 793                                              |
| Innodb_os_log_pending_fsyncs                  | 0                                                |
| Innodb_os_log_pending_writes                  | 0                                                |
| Innodb_os_log_written                         | 664064                                           |
| Innodb_page_size                              | 16384                                            |
| Innodb_pages_created                          | 56                                               |
| Innodb_pages_read                             | 988                                              |
| Innodb_pages_written                          | 1936                                             |
| Innodb_row_lock_current_waits                 | 0                                                |
| Innodb_row_lock_time                          | 0                                                |
| Innodb_row_lock_time_avg                      | 0                                                |
| Innodb_row_lock_time_max                      | 0                                                |
| Innodb_row_lock_waits                         | 0                                                |
| Innodb_rows_deleted                           | 2                                                |
| Innodb_rows_inserted                          | 19219                                            |
| Innodb_rows_read                              | 249102                                           |
| Innodb_rows_updated                           | 77                                               |
| Innodb_num_open_files                         | 81                                               |
| Innodb_truncated_status_writes                | 0                                                |
| Innodb_available_undo_logs                    | 128                                              |
| Key_blocks_not_flushed                        | 0                                                |
| Key_blocks_unused                             | 12751                                            |
| Key_blocks_used                               | 645                                              |
| Key_read_requests                             | 321877                                           |
| Key_reads                                     | 648                                              |
| Key_write_requests                            | 196                                              |
| Key_writes                                    | 150                                              |
| Locked_connects                               | 0                                                |
| Max_execution_time_exceeded                   | 0                                                |
| Max_execution_time_set                        | 0                                                |
| Max_execution_time_set_failed                 | 0                                                |
| Max_used_connections                          | 3                                                |
| Max_used_connections_time                     | 2018-06-24 23:43:48                              |
| Not_flushed_delayed_rows                      | 0                                                |
| Ongoing_anonymous_transaction_count           | 0                                                |
| Open_files                                    | 229                                              |
| Open_streams                                  | 0                                                |
| Open_table_definitions                        | 206                                              |
| Open_tables                                   | 786                                              |
| Opened_files                                  | 502                                              |
| Opened_table_definitions                      | 208                                              |
| Opened_tables                                 | 817                                              |
| Performance_schema_accounts_lost              | 0                                                |
| Performance_schema_cond_classes_lost          | 0                                                |
| Performance_schema_cond_instances_lost        | 0                                                |
| Performance_schema_digest_lost                | 0                                                |
| Performance_schema_file_classes_lost          | 0                                                |
| Performance_schema_file_handles_lost          | 0                                                |
| Performance_schema_file_instances_lost        | 0                                                |
| Performance_schema_hosts_lost                 | 0                                                |
| Performance_schema_index_stat_lost            | 0                                                |
| Performance_schema_locker_lost                | 0                                                |
| Performance_schema_memory_classes_lost        | 0                                                |
| Performance_schema_metadata_lock_lost         | 0                                                |
| Performance_schema_mutex_classes_lost         | 0                                                |
| Performance_schema_mutex_instances_lost       | 0                                                |
| Performance_schema_nested_statement_lost      | 0                                                |
| Performance_schema_prepared_statements_lost   | 0                                                |
| Performance_schema_program_lost               | 0                                                |
| Performance_schema_rwlock_classes_lost        | 0                                                |
| Performance_schema_rwlock_instances_lost      | 0                                                |
| Performance_schema_session_connect_attrs_lost | 0                                                |
| Performance_schema_socket_classes_lost        | 0                                                |
| Performance_schema_socket_instances_lost      | 0                                                |
| Performance_schema_stage_classes_lost         | 0                                                |
| Performance_schema_statement_classes_lost     | 0                                                |
| Performance_schema_table_handles_lost         | 0                                                |
| Performance_schema_table_instances_lost       | 0                                                |
| Performance_schema_table_lock_stat_lost       | 0                                                |
| Performance_schema_thread_classes_lost        | 0                                                |
| Performance_schema_thread_instances_lost      | 0                                                |
| Performance_schema_users_lost                 | 0                                                |
| Prepared_stmt_count                           | 0                                                |
| Qcache_free_blocks                            | 1                                                |
| Qcache_free_memory                            | 16760152                                         |
| Qcache_hits                                   | 0                                                |
| Qcache_inserts                                | 0                                                |
| Qcache_lowmem_prunes                          | 0                                                |
| Qcache_not_cached                             | 16355                                            |
| Qcache_queries_in_cache                       | 0                                                |
| Qcache_total_blocks                           | 1                                                |
| Queries                                       | 19164                                            |
| Questions                                     | 19155                                            |
| Select_full_join                              | 0                                                |
| Select_full_range_join                        | 0                                                |
| Select_range                                  | 2677                                             |
| Select_range_check                            | 0                                                |
| Select_scan                                   | 2098                                             |
| Slave_open_temp_tables                        | 0                                                |
| Slow_launch_threads                           | 0                                                |
| Slow_queries                                  | 0                                                |
| Sort_merge_passes                             | 12                                               |
| Sort_range                                    | 4859                                             |
| Sort_rows                                     | 244452                                           |
| Sort_scan                                     | 854                                              |
| Ssl_accept_renegotiates                       | 0                                                |
| Ssl_accepts                                   | 0                                                |
| Ssl_callback_cache_hits                       | 0                                                |
| Ssl_cipher                                    |                                                  |
| Ssl_cipher_list                               |                                                  |
| Ssl_client_connects                           | 0                                                |
| Ssl_connect_renegotiates                      | 0                                                |
| Ssl_ctx_verify_depth                          | 0                                                |
| Ssl_ctx_verify_mode                           | 0                                                |
| Ssl_default_timeout                           | 0                                                |
| Ssl_finished_accepts                          | 0                                                |
| Ssl_finished_connects                         | 0                                                |
| Ssl_server_not_after                          |                                                  |
| Ssl_server_not_before                         |                                                  |
| Ssl_session_cache_hits                        | 0                                                |
| Ssl_session_cache_misses                      | 0                                                |
| Ssl_session_cache_mode                        | NONE                                             |
| Ssl_session_cache_overflows                   | 0                                                |
| Ssl_session_cache_size                        | 0                                                |
| Ssl_session_cache_timeouts                    | 0                                                |
| Ssl_sessions_reused                           | 0                                                |
| Ssl_used_session_cache_entries                | 0                                                |
| Ssl_verify_depth                              | 0                                                |
| Ssl_verify_mode                               | 0                                                |
| Ssl_version                                   |                                                  |
| Table_locks_immediate                         | 11962                                            |
| Table_locks_waited                            | 0                                                |
| Table_open_cache_hits                         | 19395                                            |
| Table_open_cache_misses                       | 817                                              |
| Table_open_cache_overflows                    | 12                                               |
| Tc_log_max_pages_used                         | 0                                                |
| Tc_log_page_size                              | 0                                                |
| Tc_log_page_waits                             | 0                                                |
| Threads_cached                                | 2                                                |
| Threads_connected                             | 1                                                |
| Threads_created                               | 3                                                |
| Threads_running                               | 1                                                |
| Uptime                                        | 2944                                             |
| Uptime_since_flush_status                     | 2944                                             |
+-----------------------------------------------+--------------------------------------------------+
353 rows in set (0.00 sec)

Read more on SHOW GLOBAL STATUS here. Read more on the values here.

I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.

Configure Ubuntu System Memory Usage

Edit /etc/sysctl.conf

Add the following to allow things to sit in ram longer

vm.vfs_cache_pressure=50

Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt

This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are."

Read these pages here and here regarding setting MySQL memory.

Reboot

shutdown -r now

Resize the disk

The 2x CPU, 4GB memory plan comes with 80GB storage allowance.  My disk at present is 50GB and I will update the size soon following this guide soon.

Upgrade disk from 50gb to 80GB soon

Quick Benchmark

I used loader.io to load 500 users to access my site in 1 minute.

HTOP showing 2x busy CPU's running at 60%

The benchmark worked with no errors.

Loader.io Success with 500 concurrent users

This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.

Conclusion

After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.

No Low memory low triggers

This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

1.0 Initial Post

Filed Under: Backup, Cloud, DB, Domain, GUI, NGINX, Performance, PHP, Scalability, Server, Ubuntu, UpCloud, Upgrade VM Tagged With: add, an, and, cpu, Disk, memory, more, on, server, Space, to, ubuntu, UpCloud, Upgrading

  • Go to page 1
  • Go to page 2
  • Go to page 3
  • Go to Next Page »

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) HTTPS (6) IoT (9) LetsEncrypt (7) Linux (20) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) PHP (13) Scalability (12) Scalable (14) Security (44) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (44) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT