• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

from

Setup a dedicated Debian subdomain (VM), Install MySQL 14 and connect to it from a WordPress on a different VM

July 21, 2018 by Simon

This is how I set up a dedicated Debian subdomain (VM), Installed MySQL 14 and connected to it from a WordPress installation on a different VM

Aside

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving fearby.com from Vultr to UpCloud.

Buy a domain name here

Domain names for just 88 cents!

Now on with the post.

Fearby.com

I will be honest, fearby.com is my play server where I can code, learn about InfoSec and share (It’s also my stroke rehab blog).

There is no faster way to learn than actually doing. The problem is my “doing” usually breaks the live site from time to time (sorry).

I really need to set up a testing environment (DEV-TEST-LIVE or GREEN-BLUE) server(s). GREEN-BLUE has advantages as I can always have a hot spare ready. All I need to do is toggle DNS and I can set the GREEN or BLUE server as the live server.

But first  I need to separate my database from my current fearby.com server and setup up a new web server. Having a Green and Blue server that uses one database server will help with near real-time production website switches.

Dedicated Database Server

I read the following ( Should MySQL and Web Server share the same server? ) at Percona Database Performance Blog. Having a separate database server should not negatively impact performance (It may even help improve speeds).

Deploy a Debian VM (not Ubuntu)

I decided to set up a Debian server instead of Ubuntu (mostly because of the good focus on stability and focus on security within Debian).

I logged into the UpCloud dashboard on my mobile phone and deployed a Debian server in 5 mins.  I will be using my existing how to setup Ubuntu on UpCloud guide (even though this is Debian).

TIP: Sign up to UpCloud using this link to get $25 free UpCloud VM credit.

Deploy Debian Sevrer

Deploy a Debian server setup steps:

  1. Login to UpCloud and go to Create server.
  2. Name your Server (use a fully qualified domain name)
  3. Add a description.
  4. Choose your data centre (Chicago for me)
  5. Choose the server specs (1x CPU, 50GB Disk, 2GB Memory, 2TB Traffic for me)
  6. Name the Primary disk
  7. Choose an operating system (Debian for me)
  8. Select an SSH Key
  9. Choose misc settings
  10. Click Deploy server

After 5 mins your server should be deployed.

After Deploy

Setup DNS

Login to your DNS provider and create DNS records to the new IP’s (IPv4 and IPv6) provided by UpCloud. It took DNS 12 hours to replicate to my in Australia.

Add a DNS record with your domain registra A NAMe = IPV4 and AAAA Name = IPv6

Setup a Firewall (at UpCloud)

I would recommend you set up a firewall at UpCloud as soon as possible (don’t forget to add the recommended UpCloud DNS IP’s and any whitelisted IP’s your firewall).

Block everything and only allow

  • Port 22: Allow known IP(s) of your ISP or VPN.
  • Port 53: Allow known UpCloud DNS servers
  • Port 80 (ALL)
  • Port 443 (ALL)
  • Port 3306 Allow your WordPress site and known IP(s) of your ISP or VPN.

Read my post on setting up a whitelisted IP on an UpCloud VM… as it is a good idea.

UpCloud thankfully has a copy firewall feature that is very handy.

Copy Firewall rules option at UpCloud

After I set up the firewall I SSH’ed into my server (I use vSSH on OSX buy you could use PUTTY).

I updated the Debian system with the following  command

sudo apt update

Get the MySQL Package

Visit http://repo.mysql.com/ and get the URL of the latest apt-config repo deb file (e.g “mysql-apt-config_0.8.9-1_all.deb”). Make a temp folder.

mkdir /temp
cd /temp

Download the MySQL deb Package

wget http://repo.mysql.com/mysql-apt-config_0.8.9-1_all.deb

Install the package

sudo dpkg -i mysql-apt-config_0.8.9-1_all.deb

Update the system again

sudo apt update

Install MySQL on Debian

sudo apt install mysql-server
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libaio1 libatomic1 libmecab2 mysql-client mysql-common mysql-community-client mysql-community-server psmisc
The following NEW packages will be installed:
libaio1 libatomic1 libmecab2 mysql-client mysql-common mysql-community-client mysql-community-server mysql-server psmisc
0 upgraded, 9 newly installed, 0 to remove and 1 not upgraded.
Need to get 37.1 MB of archives.
After this operation, 256 MB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-community-client amd64 5.7.22-1debian9 [8886 kB]
Get:2 http://deb.debian.org/debian stretch/main amd64 mysql-common all 5.8+1.0.2 [5608 B]
Get:3 http://deb.debian.org/debian stretch/main amd64 libaio1 amd64 0.3.110-3 [9412 B]
Get:4 http://deb.debian.org/debian stretch/main amd64 libatomic1 amd64 6.3.0-18+deb9u1 [8966 B]
Get:5 http://deb.debian.org/debian stretch/main amd64 psmisc amd64 22.21-2.1+b2 [123 kB]
Get:6 http://deb.debian.org/debian stretch/main amd64 libmecab2 amd64 0.996-3.1 [256 kB]
Get:7 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-client amd64 5.7.22-1debian9 [12.4 kB]
Get:8 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-community-server amd64 5.7.22-1debian9 [27.8 MB]
Get:9 http://repo.mysql.com/apt/debian stretch/mysql-5.7 amd64 mysql-server amd64 5.7.22-1debian9 [12.4 kB]
Fetched 37.1 MB in 12s (3023 kB/s)
Preconfiguring packages ...
Selecting previously unselected package mysql-common.
(Reading database ... 34750 files and directories currently installed.)
Preparing to unpack .../0-mysql-common_5.8+1.0.2_all.deb ...
Unpacking mysql-common (5.8+1.0.2) ...
Selecting previously unselected package libaio1:amd64.
Preparing to unpack .../1-libaio1_0.3.110-3_amd64.deb ...
Unpacking libaio1:amd64 (0.3.110-3) ...
Selecting previously unselected package libatomic1:amd64.
Preparing to unpack .../2-libatomic1_6.3.0-18+deb9u1_amd64.deb ...
Unpacking libatomic1:amd64 (6.3.0-18+deb9u1) ...
Selecting previously unselected package mysql-community-client.
Preparing to unpack .../3-mysql-community-client_5.7.22-1debian9_amd64.deb ...
Unpacking mysql-community-client (5.7.22-1debian9) ...
Selecting previously unselected package mysql-client.
Preparing to unpack .../4-mysql-client_5.7.22-1debian9_amd64.deb ...
Unpacking mysql-client (5.7.22-1debian9) ...
Selecting previously unselected package psmisc.
Preparing to unpack .../5-psmisc_22.21-2.1+b2_amd64.deb ...
Unpacking psmisc (22.21-2.1+b2) ...
Selecting previously unselected package libmecab2:amd64.
Preparing to unpack .../6-libmecab2_0.996-3.1_amd64.deb ...
Unpacking libmecab2:amd64 (0.996-3.1) ...
Selecting previously unselected package mysql-community-server.
Preparing to unpack .../7-mysql-community-server_5.7.22-1debian9_amd64.deb ...
Unpacking mysql-community-server (5.7.22-1debian9) ...
Selecting previously unselected package mysql-server.
Preparing to unpack .../8-mysql-server_5.7.22-1debian9_amd64.deb ...
Unpacking mysql-server (5.7.22-1debian9) ...
Setting up libatomic1:amd64 (6.3.0-18+deb9u1) ...
Setting up psmisc (22.21-2.1+b2) ...
Setting up mysql-common (5.8+1.0.2) ...
update-alternatives: using /etc/mysql/my.cnf.fallback to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Setting up libmecab2:amd64 (0.996-3.1) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Setting up libaio1:amd64 (0.3.110-3) ...
Processing triggers for systemd (232-25+deb9u4) ...
Processing triggers for man-db (2.7.6.1-2) ...
Setting up mysql-community-client (5.7.22-1debian9) ...
Setting up mysql-client (5.7.22-1debian9) ...
Setting up mysql-community-server (5.7.22-1debian9) ...
update-alternatives: using /etc/mysql/mysql.cnf to provide /etc/mysql/my.cnf (my.cnf) in auto mode
Created symlink /etc/systemd/system/multi-user.target.wants/mysql.service -> /lib/systemd/system/mysql.service.
Setting up mysql-server (5.7.22-1debian9) ...
Processing triggers for libc-bin (2.24-11+deb9u3) ...
Processing triggers for systemd (232-25+deb9u4) ...

Secure MySQL

You can secure the MySQL server deployment (set options as needed)

sudo mysql_secure_installation

Enter password for user root:
********************************************
VALIDATE PASSWORD PLUGIN can be used to test passwords
and improve security. It checks the strength of password
and allows the users to set only those passwords which are
secure enough. Would you like to setup VALIDATE PASSWORD plugin?

Press y|Y for Yes, any other key for No: No
Using existing password for root.
Change the password for root ? ((Press y|Y for Yes, any other key for No) : No

... skipping.
By default, a MySQL installation has an anonymous user,
allowing anyone to log into MySQL without having to have
a user account created for them. This is intended only for
testing, and to make the installation go a bit smoother.
You should remove them before moving into a production
environment.

Remove anonymous users? (Press y|Y for Yes, any other key for No) : Yes
Success.

Normally, root should only be allowed to connect from
'localhost'. This ensures that someone cannot guess at
the root password from the network.

Disallow root login remotely? (Press y|Y for Yes, any other key for No) : No

... skipping.
By default, MySQL comes with a database named 'test' that
anyone can access. This is also intended only for testing,
and should be removed before moving into a production
environment.

Remove test database and access to it? (Press y|Y for Yes, any other key for No) : Yes
- Dropping test database...
Success.

- Removing privileges on test database...
Success.

Reloading the privilege tables will ensure that all changes
made so far will take effect immediately.

Reload privilege tables now? (Press y|Y for Yes, any other key for No) : Yes
Success.

All done!

Install NGINX

I installed NGINX to allow Adminer MySQL GUI to be used

I ran these commands to install NGINX.

sudo apt update
sudo apt upgrade
sudo apt-get install nginx

I edited my NGINX configuration as required.

  • Set a web server root
  • Set desired headers
  • Optimized NGINX (see past guides here, here and here)

I reloaded NGINX

sudo nginx -t
sudo nginx -s reload
sudo systemctl restart nginx

Install PHP

I followed this guide to install PHP on Debian.

sudo apt update
sudo apt upgrade

sudo apt install ca-certificates apt-transport-https
wget -q https://packages.sury.org/php/apt.gpg -O- | sudo apt-key add -
echo "deb https://packages.sury.org/php/ stretch main" | sudo tee /etc/apt/sources.list.d/php.list

sudo apt update
sudo apt install php7.2
sudo apt install php-pear php7.2-curl php7.2-dev php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml php7.2-cli php7.2-common

Install PHP FPM

apt-get install php7.2-fpm

Increase Upload Limits

You may need to temporarily increase upload limits in NGINX and PHP before you can restore a WordPress database. My feabry.com blog is about 87MB.

Add “client_max_body_size 100M;” to “/etc/nginx/nginx.conf”

Add the following to “/etc/php/7.2/fpm/php.ini”

  • post_max_size = 100M
  • upload_max_filesize = 100M

Restore a backup of my MySQL database in MySQL

You can now use Adminer to restore your blog to MySQL. Read my post here on Adminer here. I used Adminer to move my WordPress site from CPanel to a self-managed server a year ago.

First login to your source server and export your desired database then login to the target server and import the database.

Firewall Check

Don’t forget to allow your WordPress site’s 2x Public IP’s and 1x Private IP to access port 3306 in your UpCloud Firewall.

How to check open ports on your current server

sudo netstat -plunt

Set MySQL Permissions

Open MySQL

mysql --host=localhost --user=root --password=***************************************************************************

I ran these statements to grant the user logging in on the nominate IP’s access to MySQL.

mysql>
GRANT ALL ON databasenmae.* TO [email protected] IDENTIFIED BY '***********sql*user*password*************';
GRANT ALL ON databasenmae.* TO [email protected] IDENTIFIED BY '***********sql*user*password*************';
GRANT ALL ON databasenmae.* TO [email protected] IDENTIFIED BY '***********sql*user*password*************';
GRANT ALL ON databasenmae.* TO [email protected] IDENTIFIED BY '***********sql*user*password*************';

Reload permissions in MySQL

FLUSH PRIVILEGES;

Allow access to the Debian machine from known IP’s

Edit “/etc/host.allow”

Additions (known safe IP’s that need access to this MySQL remotely).

mysqld : IPv4Server1PublicAddress : allow
mysqld : IPv4Server1PrivateAddress : allow
mysqld : IPv4Server2PublicAddress : allow
mysqld : IPv4Server1PrivateAddress : allow

mysqld : ALL : deny

Tell MySQL to listen on

Edit “/etc/mysql/my.cnf”

Added..

[mysqld]
user = mysql
pid-file = /var/run/mysqld/mysqld.pid
socket = /var/run/mysqld/mysqld.sock
port = 3306
basedir = /usr
datadir = /var/lib/mysql
tmpdir = /tmp
language = /usr/share/mysql/English
bind-address = DebianServersIntenalIPv4Address

I guess you could change the port to something random???

Restart MySQL

sudo service mysql restart

Install a second local firewall on Debian

Install ufw

sudo apt-get instal ufw

Do add the IP of your desired server or VPN to access SSH

sudo ufw allow from 123.123.123.123 to any port 22

Do add the IP of your desired server or VPN to access WWW

sudo ufw allow from 123.123.123.123 to any port 80

Now add the known IP’s (e.g any web servers public (IPv4/IPv6) or Private IP’s) that you wish to grant access to MySQL (e.g the source website that used to have MySQL)

sudo ufw allow from 123.123.123.123 to any port 3306

Do add UpCloud DNS Servers to your firewall

sudo ufw allow from 94.237.127.9 to any port 53
sudo ufw allow from 94.237.40.9 to any port 53
sudo ufw allow from 2a04:3544:53::1 to any port 53
sudo ufw allow from 2a04:3540:53::1 to any port 53

Add all other rules as needed (if you stuff up and lock your self out you can login to the server with the Console on UpCloud)

Restart the ufw firewall

sudo ufw disable
sudo ufw enable

Prevent MySQL starting on the source server

Now we can shut down MySQL on the source server (leave it there just in case).

Edit “/etc/init/mysql.conf”

Comment out the line that contains “start on ” and save the file

and run

sudo systemctl disable mysql

Reboot

shutdown -r now

Stop and Disable NGINX on the new DB server

We don’t need NGINX running now the database has been imported with Adminer.

Stop and prevent NGINX from starting up on startup.

/etc/init.d/nginx stop
sudo update-rc.d -f nginx disable
sudo systemctl disable nginx

Check to see if MySQL is Disabled

service mysql status
* mysql.service - MySQL Community Server
Loaded: loaded (/lib/systemd/system/mysql.service; disabled; vendor preset: enabled)
Active: inactive (dead)

Yep

Test access to the database server in PHP code

Add to dbtest.php

<em>SELECT guid FROM wp_posts</em>()<br />
<ul><?php

//External IP (charged after quota hit)
//$servername = 'db.yourserver.com';

//Private IP (free)
//$servername = '10.x.x.x';

$username = 'username';
$password = '********your*password*********';
$dbname = 'database';

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}

$sql = 'SELECT guid FROM wp_posts';
$result = $conn->query($sql);

if ($result->num_rows > 0) {
    // output data of each row
    while($row = $result->fetch_assoc()) {
        echo $row["guid"] . "<br>";
    }
} else {
    echo "0 results";
}
$conn->close();
?></ul>
Done

Check for open ports.

You can install nmap on another server and scan for open ports

Install nmap

sudo apt-get install nmap

Scan a server for open ports with nmap

You should see this on a server that has access to see port 3306 (port 3306 should not be visible by non-whitelisted IP’s).  Port 3shouldoudl not be seen via everyone.

sudo nmap -PN db.yourserver.com

Starting Nmap 7.40 ( https://nmap.org ) at 2018-07-20 14:15 UTC
Nmap scan report for db.yourserver.com (IPv4IP)
Host is up (0.0000070s latency).
Other addresses for db.yourserver.com (not scanned): IPv6IP
Not shown: 997 closed ports
PORT     STATE SERVICE
3306/tcp open  mysql

You should see something like this on a server that has access to see port 80/443 (a web server)

sudo nmap -PN yourserver.com

Starting Nmap 7.40 ( https://nmap.org ) at 2018-07-20 14:18 UTC
Nmap scan report for db.yourserver.com (IPv4IP)
Host is up (0.0000070s latency).
Other addresses for db.yourserver.com (not scanned): IPv6IP
Not shown: 997 closed ports
PORT     STATE SERVICE
80/tcp   open  http
443/tcp   open  https

I’d recommend you use a service like https://pentest-tools.com/network-vulnerability-scanning/tcp-port-scanner-online-nmap# to check for open ports.  https://hackertarget.com/tcp-port-scan/ is a great tool too.

https://www.infobyip.com/tcpportchecker.php is also a free port checker that you can use to verify individual closed ports.

Screeshot of https://www.infobyip.com/tcpportchecker.php

Hardening MySQL and Debian

Read: https://www.debian.org/doc/manuals/securing-debian-howto/ap-checklist.en.html

Configuring WordPress use the dedicated Debian VM

On the source server that used to have MySQL edit your wp-config.php file for WordPress.

Remove

define('DB_HOST', 'localhost');

add (read the update below, I changed the DNS IP to the Private IP to have free traffic)

//Oriinal localhost
//define('DB_HOST', 'localhost');

//New external host via DNS (Charged after quota hit)
//define('DB_HOST', 'db.fearby.com');

//New external host via Private IP (Free)
define('DB_HOST','10.x.x.x');

Restart NGINX

sudo nginx -t
sudo nginx -s reload
sudo systemctl restart nginx

Restart PHP-FPM

service php7.2-fpm restart

Conclusion

Nice, I seem to have shaved off 0.3 seconds in load times (25% improvement)

1sec gtmtrix load time

Update: Using a Private IP or Public IP between WordPress and MySQL servers

After I released this blog post (version 1.0 with no help from UpCloud) UpCloud contacted me and said the following.

Hello Simon,

I notice there's no mention of using the private network IPs. Did you know that we automagically assign you one when you deploy with our templates. The private network works out of the box without additional configuration, you can use that communicate between your own cloud servers and even across datacentres.

There's no bandwidth charge when communicating over private network, they do not go through public internet as well. With this, you can easily build high redundant setups.

Let me know if you have any other questions.

--
Kelvin from UpCloud

I will have updated my references in this post and replace the public IP address (that is linked to DNS record for db.fearby.com) and instead use the private ip address (e.g 10.x.x.x), your servers private IP address is listed against the public IPv$ and IPv6 address.

I checked that the local ufw firewall did indeed allow the private IP access to MySQL.

sudo ufw status numbered |grep 10.x.x.x
[27] 3306                       ALLOW IN    10.x.x.x

On my new Debian MySQL server, I edited the file /etc/mysql/my.cnf and changed the IP to the private IP and not the public IP.

Now it looked like

[mysqld]
user            = mysql
pid-file        = /var/run/mysqld/mysqld.pid
socket          = /var/run/mysqld/mysqld.sock
port            = 3306
basedir         = /usr
datadir         = /var/lib/mysql
tmpdir          = /tmp
language        = /usr/share/mysql/English
bind-address    = 10.x.x.x

(10.x.x.x  my Debian servers private IP)

On my WordPress instance, I edited the file  /www-root/wp-config.php

I added the new private host

//Oriinal localhost
//define('DB_HOST', 'localhost');

//New external host via DNS (Charged after quota hit)
//define('DB_HOST', 'db.fearby.com');

//New external host via Private IP (Free)
define('DB_HOST','10.x.x.x');

(10.x.x.x  my Debian servers private IP)

Alos on Debian/MySQL ensure you have granted access to the private IP of the WordPress server

Edit /etc/host.allow

Add

mysqld : 10.x.x.x : allow

Restart MySQL

sudo systemctl restart mysql

TIP: Enable UpCloud Backups

Do setup automatic backups (and or take manual backups). Backups are an extra charge but are essential IMHO.

UpCloud backups

Troubleshooting

If you can’t access MySQL log back into MySQL

mysql --host=localhost --user=root --password=***************************************************************************

and run

GRANT ALL PRIVILEGES ON *.* TO [email protected]'%' IDENTIFIED BY '***********sql*user*password*************''; FLUSH PRIVILEGES;

Reboot

Lower Upload Limits

Don’t forget to lower file upload sizes in NGINX and PHP (e.g 2M) now that the database has been restored.

I hope this guide helps someone.

TIP: Sign up to UpCloud using this link to get $25 free UpCloud VM credit.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.6 Changed Public IP use to private IP use to ensure we are not charged when the serves sage goes over the quota

v1.5 Fixed 03 type (should have been 0.3)

v1.4 added disable nginx info

v1.3 added https://www.infobyip.com/tcpportchecker.php

v1.1 added https://hackertarget.com/tcp-port-scan/

v1.0 Initial Post

Filed Under: Debian, MySQL, VM, Wordpress Tagged With: 14, a, and, Connect, debian, dedicated, different, from, install, MySQL, Setup, Subdomain, to, vm, wordpress

Moving an Ubuntu 16.04 VM on Vultr from one data centre to another via snapshots

April 17, 2018 by Simon

This guide will show how you can move an Ubuntu VM server domain between Vultr data centres via snapshots.

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Sometimes you need to move a sever between locations and/or upgrade the server (to have more memory t install WordPress).

Moving an existing Vultr server

If you don’t have an Ubuntu server click here (follow this guide).

Login to Vultr and specify a source server, click Snapshots and click Take Snapshot.

Make snapshot

Wait for the snapshot to finish (It may take 1 hour).

Snapshot Started

Great, the snapshot is done.

Snapshot Ready

Now I can create a new server (in a different data centre).

Add

Deploy New Instance

Choose a location (Australia is at capacity, so I’ll deploy to Silicon Valley then move again in a few weeks), choose the snapshot to restore, choose a plan, I enabled IPV6/Auto Backups and Private Networking.

TIP: The password for the server will be the same as the source server so write it down.

Deploy

Click Deploy Now

Deploy

After a few minutes, you can see the new servers IP address, you can log in to your domain name provider (in my case Namecheap) and update the target IPV4 and IPV6 address.

You can find IPV4 and IPV6 addresses by opening your server, clicking settings then IPxV4 or IPV6.

ip

You will need to update Vultr DNS settings (login to Vultr, Click Servers, Click DNS then edit your existing Domain DNS entry).  Add you’re new serves IP addresses.

Vultr DNS

Update: I added an IPV6/AAAA record too.

Wait for DNS Replication

Goto https://www.whatsmydns.net/ and check the global DNS propagation for your new domain’s server.

DNS Propigation

If you are happy that the server has been migrated (snapshot restored) and that the domain DNS is pointing to your new server you can delete the old server in the Vultr server list.

Servers

Post-Migrate Actions

  • Setup Daily backups.
  • Review firewall settings (guide here).
  • Optional: Install MySQL
  • Optional: Install PHP
  • Optional: Install PHP Pooled Connections
  • Optional: Install WordPress
  • Optional: Install WordPress CDN
  • Optional: Configure Cloudflare
  • etc

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Vultr Link

v1.0 Initial post

Filed Under: Linux, Migrate, Server, Ubuntu, Vultr, Wordpress Tagged With: 16.04, an, another, center, data, from, Moving, New Jersey, on, snapshot, sydney, to, ubuntu, vm, vultr

Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

April 10, 2018 by Simon

This guide will help you install and setup the pm2 NodejJS process monitor PM2 from Keymetrics.io for free and manage your node apps performance and exceptions.

What is PM2?

PM2 is a production process manager for Node.js applications with a built-in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks. This is the steps I used on Ubuntu 16.04. This is NOT a paid endorsement (just self-documenting).

Key Features of PM2

PM2 offers web-based monitoring dashboard, exception reporting, load balancer, CPU and memory monitoring, transaction tracer and much more for NodeJS apps.

pm2-features

What is PM2?

Official page: http://pm2.keymetrics.io/

More info https://www.npmjs.com/package/pm2

Install PM2

npm install pm2 -g

Install Output

npm install pm2 -g
/usr/bin/pm2 -> /usr/lib/node_modules/pm2/bin/pm2
/usr/bin/pm2-dev -> /usr/lib/node_modules/pm2/bin/pm2-dev
/usr/bin/pm2-docker -> /usr/lib/node_modules/pm2/bin/pm2-docker
/usr/bin/pm2-runtime -> /usr/lib/node_modules/pm2/bin/pm2-runtime
/usr/lib
└─┬ [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ ├── [email protected]
  │ ├── [email protected]
  │ ├─┬ [email protected]
  │ │ └── [email protected]
  │ ├── [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ ├─┬ [email protected]
  │ │ └─┬ [email protected]
  │ │   ├── [email protected]
  │ │   ├─┬ [email protected]
  │ │   │ └─┬ [email protected]
  │ │   │   ├── [email protected]
  │ │   │   └── [email protected]
  │ │   ├─┬ [email protected]
  │ │   │ ├── [email protected]
  │ │   │ └─┬ [email protected]
  │ │   │   └── [email protected]
  │ │   ├─┬ [email protected]
  │ │   │ ├─┬ [email protected]
  │ │   │ │ └─┬ [email protected]
  │ │   │ │   ├── [email protected]
  │ │   │ │   └── [email protected]
  │ │   │ ├─┬ [email protected]
  │ │   │ │ ├── [email protected]
  │ │   │ │ ├── [email protected]
  │ │   │ │ ├── [email protected]
  │ │   │ │ └── [email protected]
  │ │   │ └── [email protected]
  │ │   ├── [email protected]
  │ │   ├── [email protected]
  │ │   ├─┬ [email protected]
  │ │   │ ├─┬ [email protected]
  │ │   │ │ └── [email protected]
  │ │   │ └── [email protected]
  │ │   ├── [email protected]
  │ │   └── [email protected]
  │ ├── [email protected]
  │ ├─┬ [email protected]
  │ │ ├── [email protected]
  │ │ ├── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ └── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├─┬ [email protected]
  │ │ │ │ └─┬ [email protected]
  │ │ │ │   └── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ └── [email protected]
  │ │ ├── [email protected]
  │ │ ├── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ ├─┬ [email protected]
  │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ │ ├── [email protected]
  │ │ │ │ │ │ └── [email protected]
  │ │ │ │ │ ├── [email protected]
  │ │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ │ └─┬ [email protected]
  │ │ │ │ │ │   └── [email protected]
  │ │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ │ └── [email protected]
  │ │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ │ └── [email protected]
  │ │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ │ └─┬ [email protected]
  │ │ │ │ │ │   └── [email protected]
  │ │ │ │ │ └─┬ [email protected]
  │ │ │ │ │   └─┬ [email protected]
  │ │ │ │ │     ├── [email protected]
  │ │ │ │ │     └── [email protected]
  │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ ├── [email protected]
  │ │ │ │ │ ├── [email protected]
  │ │ │ │ │ └─┬ [email protected]
  │ │ │ │ │   ├── [email protected]
  │ │ │ │ │   └─┬ [email protected]
  │ │ │ │ │     ├── [email protected]
  │ │ │ │ │     ├── [email protected]
  │ │ │ │ │     └── [email protected]
  │ │ │ │ ├── [email protected]
  │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ └─┬ [email protected]
  │ │ │ │ │   ├── [email protected]
  │ │ │ │ │   └── [email protected]
  │ │ │ │ ├─┬ [email protected]
  │ │ │ │ │ ├── [email protected]
  │ │ │ │ │ └── [email protected]
  │ │ │ │ └── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├─┬ [email protected]
  │ │ │ │ └─┬ [email protected]
  │ │ │ │   ├─┬ [email protected]
  │ │ │ │   │ └── [email protected]
  │ │ │ │   ├─┬ [email protected]
  │ │ │ │   │ └── [email protected]
  │ │ │ │   └── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├─┬ source-map-resolve[email protected]
  │ │ │ │ ├── [email protected]
  │ │ │ │ ├── [email protected]
  │ │ │ │ ├── [email protected]
  │ │ │ │ ├── [email protected]
  │ │ │ │ └── [email protected]
  │ │ │ └── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ ├─┬ [email protected]
  │ │ │ │ └─┬ [email protected]
  │ │ │ │   ├── [email protected]
  │ │ │ │   └── [email protected]
  │ │ │ └─┬ [email protected]
  │ │ │   └── [email protected]
  │ │ ├── [email protected]
  │ │ └─┬ [email protected]
  │ │   └─┬ [email protected]
  │ │     └── [email protected]
  │ ├─┬ [email protected]
  │ │ ├── [email protected]
  │ │ └── [email protected]
  │ ├── [email protected]
  │ ├─┬ [email protected]
  │ │ └── [email protected]
  │ ├─┬ [email protected]
  │ │ └── [email protected]
  │ ├─┬ [email protected]
  │ │ └── [email protected]
  │ ├── [email protected]
  │ ├─┬ [email protected]
  │ │ ├── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ └─┬ [email protected]
  │ │ │   ├── [email protected]
  │ │ │   └── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ ├── [email protected]
  │ │ │ └── [email protected]
  │ │ └── [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ ├── [email protected]
  │ ├─┬ [email protected]
  │ │ └── [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ ├── [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ ├── [email protected]
  │ ├── [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ ├── [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  ├─┬ [email protected]
  │ ├── [email protected]
  │ ├── [email protected]
  │ └─┬ [email protected]
  │   ├─┬ [email protected]
  │   │ ├── [email protected]
  │   │ └── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   ├── [email protected]
  │   └── [email protected]
  ├─┬ [email protected]
  │ └─┬ [email protected]
  │   └── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ ├─┬ [email protected]
  │ │ ├── [email protected]
  │ │ ├─┬ [email protected]
  │ │ │ └── [email protected]
  │ │ └── [email protected]
  │ ├── [email protected]
  │ └─┬ [email protected]
  │   └─┬ [email protected]
  │     └── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  ├── [email protected]
  ├── [email protected]
  ├─┬ [email protected]
  │ └── [email protected]
  └─┬ [email protected]
    └─┬ [email protected]
      └── [email protected]

PM2 Pricing

PM2 appears to be for high-end apps but I am only using the free version or PM2 (thanks KeyMetrics)

pm2-pricing

Create a bucket for your node app

Login to keymetrics.io,

Click Generate New Bucket

Create New Bucket

Give the bucket a name etc.

Node Bucket Name

You can now link your bucket with your local pm2 installation (keep the keys private (this one no longer exists))

pm2-link

Linking your local pm2 installation with your keymetrics bucket

pm2 link l3brztzboz25him i6kofelsyfo7xrd
[KM] Connecting
[Monitoring Enabled] Dashboard access: https://app.keymetrics.io/#/r/i6kofelsyfo7xrd

To add an existing node app to PM2 type the following.

cd /your-node-application-path/
pm2 start yourapp.js -i 0 --name "myappname"

You can view node apps that pm2 is managing by typing

pm2 status

I had a two CPU VM and I found that the app I added was added to each of the two CPU (I only needed one) so I needed to delete the second app on my second core

pm2 delete 1

Restart the API

pm2 restart myappname

You can add a single node apps one 1, 3 or max available CPU’s

# Start the maximum processes depending on available CPUs
pm2 start app.js -i 0

# Start the maximum processes -1 depending on available CPUs
pm2 start app.js -i -1

# Start 3 processes
pm2 start app.js -i 3

Again, to add an existing node app to PM2 type the following.

cd /your-node-application-path/
pm2 start yourapp.js -i 0 --name "myappname"

Now you can view node app data online. If you don’t have a node app ready you can use the test app.

monitor output

You can monitor your node app locally too from the CLI.

local monitoring

You can also view a demo bucket at keymetrix.io

pm2-demo-bucket

PM2’s one age documentation can be found here.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial post

Filed Under: API, Automation, Cloud, Free, NGINX, NodeJS, Scalability, Server, Ubuntu, Vultr Tagged With: and, apps, background, Deploying, from, in, keymetrics.io, monitoring, NodeJS, the, with PM2

How to purchase your own domain name and set up a web server from $5 a month

March 28, 2018 by Simon

This guide will show technically minded people how you can purchase your own domain name, set up a web server on Vultr with an online store using WordPress/WooCommerce from $5 a month. Warning this post is technical (if you have never used SSH, Ubuntu, Linux Command Line, hate risk or are not patient then this is NOT the guide you are after).

I personally recommend (not a paid endorsement) the free WooCommerce plugin for the free WordPress.org CMS on the free Ubuntu Operating system with the free NGINX web server and the free MYSQL database engine and free SSL certificates from Lets Encrypt.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Sorry for using the word free a lot but I like free things.  One of the benefits of a using a self-managed server is you get the option to install free software and configure the server how you want and secure it how you want. Truth be told managed ho (e.g CPanel, etc) are in the business of making money via monthly feed, expensive SSL certificates, taxing your transactions or pushing you to higher-priced tiers.

Legend:

  • Self Managed Server = A server that you create, you configure patch and support (all the reward and risk is owned by you and costs are low).
  • Hosted Server = A server you have partial control of and the hosts manage the server and support (You hand away all risk and most of the control and pay for support/features).

I moved to a self-managed server after I was paying $25/m for a poorly performing website and $150/y for a poor quality  SSL certificate and a slice of a server that seemed to always say “Usage Limit Exceeded”. Why pay for an insecure website that my visitors could not view because the usage limit was exceeded.

Bad CPanel SSL Certificate

fyi: Fearby.com costs me $10 a month for a server and $5/m for CDN abilities.

CPanel hosts are an option when you don’t want to self-manage a service and take on the hassle but be prepared for server limitations (The image below was taken on an older CPanel based hosts before I moved to a self-managed Vultr server)

cpenal_usage_exceeded

I recently discovered a well known and established website hosting service (that I used to use) and a friend is still using is insecure. My friend’s site has a static website on it but the server underneath was very old and insecure. Having a secure web server should be at the top of your list with any self-managed or hosted website (this will help search engine optimization and prevent risks to your website visitors).

Static Website

Sites like Virus Total, SSL Labs and Alexa Site Info , Qualys are good ways to review a site’s credibility.

fyi: The awesome https://seositecheckup.com/ is awesome for evaluating our sites SEO score.

Before we set up a server with WordPress on your own server let’s quickly look at the alternative commercial ready to go website builders.

Alternative (paid) DIY Website Builders

The following leading commercial sites will allow you to build a site online.

  • https://www.wix.com/
  • https://www.squarespace.com/
  • https://www.shopify.com.au/
  • https://www.weebly.com/au
  • https://www.wordpress.com

In my opinion, five things matter with setting up site online website.

  • Setup Costs, Monthly Cost and Commissions (what are the hidden charges)
  • Security (having a food SSL Certificate is key to having a good organic traffic from search engines)
  • Site Speed (Having a slow site will impact search engine optimization and drive visitors away)
  • Accessibility (if your site is not WCAG accessible it will not rank high on search engines).
  • Control (will you be able to do everything you want too, nothing worse than going so far and being limited)

Ok, let’s see how much it will cost to set up a simple business site on the sites above.

Wix 

Setup: Goto https://www.wix.com/, Login to Wix, click Create Site, click Business, Click Choose a Template, Edit the page, Click Save, Click “Connect your own customized domain“, Click “Connect a domain you already own“.

I was redirected to a Wix plan pricing page where I need to choose a plan to continue. From what I researched you cant control HTML on Wix so can’t add a MailChimp newsletter signup form so you would have to go with the $24.5/m option to enable Email Campaigns.

Wix Plans Chooser

I could not see information about included SSL certificates, SEO or other chargers.  SSL is free after you pay right?

The Wix editor appears OK (it may take a bit of learning though).

Wix Editoer

I clicked publish and the site was live

Wix site published

A quick check of the SSL, Accessibility and SEO and no obvious deal breakers here apart from the price and platform lock-in.

Wix Checkup

I performed a security check on the site with https://freescan.qualys.com (passed)

Conclusion: I hear Wix templates are hard to change so choose your template wisely, A large collection of apps are available that you can add to the site.

Although Wix was nice and it does include a full-featured look at the engine it is not for me ($24/m USD is too expensive).

Squarespace 

Squarespace basic websites cost $16/$25 a month or $34/52 for online stores: https://www.squarespace.com/pricing/

SquareSpace Pricing

Setup a Squarespace website: Goto https://www.squarespace.com/, Click Start a Free Trial, Choose a Template, Create an Account (a quick read of the terms of service and privacy policy, #scary), SpareSpace sites are pre-published?

Square Space Build

Loading the webpage on a non-logged-in (with SquareSpace login) browser displays a trial warning.  Trial pages are essentially restricted (unlike Wix).

Login Challenge

The mobile view does not match the template?  I guess the chosen template is more of a vibe and not a template.

Mobile view

Setting up a Squarespace website may take some time. Squarespace does have some nifty advance options in a slide-out menu though.

Squarespace Settings

Because the public view of the page is restricted I cannot scan it with WCAG accessibility tools. Scanning the site performance speeds with gtmetrix also fails.

Performance Rejected

Squarespace is well known to be difficult to set up a website when compared to other drag and drop editors (but Squarespace sites do look nice).

I am not paying $54/m for a website so let’s move on.

Shopify

Shopify Setup: Goto https://shopify.com and click Create, Sign up and enter your store name. Complete the wizard. 

Shopify

Choose a Shopify Plan

Shopify Plan

Scalping transactions, no thanks. let’s move on.

Weebly

Weebly Setup: Goto https://www.weebly.com/au and click Get Started under Create Store. Enter your account details and click Create Your Site, enter the name of the store, Click I’m just trying Weebly, click the type of product you will be selling.

Weebly Site Setup

Weebly Setup

Theme Selection

Theme Select

Choose a Domain

Domain Select

Publish the site

Publish

Clicking publish appears to be a dead end.

Verify Weebly

“Please contact Weebly Support to verify your account”, No Thanks, let’s move on.

One candidate remains and that is WordPress hosted (wordpress.com not wordpress.org).

WordPress.com

WordPress.com offer hosted plans for WordPress in the cloud.

Setup a WordPress site, the only one that removes WordPress branding and allows third-party plugins to be installed it the Business plans for $33 a month.

WordPress Plans

Setup Basics

Wordpress

Choose a WordPress theme.

Choose Theme

Assign a Domain

WordPress Domain

In order to buy a domain, you need to log in (top right) with an account

My working WordPress account (is no longer working), it was in my password manager.

wordpresscomerror

I seem to be stuck in a signup loop

Wordpress

Time to move on. Time to set up my own server on Vultr and setup WordPress and  WooCommerce,

But, before we do, let’s ensure our name is secure online.

Search for your Name/Brand

Do search for your website (or thing) in search engines to see if your name is already taken, don’t buy a domain that is owned or has IP or trademark presence. It is a  good idea to use sites like https://namechk.com/ to see if your site or social media is already taken.

https://namechk.com/

namechk.com will allow you to search for name availability online.  The name “mything” is not fully available online.

https://namechk.com/ 2

You will want to see all green squares (name available) below before buying a domain name. This looks better.

Namechk ok

I would recommend you create your social media accounts before or right after buying your domain. Sites like Twitter will insist on short usernames names so get your social media sites first.

Trademark and Brand Search

Also, perform a trademark and IP search.

Australian Trademark Search: https://search.ipaustralia.gov.au/trademarks/search/quick

United States Trademark Database: https://www.uspto.gov/trademarks-application-process/search-trademark-database

Global brand Search: http://www.wipo.int/branddb/en/

etc

Self Managed Warning

I tend to go the “self-managed server route” and install the free WordPress CMS because:

  • I can.
  • I am tight.
  • I like having full control (usually the best features for online web hosts are hidden behind subscriber tiers, you can install and do whatever you want on your own server like build API’s, distributed MySQL servers, install MongoDB or Redis , use up to date PHP etc).
  • I have been stung by CPanel hosts charging $150/y for a crappy SSL certificate (You can set up your own SSL certificate for $0 and set up super secure SSL rules).
  • I can manage WordPress via the command line
  • I can upgrade the server and restore it whenever I want.
  • I can manage my own server performance (e.g setup PHP child workers) or install a Content Delivery Network.
  • I can direct domain email to google G Suite, see pricing here.
  • etc.

There are many reasons why you would not want to “self-manage” your own server

  • Technical Requirements (and time to support).
  • Higher Risk.
  • Applying Updates and Patches.
  • etc.

Being technically minded and choosing a “self-managed web servers” can take away time from the fun stuff like SEO, Site Design, customer needs, branding etc.

Self Managed Costs

For $5 a month you can buy a server with enough memory to install WordPress (cheaper if you don’t need WordPress)

Vultr is great. Vultr does have ready to go servers that you can deploy that have WordPress all set up.

wordpress-template

The Vultr template above does use the Centos OS (read my guide setting up Centos on a different service provider here) but I prefer to manually setup a server with Ubuntu 16.04 OS on Vultr.

With $5 server you can do what you want with it.  I have blogged before about setting up your own Server. e.g Installing Centos and Ubuntu server on Digital Ocean.  Digital Ocean does not have data centres in Australia and this kills scalability. AWS is good but 4x the price of Vultr. I have blogged about setting up and AWS server here (and upgrading an AWS instance). I tried to check out Alibaba Cloud but the verification process was broken so I decided to check our Vultr.

Manual Setup of Vultr on an Ubuntu 16.04 server

  • Deploy a Vultr Server – Guide here  ($2.5/m to New Jersey or Florida or $5/m to Sydney,  I would recommend you opt-in for the auto backup for $0.50c/m and $1/m respectively).
  • Setup NGINX.
  • Setup PHP and PHP-FPM (see guide above), consider adding PHP child workers.
  • Setup and secure MySQL (see guide above), create a database for WordPress to use.
  • Instal Adminder MySQL GUI (guide here).
  • Setup a free Lets Encrypt SSL certificate (guide here).
  • Install WordPress (and Jetpack plugin).
  • Install WordPress CLI.
  • Instal the WooCommerce Storefront WordPress Theme.
  • Install WooCommerce Plugin.
  • Secure Ubuntu.
  • Also consider linking your domain to Cloudflare to boost performance, scanning your site with Qualys Freescan and OWASP ZAP).
  • Consider setting up a WordPress image compressor and CDN plugin. like EWWW.io

Manual WooCommerce Plugin Setup

Once you setup Woocommerce you can set up the store defaults. Go to the WordPress dashboard and click WooCommerce Settings

Woo Commerce Settings

Settings – General

General

  • Set Address, City and State and Postcode
  • Set allowed countries to sell in (e.g Australia)
  • Set allowed countries to ship items to (e.g Australia)
  • Set Enable Taxes
  • Set Currency
  • etc

Settings – Products

Products

  • Set Weight
  • Set Dimensions
  • Enable Product Reviews
  • Enable Star Ratings on Reviews
  • etc

Settings – Shipping

Shipping

  • Enable Shipping Calculator
  • Add Shipping Classes
  • Shipping Zones
  • etc.

Settings – Checkout

Settings checkout

  • Force Secure Checkup
  • Create a Terms and Conditions page (and set).
  • etc

Settings – Account

Accounts

  • Set Account Options
  • etc

Settings – Emails

Emails

  • Set Email Preferences
  • Set Email Header Image
  • Set Email Colour
  • Set Footer Text
  • etc

Settings – API

API

  • API can be disabled if you don’t need it.

Optional Actions

  • Setup Yoast Plugin
  • Setup other plugins

Instaling a Woo Commerce Child Theme

Go to https://woocommerce.com/product-category/themes/storefront-child-theme-themes/ and choose a theme.

Themes

Purchase and Install the desired child theme (I uploaded it to my /wp-content/themes/ folder with forklift). I chose a free deli theme.

Goto your WordPress then themes folder and activate your new child theme.

Activate Theme

Post Site Setup

Just because your site is live does not mean you can rest.

SEO Optimization

Do use sites like https://seositecheckup.com/ and follow recommended actions to improve your SEO like updating meta tags.

More Reading

Attaching an email to your domain

You can pay $5 a month and link a G Suite email to your domain.

  • Dedicated professional Google G Suite email account for $5 a month with 30GB storage (If you don’t want ot to buy a G Suite email and link it to your domain then you don’t need this).

Once you have a G Suite account you can link other domains (and domain emails) to it. You can login to your G Suite emails via G Mail and send emails from apps or the command line.

Why Vultr

I use the server host Vultr as they have data centres all around the world and the support of great, Digital Ocean is good too but they don’t have data centres in my country (Australia). Vultr allows you to deploy all over the world upgrade servers, move servers, add storage and restore servers.

Alternatively, you can buy a $2.5/m server and generate  a static website

I use the Platforma Web HTML generator to build mobile and WCAG compliant websites.

Buying a domain,  I buy my domains from https://www.namecheap.com/  it is a good idea to look for coupons first at https://www.namecheap.com/promos/coupons.aspx before buying a domain.

Once you buy a domain you can point it to a Vultr server and upload your website.

I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v.1.2 WordPress WooCommerce

v1.1 SEO

v1.0 Initial Draft

Filed Under: Ubuntu, VM, Vultr, Website, Wordpress Tagged With: $5, a, and, domain, from, How, month, name, own, purchase, seo, server, set up, to, web, your

Backing up OSX or an Ubuntu server with Backblaze B2 Cloud Storage from the Command Line

March 14, 2018 by Simon

This computer will show you can back up computer or server with Backblaze B2 Cloud Storage from the Command Line n OSX and Ubuntu.

This post is still being written. I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Also, I have blogged about how you can add block storage to a Vultr server, backup and restore snapshots , syncing files with rsync along with using GitHub and Bitbucket but what do you do if you need to backup large amounts of data?

Backblaze has a Cloud storage solution that costs as low as $0.005c a GB (a month), The first 10G is free. Backblaze say “From bytes to petabytes Backblaze B2 is the lowest cost high-performance cloud storage in the world. ”

Back Blaze have open sourced internal drive enclosure designs and drive failure stats and it’s time I gave them a try.

Goto https://www.backblaze.com

Backblaze

Create or sign in.

Backblaze Login

After you login got the dashboard.

B2 Cloud

Click Backblaze B2 Cloud Storage

Activate B2 Cloud

Signup

Create a Bucket

Create Bucket

Name the bucket (long names with a GUID are good).

Name the Bucket

You can rename the bucket here and change public/private and or upload/download files manually.

Manage Bucket

The first thing I did was limit the versions of files under the lifecycle settings for the bucket.

Version Settings

Now I created a series of subfolders to store files from different servers (I could have used many buckets but one bucket will do).

Folders

I can upload files via the Backblaze bucket GUI if I needed to.

Upload and Download

Back Blaze has a command line tool for uploading: https://www.backblaze.com/b2/docs/quick_command_line.html

Install Steps

Backblaze state “The B2 command-line tool is available from the Python Package Index (PyPI) using the standard pip installation tool. Your first step is to make sure that you have either Python 2 (2.6 or later) or Python 3 (3.2 or later) installed.”

I have Python 2.7 installed

python --version
Python 2.7.10

Install PIP

sudo easy_install pip
Password:
Searching for pip
Best match: pip 1.5.6
Processing pip-1.5.6-py2.7.egg
pip 1.5.6 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Using /Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

I ran into issues updating b2 CLI

sudo pip install --upgrade b2
Requirement already up-to-date: b2 in /Library/Python/2.7/site-packages
Requirement already up-to-date: arrow>=0.8.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: logfury>=0.1.2 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: requests>=2.9.1 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: six>=1.10 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: tqdm>=4.5.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: futures>=3.0.5 in /Library/Python/2.7/site-packages (from b2)
Downloading/unpacking python-dateutil from https://pypi.python.org/packages/bc/c5/3449988d33baca4e9619f49a14e28026399b0a8c32817e28b503923a04ab/python_dateutil-2.7.0-py2.py3-none-any.whl#md5=5a86a548fe776cc079bf4a835473e3f8 (from arrow>=0.8.0->b2)
  Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207kB): 207kB downloaded
Installing collected packages: python-dateutil
  Found existing installation: python-dateutil 1.5
    Uninstalling python-dateutil:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

I tried installing via the alternative method (with no luck)

git clone https://github.com/Backblaze/B2_Command_Line_Tool.git
Cloning into 'B2_Command_Line_Tool'...
remote: Counting objects: 5084, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 5084 (delta 1), reused 1 (delta 0), pack-reused 5076
Receiving objects: 100% (5084/5084), 1.25 MiB | 689.00 KiB/s, done.
Resolving deltas: 100% (3622/3622), done.
cd B2_Command_Line_Tool/

I tried running the setup script (with no luck)

sudo python setup.py install
setuptools 20.2 or later is required. To fix, try running: pip install "setuptools>=20.2"

Upgrading setup tools also failed

sudo pip install "setuptools>=20.2"
Downloading/unpacking setuptools>=20.2
  Downloading setuptools-38.5.2-py2.py3-none-any.whl (490kB): 490kB downloaded
Installing collected packages: setuptools
  Found existing installation: setuptools 18.5
    Uninstalling setuptools:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

Backing up a Mac via command line with B2

More to come when I can get B2 CLI Installed.

Backing up an Ubuntu machine via command line with B2

More to come when I can get B2 CLI Installed.

Update

My ticket with Backblaze was automatically closed with this note “If the issue is persisting, it may be easiest to map the installation to the user folder, rather than the system level.”

No ideas how but something to research.

Ask a question or the recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V1.1 ticket closed

v1.0 Initial post

Filed Under: Backup Tagged With: an, B2, backblaze, Backing, cloud, command, from, line, or, OSX, server, storage, the, ubuntu, up, with

Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl

December 3, 2017 by Simon

Below is my quick guide to showing how I recover lost files on a removable (or internal drive) with software on Windows. This is not a paid advert but this is what works for me (and has worked with everything I have thrown at it). Tell me what works for you (especially on OSX).

“Helping one person might not change the world but it could change the world for one person” – Buddha.

I hope this guide helps someone, I have never seen someone happier than when I have been able to restore lost files or photos for someone. Not everyone backs up data (read my guide here).

Before you start

Don’t touch (format, access or install software) onto drives and USB drive that you want to restore files from or you may overwrite files on that drive. Doing this will potentially block the ability to restore files. Electronics die, software gets corrupted, this can be complex so If in doubt seek professional help and advice before proceeding. This is advice only and it’s your call with your data, Restoring files on damaged magnetic or USB drives can damage the data storage platter or storage chips, so proceed at your own risk. (legal disclaimer over, good luck).

More on how file systems and storage work

How File Systems Work

How the File Allocation Table Works

Installing DIY Data Recovery

The DIY Data Recovery software can restore images (photos) from a UBS, SD card or files (images or non-images) from data or operating system drives. If you have a system or data drive from a PC you can plug it into another PC running DIY Dat Recovery and scan and restore from the possibly bad drive to a good host drive, never restore files to a bad drive from a bad drive. I am an Apple Mac user now but only know of this Windows method for restoring files, please let me know of an Apple Mac way to restore files on FAT, NTFS and other file systems, please.

I usually connect USB drives directly or connect SATA drives via a small caddy

On a working Windows computer install DIY Data Recovery software from here.

Install

Plugin your dead USB or another drive to restore files from (do not interact with the drive once Windows can see it).

Insert Drive

Do not format or modify the drive if prompted (cancel this screen).

Dont Format

Sometimes Windows will offer to fix it (do not interact with this dialogue).

From memory, you can follow the steps below on a free trial version but you will need a paid version (serial number) to actually restore files. The free trial will show files that it can restore though. I paid about $60 for the software a few years ago, it is currently 59 EUROS. Enter your serial number if you have one.

Register iRecovery

After you register it (or continue with the free trial) select the drive to scan for files and click Next.

Select Drive

Here is another unreadable SATA drive

DIY Data Recovery will scan each sector and try and find files. On a smaller healthy drive, the scan won’t take long, on a larger drive (e.g a 2TB NTFS drive it may take 18+ hours).

In this case, we can see the USB drive has two bad sectors (red squares) at the front of the USB stick in the file system sectors (green squares). This possibly happened when the USB was pulled out when the operating system was writing to the USB stick. If the USB was dying the bad sectors would be at random positions.

Data fragments are listed as blue squares.

Let it scan

Here is another sample scan results

When the scan is done click ‘Save” to save the found sectors state (saving the state can save you time later).

Disclaimer: Clicking save does not save lost files it just saves the current scan state.

choose Files to recover

You can click the legend button to see the state of each file (Not processed, Presumably valid, Invalid, bad sectors etc)

Tick the files you want to be restored.

Legend

FYI: You will see some files you know, some files you forgot and some weird files. Restoring files is hit and miss and you may never see files again (do backup). Benjamin Franklin once said “Failing to plan is planning to fail”, You can also say “Failing to backup is guaranteeing lost data”.

Restoring Files

Do

  • Create a folder on your working Desktop to restore files to.
  • Do increase Read and Write cache
  • Do set longer Timeouts (e.g 1000, 5000, 10000, 60000)

Restore

Click “Start copying the selected files” to restore.

restore

Hopefully, you will have files restored, if not seek professional help. I’d recommend keeping the corrupt or source drive in a drawer, you may be able to use other tools to restore more files at a later date.

Restored Files

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V1.1 Added more screenshot from a second DATA drive scan

v1.0 Initial Post

Filed Under: Restore Tagged With: a, diydatarecovery.nl, FAT32, files, from, iRecover, Linux EXT, Linux XFS, lost, NTFS, on, or, Restoring, volume, Windows FAT, with

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) HTTPS (6) IoT (9) LetsEncrypt (7) Linux (20) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) PHP (13) Scalability (12) Scalable (14) Security (44) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (44) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT