an

Adding two sub domains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04

June 27, 2018 by Simon

Here is how I added two subdomains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

UpCloud performance is great.

Buy a domain name from Namecheap here.

Goal(s)

Setup 2x subdomains on https://fearby.com

– Sub Domain #1: https://test.fearby.com (pointing to a dedicated UpCloud VM in Singapore for testing).

– Sub Domain #2: https://audit.fearby.com (pointing to a sub-website on the NGINX/VM that runs https://fearby.com )

Let’s set up the first Sub Domain (dedicated VM) and SSL

Backup

Do backup your server first.

I created a second server ($5 month or $0.07c hour 1,024MB Memory, 25GB Disk, 1024 GB Month Data Transfer) at UpCloud. If you don’t already have an account at UpCloud use this link to signup and get $25 free credit ( https://www.upcloud.com/register/?promo=D84793 ). Read my blog post on why UpCloud is awesome and how I moved my domain to UpCloud.

Once I spun up a second server I obtained the IPv4 and IPv6 IP addresses of the new “test” VM from the UpCloud dashboard.

IPV4 IP: 94.237.65.54
IPV6 IP: 2a04:3543:1000:2310:24b7:7cff:fe92:468c

1 2	IPV4 IP: 94.237.65.54 IPV6 IP: 2a04:3543:1000:2310:24b7:7cff:fe92:468c

DNS

These DNS records were already in place with my DNS provider (Cloudflare).

A fearby.com 209.50.48.88
AAAA fearby.com 2605:7380:1000:1310:24b7:7cff:fe92:0d64

1 2	A fearby.com 209.50.48.88 AAAA fearby.com 2605:7380:1000:1310:24b7:7cff:fe92:0d64

I added these DNS records for the subdomains.

I added a new A NAME record for the new shared NGINX subdomain (for https://audit.fearby.com), this subdomain will be a sub-website that is running off the same server as https://fearby.com

A audit 209.50.48.88
AAAA audit 2605:7380:1000:1310:24b7:7cff:fe92:0d64

1 2	A audit 209.50.48.88 AAAA audit 2605:7380:1000:1310:24b7:7cff:fe92:0d64

I added another set of records for the new dedicated VM subdomain (for https://test.fearby.com)

A test 94.237.65.54
AAAA test 2a04:3543:1000:2310:24b7:7cff:fe92:468c

1 2	A test 94.237.65.54 AAAA test 2a04:3543:1000:2310:24b7:7cff:fe92:468c

I waited for DNS to replicate around the globe by watching https://www.whatsmydns.net/

Setup a Firewall

On the new dedicated https://test.fearby.com VM, I installed the ufw firewall.

sudo apt-get install ufw

1	sudo apt-get install ufw

I configured the firewall to allow minimum ports (and added whitelisted IP for port 22 and added UpCloud DNS servers). I will lock this down some more later.

TIP: If your ISP does not offer a dedicated IP try a VPN. I use https://cyberghostvpn.com on OSX and Android.

Firewall rules.

sudo ufw status numbered

     To                         Action      From
     --                         ------      ----
[ 1] 22                         ALLOW IN    x.x.x.x
[ 2] 80                         ALLOW IN    Anywhere
[ 3] 443                        ALLOW IN    Anywhere
[ 4] 53                         ALLOW IN    93.237.127.9
[ 5] 53                         ALLOW IN    93.237.40.9
[ 6] 25                         DENY IN     Anywhere
[ 7] 80 (v6)                    ALLOW IN    Anywhere (v6)
[ 8] 443 (v6)                   ALLOW IN    Anywhere (v6)
[ 9] 53                         ALLOW IN    2a04:3540:53::1
[10] 53                         ALLOW IN    2a04:3544:53::1
[11] 22                         ALLOW IN    x.x.x.x.x.x.x.x.x
[12] 25 (v6)                    DENY IN     Anywhere (v6)

sudo ufw status numbered

To Action From

-- ------ ----

[ 1] 22 ALLOW IN x.x.x.x

[ 2] 80 ALLOW IN Anywhere

[ 3] 443 ALLOW IN Anywhere

[ 4] 53 ALLOW IN 93.237.127.9

[ 5] 53 ALLOW IN 93.237.40.9

[ 6] 25 DENY IN Anywhere

[ 7] 80 (v6) ALLOW IN Anywhere (v6)

[ 8] 443 (v6) ALLOW IN Anywhere (v6)

[ 9] 53 ALLOW IN 2a04:3540:53::1

[10] 53 ALLOW IN 2a04:3544:53::1

[11] 22 ALLOW IN x.x.x.x.x.x.x.x.x

[12] 25 (v6) DENY IN Anywhere (v6)

I enabled the firewall.

sudo ufw enable

1	sudo ufw enable

Install NGINX (on https://test.fearby.com)

On the new dedicated https://test.fearby.com VM I…

Created a new www root

mkdir /www-root

1	mkdir /www-root

Set permissions

sudo chown -R www-data:www-data /www-root

1	sudo chown -R www-data:www-data /www-root

Installed NGINX

sudo apt-get update
sudo apt-get install nginx

1 2	sudo apt-get update sudo apt-get install nginx

I created a placeholder webpage

sudo nano /www-root/index.html

1	sudo nano /www-root/index.html

Configured the root value in /etc/nginx/sites-available/default

Created a symbolic link of the nginx config

sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default

1	sudo ln -s /etc/nginx/sites-available/default /etc/nginx/sites-enabled/default

Lets Encrypt SSL

I have previously setup Lets encrypt on Ubuntu 16.04 but not 18.04. Certbot had info on setting up Lets Encrypt for 14.x 16.x and 17.x but not 18.x

Full credit for the SSL steps goes to @Linuxize ( tips on setting up Lets Encrypt on Ubuntu 18.04 ). Check out https://linuxize.com/

I installed Lets Encrypt certbot

sudo apt update
sudo apt install certbot

1 2	sudo apt update sudo apt install certbot

I created a new Diffie–Hellman key

mkdir -p /etc/ssl/certs/
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

1 2	mkdir -p /etc/ssl/certs/ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Map requests to http://test.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).

mkdir -p /var/lib/letsencrypt/.well-known
chgrp www-data /var/lib/letsencrypt
chmod g+s /var/lib/letsencrypt

mkdir -p /var/lib/letsencrypt/.well-known

chgrp www-data /var/lib/letsencrypt

chmod g+s /var/lib/letsencrypt

Create a /etc/nginx/snippets/letsencrypt.conf on http://test.fearby.com and enforce the redirect.

location ^~ /.well-known/acme-challenge/ {
  allow all;
  root /var/lib/letsencrypt/;
  default_type "text/plain";
  try_files $uri =404;
}

location ^~ /.well-known/acme-challenge/ {

allow all;

root /var/lib/letsencrypt/;

default_type "text/plain";

try_files $uri =404;

}

Create a /etc/nginx/snippets/ssl.conf file on http://test.fearby.com

ssl_dhparam /etc/ssl/certs/dhparam.pem;

ssl_session_timeout 1d;
ssl_session_cache shared:SSL:50m;
ssl_session_tickets off;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';
ssl_prefer_server_ciphers on;

ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 30s;

add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload";
add_header X-Frame-Options SAMEORIGIN;
add_header X-Content-Type-Options nosniff;

ssl_dhparam /etc/ssl/certs/dhparam.pem;

ssl_session_timeout 1d;

ssl_session_cache shared:SSL:50m;

ssl_session_tickets off;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:AES128-GCM-SHA256:AES256-GCM-SHA384:AES128-SHA256:AES256-SHA256:AES128-SHA:AES256-SHA:DES-CBC3-SHA:!DSS';

ssl_prefer_server_ciphers on;

ssl_stapling on;

ssl_stapling_verify on;

resolver 8.8.8.8 8.8.4.4 valid=300s;

resolver_timeout 30s;

add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload";

add_header X-Frame-Options SAMEORIGIN;

add_header X-Content-Type-Options nosniff;

Let’s get a certificate

sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d test.fearby.com

1	sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d test.fearby.com

Certificates have been created 🙂

ls -al /etc/letsencrypt/live/test.fearby.com/
total 12
drwxr-xr-x 2 user user 4096 Jun 26 11:30 .
drwx------ 3 user user 4096 Jun 26 11:30 ..
-rw-r--r-- 1 user user  543 Jun 26 11:30 README
lrwxrwxrwx 1 user user   39 Jun 26 11:30 cert.pem -> ../../archive/test.fearby.com/cert1.pem
lrwxrwxrwx 1 user user   40 Jun 26 11:30 chain.pem -> ../../archive/test.fearby.com/chain1.pem
lrwxrwxrwx 1 user user   44 Jun 26 11:30 fullchain.pem -> ../../archive/test.fearby.com/fullchain1.pem
lrwxrwxrwx 1 user user   42 Jun 26 11:30 privkey.pem -> ../../archive/test.fearby.com/privkey1.pem

ls -al /etc/letsencrypt/live/test.fearby.com/

total 12

drwxr-xr-x 2 user user 4096 Jun 26 11:30 .

drwx------ 3 user user 4096 Jun 26 11:30 ..

-rw-r--r-- 1 user user 543 Jun 26 11:30 README

lrwxrwxrwx 1 user user 39 Jun 26 11:30 cert.pem -> ../../archive/test.fearby.com/cert1.pem

lrwxrwxrwx 1 user user 40 Jun 26 11:30 chain.pem -> ../../archive/test.fearby.com/chain1.pem

lrwxrwxrwx 1 user user 44 Jun 26 11:30 fullchain.pem -> ../../archive/test.fearby.com/fullchain1.pem

lrwxrwxrwx 1 user user 42 Jun 26 11:30 privkey.pem -> ../../archive/test.fearby.com/privkey1.pem

Now lets edit “/etc/nginx/sites-available/default” on https://test.fearby.com VM and add the cert paths.

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        listen 443 ssl http2;
        listen [::]:443 ssl http2;

        if ($scheme != "https") {
                return 301 https://$host$request_uri;
        }

        ssl_certificate /etc/letsencrypt/live/test.fearby.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/test.fearby.com/privkey.pem;
        ssl_trusted_certificate /etc/letsencrypt/live/test.fearby.com/chain.pem;

        include snippets/ssl.conf;

        #ssl_stapling on; # Requires nginx >= 1.3.7
        # add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";

        add_header X-Frame-Options DENY;
        add_header X-Content-Type-Options nosniff;
        add_header X-XSS-Protection "1; mode=block";

        root /www-root/;

        include snippets/letsencrypt.conf;

        index index.html;

        server_name test.fearby.com;

        location / {
                try_files $uri $uri/ =404;
        }
}

server {

listen 80 default_server;

listen [::]:80 default_server;

listen 443 ssl http2;

listen [::]:443 ssl http2;

if ($scheme != "https") {

return 301 https://$host$request_uri;

}

ssl_certificate /etc/letsencrypt/live/test.fearby.com/fullchain.pem;

ssl_certificate_key /etc/letsencrypt/live/test.fearby.com/privkey.pem;

ssl_trusted_certificate /etc/letsencrypt/live/test.fearby.com/chain.pem;

include snippets/ssl.conf;

#ssl_stapling on; # Requires nginx >= 1.3.7

# add_header Strict-Transport-Security "max-age=63072000; includeSubdomains; preload";

add_header X-Frame-Options DENY;

add_header X-Content-Type-Options nosniff;

add_header X-XSS-Protection "1; mode=block";

root /www-root/;

include snippets/letsencrypt.conf;

index index.html;

server_name test.fearby.com;

location / {

try_files $uri $uri/ =404;

}

Reload NGINX

sudo systemctl reload nginx

1	sudo systemctl reload nginx

sudo nginx -t
sudo nginx -s reload
sudo systemctl reload nginx

sudo nginx -t

sudo nginx -s reload

sudo systemctl reload nginx

Now let’s setup the second subdomain (subsite off https://fearby.com) and SSL

I already have NGINX on https://fearby.com set up a second site.

DNS

We have already setup a DNS record for https://audit.fearby.com (above)

Firewall

Already configured at https://fearby.com

SSL

Because I had an existing Comodo certificate on https://fearby.com I am going to repeat the steps above to generate a new certificate but save the NGINX config to /etc/nginx/sites-available/audit.fearby.com (this activates the second site)

TIP: Follow the Linuxize guide here (for creating ssl.conf, letsencrypt.conf etc config files), Do a backup and restore if need be.

I created a new Diffie–Hellman key

mkdir -p /etc/ssl/certs/
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

1 2	mkdir -p /etc/ssl/certs/ sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 2048

Let’s get a certificate

sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d audit.fearby.com

1	sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d audit.fearby.com

Configure NGINX

Map requests to http://audit.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).

mkdir -p /var/lib/letsencrypt/.well-known
chgrp www-data /var/lib/letsencrypt
chmod g+s /var/lib/letsencrypt

mkdir -p /var/lib/letsencrypt/.well-known

chgrp www-data /var/lib/letsencrypt

chmod g+s /var/lib/letsencrypt

I created a new NGINX site ( /etc/nginx/sites-available/audit.fearby.com )

#proxy_cache_path /tmp/nginx-cache keys_zone=one:10m;#
server {
        root /www-audit-root;

        # Listen Ports
        listen 80;
        listen [::]:80;
        listen 443 ssl http2;
        listen [::]:443 ssl http2;

        # Default File
        index index.html index.php index.htm;

        # Server Name
        server_name audit.fearby.com;

        include snippets/letsencrypt.conf;

        location / {
                try_files $uri $uri/ =404;
        }

        ssl_certificate /etc/letsencrypt/live/audit.fearby.com/fullchain.pem;
        ssl_certificate_key /etc/letsencrypt/live/audit.fearby.com/privkey.pem;
        ssl_trusted_certificate /etc/letsencrypt/live/audit.fearby.com/chain.pem;

        ssl_dhparam /etc/ssl/certs/auditdhparam.pem;

        ssl_session_timeout 1d;
        #ssl_session_cache shared:SSL:50m;
        ssl_session_tickets off;

        ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
        ssl_ciphers 'ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA38$

        ssl_prefer_server_ciphers on;

        ssl_stapling on;
        ssl_stapling_verify on;

        #resolver 8.8.8.8 8.8.4.4 valid=300s;
        #resolver_timeout 30s;

        add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload";
        add_header X-Frame-Options SAMEORIGIN;
        add_header X-Content-Type-Options nosniff;

        if ($scheme != "https") {
                return 301 https://$host$request_uri;
        }
}

#proxy_cache_path /tmp/nginx-cache keys_zone=one:10m;#

server {

root /www-audit-root;

# Listen Ports

listen 80;

listen [::]:80;

listen 443 ssl http2;

listen [::]:443 ssl http2;

# Default File

index index.html index.php index.htm;

# Server Name

server_name audit.fearby.com;

include snippets/letsencrypt.conf;

location / {

try_files $uri $uri/ =404;

}

ssl_certificate /etc/letsencrypt/live/audit.fearby.com/fullchain.pem;

ssl_certificate_key /etc/letsencrypt/live/audit.fearby.com/privkey.pem;

ssl_trusted_certificate /etc/letsencrypt/live/audit.fearby.com/chain.pem;

ssl_dhparam /etc/ssl/certs/auditdhparam.pem;

ssl_session_timeout 1d;

#ssl_session_cache shared:SSL:50m;

ssl_session_tickets off;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;

ssl_prefer_server_ciphers on;

ssl_stapling on;

ssl_stapling_verify on;

#resolver 8.8.8.8 8.8.4.4 valid=300s;

#resolver_timeout 30s;

add_header Strict-Transport-Security "max-age=15768000; includeSubdomains; preload";

add_header X-Frame-Options SAMEORIGIN;

add_header X-Content-Type-Options nosniff;

if ($scheme != "https") {

return 301 https://$host$request_uri;

}

I created a symbolic link of the config file

sudo ln -s /etc/nginx/sites-available/audit.fearby.com /etc/nginx/sites-enabled/audit.fearby.com

1	sudo ln -s /etc/nginx/sites-available/audit.fearby.com /etc/nginx/sites-enabled/audit.fearby.com

Reload NGINX

sudo systemctl reload nginx

1	sudo systemctl reload nginx

sudo nginx -t
sudo nginx -s reload
sudo systemctl reload nginx

sudo nginx -t

sudo nginx -s reload

sudo systemctl reload nginx

How to test the certificate renewal

sudo certbot renew --dry-run

1	sudo certbot renew --dry-run

Automate the renewal in crontab (every 12 hours)

I set this crontab entry up on https://fearby.com and https://test.fearby.com

crontab -e
0 */12 * * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew --renew-hook "systemctl reload nginx"

1 2	crontab -e 0 /12 * * root test -x /usr/bin/certbot -a \! -d /run/systemd/system && perl -e 'sleep int(rand(3600))' && certbot -q renew --renew-hook "systemctl reload nginx"

Conclusion

Yes, I haVe 2 subdomains (1x dedicated VM and the other is a sub-website off an existing server) with SSL certificates.

Ping Results

ping -c 4 fearby.com
PING fearby.com (209.50.48.88): 56 data bytes
64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=220.000 ms
64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=290.602 ms
64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=311.938 ms
64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=330.841 ms

--- fearby.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 220.000/288.345/330.841/41.948 ms

ping -c 4 test.fearby.com
PING test.fearby.com (94.237.65.54): 56 data bytes
64 bytes from 94.237.65.54: icmp_seq=0 ttl=44 time=333.590 ms
64 bytes from 94.237.65.54: icmp_seq=1 ttl=44 time=252.433 ms
64 bytes from 94.237.65.54: icmp_seq=2 ttl=44 time=271.153 ms
64 bytes from 94.237.65.54: icmp_seq=3 ttl=44 time=292.685 ms

--- test.fearby.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 252.433/287.465/333.590/30.200 ms

ping -c 4 audit.fearby.com
PING audit.fearby.com (209.50.48.88): 56 data bytes
64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=281.662 ms
64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=307.676 ms
64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=227.985 ms
64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=215.566 ms

--- audit.fearby.com ping statistics ---
4 packets transmitted, 4 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 215.566/258.222/307.676/37.845 ms

ping -c 4 fearby.com

PING fearby.com (209.50.48.88): 56 data bytes

64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=220.000 ms

64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=290.602 ms

64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=311.938 ms

64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=330.841 ms

--- fearby.com ping statistics ---

4 packets transmitted, 4 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 220.000/288.345/330.841/41.948 ms

ping -c 4 test.fearby.com

PING test.fearby.com (94.237.65.54): 56 data bytes

64 bytes from 94.237.65.54: icmp_seq=0 ttl=44 time=333.590 ms

64 bytes from 94.237.65.54: icmp_seq=1 ttl=44 time=252.433 ms

64 bytes from 94.237.65.54: icmp_seq=2 ttl=44 time=271.153 ms

64 bytes from 94.237.65.54: icmp_seq=3 ttl=44 time=292.685 ms

--- test.fearby.com ping statistics ---

4 packets transmitted, 4 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 252.433/287.465/333.590/30.200 ms

ping -c 4 audit.fearby.com

PING audit.fearby.com (209.50.48.88): 56 data bytes

64 bytes from 209.50.48.88: icmp_seq=0 ttl=44 time=281.662 ms

64 bytes from 209.50.48.88: icmp_seq=1 ttl=44 time=307.676 ms

64 bytes from 209.50.48.88: icmp_seq=2 ttl=44 time=227.985 ms

64 bytes from 209.50.48.88: icmp_seq=3 ttl=44 time=215.566 ms

--- audit.fearby.com ping statistics ---

4 packets transmitted, 4 packets received, 0.0% packet loss

round-trip min/avg/max/stddev = 215.566/258.222/307.676/37.845 ms

Webpage Results

Troubleshooting

If you are having troubles generating the initial certificate check that you have not blocked port 80 and don’t have “Strict-Transport-Security” heavers enabled.

sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d g
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
http-01 challenge for yoursubdomain.domain.com
Using the webroot path /var/lib/letsencrypt for all unmatched domains.
Waiting for verification...
Cleaning up challenges
Failed authorization procedure. yoursubdomain.domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficzLlmg_w6Tc: q%!(EXTRA string=<html>
<head><title>404 Not Found</title></head>
<body bgcolor="white">
<center><h1>404 Not Found</h1></center>
<hr><center>)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: yoursubdomain.domain.com
   Type:   unauthorized
   Detail: Invalid response from
   http://yoursubdomain.domain.com/.well-known/acme-challenge/_QA3jblEydx5mE8I8OdRsd2EdHIj4R-przLlmg_w6Tc:
   q%!(EXTRA string=<html>
   <head><title>404 Not Found</title></head>
   <body bgcolor="white">
   <center><h1>404 Not Found</h1></center>
   <hr><center>)

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A record(s) for that domain
   contain(s) the right IP address.

sudo certbot certonly --agree-tos --email your@email.com --webroot -w /var/lib/letsencrypt/ -d g

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Obtaining a new certificate

Performing the following challenges:

http-01 challenge for yoursubdomain.domain.com

Using the webroot path /var/lib/letsencrypt for all unmatched domains.

Waiting for verification...

Cleaning up challenges

Failed authorization procedure. yoursubdomain.domain.com (http-01): urn:acme:error:unauthorized :: The client lacks sufficzLlmg_w6Tc: q%!(EXTRA string=<html>

<head><title>404 Not Found</title></head>

<center><h1>404 Not Found</h1></center>

<hr><center>)

IMPORTANT NOTES:

- The following errors were reported by the server:

Domain: yoursubdomain.domain.com

Type: unauthorized

Detail: Invalid response from

http://yoursubdomain.domain.com/.well-known/acme-challenge/_QA3jblEydx5mE8I8OdRsd2EdHIj4R-przLlmg_w6Tc:

q%!(EXTRA string=<html>

<head><title>404 Not Found</title></head>

<center><h1>404 Not Found</h1></center>

<hr><center>)

To fix these errors, please make sure that your domain name was

entered correctly and the DNS A record(s) for that domain

contain(s) the right IP address.

I re-ran the certbot command but pointed to the real /www-root (not/var/lib/letsencrypt/)

Create a new

mkdir /www-root/.well-known/
mkdir /www-root/.well-known/acme-challenge/
sudo certbot certonly --agree-tos --email your@email.com --webroot -w /www-root -d yoursubdomain.domain.com

mkdir /www-root/.well-known/

mkdir /www-root/.well-known/acme-challenge/

sudo certbot certonly --agree-tos --email your@email.com --webroot -w /www-root -d yoursubdomain.domain.com

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

Revision History

v1.1 Troubleshooting

v1.0 Initial Post

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

June 25, 2018 by Simon

Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here). To further increase the speed of the WordPress I have configured the OS to use memory over the disk. About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).

Read the full article here: https://fearby.com/article/upgrading-an-ubuntu-server-on-upcloud-to-add-more-cpu-memory-and-disk-space/

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

June 25, 2018 by Simon

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).

Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Spoiler: UpCloud performance is great.

Why Upgrade

I have 1 CPU, 1 GB memory and 50GB storage and it is running well? I have PHP child workers set up and have set up the preferred use of memory over swap file usage.

I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.

How to Upgrade

This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.

UpCloud Pricing: https://www.upcloud.com/pricing/

Upgrade an UpCloud VM

I shut down my existing VM. Read this guide to setup a VM.

shutdown -P

1	shutdown -P

Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.

I confirmed the upgrade options (2x CPU, 4096 MB Memory).

Click Update

After 10 seconds you can start your server from the UpCloud server.

I confirmed the CPU upgrade was visible in the VM

cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.978
cache size      : 16384 KB
physical id     : 0
siblings        : 2
core id         : 0
cpu cores       : 2
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 6199.95
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

processor       : 1
vendor_id       : GenuineIntel
cpu family      : 6
model           : 63
model name      : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz
stepping        : 2
microcode       : 0x1
cpu MHz         : 3099.978
cache size      : 16384 KB
physical id     : 0
siblings        : 2
core id         : 1
cpu cores       : 2
apicid          : 1
initial apicid  : 1
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 6199.95
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

cat /proc/cpuinfo

processor : 0

vendor_id : GenuineIntel

cpu family : 6

model : 63

model name : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz

stepping : 2

microcode : 0x1

cpu MHz : 3099.978

cache size : 16384 KB

physical id : 0

siblings : 2

core id : 0

cpu cores : 2

apicid : 0

initial apicid : 0

fpu : yes

fpu_exception : yes

cpuid level : 13

wp : yes

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon rep_good nopl xtopology cpuid pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single pti fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat

bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass

bogomips : 6199.95

clflush size : 64

cache_alignment : 64

address sizes : 40 bits physical, 48 bits virtual

power management:

processor : 1

vendor_id : GenuineIntel

cpu family : 6

model : 63

model name : Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz

stepping : 2

microcode : 0x1

cpu MHz : 3099.978

cache size : 16384 KB

physical id : 0

siblings : 2

core id : 1

cpu cores : 2

apicid : 1

initial apicid : 1

fpu : yes

fpu_exception : yes

cpuid level : 13

wp : yes

bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass

bogomips : 6199.95

clflush size : 64

cache_alignment : 64

address sizes : 40 bits physical, 48 bits virtual

power management:

Software Tweaks Post Upgrade.

I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.

worker_processes auto;
worker_cpu_affinity auto;

1 2	worker_processes auto; worker_cpu_affinity auto;

I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase the memory usage and child workers. I doubled child workers and max memeory limit.

memory_limit = 3072M
pm.max_children = 80
pm.start_servers = 30
pm.min_spare_servers = 10
pm.max_spare_servers = 30

memory_limit = 3072M

pm.max_children = 80

pm.start_servers = 30

pm.min_spare_servers = 10

pm.max_spare_servers = 30

I restarted NGINX and PHP

nginx -t
nginx -s reload
/etc/init.d/nginx restart
service php7.2-fpm restart

nginx -t

nginx -s reload

/etc/init.d/nginx restart

service php7.2-fpm restart

I tweaked WordPress max memory limits

define( 'WP_MEMORY_LIMIT','3072M');
define( 'WP_MAX_MEMORY_LIMIT','3072M');

1 2	define( 'WP_MEMORY_LIMIT','3072M'); define( 'WP_MAX_MEMORY_LIMIT','3072M');

MySQL Tweaks: I logged into MySQL

mysql -u root -p

1	mysql -u root -p

I ran “SHOW GLOBAL STATUS” to view stats

mysql> SHOW GLOBAL STATUS;
+-----------------------------------------------+--------------------------------------------------+
| Variable_name                                 | Value                                            |
+-----------------------------------------------+--------------------------------------------------+
| Aborted_clients                               | 0                                                |
| Aborted_connects                              | 0                                                |
| Binlog_cache_disk_use                         | 0                                                |
| Binlog_cache_use                              | 0                                                |
| Binlog_stmt_cache_disk_use                    | 0                                                |
| Binlog_stmt_cache_use                         | 0                                                |
| Bytes_received                                | 3179986                                          |
| Bytes_sent                                    | 223872114                                        |
| Com_admin_commands                            | 0                                                |
| Com_assign_to_keycache                        | 0                                                |
| Com_alter_db                                  | 0                                                |
| Com_alter_db_upgrade                          | 0                                                |
| Com_alter_event                               | 0                                                |
| Com_alter_function                            | 0                                                |
| Com_alter_instance                            | 0                                                |
| Com_alter_procedure                           | 0                                                |
| Com_alter_server                              | 0                                                |
| Com_alter_table                               | 0                                                |
| Com_alter_tablespace                          | 0                                                |
| Com_alter_user                                | 0                                                |
| Com_analyze                                   | 0                                                |
| Com_begin                                     | 0                                                |
| Com_binlog                                    | 0                                                |
| Com_call_procedure                            | 0                                                |
| Com_change_db                                 | 284                                              |
| Com_change_master                             | 0                                                |
| Com_change_repl_filter                        | 0                                                |
| Com_check                                     | 0                                                |
| Com_checksum                                  | 0                                                |
| Com_commit                                    | 0                                                |
| Com_create_db                                 | 0                                                |
| Com_create_event                              | 0                                                |
| Com_create_function                           | 0                                                |
| Com_create_index                              | 0                                                |
| Com_create_procedure                          | 0                                                |
| Com_create_server                             | 0                                                |
| Com_create_table                              | 0                                                |
| Com_create_trigger                            | 0                                                |
| Com_create_udf                                | 0                                                |
| Com_create_user                               | 0                                                |
| Com_create_view                               | 0                                                |
| Com_dealloc_sql                               | 0                                                |
| Com_delete                                    | 18                                               |
| Com_delete_multi                              | 0                                                |
| Com_do                                        | 0                                                |
| Com_drop_db                                   | 0                                                |
| Com_drop_event                                | 0                                                |
| Com_drop_function                             | 0                                                |
| Com_drop_index                                | 0                                                |
| Com_drop_procedure                            | 0                                                |
| Com_drop_server                               | 0                                                |
| Com_drop_table                                | 0                                                |
| Com_drop_trigger                              | 0                                                |
| Com_drop_user                                 | 0                                                |
| Com_drop_view                                 | 0                                                |
| Com_empty_query                               | 0                                                |
| Com_execute_sql                               | 0                                                |
| Com_explain_other                             | 0                                                |
| Com_flush                                     | 0                                                |
| Com_get_diagnostics                           | 0                                                |
| Com_grant                                     | 0                                                |
| Com_ha_close                                  | 0                                                |
| Com_ha_open                                   | 0                                                |
| Com_ha_read                                   | 0                                                |
| Com_help                                      | 0                                                |
| Com_insert                                    | 342                                              |
| Com_insert_select                             | 0                                                |
| Com_install_plugin                            | 0                                                |
| Com_kill                                      | 0                                                |
| Com_load                                      | 0                                                |
| Com_lock_tables                               | 0                                                |
| Com_optimize                                  | 0                                                |
| Com_preload_keys                              | 0                                                |
| Com_prepare_sql                               | 0                                                |
| Com_purge                                     | 0                                                |
| Com_purge_before_date                         | 0                                                |
| Com_release_savepoint                         | 0                                                |
| Com_rename_table                              | 0                                                |
| Com_rename_user                               | 0                                                |
| Com_repair                                    | 0                                                |
| Com_replace                                   | 0                                                |
| Com_replace_select                            | 0                                                |
| Com_reset                                     | 0                                                |
| Com_resignal                                  | 0                                                |
| Com_revoke                                    | 0                                                |
| Com_revoke_all                                | 0                                                |
| Com_rollback                                  | 0                                                |
| Com_rollback_to_savepoint                     | 0                                                |
| Com_savepoint                                 | 0                                                |
| Com_select                                    | 16358                                            |
| Com_set_option                                | 849                                              |
| Com_signal                                    | 0                                                |
| Com_show_binlog_events                        | 0                                                |
| Com_show_binlogs                              | 0                                                |
| Com_show_charsets                             | 0                                                |
| Com_show_collations                           | 0                                                |
| Com_show_create_db                            | 0                                                |
| Com_show_create_event                         | 0                                                |
| Com_show_create_func                          | 0                                                |
| Com_show_create_proc                          | 0                                                |
| Com_show_create_table                         | 0                                                |
| Com_show_create_trigger                       | 0                                                |
| Com_show_databases                            | 3                                                |
| Com_show_engine_logs                          | 0                                                |
| Com_show_engine_mutex                         | 0                                                |
| Com_show_engine_status                        | 0                                                |
| Com_show_events                               | 0                                                |
| Com_show_errors                               | 0                                                |
| Com_show_fields                               | 921                                              |
| Com_show_function_code                        | 0                                                |
| Com_show_function_status                      | 0                                                |
| Com_show_grants                               | 0                                                |
| Com_show_keys                                 | 1                                                |
| Com_show_master_status                        | 0                                                |
| Com_show_open_tables                          | 0                                                |
| Com_show_plugins                              | 0                                                |
| Com_show_privileges                           | 0                                                |
| Com_show_procedure_code                       | 0                                                |
| Com_show_procedure_status                     | 0                                                |
| Com_show_processlist                          | 0                                                |
| Com_show_profile                              | 0                                                |
| Com_show_profiles                             | 0                                                |
| Com_show_relaylog_events                      | 0                                                |
| Com_show_slave_hosts                          | 0                                                |
| Com_show_slave_status                         | 0                                                |
| Com_show_status                               | 6                                                |
| Com_show_storage_engines                      | 0                                                |
| Com_show_table_status                         | 0                                                |
| Com_show_tables                               | 2                                                |
| Com_show_triggers                             | 0                                                |
| Com_show_variables                            | 6                                                |
| Com_show_warnings                             | 1                                                |
| Com_show_create_user                          | 0                                                |
| Com_shutdown                                  | 0                                                |
| Com_slave_start                               | 0                                                |
| Com_slave_stop                                | 0                                                |
| Com_group_replication_start                   | 0                                                |
| Com_group_replication_stop                    | 0                                                |
| Com_stmt_execute                              | 4                                                |
| Com_stmt_close                                | 4                                                |
| Com_stmt_fetch                                | 0                                                |
| Com_stmt_prepare                              | 4                                                |
| Com_stmt_reset                                | 0                                                |
| Com_stmt_send_long_data                       | 4                                                |
| Com_truncate                                  | 2                                                |
| Com_uninstall_plugin                          | 0                                                |
| Com_unlock_tables                             | 0                                                |
| Com_update                                    | 70                                               |
| Com_update_multi                              | 0                                                |
| Com_xa_commit                                 | 0                                                |
| Com_xa_end                                    | 0                                                |
| Com_xa_prepare                                | 0                                                |
| Com_xa_recover                                | 0                                                |
| Com_xa_rollback                               | 0                                                |
| Com_xa_start                                  | 0                                                |
| Com_stmt_reprepare                            | 0                                                |
| Connection_errors_accept                      | 0                                                |
| Connection_errors_internal                    | 0                                                |
| Connection_errors_max_connections             | 0                                                |
| Connection_errors_peer_address                | 0                                                |
| Connection_errors_select                      | 0                                                |
| Connection_errors_tcpwrap                     | 0                                                |
| Connections                                   | 292                                              |
| Created_tmp_disk_tables                       | 1124                                             |
| Created_tmp_files                             | 30                                               |
| Created_tmp_tables                            | 1369                                             |
| Delayed_errors                                | 0                                                |
| Delayed_insert_threads                        | 0                                                |
| Delayed_writes                                | 0                                                |
| Flush_commands                                | 1                                                |
| Handler_commit                                | 6094                                             |
| Handler_delete                                | 33                                               |
| Handler_discover                              | 0                                                |
| Handler_external_lock                         | 38571                                            |
| Handler_mrr_init                              | 0                                                |
| Handler_prepare                               | 0                                                |
| Handler_read_first                            | 2299                                             |
| Handler_read_key                              | 134761                                           |
| Handler_read_last                             | 237                                              |
| Handler_read_next                             | 310119                                           |
| Handler_read_prev                             | 2733                                             |
| Handler_read_rnd                              | 222350                                           |
| Handler_read_rnd_next                         | 472820                                           |
| Handler_rollback                              | 0                                                |
| Handler_savepoint                             | 0                                                |
| Handler_savepoint_rollback                    | 0                                                |
| Handler_update                                | 15605                                            |
| Handler_write                                 | 17310                                            |
| Innodb_buffer_pool_dump_status                | Dumping of buffer pool not started               |
| Innodb_buffer_pool_load_status                | Buffer pool(s) load completed at 180624 23:38:01 |
| Innodb_buffer_pool_resize_status              |                                                  |
| Innodb_buffer_pool_pages_data                 | 1035                                             |
| Innodb_buffer_pool_bytes_data                 | 16957440                                         |
| Innodb_buffer_pool_pages_dirty                | 0                                                |
| Innodb_buffer_pool_bytes_dirty                | 0                                                |
| Innodb_buffer_pool_pages_flushed              | 1936                                             |
| Innodb_buffer_pool_pages_free                 | 7144                                             |
| Innodb_buffer_pool_pages_misc                 | 13                                               |
| Innodb_buffer_pool_pages_total                | 8192                                             |
| Innodb_buffer_pool_read_ahead_rnd             | 0                                                |
| Innodb_buffer_pool_read_ahead                 | 0                                                |
| Innodb_buffer_pool_read_ahead_evicted         | 0                                                |
| Innodb_buffer_pool_read_requests              | 306665                                           |
| Innodb_buffer_pool_reads                      | 950                                              |
| Innodb_buffer_pool_wait_free                  | 0                                                |
| Innodb_buffer_pool_write_requests             | 26509                                            |
| Innodb_data_fsyncs                            | 1229                                             |
| Innodb_data_pending_fsyncs                    | 0                                                |
| Innodb_data_pending_reads                     | 0                                                |
| Innodb_data_pending_writes                    | 0                                                |
| Innodb_data_read                              | 16273920                                         |
| Innodb_data_reads                             | 1078                                             |
| Innodb_data_writes                            | 2857                                             |
| Innodb_data_written                           | 53379584                                         |
| Innodb_dblwr_pages_written                    | 1275                                             |
| Innodb_dblwr_writes                           | 109                                              |
| Innodb_log_waits                              | 0                                                |
| Innodb_log_write_requests                     | 450                                              |
| Innodb_log_writes                             | 585                                              |
| Innodb_os_log_fsyncs                          | 793                                              |
| Innodb_os_log_pending_fsyncs                  | 0                                                |
| Innodb_os_log_pending_writes                  | 0                                                |
| Innodb_os_log_written                         | 664064                                           |
| Innodb_page_size                              | 16384                                            |
| Innodb_pages_created                          | 56                                               |
| Innodb_pages_read                             | 988                                              |
| Innodb_pages_written                          | 1936                                             |
| Innodb_row_lock_current_waits                 | 0                                                |
| Innodb_row_lock_time                          | 0                                                |
| Innodb_row_lock_time_avg                      | 0                                                |
| Innodb_row_lock_time_max                      | 0                                                |
| Innodb_row_lock_waits                         | 0                                                |
| Innodb_rows_deleted                           | 2                                                |
| Innodb_rows_inserted                          | 19219                                            |
| Innodb_rows_read                              | 249102                                           |
| Innodb_rows_updated                           | 77                                               |
| Innodb_num_open_files                         | 81                                               |
| Innodb_truncated_status_writes                | 0                                                |
| Innodb_available_undo_logs                    | 128                                              |
| Key_blocks_not_flushed                        | 0                                                |
| Key_blocks_unused                             | 12751                                            |
| Key_blocks_used                               | 645                                              |
| Key_read_requests                             | 321877                                           |
| Key_reads                                     | 648                                              |
| Key_write_requests                            | 196                                              |
| Key_writes                                    | 150                                              |
| Locked_connects                               | 0                                                |
| Max_execution_time_exceeded                   | 0                                                |
| Max_execution_time_set                        | 0                                                |
| Max_execution_time_set_failed                 | 0                                                |
| Max_used_connections                          | 3                                                |
| Max_used_connections_time                     | 2018-06-24 23:43:48                              |
| Not_flushed_delayed_rows                      | 0                                                |
| Ongoing_anonymous_transaction_count           | 0                                                |
| Open_files                                    | 229                                              |
| Open_streams                                  | 0                                                |
| Open_table_definitions                        | 206                                              |
| Open_tables                                   | 786                                              |
| Opened_files                                  | 502                                              |
| Opened_table_definitions                      | 208                                              |
| Opened_tables                                 | 817                                              |
| Performance_schema_accounts_lost              | 0                                                |
| Performance_schema_cond_classes_lost          | 0                                                |
| Performance_schema_cond_instances_lost        | 0                                                |
| Performance_schema_digest_lost                | 0                                                |
| Performance_schema_file_classes_lost          | 0                                                |
| Performance_schema_file_handles_lost          | 0                                                |
| Performance_schema_file_instances_lost        | 0                                                |
| Performance_schema_hosts_lost                 | 0                                                |
| Performance_schema_index_stat_lost            | 0                                                |
| Performance_schema_locker_lost                | 0                                                |
| Performance_schema_memory_classes_lost        | 0                                                |
| Performance_schema_metadata_lock_lost         | 0                                                |
| Performance_schema_mutex_classes_lost         | 0                                                |
| Performance_schema_mutex_instances_lost       | 0                                                |
| Performance_schema_nested_statement_lost      | 0                                                |
| Performance_schema_prepared_statements_lost   | 0                                                |
| Performance_schema_program_lost               | 0                                                |
| Performance_schema_rwlock_classes_lost        | 0                                                |
| Performance_schema_rwlock_instances_lost      | 0                                                |
| Performance_schema_session_connect_attrs_lost | 0                                                |
| Performance_schema_socket_classes_lost        | 0                                                |
| Performance_schema_socket_instances_lost      | 0                                                |
| Performance_schema_stage_classes_lost         | 0                                                |
| Performance_schema_statement_classes_lost     | 0                                                |
| Performance_schema_table_handles_lost         | 0                                                |
| Performance_schema_table_instances_lost       | 0                                                |
| Performance_schema_table_lock_stat_lost       | 0                                                |
| Performance_schema_thread_classes_lost        | 0                                                |
| Performance_schema_thread_instances_lost      | 0                                                |
| Performance_schema_users_lost                 | 0                                                |
| Prepared_stmt_count                           | 0                                                |
| Qcache_free_blocks                            | 1                                                |
| Qcache_free_memory                            | 16760152                                         |
| Qcache_hits                                   | 0                                                |
| Qcache_inserts                                | 0                                                |
| Qcache_lowmem_prunes                          | 0                                                |
| Qcache_not_cached                             | 16355                                            |
| Qcache_queries_in_cache                       | 0                                                |
| Qcache_total_blocks                           | 1                                                |
| Queries                                       | 19164                                            |
| Questions                                     | 19155                                            |
| Select_full_join                              | 0                                                |
| Select_full_range_join                        | 0                                                |
| Select_range                                  | 2677                                             |
| Select_range_check                            | 0                                                |
| Select_scan                                   | 2098                                             |
| Slave_open_temp_tables                        | 0                                                |
| Slow_launch_threads                           | 0                                                |
| Slow_queries                                  | 0                                                |
| Sort_merge_passes                             | 12                                               |
| Sort_range                                    | 4859                                             |
| Sort_rows                                     | 244452                                           |
| Sort_scan                                     | 854                                              |
| Ssl_accept_renegotiates                       | 0                                                |
| Ssl_accepts                                   | 0                                                |
| Ssl_callback_cache_hits                       | 0                                                |
| Ssl_cipher                                    |                                                  |
| Ssl_cipher_list                               |                                                  |
| Ssl_client_connects                           | 0                                                |
| Ssl_connect_renegotiates                      | 0                                                |
| Ssl_ctx_verify_depth                          | 0                                                |
| Ssl_ctx_verify_mode                           | 0                                                |
| Ssl_default_timeout                           | 0                                                |
| Ssl_finished_accepts                          | 0                                                |
| Ssl_finished_connects                         | 0                                                |
| Ssl_server_not_after                          |                                                  |
| Ssl_server_not_before                         |                                                  |
| Ssl_session_cache_hits                        | 0                                                |
| Ssl_session_cache_misses                      | 0                                                |
| Ssl_session_cache_mode                        | NONE                                             |
| Ssl_session_cache_overflows                   | 0                                                |
| Ssl_session_cache_size                        | 0                                                |
| Ssl_session_cache_timeouts                    | 0                                                |
| Ssl_sessions_reused                           | 0                                                |
| Ssl_used_session_cache_entries                | 0                                                |
| Ssl_verify_depth                              | 0                                                |
| Ssl_verify_mode                               | 0                                                |
| Ssl_version                                   |                                                  |
| Table_locks_immediate                         | 11962                                            |
| Table_locks_waited                            | 0                                                |
| Table_open_cache_hits                         | 19395                                            |
| Table_open_cache_misses                       | 817                                              |
| Table_open_cache_overflows                    | 12                                               |
| Tc_log_max_pages_used                         | 0                                                |
| Tc_log_page_size                              | 0                                                |
| Tc_log_page_waits                             | 0                                                |
| Threads_cached                                | 2                                                |
| Threads_connected                             | 1                                                |
| Threads_created                               | 3                                                |
| Threads_running                               | 1                                                |
| Uptime                                        | 2944                                             |
| Uptime_since_flush_status                     | 2944                                             |
+-----------------------------------------------+--------------------------------------------------+
353 rows in set (0.00 sec)

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

342

343

344

345

346

347

348

349

350

351

352

353

354

355

356

357

358

359

mysql> SHOW GLOBAL STATUS;

+-----------------------------------------------+--------------------------------------------------+

| Variable_name | Value |

+-----------------------------------------------+--------------------------------------------------+

| Aborted_clients | 0 |

| Aborted_connects | 0 |

| Binlog_cache_disk_use | 0 |

| Binlog_cache_use | 0 |

| Binlog_stmt_cache_disk_use | 0 |

| Binlog_stmt_cache_use | 0 |

| Bytes_received | 3179986 |

| Bytes_sent | 223872114 |

| Com_admin_commands | 0 |

| Com_assign_to_keycache | 0 |

| Com_alter_db | 0 |

| Com_alter_db_upgrade | 0 |

| Com_alter_event | 0 |

| Com_alter_function | 0 |

| Com_alter_instance | 0 |

| Com_alter_procedure | 0 |

| Com_alter_server | 0 |

| Com_alter_table | 0 |

| Com_alter_tablespace | 0 |

| Com_alter_user | 0 |

| Com_analyze | 0 |

| Com_begin | 0 |

| Com_binlog | 0 |

| Com_call_procedure | 0 |

| Com_change_db | 284 |

| Com_change_master | 0 |

| Com_change_repl_filter | 0 |

| Com_check | 0 |

| Com_checksum | 0 |

| Com_commit | 0 |

| Com_create_db | 0 |

| Com_create_event | 0 |

| Com_create_function | 0 |

| Com_create_index | 0 |

| Com_create_procedure | 0 |

| Com_create_server | 0 |

| Com_create_table | 0 |

| Com_create_trigger | 0 |

| Com_create_udf | 0 |

| Com_create_user | 0 |

| Com_create_view | 0 |

| Com_dealloc_sql | 0 |

| Com_delete | 18 |

| Com_delete_multi | 0 |

| Com_do | 0 |

| Com_drop_db | 0 |

| Com_drop_event | 0 |

| Com_drop_function | 0 |

| Com_drop_index | 0 |

| Com_drop_procedure | 0 |

| Com_drop_server | 0 |

| Com_drop_table | 0 |

| Com_drop_trigger | 0 |

| Com_drop_user | 0 |

| Com_drop_view | 0 |

| Com_empty_query | 0 |

| Com_execute_sql | 0 |

| Com_explain_other | 0 |

| Com_flush | 0 |

| Com_get_diagnostics | 0 |

| Com_grant | 0 |

| Com_ha_close | 0 |

| Com_ha_open | 0 |

| Com_ha_read | 0 |

| Com_help | 0 |

| Com_insert | 342 |

| Com_insert_select | 0 |

| Com_install_plugin | 0 |

| Com_kill | 0 |

| Com_load | 0 |

| Com_lock_tables | 0 |

| Com_optimize | 0 |

| Com_preload_keys | 0 |

| Com_prepare_sql | 0 |

| Com_purge | 0 |

| Com_purge_before_date | 0 |

| Com_release_savepoint | 0 |

| Com_rename_table | 0 |

| Com_rename_user | 0 |

| Com_repair | 0 |

| Com_replace | 0 |

| Com_replace_select | 0 |

| Com_reset | 0 |

| Com_resignal | 0 |

| Com_revoke | 0 |

| Com_revoke_all | 0 |

| Com_rollback | 0 |

| Com_rollback_to_savepoint | 0 |

| Com_savepoint | 0 |

| Com_select | 16358 |

| Com_set_option | 849 |

| Com_signal | 0 |

| Com_show_binlog_events | 0 |

| Com_show_binlogs | 0 |

| Com_show_charsets | 0 |

| Com_show_collations | 0 |

| Com_show_create_db | 0 |

| Com_show_create_event | 0 |

| Com_show_create_func | 0 |

| Com_show_create_proc | 0 |

| Com_show_create_table | 0 |

| Com_show_create_trigger | 0 |

| Com_show_databases | 3 |

| Com_show_engine_logs | 0 |

| Com_show_engine_mutex | 0 |

| Com_show_engine_status | 0 |

| Com_show_events | 0 |

| Com_show_errors | 0 |

| Com_show_fields | 921 |

| Com_show_function_code | 0 |

| Com_show_function_status | 0 |

| Com_show_grants | 0 |

| Com_show_keys | 1 |

| Com_show_master_status | 0 |

| Com_show_open_tables | 0 |

| Com_show_plugins | 0 |

| Com_show_privileges | 0 |

| Com_show_procedure_code | 0 |

| Com_show_procedure_status | 0 |

| Com_show_processlist | 0 |

| Com_show_profile | 0 |

| Com_show_profiles | 0 |

| Com_show_relaylog_events | 0 |

| Com_show_slave_hosts | 0 |

| Com_show_slave_status | 0 |

| Com_show_status | 6 |

| Com_show_storage_engines | 0 |

| Com_show_table_status | 0 |

| Com_show_tables | 2 |

| Com_show_triggers | 0 |

| Com_show_variables | 6 |

| Com_show_warnings | 1 |

| Com_show_create_user | 0 |

| Com_shutdown | 0 |

| Com_slave_start | 0 |

| Com_slave_stop | 0 |

| Com_group_replication_start | 0 |

| Com_group_replication_stop | 0 |

| Com_stmt_execute | 4 |

| Com_stmt_close | 4 |

| Com_stmt_fetch | 0 |

| Com_stmt_prepare | 4 |

| Com_stmt_reset | 0 |

| Com_stmt_send_long_data | 4 |

| Com_truncate | 2 |

| Com_uninstall_plugin | 0 |

| Com_unlock_tables | 0 |

| Com_update | 70 |

| Com_update_multi | 0 |

| Com_xa_commit | 0 |

| Com_xa_end | 0 |

| Com_xa_prepare | 0 |

| Com_xa_recover | 0 |

| Com_xa_rollback | 0 |

| Com_xa_start | 0 |

| Com_stmt_reprepare | 0 |

| Connection_errors_accept | 0 |

| Connection_errors_internal | 0 |

| Connection_errors_max_connections | 0 |

| Connection_errors_peer_address | 0 |

| Connection_errors_select | 0 |

| Connection_errors_tcpwrap | 0 |

| Connections | 292 |

| Created_tmp_disk_tables | 1124 |

| Created_tmp_files | 30 |

| Created_tmp_tables | 1369 |

| Delayed_errors | 0 |

| Delayed_insert_threads | 0 |

| Delayed_writes | 0 |

| Flush_commands | 1 |

| Handler_commit | 6094 |

| Handler_delete | 33 |

| Handler_discover | 0 |

| Handler_external_lock | 38571 |

| Handler_mrr_init | 0 |

| Handler_prepare | 0 |

| Handler_read_first | 2299 |

| Handler_read_key | 134761 |

| Handler_read_last | 237 |

| Handler_read_next | 310119 |

| Handler_read_prev | 2733 |

| Handler_read_rnd | 222350 |

| Handler_read_rnd_next | 472820 |

| Handler_rollback | 0 |

| Handler_savepoint | 0 |

| Handler_savepoint_rollback | 0 |

| Handler_update | 15605 |

| Handler_write | 17310 |

| Innodb_buffer_pool_dump_status | Dumping of buffer pool not started |

| Innodb_buffer_pool_load_status | Buffer pool(s) load completed at 180624 23:38:01 |

| Innodb_buffer_pool_resize_status | |

| Innodb_buffer_pool_pages_data | 1035 |

| Innodb_buffer_pool_bytes_data | 16957440 |

| Innodb_buffer_pool_pages_dirty | 0 |

| Innodb_buffer_pool_bytes_dirty | 0 |

| Innodb_buffer_pool_pages_flushed | 1936 |

| Innodb_buffer_pool_pages_free | 7144 |

| Innodb_buffer_pool_pages_misc | 13 |

| Innodb_buffer_pool_pages_total | 8192 |

| Innodb_buffer_pool_read_ahead_rnd | 0 |

| Innodb_buffer_pool_read_ahead | 0 |

| Innodb_buffer_pool_read_ahead_evicted | 0 |

| Innodb_buffer_pool_read_requests | 306665 |

| Innodb_buffer_pool_reads | 950 |

| Innodb_buffer_pool_wait_free | 0 |

| Innodb_buffer_pool_write_requests | 26509 |

| Innodb_data_fsyncs | 1229 |

| Innodb_data_pending_fsyncs | 0 |

| Innodb_data_pending_reads | 0 |

| Innodb_data_pending_writes | 0 |

| Innodb_data_read | 16273920 |

| Innodb_data_reads | 1078 |

| Innodb_data_writes | 2857 |

| Innodb_data_written | 53379584 |

| Innodb_dblwr_pages_written | 1275 |

| Innodb_dblwr_writes | 109 |

| Innodb_log_waits | 0 |

| Innodb_log_write_requests | 450 |

| Innodb_log_writes | 585 |

| Innodb_os_log_fsyncs | 793 |

| Innodb_os_log_pending_fsyncs | 0 |

| Innodb_os_log_pending_writes | 0 |

| Innodb_os_log_written | 664064 |

| Innodb_page_size | 16384 |

| Innodb_pages_created | 56 |

| Innodb_pages_read | 988 |

| Innodb_pages_written | 1936 |

| Innodb_row_lock_current_waits | 0 |

| Innodb_row_lock_time | 0 |

| Innodb_row_lock_time_avg | 0 |

| Innodb_row_lock_time_max | 0 |

| Innodb_row_lock_waits | 0 |

| Innodb_rows_deleted | 2 |

| Innodb_rows_inserted | 19219 |

| Innodb_rows_read | 249102 |

| Innodb_rows_updated | 77 |

| Innodb_num_open_files | 81 |

| Innodb_truncated_status_writes | 0 |

| Innodb_available_undo_logs | 128 |

| Key_blocks_not_flushed | 0 |

| Key_blocks_unused | 12751 |

| Key_blocks_used | 645 |

| Key_read_requests | 321877 |

| Key_reads | 648 |

| Key_write_requests | 196 |

| Key_writes | 150 |

| Locked_connects | 0 |

| Max_execution_time_exceeded | 0 |

| Max_execution_time_set | 0 |

| Max_execution_time_set_failed | 0 |

| Max_used_connections | 3 |

| Max_used_connections_time | 2018-06-24 23:43:48 |

| Not_flushed_delayed_rows | 0 |

| Ongoing_anonymous_transaction_count | 0 |

| Open_files | 229 |

| Open_streams | 0 |

| Open_table_definitions | 206 |

| Open_tables | 786 |

| Opened_files | 502 |

| Opened_table_definitions | 208 |

| Opened_tables | 817 |

| Performance_schema_accounts_lost | 0 |

| Performance_schema_cond_classes_lost | 0 |

| Performance_schema_cond_instances_lost | 0 |

| Performance_schema_digest_lost | 0 |

| Performance_schema_file_classes_lost | 0 |

| Performance_schema_file_handles_lost | 0 |

| Performance_schema_file_instances_lost | 0 |

| Performance_schema_hosts_lost | 0 |

| Performance_schema_index_stat_lost | 0 |

| Performance_schema_locker_lost | 0 |

| Performance_schema_memory_classes_lost | 0 |

| Performance_schema_metadata_lock_lost | 0 |

| Performance_schema_mutex_classes_lost | 0 |

| Performance_schema_mutex_instances_lost | 0 |

| Performance_schema_nested_statement_lost | 0 |

| Performance_schema_prepared_statements_lost | 0 |

| Performance_schema_program_lost | 0 |

| Performance_schema_rwlock_classes_lost | 0 |

| Performance_schema_rwlock_instances_lost | 0 |

| Performance_schema_session_connect_attrs_lost | 0 |

| Performance_schema_socket_classes_lost | 0 |

| Performance_schema_socket_instances_lost | 0 |

| Performance_schema_stage_classes_lost | 0 |

| Performance_schema_statement_classes_lost | 0 |

| Performance_schema_table_handles_lost | 0 |

| Performance_schema_table_instances_lost | 0 |

| Performance_schema_table_lock_stat_lost | 0 |

| Performance_schema_thread_classes_lost | 0 |

| Performance_schema_thread_instances_lost | 0 |

| Performance_schema_users_lost | 0 |

| Prepared_stmt_count | 0 |

| Qcache_free_blocks | 1 |

| Qcache_free_memory | 16760152 |

| Qcache_hits | 0 |

| Qcache_inserts | 0 |

| Qcache_lowmem_prunes | 0 |

| Qcache_not_cached | 16355 |

| Qcache_queries_in_cache | 0 |

| Qcache_total_blocks | 1 |

| Queries | 19164 |

| Questions | 19155 |

| Select_full_join | 0 |

| Select_full_range_join | 0 |

| Select_range | 2677 |

| Select_range_check | 0 |

| Select_scan | 2098 |

| Slave_open_temp_tables | 0 |

| Slow_launch_threads | 0 |

| Slow_queries | 0 |

| Sort_merge_passes | 12 |

| Sort_range | 4859 |

| Sort_rows | 244452 |

| Sort_scan | 854 |

| Ssl_accept_renegotiates | 0 |

| Ssl_accepts | 0 |

| Ssl_callback_cache_hits | 0 |

| Ssl_cipher | |

| Ssl_cipher_list | |

| Ssl_client_connects | 0 |

| Ssl_connect_renegotiates | 0 |

| Ssl_ctx_verify_depth | 0 |

| Ssl_ctx_verify_mode | 0 |

| Ssl_default_timeout | 0 |

| Ssl_finished_accepts | 0 |

| Ssl_finished_connects | 0 |

| Ssl_server_not_after | |

| Ssl_server_not_before | |

| Ssl_session_cache_hits | 0 |

| Ssl_session_cache_misses | 0 |

| Ssl_session_cache_mode | NONE |

| Ssl_session_cache_overflows | 0 |

| Ssl_session_cache_size | 0 |

| Ssl_session_cache_timeouts | 0 |

| Ssl_sessions_reused | 0 |

| Ssl_used_session_cache_entries | 0 |

| Ssl_verify_depth | 0 |

| Ssl_verify_mode | 0 |

| Ssl_version | |

| Table_locks_immediate | 11962 |

| Table_locks_waited | 0 |

| Table_open_cache_hits | 19395 |

| Table_open_cache_misses | 817 |

| Table_open_cache_overflows | 12 |

| Tc_log_max_pages_used | 0 |

| Tc_log_page_size | 0 |

| Tc_log_page_waits | 0 |

| Threads_cached | 2 |

| Threads_connected | 1 |

| Threads_created | 3 |

| Threads_running | 1 |

| Uptime | 2944 |

| Uptime_since_flush_status | 2944 |

+-----------------------------------------------+--------------------------------------------------+

353 rows in set (0.00 sec)

Read more on SHOW GLOBAL STATUS here. Read more on the values here.

I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.

Configure Ubuntu System Memory Usage

Edit /etc/sysctl.conf

Add the following to allow things to sit in ram longer

vm.vfs_cache_pressure=50

1	vm.vfs_cache_pressure=50

Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt

This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are."

This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim. Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are."

Read these pages here and here regarding setting MySQL memory.

Reboot

shutdown -r now

1	shutdown -r now

Resize the disk

The 2x CPU, 4GB memory plan comes with 80GB storage allowance. My disk at present is 50GB and I will update the size soon following this guide soon.

Quick Benchmark

I used loader.io to load 500 users to access my site in 1 minute.

The benchmark worked with no errors.

This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.

Conclusion

After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.

This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

Revision History

1.0 Initial Post

How to install Windows 10 Pro alongside an OSX partition with Apple Boot Camp

April 20, 2018 by Simon

This guide will show you how to install Windows 10 Pro alongside an OSX partition with Apple Boot Camp.

I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. I use OSX to develop and occasionally need to use Windows to achieve some development tasks, this is how you can install Windows on an Apple PC.

Apple Boot Camp

Apple Bootcamp software can be found here: https://support.apple.com/en-au/HT201468

Bootcamp Steps.

You can download a Windows 10 ISO here: https://www.microsoft.com/en-au/software-download/windows10

You will need to plug in a 16GB or larger USB key to use as an install medium.

Download Boot Camp

Download AppBoot Camp here: https://support.apple.com/en-au/boot-camp

Start Apple Boot Camp, click Continue, Tick all options, select the Windows 10 ISO file (beware Bootcamp will select the first ISO file it finds, ensure your USB is selected, Click Continue (tip: You may need to er format and prepare the USB key 10 times, Apple Bootcamp is not the most reliable program), It may take 6 hours to copy files pre Window Setup).

Choose your Windows partition size (at least 200GB is ideas)

When the Bootcamp Wizard is complete you can reboot into Windows (automatic at the end of the wizard)

Enter the desired details in each step (e.g Local, Windows Key, Windows Version, Partition and Country etc)

Loads more steps like Keyboard Layout, Domain, Microsoft Account, Password and Cortana and Privacy etc.

Windows is now installed, Apple Bootcamp will set up appropriate drivers for your Mac,

While this is happening I will install Google Chrome Canary.

Bootcamp has now finished setting up drivers.

Windows 10 Start menu (I prefer OSX’s simplicity)

You can set up your prefered Startup disk in the Apple System Preferences or press the Option key on startup and choose a partition to boot.

Update: Open a Windows 10 Boot Camp Installation on OSX in Parallels (like a VM)

Read this guide will show you how you can open a Windows 10 Boot Camp Installation on OSX in Parallels (like a VM).

I hope this guide helps someone.

Ask a question or recommend an article

Revision History

v1.1 Added Open Bootcams as VM

v1.0 Initial post

Moving an Ubuntu 16.04 VM on Vultr from one data centre to another via snapshots

April 17, 2018 by Simon

This guide will show how you can move an Ubuntu VM server domain between Vultr data centres via snapshots.

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Sometimes you need to move a sever between locations and/or upgrade the server (to have more memory t install WordPress).

Moving an existing Vultr server

If you don’t have an Ubuntu server click here (follow this guide).

Login to Vultr and specify a source server, click Snapshots and click Take Snapshot.

Wait for the snapshot to finish (It may take 1 hour).

Great, the snapshot is done.

Now I can create a new server (in a different data centre).

Deploy New Instance

Choose a location (Australia is at capacity, so I’ll deploy to Silicon Valley then move again in a few weeks), choose the snapshot to restore, choose a plan, I enabled IPV6/Auto Backups and Private Networking.

TIP: The password for the server will be the same as the source server so write it down.

Click Deploy Now

After a few minutes, you can see the new servers IP address, you can log in to your domain name provider (in my case Namecheap) and update the target IPV4 and IPV6 address.

You can find IPV4 and IPV6 addresses by opening your server, clicking settings then IPxV4 or IPV6.

You will need to update Vultr DNS settings (login to Vultr, Click Servers, Click DNS then edit your existing Domain DNS entry). Add you’re new serves IP addresses.

Update: I added an IPV6/AAAA record too.

Wait for DNS Replication

Goto https://www.whatsmydns.net/ and check the global DNS propagation for your new domain’s server.

If you are happy that the server has been migrated (snapshot restored) and that the domain DNS is pointing to your new server you can delete the old server in the Vultr server list.

Post-Migrate Actions

Setup Daily backups.
Review firewall settings (guide here).
Optional: Install MySQL
Optional: Install PHP
Optional: Install PHP Pooled Connections
Optional: Install WordPress
Optional: Install WordPress CDN
Optional: Configure Cloudflare
etc

I hope this guide helps someone.

Ask a question or recommend an article

Revision History

v1.1 Vultr Link

v1.0 Initial post

Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare

April 5, 2018 by Simon

This guide will show you how to enable the latest Transport Layer Security (TLS) 1.3 protocol with it’s predecessor Secure Sockets Layer (SSL) with NGINX and OpenSSL for better website security on an Ubuntu 16.04 server

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Making sure your server is up to date and running the latest SSL software is important. I have updated Open SSL before and blogged about this here. Do backup your server before changing settings and if you use Cloudflare (if you don’t do it now) enable Development Mode (and disable caching until changes are made).

For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

TLS 1.3 is the latest SSL security protocol that can be used between clients and servers to encrypt connections on the web.

TLS 1.3 uptake is only 60% according to https://caniuse.com/#search=TLS%201.3

Read why TLS 1.3 is important and news on TLS 1.3 can be found here: https://www.openssl.org/blog/blog/2018/02/08/tlsv1.3/

The Good and Bad

Done be like this commercial site with very poor security (tested with SSL labs and asafaweb)

Here is what the top 1 million sites do

Here it is!! Alexa Top 1 Million Analysis – February 2018 https://t.co/TjBHNX7zTi
— Scott Helme (@Scott_Helme) February 26, 2018

Installing Open SSL on Ubuntu

Connect to your Ubuntu 16.04 server via SSH (I connected to my Vultr server)

Check what version of OpenSSL you have? My OpenSSL is out of date.

# openssl version
OpenSSL 1.1.0g  2 Nov 2017

1 2	# openssl version OpenSSL 1.1.0g 2 Nov 2017

Tip: What Ciphers does your Open SSL Support?

openssl ciphers -s -v
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH       Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-ECDSA-AES256-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA1
ECDHE-RSA-AES256-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
ECDHE-ECDSA-AES128-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
ECDHE-RSA-AES128-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1

openssl ciphers -s -v

ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD

ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD

DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(256) Mac=AEAD

ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD

ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=RSA Enc=CHACHA20/POLY1305(256) Mac=AEAD

DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH Au=RSA Enc=CHACHA20/POLY1305(256) Mac=AEAD

ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD

ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD

DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(128) Mac=AEAD

ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384

ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384

DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(256) Mac=SHA256

ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256

ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256

DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(128) Mac=SHA256

ECDHE-ECDSA-AES256-SHA TLSv1 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA1

ECDHE-RSA-AES256-SHA TLSv1 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1

DHE-RSA-AES256-SHA SSLv3 Kx=DH Au=RSA Enc=AES(256) Mac=SHA1

ECDHE-ECDSA-AES128-SHA TLSv1 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA1

ECDHE-RSA-AES128-SHA TLSv1 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA1

DHE-RSA-AES128-SHA SSLv3 Kx=DH Au=RSA Enc=AES(128) Mac=SHA1

AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD

AES128-GCM-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(128) Mac=AEAD

AES256-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA256

AES128-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA256

AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1

AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1

Time to update Open SSL

OpenSSL 1.1.1 beta is available and supports TLS 1.3 but it is n BETA form. OpenSSL code is available here.

I did the following to download and build the latest version of OpenSSL.

mkdir /openssltemp
cd /openssltemp
sudo git clone git://git.openssl.org/openssl.git
cd openssl/
./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl -Wl,-rpath,/usr/local/ssl/lib
make
sudo make install

mkdir /openssltemp

cd /openssltemp

sudo git clone git://git.openssl.org/openssl.git

cd openssl/

./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl -Wl,-rpath,/usr/local/ssl/lib

make

sudo make install

I tried to check the open SSL version but had an error?

openssl version 
openssl: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)
openssl: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)

openssl version

openssl: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)

openssl: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)

A quick GitHub ticket revealed I needed to set a path variable.

export LD_LIBRARY_PATH=/usr/local/lib
echo "export LD_LIBRARY_PATH=/usr/local/bin/openssl" >> ~/.bashrc

1 2	export LD_LIBRARY_PATH=/usr/local/lib echo "export LD_LIBRARY_PATH=/usr/local/bin/openssl" >> ~/.bashrc

Open SSL now reports it’s version.

openssl version
OpenSSL 1.1.1-pre3 (beta) 20 Mar 2018

1 2	openssl version OpenSSL 1.1.1-pre3 (beta) 20 Mar 2018

What version NGINX do you have (1.13 supports TLS 1.3) read here

# nginx -v
nginx version: nginx/1.13.9

1 2	# nginx -v nginx version: nginx/1.13.9

Backup your NGINX

Do backup your server files and take a snapshot if need be. I am not responsible;e for a broken server,

sudo cp -R /etc/nginx/ /nginx-backup-26thMar-2018

1	sudo cp -R /etc/nginx/ /nginx-backup-26thMar-2018

Edit NGINX Configuration

Update NGINX configuration: /etc/nginx/sites-available/default

ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ecdh_curve secp384r1;

ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256;

ssl_prefer_server_ciphers on;

ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;

ssl_ecdh_curve secp384r1;

tip: Review other NGINX hardening settings here. Also remove TLSv1.0

I tested my NGINX config loaded them and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restrat

nginx -t

nginx -s reload

/etc/init.d/nginx restrat

Check the status of NGINX

# /etc/init.d/nginx status

[ ok ] Restarting nginx (via systemctl): nginx.service.
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) 
     Docs: man:nginx(8)
  Process: 15154 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
  Process: 15162 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
  Process: 15159 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
 Main PID: 15166 (nginx)
    Tasks: 4
   Memory: 2.3M
      CPU: 27ms
   CGroup: /system.slice/nginx.service
           ├─15166 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─15170 nginx: worker process
           ├─15171 nginx: cache manager process
           └─15172 nginx: cache loader process

# /etc/init.d/nginx status

[ ok ] Restarting nginx (via systemctl): nginx.service.

● nginx.service - A high performance web server and a reverse proxy server

Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)

Active: active (running)

Docs: man:nginx(8)

Process: 15154 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)

Process: 15162 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)

Process: 15159 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)

Main PID: 15166 (nginx)

Tasks: 4

Memory: 2.3M

CPU: 27ms

CGroup: /system.slice/nginx.service

├─15166 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;

├─15170 nginx: worker process

├─15171 nginx: cache manager process

└─15172 nginx: cache loader process

If you have configured Cloudflare then log in and enable TLS support.

Enable TLS 1.3 in Chrome by visiting chrome://flags/#tls13-variant This should be automatic in later versions of Chrome and other browsers.

Verify TLS

I used the developer tools in Chrome to confirm the page was verified in TLS 1.3.

Updated to 1.1.1-pre6-dev

mkdir /temp
cd /temp
sudo git clone https://github.com/openssl/openssl.git
cd openssl/
./config --prefix=/usr/local --openssldir=/usr/local -Wl,-rpath,/usr/local
make
sudo make install
openssl
OpenSSL> version
OpenSSL 1.1.1-pre6-dev  xx XXX xxxx
OpenSSL> exit

mkdir /temp

cd /temp

sudo git clone https://github.com/openssl/openssl.git

cd openssl/

./config --prefix=/usr/local --openssldir=/usr/local -Wl,-rpath,/usr/local

make

sudo make install

openssl

OpenSSL> version

OpenSSL 1.1.1-pre6-dev xx XXX xxxx

OpenSSL> exit

Don’t forget to test your SSL strength with https://dev.ssllabs.com/ssltest/

I hope this guide helps someone.

Ask a question or recommend an article

Revision History

v1.3 added bad ssl cert.

v1.2 ssl test v1.1 updated to 1.1.1-pre6-dev

v1.0 Initial post

Backing up OSX or an Ubuntu server with Backblaze B2 Cloud Storage from the Command Line

March 14, 2018 by Simon

This computer will show you can back up computer or server with Backblaze B2 Cloud Storage from the Command Line n OSX and Ubuntu.

This post is still being written. I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Also, I have blogged about how you can add block storage to a Vultr server, backup and restore snapshots , syncing files with rsync along with using GitHub and Bitbucket but what do you do if you need to backup large amounts of data?

Backblaze has a Cloud storage solution that costs as low as $0.005c a GB (a month), The first 10G is free. Backblaze say “From bytes to petabytes Backblaze B2 is the lowest cost high-performance cloud storage in the world. ”

Back Blaze have open sourced internal drive enclosure designs and drive failure stats and it’s time I gave them a try.

Goto https://www.backblaze.com

Create or sign in.

After you login got the dashboard.

Click Backblaze B2 Cloud Storage

Create a Bucket

Name the bucket (long names with a GUID are good).

You can rename the bucket here and change public/private and or upload/download files manually.

The first thing I did was limit the versions of files under the lifecycle settings for the bucket.

Now I created a series of subfolders to store files from different servers (I could have used many buckets but one bucket will do).

I can upload files via the Backblaze bucket GUI if I needed to.

Back Blaze has a command line tool for uploading: https://www.backblaze.com/b2/docs/quick_command_line.html

Install Steps

Backblaze state “The B2 command-line tool is available from the Python Package Index (PyPI) using the standard pip installation tool. Your first step is to make sure that you have either Python 2 (2.6 or later) or Python 3 (3.2 or later) installed.”

I have Python 2.7 installed

python --version
Python 2.7.10

1 2	python --version Python 2.7.10

Install PIP

sudo easy_install pip
Password:
Searching for pip
Best match: pip 1.5.6
Processing pip-1.5.6-py2.7.egg
pip 1.5.6 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Using /Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

sudo easy_install pip

Password:

Searching for pip

Best match: pip 1.5.6

Processing pip-1.5.6-py2.7.egg

pip 1.5.6 is already the active version in easy-install.pth

Installing pip script to /usr/local/bin

Installing pip2.7 script to /usr/local/bin

Installing pip2 script to /usr/local/bin

Using /Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg

Processing dependencies for pip

Finished processing dependencies for pip

I ran into issues updating b2 CLI

sudo pip install --upgrade b2
Requirement already up-to-date: b2 in /Library/Python/2.7/site-packages
Requirement already up-to-date: arrow>=0.8.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: logfury>=0.1.2 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: requests>=2.9.1 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: six>=1.10 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: tqdm>=4.5.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: futures>=3.0.5 in /Library/Python/2.7/site-packages (from b2)
Downloading/unpacking python-dateutil from https://pypi.python.org/packages/bc/c5/3449988d33baca4e9619f49a14e28026399b0a8c32817e28b503923a04ab/python_dateutil-2.7.0-py2.py3-none-any.whl#md5=5a86a548fe776cc079bf4a835473e3f8 (from arrow>=0.8.0->b2)
  Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207kB): 207kB downloaded
Installing collected packages: python-dateutil
  Found existing installation: python-dateutil 1.5
    Uninstalling python-dateutil:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

sudo pip install --upgrade b2

Requirement already up-to-date: b2 in /Library/Python/2.7/site-packages

Requirement already up-to-date: arrow>=0.8.0 in /Library/Python/2.7/site-packages (from b2)

Requirement already up-to-date: logfury>=0.1.2 in /Library/Python/2.7/site-packages (from b2)

Requirement already up-to-date: requests>=2.9.1 in /Library/Python/2.7/site-packages (from b2)

Requirement already up-to-date: six>=1.10 in /Library/Python/2.7/site-packages (from b2)

Requirement already up-to-date: tqdm>=4.5.0 in /Library/Python/2.7/site-packages (from b2)

Requirement already up-to-date: futures>=3.0.5 in /Library/Python/2.7/site-packages (from b2)

Downloading/unpacking python-dateutil from https://pypi.python.org/packages/bc/c5/3449988d33baca4e9619f49a14e28026399b0a8c32817e28b503923a04ab/python_dateutil-2.7.0-py2.py3-none-any.whl#md5=5a86a548fe776cc079bf4a835473e3f8 (from arrow>=0.8.0->b2)

Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207kB): 207kB downloaded

Installing collected packages: python-dateutil

Found existing installation: python-dateutil 1.5

Uninstalling python-dateutil:

Cleaning up...

Exception:

Traceback (most recent call last):

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main

status = self.run(options, args)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run

requirement_set.install(install_options, global_options, root=options.root_path)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install

requirement.uninstall(auto_confirm=True)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall

paths_to_remove.remove(auto_confirm)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove

renames(path, new_path)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames

shutil.move(old, new)

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move

copytree(src, real_dst, symlinks=True)

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree

raise Error, errors

Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

I tried installing via the alternative method (with no luck)

git clone https://github.com/Backblaze/B2_Command_Line_Tool.git
Cloning into 'B2_Command_Line_Tool'...
remote: Counting objects: 5084, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 5084 (delta 1), reused 1 (delta 0), pack-reused 5076
Receiving objects: 100% (5084/5084), 1.25 MiB | 689.00 KiB/s, done.
Resolving deltas: 100% (3622/3622), done.

git clone https://github.com/Backblaze/B2_Command_Line_Tool.git

Cloning into 'B2_Command_Line_Tool'...

remote: Counting objects: 5084, done.

remote: Compressing objects: 100% (8/8), done.

remote: Total 5084 (delta 1), reused 1 (delta 0), pack-reused 5076

Receiving objects: 100% (5084/5084), 1.25 MiB | 689.00 KiB/s, done.

Resolving deltas: 100% (3622/3622), done.

cd B2_Command_Line_Tool/

1	cd B2_Command_Line_Tool/

I tried running the setup script (with no luck)

sudo python setup.py install
setuptools 20.2 or later is required. To fix, try running: pip install "setuptools>=20.2"

1 2	sudo python setup.py install setuptools 20.2 or later is required. To fix, try running: pip install "setuptools>=20.2"

Upgrading setup tools also failed

sudo pip install "setuptools>=20.2"
Downloading/unpacking setuptools>=20.2
  Downloading setuptools-38.5.2-py2.py3-none-any.whl (490kB): 490kB downloaded
Installing collected packages: setuptools
  Found existing installation: setuptools 18.5
    Uninstalling setuptools:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

sudo pip install "setuptools>=20.2"

Downloading/unpacking setuptools>=20.2

Downloading setuptools-38.5.2-py2.py3-none-any.whl (490kB): 490kB downloaded

Installing collected packages: setuptools

Found existing installation: setuptools 18.5

Uninstalling setuptools:

Cleaning up...

Exception:

Traceback (most recent call last):

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main

status = self.run(options, args)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run

requirement_set.install(install_options, global_options, root=options.root_path)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install

requirement.uninstall(auto_confirm=True)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall

paths_to_remove.remove(auto_confirm)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove

renames(path, new_path)

File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames

shutil.move(old, new)

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move

copytree(src, real_dst, symlinks=True)

File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree

raise Error, errors

Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

Backing up a Mac via command line with B2

More to come when I can get B2 CLI Installed.

Backing up a an Ubuntu machine via command line with B2

More to come when I can get B2 CLI Installed.

Update

My ticket with Backblaze was automatically closed with this note “If the issue is persisting, it may be easiest to map the installation to the user folder, rather than the system level.”

No ideas how but something to research.

Ask a question or the recommend an article

Revision History

V1.1 ticket closed

v1.0 Initial post

How to start-up (flesh out) an app idea

February 3, 2018 by Simon

How do you start up a project idea? How much time and effort do you put into product planning, prototyping, product development and product support?

Below is a rough guide to how you should start up a project (exec summary: start with defining the problem(s) and solution(s)).

I have blogged about app workflow here (mostly for public apps).

Choose a Methodology

Early on you should choose a project management methodology, Agile is a popular (no lock-in) methodology compared to the older/less-flexible Waterfall or PRINCE2 methodologies.

I am a fan of the Agile methodology as it allows for inevitable frequent and required pivots over the more rigid locked down methodologies of the past.

Agile Software Development Manifesto

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

Agile Methodology and Practices (Stages)

1. Inception Phase

Form Initial Team (Stakeholders)
Develop Common Project Vision (What is the problem)
Align with Enterprise Direction (Integrate where possible)
Explore Initial Scope (MoSCoW Method)
Identify Initial Technical Strategy (testing, draft, final, support)
Develop Initial Release Plan (Goals)
Form Work Environment (remote, onsite etc)
Secure Funding (after whiteboarding)
Identify Risks (what if…)

2. Construction Phase

Produce Potentially Consumable Solution (Minimum Viable Product/Minimum Value)
Address Changing Stakeholder Needs (Pivot)
Move Closer to Deployable Release (Iterate)
Improve Quality (Measure)
Prove Architecture Early (Test)

3. Transition Phase

Ensure Solution Is Consumable (Signoff)
Deploy Solution (Test and Deploy)

4. Ongoing Goals

Fulfill Project Mission (Complete Must-have backlog)
Grow Team Members
Address Risk
Improve Team Process and Environment
Leverage/Enhance Existing Infrastructure

First, you will need to define the problem(s)

First, decide on what the problem(s) are and are aiming to solve? After the problem(s) are known you can discuss and validate possible solutions before any coding is done (billed). The business requirement will drive the technical requirements/directions,.

Letting technology goals drive the project is a bad idea. Why do you need XYZ? Is an app required after all? Does an “off the shelf solution” exist already? Can existing processes or software be tweaked?

You will need stakeholders (or stakeholder proxies)

Choose customer stakeholders (or customer proxies) that are engaged and positive, build a team of about 5 people who can drive the decisions and who understand the problems. Involve key stakeholders early and capture problems in a formal setting

Identify the stakeholders and establish a team
Document the problems (as it is and not how it should be)
Identify and specify improvement points
Plan solutions and Iterate

Improving a process or product should be measurable and agreed. The main aim of a customer/stakeholder is to create validated user stories that define the problem(s) and solution(s) from the end users perspective.

Example user story formats (snip)

As a <role>, I want <capability> so that <receive benefit>
In order to <receive benefit> as a <role>, I want <goal/desire>
As a <role>, I want <goal/desire>
As <persona>, I want <what?> so that <why?>
etc

Whiteboarding

Whiteboarding is a great way to get everyone on board (and on the same page). One person may assume that something is a certain way but another person from the trenches may say it is not the case at all. It is key that problems reported lower down the managerial structure are listen to and be valued. Accept any problems and add them to the backlog (prioritizing comes later).

Whiteboarding should build a backlog of ideas and common understanding of what the landscape of issues are. I use a tool called Mind Node from https://mindnode.com/ to map things electronically. Within minutes you can have a map of connected or not connected areas of interest.

Backlog

A backlog is a place where you can document issues and break issues down into tasks. I have blogged before about Developing software and staying on track. You can use software like Trello, JIRA or Excel or notepad.

Backlogs should have crazy ideas and things that will never be developed. Agile development iterations should only pick the best items to work on.

Trello offers basic shared lists management

Jira offers a more complex Agile backlog management suite

Mockup Prototypes

It is important that you develop validate ideas with on device/screen with mockups/prototypes before coding something.

I use Platforma web and Platforma mobile prototyping tools with Adobe XD to prototype before development.

Agile Terminology in Backlogs

Whiteboarding and Backlogs can allow you to capture Epics, Stories and Tasks. Once tasks are prioritized they can be picked up and complete in Agile work sprints.

Epics are the bigger user deliverable objectives like a “usable web-based solution”, “usable iOS mobile app”, “user Android Mobile app” or other long-term objectives.
Stories are smaller things like “deliver back-end server infrastructure”, “Deploy a working prototype of feature X”, “Build an API (Application Programmer Interface)”. A user story can usually be completed in 2 weeks.
Tasks are often subtasks to stories and can be done quite quickly. A task could be “Change the colour scheme”, “Set up a testing environment” or “Deploy the alpha release”.

Prioritize the Backlog Often

Prioritize items in the backlog often using the MoSCoW Method.

Must-Have
Should Have
Could Have
Won’t Have

Do not work on won’t have and focus on Must-Haves first.

Define the Solution and Minimum Success Criteria

The following graphic is well known within the Project Management community.

How would you feel as the customer knowing that you have not been understood and that you may not get what you need?

How would you feel as the developer not knowing fully what you need to deliver?

No one likes to double back or waste time and resources unnecessarily. Define and plan early. “Failing to plan is planning to fail.” Benjamin Franklin

Technology Requirements

The backlog will drive the requirements; e.g:

mobile app or web app
type of technology needed
the location and number of servers
number of prototypes
scalability and redundancy

Knowing the requirements drives the technology. Technology drives the work and the budget.

TIP: Be wary of contractors that quote before listening to your full requirements. I was once told by a contractor that their main job was to “Con” “and Insult” you. If you accept a quote before saying what you need is a guarantee that you are paying a load of profit.

Now What

If you have defined the problem well, have good user stories and validated with stakeholders you are on the right path.

Be prepared to pivot and set frequent milestones to launch value (RERO).

Obviously centralised web-based technology is easily updatable compared to compiled and distributed mobile apps so choosing the right technology early is key to success.

Security and Quality

This is the project triangle where you can choose two sides (not three)

Quality is a given so you must choose that, the other given is secure in my opinion. I have blogged before about securing Ubuntu in the cloud (checklist), running security audits, installing WordPress WordFence Firewalls and WordPress Anti Malware plugins along with updating OpenSSL and applying kernel patches to protect against Spectre and Meltdown securing issues. Supporting an app is serious stuff and often deployment is the start of the real work.

Maintenance

Will you need to recompile apps to meet iOS or Android requirements?

How often will you release updates? What percentage of devices do you want to reach?

What “service level” will you set for concurrent users, uptime and failover? What priority will you give to backups and snapshots of the environment in case you need to restore to a previous state?

Past Lesson learned

Obviously, the larger the project and higher levels of storage/reliability/security the more checks and balances need to happen.

The Australian Bureau of Statistics contracted IBM to handle the Australian online census in 2016 and the major failure and report into what the identified problems were was an interesting read.

The ABS website was pulled down on the night of the study, with it then being unavailable for about 40 hours.
The ABS is judged to have failed to communicate with the public properly about major
IBM is deemed to have failed to properly test its disaster recovery processes
IBM had failed to properly plan for how to bring systems back up again.
IBM’s failure to have tested a router restart, or have a backup synchronised and in place, appears to have been significant contributing factors to the failure of the eCensus
In his report, meanwhile, Mr MacGibbon also found that the long-term, almost exclusive, relationship between the ABS and IBM had contributed to the problems by meaning any external questioning or oversight was absent.
Read more about the report.

As a minimum, you should plan and discuss early on about the performance, reliability and uptime targets for any app. Personally I have moved between multiple cloud providers (CloudAnt, Digital Ocean, AWS and Vultr) to comfortably meet cost v performance (each app WILL have different requirements)

Type of app

The type of app (Web, Desktop, Mobile) you will need will come after discussing the problem, focus on the problem(s) and not the solution(s).

Conclusion

Do’s

Keep Source ownership and IP (never let contractors own the code or processes)
Insist on frequent handover often
Do plan for updated version with major OS releases
Do secure the app
Do plan for maintenance of the app.

Don’t

Lock into expensive maintenance contracts.
Ensure the vendor is passionate about solving your problem and not the invoice they will submit.
etc

I hope this guide helps someone.

More Reading

Making your first Android app

Making your first cross-platform mobile app

Making your first cross-platform app with Electron

Ask a question or recommend an article

Revision History

v1.1 added More Reading

v1.0 Initial post

Add email alias to an existing GSuite email account

January 14, 2018 by Simon

This short post will show you how (and why) you should add email aliases to email accounts for previously added domains.

Here are my two posts related to setting up a new suite account and adding a second domain to an existing GSuite account. I used to have my own C Panel domain but moved all servers to self-managed servers first on Digital Ocean then AWS then Vultr. Vultr is now my preferred hosts as they are affordable and have server locations in Australia.

Click Users

I tried to add an email alias to existing 2x domains in Google Suite (the help did not align with the screens I was seeing in my admin consoles), there was no email alias section)??

Official Google Help: https://support.google.com/domains/answer/6304345?hl=en-AU

Snip:

Every user in a G Suite account has a <em>primary address</em> for signing in to their account and receiving mail. If a user wants another email address, you can create an <strong>email alias</strong> for them.

<strong>Why use an alias? </strong> If ann@solarmora.com wants a sales address to post to the company website, you can create the alias <strong>sales@solarmora.com</strong>. Mail sent to either address then appears in Ann's Gmail inbox.

You can add up to 30 email aliases for each user. Users continue to sign in to their G Suite account with their primary address, not an email alias.

Every user in a G Suite account has a <em>primary address</em> for signing in to their account and receiving mail. If a user wants another email address, you can create an <strong>email alias</strong> for them.

<strong>Why use an alias? </strong> If ann@solarmora.com wants a sales address to post to the company website, you can create the alias <strong>sales@solarmora.com</strong>. Mail sent to either address then appears in Ann's Gmail inbox.

You can add up to 30 email aliases for each user. Users continue to sign in to their G Suite account with their primary address, not an email alias.

I fired up the G Suite support

I wanted to add the following email aliases of “security@..”, “admin@..” and “root@..” to an existing email on 2 domains to support security disclosures (post soon (nope Troy Hunt beat me, read his post on security disclosure here)).

While I was waiting for the support chat I figured out the email alias thing, it turned out I needed to click the “Account” header on the user screen (I did not realise “account word was clickable).

Solution: How to add Aliases

You need to click the account heading (It’s a link apparently).

When the Ajax content loads you can scroll down to alias section.

Tip: There is no “Apply” or “OK” button when adding aliases so just press enter.

Now if you want to be able to send as the new alias you need to add the aliases in Gmail.

Now you can send via this email alias in GMail or via CLI via the primary account 🙂

sendemail -f root@yourdomain.com -t to@email.com -u "Test email via alias" -s smtp.gmail.com:587 -o tls=yes -xu simon@yourdomain.com -xp YourPa$$w0rdH3r3 -m "Test email via alias"

1	sendemail -f root@yourdomain.com -t to@email.com -u "Test email via alias" -s smtp.gmail.com:587 -o tls=yes -xu simon@yourdomain.com -xp YourPa$$w0rdH3r3 -m "Test email via alias"

Read my blog post on sending email from the command line via G Suite.

Donate and make this blog better

Ask a question or recommend an article

Revision History
v1.1 Added Troy Hunt “Data Breach Disclosures” link

v1.0 Working Copy.

Hope this helps someone.

Adding a second domain to an existing GSuite account and creating an email address

December 14, 2017 by Simon

Below is my post (personal view, not paid view) on how to add a second domain to an existing gsuite account and creating an email address and sending emails from a client tor CLI.

Jan 2018 Update

Post on adding email aliases to G Suite

Main

Read my post here if you have never created or managed a GSuite account to take control of mail for a domain. This guide will build on this previous guide and show you how to add a second domain to an existing GSuite account. Read this guide if you need a Namecheap domain and or an Ubuntu VM to host a website. I really like deploying WordPress via Command Line (CLI), read my guide <

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

"If you're not still learning, you're already dying."
- Ryan Holiday - Ego is the Enemy

Follow me on Twitter: @FearbySoftware

View All Posts. Become a Patron

an

Adding two sub domains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

How to install Windows 10 Pro alongside an OSX partition with Apple Boot Camp

Moving an Ubuntu 16.04 VM on Vultr from one data centre to another via snapshots

Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare

How to start-up (flesh out) an app idea

Add email alias to an existing GSuite email account

Adding a second domain to an existing GSuite account and creating an email address