• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

website

Recovering a Dead Nginx, Mysql, PHP WordPress website

July 10, 2021 by Simon

(laptrinhx.com – do not steal this post)

In early 2021 www.fearby.com died and it was all my fault. Here is my breakdown of the events.

On the 4th of January 2021, I woke to see my website not loading.

Cloud flare reporting my websitre was unavailable.

https://www.fearby.com had 2 servers.

  • Web server (www.fearby.com)
  • Database server (db.fearby.com)

Upon investigating why my website was down I found out that WordPress could not talk to the database server. I tried to log into the database server via SSH failed (no response). I tried logging into my db.fearby.com server via the root console and it too did not work.

I was locked out of my own server and memory told me it was caused by my be playing with fail2ban and other system auditing tools a few months earlier.

I tried restoring the db.fearby.com serer from backups (one at a time). I had the last 7 days as individual backups.  I had no luck, all of my backups were no good (I sat on the problem too long).

View of the last 7 days of backups.

In mid-2020 I locked myself out of db.fearby.com (SSH and root console) because I setup an aggressive fail2ban, AIDE intrusion detection system(s) and firewall rules. I could no longer access my db.fearby.com server via SSH or the root console.  The database server was still operational and I foolishly left it running (with no access).

I did not know how to (or had enough time) reset the root password on the Debian server. I was unable to think of a fix to restore my website. I should have reset the root password, it is easy to do thanks to a post from Janne Roustemaa – How to reset root password on cloud server.

A few months ago I finally found out how to reset the root password of a Debian server.

How to Reset the Root Password on a Debian server on UpCloud

I followed Janne Roustemaa’s guide here: How to reset root password on cloud server.

I logged into the Up Cloud Hub.

UpCloud Hub (login page)

I shut down db.fearby.com from the UpCloud Dashboard.

I created a backup of db.fearby.com (just in case). I upgraded my backup plan from 1 backup every 7 days to daily backups for 7 days and weekly backups for 1 month.

Database Backup plan selection

Deploy a temporary server to reset the root password

I deployed a new temporary (cheap) server alongside the dead server in Chicago.

Get $25 free credit on UpCloud and deploy your own server: Use this link to get $25 credit (new UpCloud users only).

Deploy $5/m server in Chicago.

I called the server “recovery.fearby.com” and set Debian 9 and the Operation System (same as the dead server).

Name: Recovery.fearby.com, Debian 9

I added the command “shutdown -h 1” to the Initialization script to ensure the server shuts down after it was deployed. 

I can only add the disk to the new server if the old db.fearby.com server is shut down.

Deploy server

I shut down db.fearby.com and recovery.fearby.com servers.

Shutting down servers GUI

Both servers have shut down

Shutdown

Detach the disk from db.fearby.com

I detach the disk from db.fearby.com server.

In the UpCloud Dashboard, I opened the db.fearby.com and clicked the resize tab

Resize Disk

I clicked the Detach button.

Detach Disk

I clicked Continue

Continue

Attach db.fearby.com disk to recovery.fearby.com

Now I attached this disk as a secondary disk onto the recovery.fearby.com server.

In the UpCloud hub I clicked Servers then selected the recovery.fearby.com server, then clicked the Resize Tab

Resize recovery.fearby.com

I scrolled down and clicked Attach existing storage

Attach Disk

Attach existing storage dialogue

Attach device Dialog

I selected the system disk from the db.fearby.com (that I detached earlier)

Attach Disk

I clicked Add a storage device button

Attach Disk dialog

Now I have attached the storage from db.fearby.com and attached it to recovery.fearby.com as a secondary disk.

2 Disks attached

Starting the recovery.fearby.com server

I started the recovery.fearby.com server by clicking Start

Start

The server is starting

Starting

When the server started, I obtained its IP and connect to it with MobaXTerm.

MobaXTerm SSH Client.

Now I can access the db.fearby.com disk.

I do not want to reset the root password until I undelete the files I need.

Viewing Disks

I ran this command to verify the attached disks.

lsblk

Two disks were visible

2 disk were visible.

Alternatively, I can view partitions with the following command

cat /proc/partitions
major minor  #blocks  name
 254        0   26214400 vda
 254        1   26213376 vda1
 254       16   52428800 vdb
 254       17   52427776 vdb1

I can see partition data with these commands

recovery.fearby.com disk: /dev/vda1

fdisk -l /dev/vda1
Disk /dev/vda1: 25 GiB, 26842497024 bytes, 52426752 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

db.fearby.com disk: /dev/vdb1

fdisk -l /dev/vdb1
Disk /dev/vdb1: 50 GiB, 53686042624 bytes, 104855552 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

I ran this command to mount the second disk

mount /dev/vdb1 /mnt

I looked in the “/mnt/Backup” folder  as this was where I had daily MySQL dumps saving too on the old system.

The folder was empty. It looks like the file system was corrupted.

Finding Deleted Files

I assumed the file system was corrupt, I installed Testdisk as I wanted to recover deleted SQL dump backups in the backup folder. 

sudo apt-get update
sudo apt-get install testdisk

I confirmed testdisk was installed

photorec --version

I ran testdisk and passed in the disk as a parameter

sudo photorec /dev/vdb1

I selected the Disk /dev/vdb1 – 53GB and pressed enter

I selected ext4 partition and pressed enter (not whole disk)

I selected ext2/ext3/ext4 filesystem and pressed enter

I selected Free to only scan in unallocated space and pressed enter

When asked to choose the recovery location to restore files to I selected /recovery (on the recovery.fearby.com disk , not the db.fearby.com disk.

I just realized that the recovery.fearby.com disk is 25Gb and the db.fearby.com disk is 50GB, lets hope I do not have more than 25GB of deleted files (or I will have to delete recovery.fearby.com and deploy a 100GB system) and run undelete again.

Recovered Files

After 20 minutes 32,809 files were recovered to /recovery/recup_dir

I pressed CTRL+C to exit photorec

I changed directory to /recovery

cd /recovery

I counted the recovered files

find . -type f | wc -l
>32810

Recovered Files were placed in sub folders.

drwxr-xr-x 68 root root  4096 Jan 19 13:28 .
drwxr-xr-x 23 root root  4096 Jan 19 13:28 ..
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.1
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.10
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.11
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.12
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.13
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.14
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.15
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.16
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.17
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.18
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.19
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.2
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.20
drwxr-xr-x  2 root root 24576 Jan 19 13:22 recup_dir.21
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.22
drwxr-xr-x  2 root root 24576 Jan 19 13:22 recup_dir.23
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.24
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.25
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.26
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.27
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.28
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.29
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.3
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.30
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.31
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.32
drwxr-xr-x  2 root root 20480 Jan 19 13:22 recup_dir.33
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.34
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.35
drwxr-xr-x  2 root root 36864 Jan 19 13:22 recup_dir.36
drwxr-xr-x  2 root root 32768 Jan 19 13:22 recup_dir.37
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.38
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.39
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.4
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.40
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.41
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.42
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.43
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.44
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.45
drwxr-xr-x  2 root root 20480 Jan 19 13:23 recup_dir.46
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.47
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.48
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.49
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.5
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.50
drwxr-xr-x  2 root root 20480 Jan 19 13:24 recup_dir.51
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.52
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.53
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.54
drwxr-xr-x  2 root root 20480 Jan 19 13:25 recup_dir.55
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.56
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.57
drwxr-xr-x  2 root root 24576 Jan 19 13:26 recup_dir.58
drwxr-xr-x  2 root root 36864 Jan 19 13:26 recup_dir.59
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.6
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.60
drwxr-xr-x  2 root root 20480 Jan 19 13:26 recup_dir.61
drwxr-xr-x  2 root root 20480 Jan 19 13:27 recup_dir.62
drwxr-xr-x  2 root root 20480 Jan 19 13:27 recup_dir.63
drwxr-xr-x  2 root root 20480 Jan 19 13:27 recup_dir.64
drwxr-xr-x  2 root root 20480 Jan 19 13:28 recup_dir.65
drwxr-xr-x  2 root root 12288 Jan 19 13:28 recup_dir.66
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.7
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.8
drwxr-xr-x  2 root root 20480 Jan 19 13:21 recup_dir.9
66 folders were recovered

I calculated the folder size

sudo  du -sh /recovery
13G     /recovery

I immediately started downloading ALL recovered files (13GB) to my local PC with MobaXTerm

On recovery.fearby.com I searched for any trace of recovered *.sql files that I was dumping daily via cron jobs

find /recovery -name "*.sql"

Many pages of recovered sql files were listed

/recovery/recup_dir.49/f36487168.sql
/recovery/recup_dir.49/f35110912.sql
/recovery/recup_dir.49/f36667392.sql
/recovery/recup_dir.49/f36995072.sql
/recovery/recup_dir.49/f35667968.sql
/recovery/recup_dir.49/f35078144.sql
/recovery/recup_dir.49/f37535744.sql
/recovery/recup_dir.49/f36913152.sql
/recovery/recup_dir.49/f33996800.sql
/recovery/recup_dir.49/f36421632.sql
/recovery/recup_dir.49/f35061760.sql
/recovery/recup_dir.49/f36143104.sql
/recovery/recup_dir.49/f36618240.sql
/recovery/recup_dir.49/f34979840.sql
/recovery/recup_dir.49/f37273600.sql
/recovery/recup_dir.49/f35995648.sql
/recovery/recup_dir.49/f36241408.sql
/recovery/recup_dir.49/f37732360.sql
/recovery/recup_dir.49/f34603008.sql
/recovery/recup_dir.49/f33980416.sql
/recovery/recup_dir.1/f0452896.sql
/recovery/recup_dir.1/f0437184.sql
/recovery/recup_dir.1/f0211232.sql
/recovery/recup_dir.17/f16547840.sql
/recovery/recup_dir.17/f16252928.sql
/recovery/recup_dir.17/f15122432.sql
/recovery/recup_dir.17/f17159720.sql
/recovery/recup_dir.17/f15089664.sql
/recovery/recup_dir.17/f15958016.sql
/recovery/recup_dir.17/f15761408.sql
/recovery/recup_dir.57/f69582848.sql
/recovery/recup_dir.57/f69533696.sql
/recovery/recup_dir.57/f69173248.sql
/recovery/recup_dir.57/f68321280.sql
/recovery/recup_dir.57/f70483968.sql
/recovery/recup_dir.57/f70746112.sql
/recovery/recup_dir.57/f68730880.sql
/recovery/recup_dir.57/f67862528.sql
/recovery/recup_dir.57/f70123520.sql
/recovery/recup_dir.57/f68337664.sql
/recovery/recup_dir.57/f70172672.sql
/recovery/recup_dir.57/f71057408.sql
/recovery/recup_dir.57/f68796416.sql
/recovery/recup_dir.57/f70533120.sql
/recovery/recup_dir.57/f69419008.sql
/recovery/recup_dir.57/f68239360.sql
/recovery/recup_dir.57/f69779456.sql
/recovery/recup_dir.57/f68255744.sql
/recovery/recup_dir.57/f67764224.sql
/recovery/recup_dir.57/f71204864.sql
/recovery/recup_dir.57/f70336512.sql
/recovery/recup_dir.57/f68501504.sql
/recovery/recup_dir.57/f67944448.sql
/recovery/recup_dir.50/f39059456.sql
/recovery/recup_dir.50/f38518784.sql
/recovery/recup_dir.50/f40206336.sql
/recovery/recup_dir.50/f40927232.sql
/recovery/recup_dir.50/f39485440.sql
/recovery/recup_dir.50/f39092224.sql
/recovery/recup_dir.50/f40861696.sql
/recovery/recup_dir.50/f39731200.sql
/recovery/recup_dir.50/f40337408.sql
/recovery/recup_dir.50/f38862848.sql
/recovery/recup_dir.50/f41664512.sql
/recovery/recup_dir.50/f41074688.sql
/recovery/recup_dir.50/f40828928.sql
/recovery/recup_dir.50/f41713664.sql
/recovery/recup_dir.50/f38092800.sql
/recovery/recup_dir.50/f39878656.sql
/recovery/recup_dir.50/f38305792.sql
/recovery/recup_dir.50/f38830080.sql
/recovery/recup_dir.50/f39534592.sql
/recovery/recup_dir.50/f39813120.sql
/recovery/recup_dir.50/f40435712.sql
/recovery/recup_dir.50/f41467904.sql
/recovery/recup_dir.50/f37901728.sql
/recovery/recup_dir.50/f38682624.sql
/recovery/recup_dir.50/f38191104.sql
/recovery/recup_dir.50/f38174720.sql
/recovery/recup_dir.50/f40878080.sql
//and many more

It would take a few hours to download 32,000 files from the other side of the world. Next time I deploy fearby.com, I will deploy it to Sydney Australia.

It looks like MobaXTerm does not copy to the selected destination that I selected when dragging and dropping files, Being impatient I scanned my system for a file that MobaXTerm had copied. MobaXTerm saves download to “%Documents%\MobaXterm\splash\tmp\dragdrop\”.

Sql file contents

I opened some of the recovered SQL files and it looks like all files were partial and were not the whole database backup. A compete mysql dump should be over 200 lines long. Dang.

I downloaded all files I could from the these folders

  • /mnt/etc/nginx
  • /mnt/var/lib/mysql
  • /mnt/Scripts

Disappointed, I sat on things for a few weeks, I thought I had lost my website.

Try 2 – Reinstall a fresh db.fearby.com

After I copied files from the old db.fearby.com serve to recovery.fearby.com, I reattached the storage to the old db.fearby.com system.

In vein, I tried mending the broken MySQL service on db.fearby.com. I tried uninstalling and reinstalling MySql on db.fearby.com (each time no luck). I was having trouble with MySQL not starting.

I had too many rabbit holes (mysql errors) to list. I thought my database was corrupt.

I was kicking myself for letting my access to db.fearby.com lapse, I was kicking myself for not having more backups.

Try 3 – Reinstall MySQL

I tried deploying a new server and setting it up, maybe from the frustration I did not do it correctly. I had HTTPS issues with CloudFlare. I gave up for a few months

Try 4 – Check for Database Corruption

I downloaded DiskInternals – MySQL Recovery and scanned my database. To my amazement, it reported no database corruption.

All tables loaded

Maybe I can recover my website?

Try 5 – Using RunCloud.io to deploy a website

Having failed with setting up a server from scratch, a friend (Hi Zach) said I should try RunCloud to deploy a server.

I tried to document everything from attaching RunCloud to UpCloud and Cloudflare’s API to deploying a server.

Long story short, RunCloud is not for me.  I had too many issues with RunCloud and IPV6, CloudFlare API Integration, No SSH access to my server, no Nginx editing capabilities with RunCloud.

RunCloud errors

I deleted the RunCloud deployed server.

Try 6 – 10 minutes deploying a serer by hand

I ended up deploying a server in 10 minutes manually without taking notes.

Summary

I deployed a server (this time to Sydney).

I Installed the UFW firewall (configured and started)

sudo apt-get install ntp

Set the date and time

sudo timedatectl set-timezone Australia/Sydney

Installed ntp time server

sudo apt-get install ntp

I Installed PHP 7.4 and PHP 7.4 FPM (I cheated and googled: https://www.cloudbooklet.com/install-php-7-4-on-debian-10/ )

I edited PHP Config

sudo nano /etc/php/7.4/fpm/php.ini

Installed NGINX webserver 

sudo apt-get install nginx

Configured NGINX (I got an CLoudflare to NGINX SSL Certificate), I also sighned up for a Cloud flare SSL certificate

sudo nano /etc/nginx/nginx.conf

Contents

worker_cpu_affinity auto;
worker_rlimit_nofile 100000;
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
        worker_connections 768;
         multi_accept on;
}

http {
        root /www-root;
        client_max_body_size 10M;

        proxy_connect_timeout 1200s;
        proxy_send_timeout 1200s;
        proxy_read_timeout 1200s;
        fastcgi_send_timeout 1200s;
        fastcgi_read_timeout 1200s;

        ##
        # Basic Settings
        ##
        sendfile on;
        tcp_nopush on;
        tcp_nodelay on;
        keepalive_timeout 65;
        types_hash_max_size 2048;

        # server_tokens off;
        # server_names_hash_bucket_size 64;
        # server_name_in_redirect off;

        include /etc/nginx/mime.types;
        default_type application/octet-stream;

        ##
        # SSL Settings
        ##

        ssl_protocols TLSv1.2 TLSv1.3;
        ssl_prefer_server_ciphers on;

        ##
        # Logging Settings
        ##

        access_log /var/log/nginx/access.log;
        error_log /var/log/nginx/error.log;

        ##
        # Gzip Settings
        ##


        gzip on;
        gzip_disable "msie6";

        gzip_vary on;
        gzip_proxied any;
        gzip_comp_level 6;
        gzip_buffers 16 8k;
        gzip_http_version 1.1;

        gzip_types text/plain application/xml;
        gzip_min_length 256;
        gzip_proxied no-cache no-store private expired auth;


        ##
        # Virtual Host Configs
        ##

        include /etc/nginx/conf.d/*.conf;
        include /etc/nginx/sites-enabled/*;
}

I edited sudo nano /etc/nginx/sites-available/default

Contents

server {
        listen 80 default_server;
        listen [::]:80 default_server;

        # SSL configuration
        #
        listen 443 ssl default_server;
        listen [::]:443 ssl default_server;

        ssl on;
        ssl_certificate /path/to/ssl/certs/cert.pem;
        ssl_certificate_key /path/to/ssl/private/key.pem;

        root /path-to-www;

        # Add index.php to the list if you are using PHP
        
        index index.html index.php;

        server_name fearby.com;

        #Security Headers
        add_header Strict-Transport-Security "max-age=31536000; includeSubDomains" always;
        add_header X-Content-Type-Options "nosniff" always;
        add_header Referrer-Policy "no-referrer-when-downgrade";
        add_header X-XSS-Protection "1; mode=block" always;
        add_header X-Frame-Options SAMEORIGIN always;
        add_header Permissions-Policy "accelerometer=(), camera=(), geolocation=(), gyroscope=(), magnetometer=(), microphone=(), payment=(), usb=()";

        # Force HTTPS
        if ($scheme != "https") {
                return 301 https://$host$request_uri;
        }

        # DENY RULES
        location ~ /\.ht {
                deny all;
        }
        location ~ ^/\.user\.ini {
                deny all;
        }
        location ~ (\.ini) {
                return 403;
        }

        if ($http_referer ~* "laptrinhx.com") {
                return 404;
        }

        if ($http_referer ~* "bdev.dev") {
                return 404;
        }

        if ($http_referer ~* "raoxyz.com") {
                return 404;
        }

        if ($http_referer ~* "congtyaz.com") {
                return 404;
        }


        location / {
                try_files $uri $uri/ /index.php?$args;
        }

        location ~ \.php$ {
                include snippets/fastcgi-php.conf;
                fastcgi_pass unix:/run/php/php7.4-fpm.sock;
        }

        # DNS
        resolver 1.1.1.1 1.0.0.1 valid=60s;
        resolver_timeout 1m;
}

I tested Nginx and PHP.

I installed MySQL (I Googled a guide)

I created a database, database user and assigned permissions for my blog.

I installed the WordPress CLI tool

I installed WordPress the using the wp-cli tool

wp core install --url=example.com --title=Example --admin_user=supervisor --admin_password=strongpassword [email protected]

When I had a blank WordPress I uploaded the blog folder that I backed up before.

I ran this command to allow Nginx to read the backed up website files

sudo chown -R www-data:www-data /path-to-www

I also uploaded the backup of my mysql database to /var/lib/mysql/oldblogdatabase

I ran this command to allow mysql to read the backed up database

sudo chown -R mysql:mysql /var/lib/mysql/oldblogdatabase

I also uploaded the following files to /var/lib/mysql

TIP: These files are very important to restore.  You cannot just copy a database in a subfolder.

  • ibdata1
  • ib_logfile0
  • ib_logfile1
  • ib_logfile1
  • ibtmp1

I ran this command to allow mysql ro read the ib* files

sudo chown -R mysql:mysql /var/lib/mysql/ib*

I was able to load my old website

Blog Up

I still have some issues to solve but it is back.

Lessons Learned

  1. Save all passwords and have backup accounts and roll back before working backups are gone.
  2. Setup a Dev Test, Pre Prod Environment and do no test on production servers.
  3. Do not delay disaster recovery actions.
  4. Do not rely on automation.
  5. Have more than a weeks worth of backups.

Get $25 free credit on UpCloud and deploy your own server: Use this link to get $25 credit (new UpCloud users only).

 

Change Log

Version 1.2

Filed Under: Uncategorized Tagged With: 404, MySQL, nginx, php, website

Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare

April 5, 2018 by Simon

This guide will show you how to enable the latest Transport Layer Security (TLS) 1.3 protocol with it’s predecessor Secure Sockets Layer (SSL) with NGINX and OpenSSL for better website security on an Ubuntu 16.04 server

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Making sure your server is up to date and running the latest SSL software is important. I have updated Open SSL before and blogged about this here.  Do back up your server before changing settings and if you use  Cloudflare (if you don’t do it now) enable Development Mode (and disable caching until changes are made).

For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

TLS 1.3 is the latest SSL security protocol that can be used between clients and servers to encrypt connections on the web.

TLS 1.3 uptake is only 60% according to https://caniuse.com/#search=TLS%201.3

TLS 1.3

Read why TLS 1.3 is important and news on TLS 1.3 can be found here: https://www.openssl.org/blog/blog/2018/02/08/tlsv1.3/

The Good and Bad

Done be like this commercial site with very poor security (tested with SSL labs and asafaweb)

Bad SSL

Here is what the top 1 million sites do

Here it is!! Alexa Top 1 Million Analysis – February 2018 https://t.co/TjBHNX7zTi

— Scott Helme (@Scott_Helme) February 26, 2018

Installing Open SSL on Ubuntu

Connect to your Ubuntu 16.04 server via SSH (I connected to my Vultr server)

Check what version of OpenSSL you have? My OpenSSL is out of date.

# openssl version
OpenSSL 1.1.0g  2 Nov 2017

Tip: What Ciphers does your Open SSL Support?

openssl ciphers -s -v
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH       Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-ECDSA-AES256-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA1
ECDHE-RSA-AES256-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
ECDHE-ECDSA-AES128-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
ECDHE-RSA-AES128-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1

Time to update Open SSL

OpenSSL 1.1.1 beta is available and supports TLS 1.3  but it is n BETA form.  OpenSSL code is available here.

I did the following to download and build the latest version of OpenSSL.

mkdir /openssltemp
cd /openssltemp
sudo git clone git://git.openssl.org/openssl.git
cd openssl/
./config --prefix=/usr/local/ssl --openssldir=/usr/local/ssl -Wl,-rpath,/usr/local/ssl/lib
make
sudo make install

I tried to check the open SSL version but had an error?

openssl version 
openssl: /usr/lib/x86_64-linux-gnu/libssl.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)
openssl: /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1: version `OPENSSL_1_1_1' not found (required by openssl)

A quick GitHub ticket revealed I needed to set a path variable.

export LD_LIBRARY_PATH=/usr/local/lib
echo "export LD_LIBRARY_PATH=/usr/local/bin/openssl" >> ~/.bashrc

Open SSL now reports it’s version.

openssl version
OpenSSL 1.1.1-pre3 (beta) 20 Mar 2018

What version NGINX do you have (1.13 supports TLS 1.3) read here

# nginx -v
nginx version: nginx/1.13.9

Backup your NGINX

Do backup your server files and take a snapshot if need be.  I am not responsible;e for a broken server,

sudo cp -R /etc/nginx/ /nginx-backup-26thMar-2018

Edit NGINX Configuration

Update NGINX configuration: /etc/nginx/sites-available/default

ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256;
ssl_prefer_server_ciphers on;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2 TLSv1.3;
ssl_ecdh_curve secp384r1;

tip: Review other NGINX hardening settings here.  Also remove TLSv1.0

I tested my NGINX config loaded them and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restart

Check the status of NGINX

# /etc/init.d/nginx status

[ ok ] Restarting nginx (via systemctl): nginx.service.
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) 
     Docs: man:nginx(8)
  Process: 15154 ExecStop=/sbin/start-stop-daemon --quiet --stop --retry QUIT/5 --pidfile /run/nginx.pid (code=exited, status=0/SUCCESS)
  Process: 15162 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
  Process: 15159 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
 Main PID: 15166 (nginx)
    Tasks: 4
   Memory: 2.3M
      CPU: 27ms
   CGroup: /system.slice/nginx.service
           ├─15166 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─15170 nginx: worker process
           ├─15171 nginx: cache manager process
           └─15172 nginx: cache loader process

If you have configured Cloudflare then log in and enable TLS support.

Cloudflare TLS Settings

Enable TLS 1.3 in Chrome by visiting chrome://flags/#tls13-variant This should be automatic in later versions of Chrome and other browsers.

Enable TLS in Chrome

Verify TLS

I used the developer tools in Chrome to confirm the page was verified in TLS 1.3.

Verify TLS

Updated to 1.1.1-pre6-dev

mkdir /temp
cd /temp
sudo git clone https://github.com/openssl/openssl.git
cd openssl/
./config --prefix=/usr/local --openssldir=/usr/local -Wl,-rpath,/usr/local
make
sudo make install
openssl
OpenSSL> version
OpenSSL 1.1.1-pre6-dev  xx XXX xxxx
OpenSSL> exit

Don’t forget to test your SSL strength with https://dev.ssllabs.com/ssltest/

SSL Test 2018

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.4 fixed typo

v1.3 added bad ssl cert.

v1.2 ssl test v1.1 updated to 1.1.1-pre6-dev

v1.0 Initial post

Filed Under: ssl Tagged With: 16.04, a, an, Cloudflare, Enabling, is, nginx, on, server, ssl, that, TLS 1.3, ubuntu, Using, website

Using the Qualys FreeScan Scanner to test your website for online vulnerabilities

March 23, 2018 by Simon

It is possible to deploy a server in minutes to hours but it can take days to secure.  What tools can you use to help identify what to secure on your website?

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line, installing a Free SSL certificate and setting up SSL security.

Security Tools

  • https://asafaweb.com/ is a good tool for quick scanning
  • Kali Linux has a number of security tools you can use.
  • You can run a system audit Lynis Audit.
  • Checking your site for vulnerabilities with Zap.
  • Run a Gravity Scan malware and supply chain scan
  • Use Qualys SSL scan to test your SSL certificate: https://www.ssllabs.com/ssltest/

Qualys

Qualys SSL Labs SSL Tester is the best tool for checking an SSL certificate strength

Most people don’t know Qualys also has another free (limited to 10 scans) vulnerability scanner for websites.

Goto https://freescan.qualys.com/ and click Start your free account.

Complete the signup form

Now check your email to login and confirm your email account

Login now from the email.

Create a password (why the 25 char max Qualys?)

Enter your website URL and click Scan

The scan can take hours

While the scan was being performed I noticed that Qualys offers alerts (I’ll check this out later): https://www.qualys.com/research/security-alerts/

Yes, the scan can take hours, take a walk or read other posts here.

The scan is almost complete

Yay, my latest scan revealed 0 High, 0 Medium and 0 Low-risk vulnerabilities.

It did report 23 informational alerts like “Firewall Detected“.

Threat Report Results

Patch Report Results

This report was empty (probably because I don’t run Windows)

Threat Report Results

The OWASP report contained partial scan results (maybe the full report is available to pro users)

Previous Scan Results

The Qualys dashboard will show all past scans.

My first scan showed a Low priority issue with the /wp-login.php page as the input fields did not have “autocomplete=”off””, I fixed this by adding “autocomplete=”off”” the removing the page (safer).

The second scan found two issues with cookies (possibly ad banner cookies) and 2 subfolders that I created in past development exercises. I deleted the two sub-folders that were not needed.

The third scan was clean.

Here is a scan of a static website of a friends server (static can be less secure if the server underneath is old or unpatched).

Static Website

Happy scanning. I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Static Web Server Scan

v1.0 Initial post

Filed Under: Firewall, LetsEncrypt, Linux, Malware, Security, Server, Ubuntu, Vulnerabilities, Vulnerability, WP Security Tagged With: for, FreeScan, online, Qualys, Scanner, test, the, to, Using, Vulnerabilities, website, your

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

March 13, 2018 by Simon

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Buy a Domain 

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Cloudflare Benefits (Free Plan)

  • DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
  • Global CDN
  • Shared SSL certificate (I disabled this and opted to use my own)
  • Access to audit logs
  • 3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Cloudflare Global Network

Setup

You will need to sign up at cloudflare.com

Cloudflare

After you create an account you will be prompted to add a siteAdd SiteCloudflare will pull your public DNS records to import.

Query DNS

You will be prompted to select a plan (I selected free)

Plan Select

Verify DNS settings to import.

DNS Import

You will now be asked to change your DNS nameservers with your domain reseller

DNS Nameservers

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Strict SSL

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

Could I kindly ask if you are reading this that you visit https://t.co/9x5TFARLCt, I am writing a @Cloudflare blog post and need to screenshot stats. Thanks in advance

— Simon Fearby (Developer) (@FearbySoftware) March 13, 2018

The Cloudflare CTO responded.  🙂

Sure thing 🙂

— John Graham-Cumming (@jgrahamc) March 13, 2018

Confirm Cloudflare link to a domain from the OSX Comand line

host -t NS fearby.com
fearby.com name server dane.ns.cloudflare.com.
fearby.com name server nora.ns.cloudflare.com.

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

Page Rules

“fearby.com.com/wp-*” Cache level: Bypass

“fearby.com.com/wp-admin/post.php*” Cache level: Bypass

“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours.  having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before).  I just need to wait for caching and minification to kick in.

DNS Improved

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

  • Load Time: 1.80s
  • First Byte 0.176s
  • Start Render 1.200s

webpagetest

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Under Attack Flow

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here.  A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

cloudflare-attack-001

Thanks, Cloudflare.

Cloudflare Pros

  • Enabling Attack mode was simple.
  • Soaked up an attack.
  • Free Tier
  • Many Reports
  • Option to force HTTPS over HTTP
  • Option to ban/challenge suspicious IP’s and set challenge timeframes.
  • Ability to setup IP firewall rules and Application Firewalls.
  • User-agent blocking
  • Lockdown URL’s to IP’s (pro feature)
  • Option to minify Javascript, CSS and HTML
  • Option to accelerate mobile links
  • Brotli compression on assets served.
  • Optio to enable BETA Rocket loader for Javascript performance tweaks.
  • Run Javascript service workers from the 120+ CDN’s
  • Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
  • HTTP/2 on, IPV6 ON
  • Option to setup load balancing/failover
  • CTO of Cloudflare responded in Twitter 🙂
  • Option to enable rate limiting (charged at 10,000 hits for $0.05c)
  • Option to block countries (pro feature)
  • Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

  • No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
  • Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
  • Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
  • WordPress draft posts are being cached even though page riles block wp-admin page caching.
  • Would be nice to have ad automatic Under Attack mode
  • Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post

Filed Under: Analytics, App, Cache, CDN, Cloud, Cloudflare, DNS, Domain, Hosting, LetsEncrypt, Marketing, Secure, Security, SEO, Server, VM, Vultr, Website, Wordpress Tagged With: a, and, Cloudflare, hosted, namecheap, on, Setting, to, up, use, vm, vultr, website

Using Chrome 65 to measure website Performance, Progressive Web Apps, Basic Practices, Accessibility and SEO

March 4, 2018 by Simon

Chrome 65 beta has added SEO to audits in the Developer Tools Audit tab.

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.

When you develop any site you need to perform regular audits to ensure it is up-to-date and compliant.  Google Chrome has audit tools built right in (Chrome 65 has SEO audit tools (yay)

Chrome 64Developer Tools – Audit tool.

Chroms 64 Audit

Chrome 64 Audit Scan options

Chrome 64 Audit Scan

Chrome 64 Audit Results

The performance results seem off given I have set up a CDN, optimized WordPress.

Chrome 64 Audit Results

Time to download Chrome 65 beta and get new audit tools.

Time to download Chrome 65 and get new audit tools

Chrome version confirmed

Chrome Version

Audit Options

Chrome 65 BETA Audit Options

Chrome 65 Audit Results

Hmmm, results are different (same website, network and time)

Chrome 64 Audit Results:

  • Performance: 45
  • Progressive Web App: 30
  • Accessibility: 86
  • Best Practice: 56
  • SEO: N/A

Chrome 65 BETA Audit Results:

  • Performance: 24
  • Progressive Web App: 45
  • Accessibility: 65
  • Best Practice: 63
  • SEO: 90

Chrome 65 BETA Audio Results
Google recommends you load pages in under 1s, I would suggest using multiple tools to indicate site speed.

https://gtmetrix.com is a great site for testing your sites speed. I used GT Metrix to move my site from 26s to 6 by setting up a WordPress CDN, moving away from CPanel to a self-managed server on Vultr then to 4s with optimising PHP by setting up child workers.

GT Metrix

https://www.webpagetest.org is also good for testing websites.

I will continue to use Chrome Audit Tools for other results like SEO

Chrome SEO Tools

That’s weird Chrome Audit Tools, I thought I did not have meta tags?

I downloaded the WP Meta SEO plugin, I use the wp command from the SLI (setup guide here)

cd /www/wp-content/plugins/
wget https://downloads.wordpress.org/plugin/wp-meta-seo.3.6.6.zip
unzip wp-meta-seo.3.6.6.zip
rm -R wp-meta-seo.3.6.6.zip

I activated the plugin in WordPress the loaded the dashboard.  WP Meta SEO asked to import data from the Yoast plugin.

meta-dashboard

Now I can see my metadata is falling behind.

wp-meta-seo

With WP Meta SEO I was able to apply individual page Meta tags.

meta-apply

I added the following meta tags with WP Meta SEO plugin for each page (the metadata below was for this page)

<meta name="viewport" content="width=device-width, initial-scale=1" />
<meta name="twitter:image" content="https://fearby-com.exactdn.com/wp-content/uploads/2018/02/php_pool_featured.jpg" />
<meta name="twitter:card" content="summary" />
<meta name="twitter:site" content="@FearbySoftware" />
<meta name="twitter:domain" content="Programming, IoT and Server Stuff" />
<meta name="twitter:description" content="How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic" />
<meta name="twitter:title" content="Setup PHP FPM on demand child workers in PHP 7.x" />
<meta property="og:image" content="https://fearby-com.exactdn.com/wp-content/uploads/2018/02/php_pool_featured.jpg" />
<meta property="og:site_name" content="Programming, IoT and Server Stuff" />
<meta property="og:description" content="How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic" />
<meta property="og:url" content="https://fearby.com/article/how-to-setup-php-fpm-on-demand-child-workers-in-php-7-x-to-increase-website-traffic/" />
<meta property="og:type" content="article" />
<meta property="og:title" content="Setup PHP FPM on demand child workers in PHP 7.x" />
<meta name="description" content="How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic" />
<meta name="keywords" content="Setup PHP FPM on demand child workers in PHP 7.x" />
<meta name="title" content="Setup PHP FPM on demand child workers in PHP 7.x" />

fyi

I use the free version of the Yoast for SEO plugin in WordPress.  The premium version has a lot more to offer, I might have to check Premium out.

Yoast Compare

I am now on the hunt for meta plugins for WordPress

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 WP-Meta SEO

v1.1 Added webpagetest.org link and results

v1.0 Initial post

Filed Under: Audit Tagged With: 65, Accessibility, and, apps, Basic, Chrome, measure, Performance, Practices, Progressive, seo, to, Using, web, website

How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic

February 26, 2018 by Simon

This blog post will show you how to setup PHP FPM on-demand child workers in PHP 7.x to increase website traffic.

My blog was experiencing a number of slow page loads and often running “sudo service php7.0-fpm restart” would resolve the problem.  I have blogged before about setting up Ubuntu Servers on AWS, Digital Ocean and Vultr but this post is about debugging and speeding up PHP on Ubuntu self-managed servers.

Background

I tried the normal tweaks in “/etc/php/7.0/fpm/php.ini” like

memory_limit = 512M

I setup servers like this.

Temporary Fix

I had even set up a temporary NGINX and php7.0-fpm restart ever 5 and 1-minute respectively until I had time to look into this.

*/5 * * * * /etc/init.d/nginx restart
* * * * * sudo service php7.0-fpm restart

Debug

I checked out the PHP7.0-fpm.log and I found the following

[25-Feb-2018 16:35:35] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 17:02:26] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 17:51:09] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 18:18:51] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 20:58:12] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 21:02:57] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 21:30:58] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 21:35:10] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it
[25-Feb-2018 23:36:28] WARNING: [pool www] server reached pm.max_children setting (5), consider raising it

Setting up a PHP-FPM pool

Read the official guide here on configuring PHP FPM pools etc.

I edited “/etc/php/7.0/fpm/pool.d/www.conf” and added the following to set up a pool of PHP-FPM servers.

; Note: This value is mandatory.
pm = dynamic

; The number ocf child processes to be created when pm is set to 'static' and the
; maximum number of child processes when pm is set to 'dynamic' or 'ondemand'.
; This value sets the limit on the number of simultaneous requests that will be
; served. Equivalent to the ApacheMaxClients directive with mpm_prefork.
; Equivalent to the PHP_FCGI_CHILDREN environment variable in the original PHP
; CGI. The below defaults are based on a server without much resources. Don't
; forget to tweak pm.* to fit your needs.
; Note: Used when pm is set to 'static', 'dynamic' or 'ondemand'
; Note: This value is mandatory.
pm.max_children = 40

; The number of child processes created on startup.
; Note: Used only when pm is set to 'dynamic'
; Default Value: min_spare_servers + (max_spare_servers - min_spare_servers) / 2
pm.start_servers = 10

; The desired minimum number of idle server processes.
; Note: Used only when pm is set to 'dynamic'
; Note: Mandatory when pm is set to 'dynamic'
pm.min_spare_servers = 5

; The number of seconds after which an idle process will be killed.
; Note: Used only when pm is set to 'ondemand'
; Default Value: 10s
pm.process_idle_timeout = 30s;

; The number of requests each child process should execute before respawning.
; This can be useful to work around memory leaks in 3rd party libraries. For
; endless request processing specify '0'. Equivalent to PHP_FCGI_MAX_REQUESTS.
; Default Value: 0
pm.max_requests = 250

You may need more or fewer child processes depending on your needs and free memory.

After editing the PHP-FPM config file restart PHP-FPM

sudo service php7.0-fpm restart

Restart Nginx

sudo /etc/init.d/nginx restart

You will be able to view the PHP child process status by typing the following

service php7.0-fpm status
● php7.0-fpm.service - The PHP 7.0 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php7.0-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Mon 2018-02-26 00:33:17 AEDT; 5min ago
     Docs: man:php-fpm7.0(8)
 Main PID: 1284 (php-fpm7.0)
   Status: "Processes active: 0, idle: 10, Requests: 56, slow: 0, Traffic: 0.2req/sec"
    Tasks: 11
   Memory: 330.1M
      CPU: 39.558s
   CGroup: /system.slice/php7.0-fpm.service
           ├─1284 php-fpm: master process (/etc/php/7.0/fpm/php-fpm.conf)
           ├─1503 php-fpm: pool www
           ├─1504 php-fpm: pool www
           ├─1505 php-fpm: pool www
           ├─1506 php-fpm: pool www
           ├─1507 php-fpm: pool www
           ├─1508 php-fpm: pool www
           ├─1509 php-fpm: pool www
           ├─1511 php-fpm: pool www
           ├─1512 php-fpm: pool www
           └─1513 php-fpm: pool www

Feb 25 10:33:16 servername systemd[1]: Starting The PHP 7.0 FastCGI Process Manager...
Feb 25 10:33:17 servername systemd[1]: Started The PHP 7.0 FastCGI Process Manager.

You can use htop (commands here) to see child PHP processes in the pool and to verify free memory.

php-pool

This command is good for watching free memory on a server

watch -n 1 'free -m'

I prefer to use up free memory (if available) and leave about 100mb free.

Every 1.0s: free -m                                                                                                            Mon Feb 26 00:47:55 2018

              total        used        free      shared  buff/cache   available
Mem:            992         518         120          40         353         280
Swap:             0           0           0

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial Post

Filed Under: PHP Tagged With: 7.x, child, demand, FPM, How, in, increase, on, php, Setup, to, traffic, website, workers

Using the free Adminer GUI for MySQL on your website

February 8, 2018 by Simon

Adminer is a free GUI tool that can you can easily install on a PHP web server. Adminer allows you to easily connect to your MySQL instance, create databases/tables/indexes/rows and backup/import databases and much more.

You can read my other posts on Useful Linux Terminal Commands and Useful OSX Terminal Commands.

I used to use phpMyAdmin to manage MySQL databases on AWS, Digital Ocean and Vultr but switched to Adminer due to forgotten issues.  You can always manage MySQL via command line but that is quite boring.

The below screenshots were taken on my local Development Mac Laptop (with optional OSX Apache SSL Setup (that reports “Not Secure” (but it is good enough to use locally)). I prefer to code in SSL and warn when SSL is not detected.

Downloading and Installing Adminer

Navigate to https://www.adminer.org/ and click Download.

Adminer GUIClick English only (.php file)

Adminer

Save the Adminder for MySQL (.php) file to your web server and give it a random name and put in a folder also with a random name (I use https://www.grc.com/passwords.htm to generate strong password).

Tip: Uploading this file to a live serve offers hackers and unauthorized people potential access to your MySQL server.  I would remove this file from live serves when you are not using it not to be sure.

Tip: Read my guide here on setting up NGINX, MySQL and PHP here.  Basically, I did this to setup MySQL on Ubuntu 16.04.

sudo apt-get install mysql-common
sudo apt-get install mysql-server
mysql --version
>mysql Ver 14.14 Distrib 5.7.19, for Linux (x86_64) using EditLine wrapper
sudo mysql_secure_installation
>Y (Valitate plugin)
>2 (Strong passwords)
>N (Don't chnage root password)
>Y (Remove anon accounts)
>Y (No remote root login)
>Y (Remove test DB)
>Y (Reload)
service mysql status
> mysql.service - MySQL Community Server

TIP: Ensure MySQL is secure and has a good root password, also consider setting up Ubuntu Firewalls and Securing Ubuntu. Also, ensure the Server is patched and does not have exploits like Spectre and meltdown.

Now you can access your Admirer php file on your Web Server (hopefully with an obfuscated name).

Randomize

Login to Adminer with your MySQL root password.

Login

Click Create databaase

Create Database

Give the database a name and choose the character coding standard (e.g UTF8 general ci). Different standards have different performance impacts too.

Save

Now that you have a database you can create a table.

Adminer

Consider adding an auto-incrementing ID and say a Key and Value varchar column.

Adminer

When the table is created you can add a row to the table.

Adminer

I created one with a “TestKey” and “TestValue” row.

Adminer

The row was inserted.

Adminer

The final thing to do is add a database user that code can connect to the database with. Click Privileges.

Adminer

Click Create user

Adminer

Tick All privileges and click Save

Adminer

Now the user is added to the database

Adminer

Let’s create a PHP file and talk to the database. Let’s use parameterized queries

<?php

date_default_timezone_set('Australia/Sydney');
echo "Last modified: " . date ("F d Y H:i:s.", getlastmod()) . "<br /><br />";

// Turn on if you need to see errors
// error_reporting(E_ALL);
// ini_set('display_errors', 0);

$dbhost = '127.0.0.1';
$dbname = 'dbtest';
$dbusername = 'dbtestuser';
$dbpassword = '*****************************************'';

$con = mysqli_connect($dbhost, $dbusername, $dbpassword, $dbname);
 
// Turn on debug stuff if you need it
// echo var_dump($con);
// printf(" - Error: %s.n", $stmt->error);
 
if($con->connect_errno > 0){

    printf(" - Error: %s.n", $stmt->error);
    die("Error: Unable to connect to MySQL");

} else {

    echo "Charset set to utf8<br />";
    mysqli_set_charset($con,"utf8");
}
 
if (!$con) {

    echo "Error: Unable to connect to MySQL (E002)" . PHP_EOL;
    echo "Debugging errno: " . mysqli_connect_errno() . PHP_EOL;
    echo "Debugging error: " . mysqli_connect_error() . PHP_EOL;
    exit;

} else {

    echo "Database Connection OK<br />";
 
    echo "&nbsp; Success: A proper connection to MySQL was made! The $dbname database is great." . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Host information: " . mysqli_get_host_info($con) . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Server Info: '" . mysqli_get_server_info($con) . "'<br />";
    echo "&nbsp; &nbsp;- Server Protocol Info : ". mysqli_get_proto_info($con) . "<br />";
    echo "&nbsp; &nbsp;- Server Version: " . mysqli_get_server_version($con) . "<br />";
    //echo " - Server Connection Stats: " . print_r(vmysqli_get_connection_stats($con)) . "<br />";
    echo "&nbsp; &nbsp;- Client Version: " . mysqli_get_client_version($con) . "<br />";
    echo "&nbsp; &nbsp;- Client Info: '" . mysqli_get_client_info() . "'<br />";
 
    echo "Ready to Query the database '$dbname'.<br />";
 
    // Input Var's that are parameterized/bound into the query statement
    $in_key = mysqli_real_escape_string($con, 'TestKey');
 
    // Output Var's that the query fills after querying the database
    // These variables will be filled with data from the current returned row
    $out_id = "";
    $out_key = "";
    $out_value = "";
 
    echo "1. About to query the database: '$dbname'<br />";
    $stmt = mysqli_stmt_init($con);

    $sql = "SELECT testid, testkey, testvalue FROM tbtest WHERE testkey = ?";
    echo "SQL: $sql (In = $in_key)<br /";

    if (mysqli_stmt_prepare($stmt, $sql)) {

            echo "2. Query Returned<br />";
            /*
                Type specification chars
                Character   Description
                i   corresponding variable has type integer
                d   corresponding variable has type double
                s   corresponding variable has type string
                b   corresponding variable is a blob and will be sent in packets
            */
            mysqli_stmt_bind_param($stmt, 's', $in_key);
            mysqli_stmt_execute($stmt);
            mysqli_stmt_bind_result($stmt, $out_id, $out_key, $out_value);
            mysqli_stmt_fetch($stmt);
     
            // Do something with the 1st returned row        
            echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//

            // Do we have more rows to process
            while($stmt->fetch()) { 
                
                    // Output returned values
                    echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//
            
            }
            mysqli_stmt_close($stmt);
            
            echo "Done<br />";
        
        } else {
        
            echo "3. Error Querying<br/>";
            printf(" - Error: %s.n", $stmt->error);
        
        }
}    
?>

Result

Adminer Results

If you don’t have a server check out my guides on AWS, Digital Ocean and Vultr.

Happy coding and I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial Version

Filed Under: MySQLGUI Tagged With: Adminer, for, free, gui, MySQL, on, the, Using, website, your

Using Platforma Web Wireframe Kit to build a website (prototype)

November 24, 2017 by Simon

I have blogged before about building a server for users to install WordPress, optimizing images in WordPress,  deploying WordPress via CLI, moving WordPress, speeding up WordPress and securing WordPress but what do you do if you want a non-WordPress site without the support hassles?

Recently I gave the https://platforma.ws iOS prototyping library extensions a test.  I was delighted to find they had a Web Wireframe Kit (generation suite) for prototyping and exporting working websites. You can try the free version or buy a licence here.

Creating a Website with Platforma Web Wireframe Kit

Goto https://platforma.ws and click HTML Generator (or click here)

You will be presented with an empty website ready for your attention.

platformaweb001

Adding Website Elements

It’s as simple as clicking a purple add button.

Add Items to Webiste

This reveals a number of HTML templates samples that you can drag and drop to your website design.

Add Element

You can then choose a category (e.g “Header”) and see the elements of the available sample.

Categories

Simply drag and drop the elements out into your design.

Drag and Drop

Now, Let’s click the purple Add  (in the top left) button and add a sample Header section, sample Contents, sample Slider, sample Body, sample first Call to Action section, sample Pricing Table, sample second Call to Action section, sample Footer section.

30 seconds later and I have generated designed a site ready to edit the exported HTML.

Designed Site

Exporting Your Site from Platforma

Click on the Export button (in the top right).

Export

I was greeted with the following export screen, this page explains the difference in export options: http://app.platforma.ws/docs/

I don’t need “node.js” or “gulp” “Advanced Version” (PUG + STYLUS) so I’ll choose “Simple Version” (HTML + CSS + JS).

Export Options

You will need to enter a licence key to continue the export.

Enter Licence Key

The website export download came down just fine.

Code

The code looks ok, I did notice that images were missing alt tags so I added those in.

Code

Any Errors in the Code (in Chrome)?

Nope, Chrome loads the code with no errors.

Loading the SIte

Testing Online

How about HTML5 and WCAG 2.0 AA

I uploaded the zip file to my server (using the scp command), I could have used SFTP.

scp /local/folder/local-file.zip [email protected]:/www/destination-folder/

I unzipped the site with

sudo apt-get install unzip
unzip filename.zip

The site loads just fine in a web browser

Load

Accessibility

I used https://achecker.ca/checker/index.php to test the site with WCAG 2.0 AA, the only remaining issues I found were in relation to the multiple H1, H2 etc tags (this can be fixed by moving the H CSS code to custom classes and removing H1, H2 etc tags altogether (and reference the custom class matching the H* tags)).

WCAG

fyi: The potential WCAG problems that were being alerted were in relation to…

  • My alt tags were potentially short
  • Potential Colour warnings
  • Potential Contrast warnings
  • Missing a “Skip to content” block
  • Reporting of placeholder graphics and alt tags (a checker is smart)
  • etc

I tested the sites HTML compliance with https://validator.w3.org/, the code passed with flying colours.

HTML5

Customizing

I could not find a way to edit the elements in the http://app.platforma.ws/# like the Platforma iOS Adobe XD Kit but you can quickly edit in your HTML after exporting (using your editor of choice like Dreamweaver, Sublime or Notepad).

Conclusion

Platforma Web Wireframe Kit is an essential tool for anyone wanting to build quick web prototype (or even live sites) website for themselves, clients etc. I am very impressed with the code created.

Read More

Check out my guide, Using Adobe XD and Platforma Web Wireframe Kit to prototype an iOS app.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Added more, fixed  a bit.

etc

Filed Under: Advice, App, iOS, UI, UX, Web Prototype, Website, Wordpress Tagged With: a, build, Kit, Platforma, prototype, to, Using, web, website, Wireframe

Improving the speed of WordPress

September 22, 2017 by Simon

This post shows my never-ending quest to speed up WordPress for free.

I have used to use WP Total Cache in the past but decided to check out what others recommended, I found this post 6 Best WordPress Caching Plugins Compared. Some WordPress Caching Plugins.

  • W3 Total cache
  • WP Fastest Cache
  • Cache Enable
  • WP Rocket
  • WP Super Cache
  • Etc

What plugin do I use?

Benchmark (No Caching Plugin)

I tested my site before installing a caching plugin with https://www.webpagetest.org/ and my site was loading in 21s (loading over 141 files).

My site loaded in a terrible 21.3 seconds. My blog is hosted on Jumba (Net Registry) on and Ultimate plan for $25 a month.

My site seems to deliver 70% images so I wonder if a page caching plugin can help?

I do run the EWWW Image Optimizer plugin to automatically compress images when I upload them to my site. Read my blog post on the EWWW Image Optimizer here.  I do keep images at a high quality to capture all details.

WP Fastest Cache Plugin

I have decided to try the WP Fastest Cache because it’s source was updated 4 hours ago compared to WP Super Cahches update 5 months ago. Both these plugins offer similar GT Metrix performance improvements and WP Fastest Cache has been tested on WordPress v4.8.

Installing WP Fastest Cache

I looked for the WP Fastest Cache Plugin on the WP plugin directory but it was not there.

I downloaded the latest version WP Fastest Cache from https://wordpress.org/plugins/wp-fastest-cache/

I upload the WP Faster Cache plugin to my site.

I Activated the plugin.

WP Fastest Cache plugin is now installed 🙂

It appears to have auto cached/indexed my site?

Now it’s time to run the same benchmark and see if the site is faster (with the same settings (Singapore chrome))?

1 of 3 test are underway.

WP Fastest Cache Results

Wow WP Fastest Cache loaded my site 2 seconds slower (Try 1 = 23 seconds, Try 2 =  21 seconds and Try 3 =  28 Seconds).

This could have been because of weekend traffic or hosting issues but this was not what I expected.

I disabled the WP Fastest Cache plugin and ran the benchmarks again and it was still 23 seconds (weekend traffic?). I re-enabled WP Fastest Cache and re-ran the test but no improvement.

My bad I think I needed to manually configure the WP Faster Cache plugin by opening the new WP Faster Cache menu on the left-hand side of the WP admin dashboard.

There I enabled caching options in the WP Faster Cache options.

I ran https://www.webpagetest.org tests again and got 16s, 18s and 16s seconds results in three tests and an A on compressed images. It appears you need to manually configure the WP Total Cache plugin after installing it (I missed this step).

I disable WP Fastest Cache and tried the WP Super Cache plugin and the test results were 29s, 24s, 24s (slower than WP Faster Cache). then tried W3 Total cache and the results were ()

I tried the W3 Total Cache plugin and the results were (30s, 16s 26s).

I Tried Autoptimize and it was tested at 45s.

It looks like WP Faster Cache is the fastest, ill turn it back on until try setup a CDN.

Fast Forward to Sept 2017

Since writing this post I have moved away from a shared C-Panel host and have moved my domain to a self-managed Vultr server closer to me, I have moved my email to Google G-Suite. I have learned how to deploy and manage WordPress by command-line tools. I have set up servers on Digital Ocean before but the servers are located in Singapore and not Sydney and latency and scalability was poor. SSL will make sites slower and servers far away will just compound the issues.

Re-enabling the WP Fastest Cache Plugin

I tried reinstalling the WP Fastest Cache plugin and for me, the plugin just slows down my site by 6 seconds.

I opened my NGINX config and got my NGINX user

sudo nano /etc/nginx/nginx.conf

My user is: www-data

I enabled the WP Fastest Cache plugin and ensured the WP Fastest Cache has ownership and access to the cache folder.

sudo chown www-data:www-data /www/wp-content/plugins/cache
sudo chown www-data:www-data /www//wp-content/plugins/cache/all
sudo chmod 755 /www/wp-content/plugins/cache *
sudo chmod 755 /www/wp-content/plugins/cache/all *

Below are the settings I use.

WP Fastest Cache

installing the WP-Optimize Plugin

I recommend setting up WP-Optimize plugin as it will optimize your database and keep things fast, it only saves me a second on my load times but this helps.

WP Optimize

WP-Optimize will allow for to review database optimizations

WP-Optimize database savings

Setting up Nginx GZip Compression

I set up my Nginx config to include

gzip on;
gzip_disable "msie6";

gzip_vary on;
gzip_proxied any;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_http_version 1.1;
gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

I set the minimum size to gzip too

gzip_min_length 20;

Benchmark with G-Zip, Caching and WP Fastest Cache 

With WP Fastest Cache I now load my site in 13.9 seconds from Singapore. Time to disable WP Fastest cache plugin as it does not seem to be helping without linking to a CDN.

With Cache plugi

Setting up Browser Caching

I also setup browser caching by editing in NginX.

sudo nano /etc/nginx/sites-available/default

Added

location ~*  \.(jpg|jpeg|png|gif|ico|svg|js|css)$ {
        expires 365d;
}

Not sure if caching CSS and JS will cause problems in future?

Benchmark with G-Zip, Caching and without WP Fastest Cache (Singapore)

I re-ran the tests and got 10.9 seconds and got a B for cached content. When in Started on C-Panel I was getting near 30s

Benchmark

Benchmark with G-Zip, Caching and without WP Fastest Cache (Sydney)

I have always benchmarked from Singapore (as Sydney was not an option when I started) but now it is.  Out f curiosity is my website load time in Sydney?

8.2 seconds. Distance does affect performance.

Google Speed Insights

Google has awesome tools to help you increase your benchmark mobile and desktop website speeds and recommend focus areas to resolve problems: https://developers.google.com/speed/pagespeed/insights/

Mobile Speed Score

Desktop Speed Score

Tips

I was getting SVG files failing compassion tests so I added the following under allowed mime types under “http gzip_types” in /etc/nginx/nginx.conf

image/svg+xml text/html+svg

Minifying JS and CSS

This needs to be done and 50% of my site files appears to be CSS and JS related.

It looks like 30%~40 of your sites google speed index is related to minified/combined JS/CSS.

Google Speed Test

I installed the Fast Velocity Minify WordPress plugin.

I ran this to install it from the command line

cd /www/wp-content/plugins#
sudo wget https://downloads.wordpress.org/plugin/fast-velocity-minify.2.2.1.zip
--2017-09-23 19:51:46--  https://downloads.wordpress.org/plugin/fast-velocity-minify.2.2.1.zip
Resolving downloads.wordpress.org (downloads.wordpress.org)... 66.155.40.187, 66.155.40.203, 66.155.40.188, ...
Connecting to downloads.wordpress.org (downloads.wordpress.org)|66.155.40.187|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 821621 (802K) [application/octet-stream]
Saving to: ‘fast-velocity-minify.2.2.1.zip’

fast-velocity-minify.2.2.1.zip 100%[=================================================>] 802.36K   830KB/s    in 1.0s

2017-09-23 19:51:47 (830 KB/s) - ‘fast-velocity-minify.2.2.1.zip’ saved [821621/821621]

Unzip

sudo unzip merge-minify-refresh.zip

I activated the plugin and set some settings

Minify Settings

Verified minify logs

Logs

Google Page Insights can now see the minifies, css, js and html

Minified

Google Page Insights – Possible Optimizations

issues

And Google Ad Words and Google Analytics appear to be holding back Google Page Insight scores

Google adwords and Analytics

I am getting a few false positives with plugins javascript but that can be resolved another day.

Pingdom (Melbourne results)

3.2 seconds, a few false positives though.

Kingdom

I was going to test with https://www.webpagetest.org/ (from Singapore) but the service kept stalling and had too many tests before me (even from Sydney).

Wait

Address First Byte Time (todo)

If I look at the first-byte load results in the waterfall view my site is taking many seconds to deliver the first byte, this lowers the performance scores about 20%. I need to set up a CDN and or configure NGINX following this guide based on this manual configuration entry (I tried some of the Nginx settings but it appears I need to compile some performance settings into Nginx).

CDN (todo)

I am sure a Content Delivery Network (CDN) will help with the whole page deliver and first-byte times but I am trying to milk as much free as possible and limit future costs. A CDN will trigger higher monthly costs (any CDN providers want to donate a temporary pro plan for review purposes).

Misc Speed Articles

  • Yoast has a good site speed article here: https://yoast.com/site-speed-tools-suggestions/
  • Nginx has a good guide on Nginx performance here: https://www.nginx.com/blog/10-tips-for-10x-application-performance/
  • Google PageSpeed tips: https://developers.google.com/speed/docs/insights/rules

Configuring Ubuntu for Performance

Preventing applications swapping for disk (read more here)

sudo nano /etc/sysctl.conf

I added this memory-related setting.

vm.swappiness = 1

This will all but prevent applications writing to disk (swap) when they are not active. I had free memory on my VM so I may as well use it.

I will monitor the free ram after reboot and play with php memory settings.

ram

Setup Lazyload for images in posts

cd /www/wp-content/plugins/
sudo wget https://downloads.wordpress.org/plugin/bj-lazy-load.zip
unzip bj-lazy-load.zip
# activate the plugin

Lazyload Plugin Settings

Lazyload

Placeholder Image ( Image: https://fearby.com/wp-content/uploads/2017/09/placeholder.jpg )

Web Performance Test from Sydney

8.4 seconds ( Score Card F A A A C, was F F F A F ).  I was getting up to 28 second load times with Net Registry C Panel servers.

Static Content is cached but Googe Ad Sence, Google Analytics, and some plugins do block the score. The front page does have some features content that has to be loaded and can’t be minified or cached much.

Sydney Results

It is obvious I need to work in the initial websitee load (DNS, CDN or SSL), there sis  3 seconds I can save here.

3 sec

Configuring PHP for Performance

todo: PHP base config.

todo: PHP caching.

Conclusion

I was expecting WP Fastest Cache to deliver faster speeds but in reality but I am getting 4 seconds faster in WordPress. I was going to configure MaxCDN but they are to expensive. Fast Velocity Minify Plugin is working a treat 🙂

I ended up ditching the shared CPanel hosted domain and setup my own server for WordPress. My site seems a lot faster now. A friend set up CloudFlare with great success, more soon. I blogged about my server setup here.

Adding browser cache and compressing and moving away from CPanel to a self-managed server helped.

The only things to try now is to use a CDN and speed up the delivery of my site and improve the First Byte Time.

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.932 added lazy load information (24th Sep 2017)

v1.952 added small changes (23rd Sep 2017)

etc

Filed Under: Blog, Cache, Cloud, Domain, Software, Wordpress Tagged With: cache, cdn, plugin, speed, website, wordpress

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) HTTPS (6) IoT (9) LetsEncrypt (7) Linux (20) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) PHP (13) Scalability (12) Scalable (14) Security (44) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (44) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT