IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

Wordpress

I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.

by

I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance. Here is what I did to set up a complete Ubuntu 18.04 system (NGINX, PHP, MySQL, WordPress etc). This is not a paid review (just me documenting my steps over 2 days).

Background (CPanel hosts)

In 1999 I hosted my first domain (www.fearby.com) on a host in Seattle (for $10 USD a month), the host used CPanel and all was good.  After a decade I was using the domain more for online development and the website was now too slow (I think I was on dial-up or ADSL 1 at the time). I moved my domain to an Australian host (for $25 a month).

After 8 years the domain host was sold and performance remained mediocre. After another year the new host was sold again and performance was terrible.

I started receiving Resource Limit Is Reached warnings (basically this was a plot by the new CPanel host to say “Pay us more and this message will go away”).

Page load times were near 30 seconds.

cpenal_usage_exceeded

The straw that broke the camel’s back was their demand of $150/year for a dodgy SSL certificate.

I needed to move to a self-managed server where I was in control.

Buying a Domain Name

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Self Managed Server

I found a good web IDE ( http://www.c9.io/ ) that allowed me to connect to a cloud VM.  C9 allowed me to open many files and terminal windows and reconnect to them later. Don’t get excited, though, as AWS has purchased C9 and it’s not the same.

C9 IDE

C9 IDE

I spun up a Digital Ocean Server at the closest data centre in Singapore. Here was my setup guide creating a Digital Ocean VM, connecting to it with C9 and configuring it. I moved my email to G Suite and moved my WordPress to Digital Ocean (other guides here and here).

I was happy since I could now send emails via CLI/code, set up free SSL certs, add second domain email to G Suite and Secure G Suite. No more usage limit errors either.

Self-managing servers require more work but it is more rewarding (flexible, faster and cheaper).  Page load times were now near 20 seconds (10-second improvement).

Latency Issue

Over 6 months, performance on Digital Ocean (in Singapore) from Australia started to drop (mentioned here).  I tried upgrading the memory but that did not help (latency was king).

Moved the website to Australia

I moved my domain to Vultr in Australia (guide here and here). All was good for a year until traffic growth started to increase.

Blog Growth

I tried upgrading the memory on Vultr and I setup PHP child workers, set up Cloudflare.

GT Metrix scores were about a “B” and Google Page Speed Scores were in the lower 40’s. Page loads were about 14 seconds (5-second improvement).

Tweaking WordPress

I set up an image compression plugin in WordPress then set up a cloud image compression and CDN Plugin from the same vendor.  Page Speed info here.

GT Metrix scores were now occasionally an “A” and Page Speed scores were in the lower 20’s. Page loads were about 3-5 seconds (10-second improvement).

A mixed bag from Vultr (more optimisation and performance improvements were needed).

This screenshot is showing poor www.gtmetrix.com scores , pool google page speed index scores and upgrading from 1GB to 2GB memory on my server.

Google Chrome Developer Console Audit Results on Vultr hosted website were not very good (I stopped checking as nothing helped).

This is a screenshot showing poor site performance (screenshot taken in Google Dev tools audit feature)

The problem was the Vultr server (400km away in Sydney) was offline (my issue) and everything above (adding more memory, adding 2x CDN’s (EWWW and Cloudflare), adding PHP Child workers etc) did not seem to help???

Enter UpCloud…

Recently, a friend sent a link to a blog article about a host called “UpCloud” who promised “Faster than SSD” performance.  This can’t be right: “Faster than SSD”? I was intrigued. I wanted to check it out as I thought nothing was faster than SSD (well, maybe RAM).

I signed up for a trial and ran a disk IO test (read the review here) and I was shocked. It’s fast. Very fast.

Summary: UpCloud was twice as fast (Disk IO and CPU) as Vultr (+ an optional $4/m firewall and $3/m for 1x backup).

This is a screenshot showing Vultr.com servers getting half the read and write disk io performance compared to upcloud.com.

fyi: Labels above are K Bytes per second. iozone loops through all file size from 4 KB to 16,348 KB and measures through the reads per second. To be honest, the meaning of the numbers doesn’t interest me, I just want to compare apples to apples.

This is am image showing iozone results breakdown chart (kbytes per sec on vertical axis, file size in horizontal axis and transfer size on third access)

(image snip from http://www.iozone.org/ which explains the numbers)

I might have to copy my website on UpCloud and see how fast it is.

Where to Deploy and Pricing

UpCloud Pricing: https://www.upcloud.com/pricing/

UpCloud Pricing

UpCloud does not have a data centre in Australia yet so why choose UpCloud?

Most of my site’s visitors are based in the US and UpCloud have disk IO twice as fast as Vultr (win-win?).  I could deploy to Chicago?

This image sows most of my visitors are in the US

My site’s traffic is growing and I need to ensure the site is fast enough in the future.

This image shows that most of my sites visitors are hitting my site on week days.

Creating an UpCloud VM

I used a friend’s referral code and signed up to create my first VM.

FYI: use my Referral code and get $25 free credit.  Sign up only takes 2 minutes.

https://www.upcloud.com/register/?promo=D84793

When you click the link above you will receive 25$ to try out serves for 3 days. You can exit his trail and deposit $10 into UpCloud.

Trial Limitations

The trial mode restrictions are as following:

* Cloud servers can only be accessed using SSH, RDP, HTTP or HTTPS protocols
* Cloud servers are not allowed to send outgoing e-mails or to create outbound SSH/RDP connections
* The internet connection is restricted to 100 Mbps (compared to 500 Mbps for non-trial accounts)
* After your 72 hours free trial, your services will be deleted unless you make a one-time deposit of $10

UpCloud Links

The UpCloud support page is located here: https://www.upcloud.com/support/

More UpCloud links to read:

Signing up to UpCloud

Navigate to https://upcloud.com/signup and add your username, password and email address and click signup.

New UpCloud Signup Page

Add your address and payment details and click proceed (you don’t need to pay anything ($1 may be charged and instantly refunded to verify the card)

Add address and payment details

That’s it, check yout email.

Signup Done

Look for the UpCloud email and click https://my.upcloud.com/

Check Email

Now login

Login to UpCloud

Now I can see a dashboard 🙂

UpCloud Dashboard

I was happy to see 24/7 support is available.

This image shows the www.upcloud.com live chat

I opted in for the new dashboard

UpCloud new new dashboard

Deploy My First UpCloud Server

This is how I deployed a server.

Note: If you are going to deploy a server consider using my referral code and get $25 credit for free.

Under the “deploy a server” widget I named the server and chose a location (I think I was supposed to use an FQDN name -e.g., “fearby.com”). The deployment worked though. I clicked continue, then more options were made available:

  1. Enter a short server description.
  2. Choose a location (Frankfurt, Helsinki, Amsterdam, Singapore, London and Chicago)
  3. Choose the number of CPU’s and amount of memory
  4. Specify disk number/names and type (MaxIOPS or HDD).
  5. Choose an Operating System
  6. Select a Timezone
  7. Define SSH Keys for access
  8. Allowed login methods
  9. Choose hardware adapter types
  10. Where the send the login password

Deploy Server

FYI: How to generate a new SSH Key (on OSX or Ubuntu)

Output

Did the key export? (yes)

> /temp# ls /temp/ -al
> drwxr-xr-x 2 root root 4096 Jun 9 15:33 .
> drwxr-xr-x 27 root root 4096 Jun 8 14:25 ..
> -rw——- 1 user user 1766 Jun 9 15:33 example_rsa
> -rw-r–r– 1 user user 396 Jun 9 15:33 example_rsa.pub

“example_rsa” is the private key and “example_rsa.pub “is the public key.

  • The public key needs to be added to the server to allow access.
  • The private key needs to be added to any local ssh program used for remote access.

Initialisation script (after deployment)

I was pleased to see an initialization script section that calls actions after the server is deployed. I configured the initialisation script to pull down a few GB of backups from my Vultr website in Sydney (files now removed).

This was my Initialisation script:

Confirm and Deploy

I clicked “Confirm and deploy” but I had an alert that said trial mode can only deploy servers up to 1024MB of memory.

This image shows I cant deploy servers with 2/GB in trial modeExiting UpCloud Trial Mode

I opened the dashboard and clicked My Account then Billing, I could see the $25 referral credit but I guess I can’t use that in Trial.

I exited trial mode by depositing $10 (USD).

View Billing Details

Make a manual 1-time deposit of $10 to exit trial mode.

Deposit $10 to exit the trial

FYI: Server prices are listed below (or view prices here).

UpCloud Pricing

Now I can go back and deploy the server with the same settings above (1x CPU, 2GB Memory, Ubuntu 18.04, MaxIOPS Storage etc)

Deployment takes a few minutes and depending on how you specified a password may be emailed to you.

UpCloud Server Deployed

The server is now deployed; now I can connect to it with my SSH program (vSSH).  Simply add the server’s IP, username, password and the SSH private key (generated above) to your ssh program of choice.

fyi: The public key contents start with “ssh-rsa”.

This image shows me connecting to my sever via ssh

I noticed that the initialisation script downloaded my 2+GB of files already. Nice.

UpCloud Billing Breakdown

I can now see on the UpCloud billing page in my dashboard that credit is deducted daily (68c); at this rate, I have 49 days credit left?

Billing Breakdown

I can manually deposit funds or set up automatic payments at any time 🙂

UpCloud Backup Options

You do not need to setup backups but in case you want to roll back (if things stuff up), it is a good idea. Backups are an additional charge.

I have set up automatic daily backups with an auto deletion after 2 days

To view backup scheduled click on your deployed server then click backup

List of UpCloud Backups

Note: Backups are charged at $0.056 for every GB stored – so $5.60 for every 100GB per month (half that for 50GB etc)

You can take manual backups at any time (and only be charged for the hour)

UpCloud Firewall Options

I set up a firewall at UpCloud to only allow the minimum number of ports (UpCloud DNS, HTTP, HTTPS and My IP to port 22).  The firewall feature is charged at $0.0056 an hour ($4.03 a month)

I love the ability to set firewall rules on incoming, destination and outgoing ports.

To view your firewall click on your deployed server then click firewall

UpCloud firewall

Update: I modified my firewall to allow inbound ICMP (IPv4/IPv6) and UDP (IPv4/IPv6) packets.

(Note: Old firewall screenshot)

Firewall Rules Allow port 80, 443 and DNS

Because my internet provider has a dynamic IP, I set up a VPN with a static IP and whitelisted it for backdoor access.

Local Ubuntu ufw Firewall

I duplicated the rules in my local ufw (2nd level) firewall (and blocked mail)

UpCloud Download Speeds

I pulled down a 1.8GB Ubuntu 18.08 Desktop ISO 3 times from gigenet.com and the file downloaded in 32 seconds (57MB/sec). Nice.

Install Common Ubuntu Packages

I installed common Ubuntu packages.

Timezone

I checked the server’s time (I thought this was auto set before I deployed)?

I reset the time to Australia/Sydney.

Now the timezone is set 🙂

Shell History

I increased the shell history.

SSH Login

I created a ~/.ssh/authorized_keys file and added my SSH public key to allow password-less logins.

I added my pubic ssh key, then exited the ssh session and logged back in. I can now log in without a password.

Install NGINX

nginx/1.14.0 is now installed.

A quick GT Metrix test.

This image shows awesome static nginx performance ratings of of 99%

Install MySQL

Run these commands to install and secure MySQL.

Securing the MySQL server deployment.
> Would you like to setup VALIDATE PASSWORD plugin?: n
> New password: **********************************************
> Re-enter new password: **********************************************
> Remove anonymous users? (Press y|Y for Yes, any other key for No) : y
> Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y
> Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y
> Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y
> Success.

I disabled the validate password plugin because I hate it.

MySQL Ver 14.14 Distrib 5.7.22 is now installed.

Set MySQL root login password type

Set MySQL root user to authenticate via “mysql_native_password”. Run the “mysql” command.

Now let’s set the root password authentication method to “mysql_native_password”

Check authentication method.

Now we need to flush permissions.

Done.

Install PHP

Install PHP 7.2

PHP 7.2.5, Zend Engine v3.2.0 with Zend OPcache v7.2.5-1 is now installed. Do update PHP frequently.

I made the following changes in /etc/php/7.2/fpm/php.ini

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 20M
> post_max_size = 20M

Install PHP Modules

sudo apt-get install php-pear php7.2-curl php7.2-dev php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml

Install PHP FPM

Configure PHP FPM config.

Edit /etc/php/7.2/fpm/php.ini

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 20M
> post_max_size = 20M

Reload php sudo service.

Install PHP Modules

Configuring NGINX

If you are not comfortable editing NGINX config files read here, here and here.

I made a new “www root” folder, set permissions and created a default html file.

I edited the “root” key in “/etc/nginx/sites-enabled/default” file and set the root a new location (e.g., “/www-root”)

I added these performance tweaks to /etc/nginx/nginx.conf

> worker_cpu_affinity auto;
> worker_rlimit_nofile 100000

I add the following lines to “http {” section in /etc/nginx/nginx.conf

Check NGINX Status

Install Open SSL that supports TLS 1.3

This is a work in progress. The steps work just fine for me on Ubuntu 16.04. but not Ubuntu 18.04.?

Installing Adminer MySQL GUI

I will use the PHP based Adminer MySQL GUI to export and import my blog from one server to another. All I needed to do is install it on both servers (simple 1 file download)

Use Adminer to Export My Blog (on Vultr)

On the original server open Adminer (http) and..

  1. Login with the MySQL root account
  2. Open your database
  3. Choose “Save” as the output
  4. Click on Export

This image shows the export of the wordpress adminer page

Save the “.sql” file.

I used Adminer on the UpCloud server to Import My Blog

FYI: Depending on the size of your database backup you may need to temporarily increase your upload and post sizes limits in PHP and NGINX before you can import your database.

Edit /etc/php/7.2/fpm/php.ini
> max_file_uploads = 100M
> post_max_size =100M

And Edit: /etc/nginx/nginx.conf
> client_max_body_size 100M;

Don’t forget to reload NGINX config and restart NGINX and PHP. Take note of the maximum allowed file size in the screenshot below. I temporarily increased my upload limits to 100MB in order to restore my 87MB blog.

Now I could open Adminer on my UpCloud server.

  1. Create a new database
  2. Click on the database and click Import
  3. Choose the SQL file
  4. Click Execute to import it

Import MuSQL backup with Adminer

Don’t forget to create a user and assign permissions (as required – check your wp-config.php file).

Import MySQL Database

Tip: Don’t forget to lower the maximum upload file size and max post size after you import your database,

Cloudflare DNS

I use Cloudflare to manage DNS, so I need to tell it about my new server.

You can get your server’s IP details from the UpCloud dashboard.

Find IP

At Cloudflare update your DNS details to point to the server’s new IPv4 (“A Record”) and IPv6 (“AAAA Record”).

Cloudflare DNS

Domain Error

I waited an hour and my website was suddenly unavailable.  At first, I thought this was Cloudflare forcing the redirection of my domain to HTTP (that was not yet set up).

DNS Not Replicated Yet

I chatted with UpCloud chat on their webpage and they kindly assisted me to diagnose all the common issues like DNS values, DNS replication, Cloudflare settings and the error was pinpointed to my NGINX installation.  All NGINX config settings were ok from what we could see?  I uninstalled NGINX and reinstalled it (and that fixed it). Thanks UpCloud Support 🙂

Reinstalled NGINX

I reinstalled NGINX and reconfigured /etc/nginx/nginx.conf (I downloaded my SSL cert from my old server just in case).

Here is my /etc/nginx/nginx.conf file.

Here is my /etc/nginx/sites-available/default file (fyi, I have not fully re-setup TLS 1.3 yet so I commented out the settings)

SSL Labs SSL Certificate Check

All good thanks to the config above.

SSL Labs

Install WP-CLI

I don’t like setting up FTP to auto-update WordPress plugins. I use the WP-CLI tool to manage WordPress installations by the command line. Read my blog here on using WP-CLI.

Download WP-CLI

Move WP-CLI to the bin folder as “wp”

Test wp

Update WordPress Plugins

Now I can run “wp plugin update” to update all WordPress plugins

Update WordPress Core

WordPress core file can be updated with “wp core update“

Troubleshooting: Use the flag “–allow-root “if wp needs higher access (unsafe action though).

Install PHP Child Workers

I edited the following file to setup PHP child workers /etc/php/7.2/fpm/pool.d/www.conf

Changes

> pm = dynamic
> pm.max_children = 40
> pm.start_servers = 15
> pm.min_spare_servers = 5
> pm.max_spare_servers = 15
> pm.process_idle_timeout = 30s;
> pm.max_requests = 500;
> php_admin_value[error_log] = /var/log/www-fpm-php.www.log
> php_admin_value[memory_limit] = 512M

Restart PHP

Test NGINX config, reload NGINX config and restart NGINX

Output (14 workers are ready)

Memory Tweak (set at your own risk)

vm.swappiness = 1

Setting swappiness to a value of 1 all but disables the swap file and tells the Operating System to aggressively use ram, a value of 10 is safer. Only set this if you have enough memory available (and free).

Possible swappiness settings:

> vm.swappiness = 0 Swap is disabled. In earlier versions, this meant that the kernel would swap only to avoid an out of memory condition when free memory will be below vm.min_free_kbytes limit, but in later versions, this is achieved by setting to 1.[2]> vm.swappiness = 1 Kernel version 3.5 and over, as well as Red Hat kernel version 2.6.32-303 and over: Minimum amount of swapping without disabling it entirely.
> vm.swappiness = 10 This value is sometimes recommended to improve performance when sufficient memory exists in a system.[3]
> vm.swappiness = 60 The default value.
> vm.swappiness = 100 The kernel will swap aggressively.

The “htop” tool is a handy memory monitoring tool to “top”

Also, you can use good old “watch” command to show near-live memory usage (auto-refreshes every 2 seconds)

Script to auto-clear the memory/cache

As a habit, I am setting up a cronjob to check when free memory falls below 100MB, then the cache is automatically cleared (freeing memory).

Script Contents: clearcache.sh

I set the cronjob to run every 15 mins, I added this to my cronjob.

Sample log output

I will check the log (/scripts/clearcache.log) in a few days and view the memory trends.

After 1/2 a day Ubuntu 18.04 is handling memory just fine, no externally triggered cache clears have happened 🙂

Free memory over time

I used https://crontab.guru/every-hour to set the right schedule in crontab.

I rebooted the VM.

Update: I now use Nixstats monitoring

Swap File

FYI: Here is a handy guide on viewing swap file usage here. I’m not using swap files so it is only an aside.

After the system rebooted I checked if the swappiness setting was active.

Yes, swappiness is set.

File System Tweaks – Write Back Cache (set at your own risk)

First, check your disk name and file system

Take note of your disk name (e.g vda1)

I used TuneFS to enable writing data to the disk before writing to the journal. tunefs is a great tool for setting file system parameters.

Warning (snip from here): “I set the mode to journal_data_writeback. This basically means that data may be written to the disk before the journal. The data consistency guarantees are the same as the ext3 file system. The downside is that if your system crashes before the journal gets written then you may lose new data — the old data may magically reappear.“

Warning this can corrupt your data. More information here.

I ran this command.

I edited my fstab to append the “writeback,noatime,nodiratime” flags for my volume after a reboot.

Edit FS Tab:

I added “writeback,noatime,nodiratime” flags to my disk options.

Updating Ubuntu Packages

Show updatable packages.

Update Packages.

Unattended Security Updates

Read more on Ubuntu 18.04 Unattended upgrades here, here and here.

Install Unattended Upgrades

Enable Unattended Upgrades.

Now I configure what packages not to auto-update.

Edit /etc/apt/apt.conf.d/50unattended-upgrades

Find “Unattended-Upgrade::Package-Blacklist” and add packages that you don’t want automatically updated, you may want to manually update these (and monitor updates).

I prefer not to auto-update critical system apps (I will do this myself).

FYI: You can find installed packages by running this command:

Enable automatic updates by editing /etc/apt/apt.conf.d/20auto-upgrades

Edit the number at the end (the number is how many days to wait before updating) of each line.

> APT::Periodic::Update-Package-Lists “1”;
> APT::Periodic::Download-Upgradeable-Packages “1”;
> APT::Periodic::AutocleanInterval “7”;
> APT::Periodic::Unattended-Upgrade “1”;

Set to “0” to disable automatic updates.

The results of unattended-upgrades will be logged to /var/log/unattended-upgrades

Update packages now.

Almost done.

I Rebooted

GT Metrix Score

I almost fell off my chair. It’s an amazing feeling hitting refresh in GT Metrix and getting sub-2-second score consistently (and that is with 17 assets loading and 361KB of HTML content)

0.9sec load times

WebPageTest.org Test Score

Nice. I am not sure why the effective use of CDN has an X rating as I have the EWWW CDN and Cloudflare. First Byte time is now a respectable “B”, This was always bad.

Update: I found out the longer you set cache delays in Cloudflare the higher the score.

Web Page Test

GT Metrix has a nice historical breakdown of load times (night and day).

Upcloud Site Speed in GTMetrix

Google Page Speed Insight Desktop Score

I benchmarked with https://developers.google.com/speed/pagespeed/insights/

This will help with future SEO rankings. It is well known that Google is pushing fast servers.

100% Desktop page speed score

Google Chrome 70 Dev Console Audit (Desktop)

100% Chrome Audit Score

This is amazing, I never expected to get this high score.  I know Google like (and are pushing) sub-1-second scores.

My site is loading so well it is time I restored some old features that were too slow on other servers

  • I disabled Lazy loading of images (this was not working on some Android devices)
  • I re-added the News Widget and news images.

GTMetrix and WebpageTest sores are still good (even after adding bloat)

Benchmarks are still good

My WordPress site is not really that small either

Large website

FYI: WordPress Plugins I use.

These are the plugins I use.

  • Autoptimize – Optimises your website, concatenating the CSS and JavaScript code, and compressing it.
  • BJ Lazy Load (Now Disabled) – Lazy image loading makes your site load faster and saves bandwidth.
  • Cloudflare – Cloudflare speeds up and protects your WordPress site.
  • Contact Form 7 – Just another contact form plugin. Simple but flexible.
  • Contact Form 7 Honeypot – Add honeypot anti-spam functionality to the popular Contact Form 7 plugin.
  • Crayon Syntax Highlighter – Supports multiple languages, themes, highlighting from a URL, local file or post text.
  • Democracy Poll – Allows creating democratic polls. Visitors can vote for more than one answer & add their own answers.
  • Display Posts Shortcode – Display a listing of posts using the shortcode
  • EWWW Image Optimizer – Reduce file sizes for images within WordPress including NextGEN Gallery and GRAND FlAGallery. Uses jpegtran, optipng/pngout, and gifsicle.
  • GDPR Cookie Consent – A simple way to show that your website complies with the EU Cookie Law / GDPR.
  • GTmetrix for WordPress – GTmetrix can help you develop a faster, more efficient, and all-around improved website experience for your users. Your users will love you for it.
  • TinyMCE Advanced – Enables advanced features and plugins in TinyMCE, the visual editor in WordPress.
  • Wordfence Security – Anti-virus, Firewall and Malware Scan
  • WP Meta SEO – WP Meta SEO is a plugin for WordPress to fill meta for content, images and main SEO info in a single view.
  • WP Performance Score Booster – Speed-up page load times and improve website scores in services like PageSpeed, YSlow, Pingdom and GTmetrix.
  • WP SEO HTML Sitemap – A responsive HTML sitemap that uses all of the settings for your XML sitemap in the WordPress SEO by Yoast Plugin.
  • WP-Optimize – WP-Optimize is WordPress’s #1 most installed optimisation plugin. With it, you can clean up your database easily and safely, without manual queries.
  • WP News and Scrolling Widgets Pro – WP News Pro plugin with six different types of shortcode and seven different types of widgets. Display News posts with various designs.
  • Yoast SEO – The first true all-in-one SEO solution for WordPress, including on-page content analysis, XML sitemaps and much more.
  • YouTube – YouTube Embed and YouTube Gallery WordPress Plugin. Embed a responsive video, YouTube channel, playlist gallery, or live stream

How I use these plugins to speed up my site.

  • I use EWWW Image Optimizer plugin to auto-compress my images and to provide a CDN for media asset deliver (pre-Cloudflare). Learn more about ExactDN and EWWW.io here.
  • I use Autoptimize plugin to optimise HTML/CSS/JS and ensure select assets are on my EWWW CDN. This plugin also removes WordPress Emojis, removed the use of Google Fonts, allows you to define pre-configured domains, Async Javascript-files etc.
  • I use BJ Lazy Load to prevent all images in a post from loading on load (and only as the user scrolls down the page).
  • GTmetrix for WordPress and Cloudflare plugins are for information only?
  • I use WP-Optimize to ensure my database is healthy and to disable comments/trackbacks and pingbacks.

Let’s Test UpCloud’s Disk IO in Chicago

Looks good to me, Read IO is a little bit lower than UpCloud’s Singapore data centre but still, it’s faster than Vultr.  I can’t wait for more data centres to become available around the world.

Why is UpCloud Disk IO so good?

I asked UpCloud on Twitter why the Disk IO was so good.

  • “MaxIOPS is UpCloud’s proprietary block-storage technology. MaxIOPS is physically redundant storage technology where all customer’s data is located in two separate physical devices at all times. UpCloud uses InfiniBand (!) network to connect storage backends to compute nodes, where customers’ cloud servers are running. All disks are enterprise-grade SSD’s. And using separate storage backends, it allows us to live migrate our customers’ cloud servers freely inside our infrastructure between compute nodes – whether it be due to hardware malfunction (compute node) or backend software updates (example CPU vulnerability and immediate patching).“

My Answers to Questions to support

Q1) What’s the difference between backups and snapshots (a Twitter user said Snapshots were a thing)

A1) Backups and snapshots are the same things with our infrastructure.

Q2) What are charges for backup of a 50GB drive?

A2) We charge $0.06 / GB of the disk being captured. But capture the whole disk, not just what was used. So for a 50GB drive, we charge $0.06 * 50 = $3/month. Even if 1GB were only used.

  • Support confirmed that each backup is charged (so 5 times manual backups are charged 5 times). Setting up a daily auto backup schedule for 2 weeks would create 14 billable backup charges.
  • I guess a 25GB server will be $1.50 a month

Q3) What are data charges if I go over my 2TB quota?

A3) Outgoing data charges are $0.056/GB after the pre-configured allowance.

Q4) What happens if my balance hits $0?

A4) You will get notification of low account balance 2 weeks in advance based on your current daily spend. When your balance reaches zero, your servers will be shut down. But they will still be charged for. You can automatically top-up if you want to assign a payment type from your Control Panel. You deposit into your balance when you want. We use a prepaid model of payment, so you need to top up before using, not billing you after usage. We give you lots of chances to top-up.

Support Tips

  • One thing to note, when deleting servers (CPU, RAM) instances, you get the option to delete the storages separately via a pop-up window. Choose to delete permanently to delete the disk, to save credit. Any disk storage lying around even unattached to servers will be billed.
  • Charges are in USD.

I think it’s time to delete my domain from Vultr in Sydney.

Deleted my Vultr domain

I deleted my Vultr domain.

Delete Vultr Server

Done.

More Reading on UpCloud

https://www.upcloud.com/documentation/faq/

UpCloud Server Status

http://status.upcloud.com

Check out my new guide on Nixstats for awesome monitoring

What I would like

  1. Ability to name individual manual backups (tag with why I backed up).
  2. Ability to push user-defined data from my VM to the dashboard
  3. Cheaper scheduled backups
  4. Sydney data centres (one day)

Update: Post UpCloud Launch Tweaks (Awesome)

I had a look at https://www.webpagetest.org/ results to see where else I can optimise webpage delivery.

Optimisation Options

Disable dasjhicons.min.css (for unauthenticated WordPress users).

Find functions.php in the www root

Edit functions.php

Add the following

// Remove dashicons in frontend for unauthenticated users
add_action( 'wp_enqueue_scripts', 'bs_dequeue_dashicons' );
function bs_dequeue_dashicons() {
    if ( ! is_user_logged_in() ) {
        wp_deregister_style( 'dashicons' );
    }
}

HTTP2 Push

I added http2 to my listening servers

I tested a http2 push page by defining this in /etc/nginx/sites-available/default 

Once I tested that push (demo here) was working I then defined two files to push that were being sent from my server

I used the WordPress Plugin Autoptimize to remove Google font usage (this removed a number of files being loaded when my page loads).

I used the WordPress Plugin WP-Optimize plugin into to remove comments and disable pingbacks and trackbacks.

WordPress wp-config.php tweaks

Add 2FA Authentication to server logins.

I recently checked out YubiCo YubiKeys and I have secured my Linux servers with 2FA prompts at login. Read the guide here. I secured my WordPress too.

Tweaks Todo

  • Compress placeholder BJ Lazy Load Image (plugin is broken)
  • Solve 2x Google Analytics tracker redirects (done, switched to Matomo)

Conclusion

I love UpCloud’s fast servers, give them a go (use my link and get $25 free credit).

I love Cloudflare for providing a fast CDN.

I love ewww.io’s automatic Image Compression and Resizing plugin that automatically handles image optimisations and pre Cloudflare/first hit CDN caching.

Read my post about server monitoring with Nixstats here.

Let the results speak for themselves (sub <1 second load times).

Results

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

2020 Update. I have stopped using Putty and WinSCP. I now use MobaXterm (a tabbed SSH client for Windows) as it is way faster than WinSCP and better than Putty. Read my review post of MobaXTerm here.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v2.1 Newer GTMetrix scores

v2.0 New UpCloud UI Update and links to new guides.

v1.9 Spelling and grammar

v1.8 Trial mode gotcha (deposit money ASAP)

v1.7 Added RSA Private key info

v1.7 – Added new firewall rules info.

v1.6 – Added more bloat to the site, still good.

v1.5 Improving Accessibility

v1.4 Added Firewall Price

v1.3 Added wp-config and plugin usage descriptions.

v1.2 Added GTMetrix historical chart.

v1.1 Fixed free typos and added final conclusion images.

v1.0 Added final results

v0.9 added more tweaks (http2 push, removing unwanted files etc)

v0.81 Draft  – Added memory usage chart and added MaxIOPS info from UpCloud.

v0.8 Draft post.

I set the cronjob to run every 15 mins, I added this to my cronjob.

 

Sample log output

 

I will check the log (/scripts/clearcache.log) in a few days and view the memory trends.

After 1/2 a day Ubuntu 18.04 is handling memory just fine, no externally triggered cache clears have happened 🙂

Free memory over time

I used https://crontab.guru/every-hour to set the right schedule in crontab.

I rebooted the VM.

Update: I now use Nixstats monitoring

Swap File

FYI: Here is a handy guide on viewing swap file usage here. I’m not using swap files so it is only an aside.

After the system rebooted I checked if the swappiness setting was active.

 

Yes, swappiness is set.

File System Tweaks – Write Back Cache (set at your own risk)

First, check your disk name and file system

 

Take note of your disk name (e.g vda1)

I used TuneFS to enable writing data to the disk before writing to the journal. tunefs is a great tool for setting file system parameters.

Warning (snip from here): “I set the mode to journal_data_writeback. This basically means that data may be written to the disk before the journal. The data consistency guarantees are the same as the ext3 file system. The downside is that if your system crashes before the journal gets written then you may loose new data — the old data may magically reappear.“

Warning this can corrupt your data. More information here.

I ran this command.

 

I edited my fstab to append the “writeback,noatime,nodiratime” flags for my volume after a reboot.

Edit FS Tab:

 

I added “writeback,noatime,nodiratime” flags to my disk options.

 

Updating Ubuntu Packages

Show updatable packages.

 

Update Packages.

 

Unattended Security Updates

Read more on Ubuntu 18.04 Unattended upgrades here, here and here.

Install Unattended Upgrades

 

Enable Unattended Upgrades.

 

Now I configure what packages not to auto update.

Edit /etc/apt/apt.conf.d/50unattended-upgrades

Find “Unattended-Upgrade::Package-Blacklist” and add packages that you don’t want automatically updated, you may want to manually update these (and monitor updates).

I prefer not to auto-update critical system apps (I will do this myself).

 

FYI: You can find installed packages by running this command:

 

Enable automatic updates by editing /etc/apt/apt.conf.d/20auto-upgrades

Edit the number at the end (the number is how many days to wait before updating) of each line.

> APT::Periodic::Update-Package-Lists “1”;
> APT::Periodic::Download-Upgradeable-Packages “1”;
> APT::Periodic::AutocleanInterval “7”;
> APT::Periodic::Unattended-Upgrade “1”;

Set to “0” to disable automatic updates.

The results of unattended-upgrades will be logged to /var/log/unattended-upgrades

Update packages now.

 

Almost done.

I Rebooted

GT Metrix Score

I almost fell off my chair. It’s an amazing feeling hitting refresh in GT Metrix and getting sub-2-second score consistently (and that is with 17 assets loading and 361KB of HTML content)

0.9sec load times

WebPageTest.org Test Score

Nice. I am not sure why the effective use of CDN has an X rating as I have the EWWW CDN and Cloudflare. First Byte time is now a respectable “B”, This was always bad.

Update: I found out the longer you set cache delays in Cloudflare the higher the score.

Web Page Test

GT Metrix has a nice historical breakdown of load times (night and day).

Upcloud Site Speed in GTMetrix

Google Page Speed Insight Desktop Score

I benchmarked with https://developers.google.com/speed/pagespeed/insights/

This will help with future SEO rankings. It is well known that Google is pushing fast servers.

100% Desktop page speed score

Google Chrome 70 Dev Console Audit (Desktop)

100% Chrome Audit Score

This is amazing, I never expected to get this high score.  I know Google like (and are pushing) sub-1-second scores.

My site is loading so well it is time I restored some old features that were too slow on other servers

  • I disabled Lazy loading of images (this was not working on some Android devices)
  • I re-added the News Widget and news images.

GTMetrix and WebpageTest sores are still good (even after adding bloat)

Benchmarks are still good

My WordPress site is not really that small either

Large website

FYI: WordPress Plugins I use.

These are the plugins I use.

  • Autoptimize – Optimises your website, concatenating the CSS and JavaScript code, and compressing it.
  • BJ Lazy Load (Now Disabled) – Lazy image loading makes your site load faster and saves bandwidth.
  • Cloudflare – Cloudflare speeds up and protects your WordPress site.
  • Contact Form 7 – Just another contact form plugin. Simple but flexible.
  • Contact Form 7 Honeypot – Add honeypot anti-spam functionality to the popular Contact Form 7 plugin.
  • Crayon Syntax Highlighter – Supports multiple languages, themes, highlighting from a URL, local file or post text.
  • Democracy Poll – Allows to create democratic polls. Visitors can vote for more than one answer & add their own answers.
  • Display Posts Shortcode – Display a listing of posts using the shortcode
  • EWWW Image Optimizer – Reduce file sizes for images within WordPress including NextGEN Gallery and GRAND FlAGallery. Uses jpegtran, optipng/pngout, and gifsicle.
  • GDPR Cookie Consent – A simple way to show that your website complies with the EU Cookie Law / GDPR.
  • GTmetrix for WordPress – GTmetrix can help you develop a faster, more efficient, and all-around improved website experience for your users. Your users will love you for it.
  • TinyMCE Advanced – Enables advanced features and plugins in TinyMCE, the visual editor in WordPress.
  • Wordfence Security – Anti-virus, Firewall and Malware Scan
  • WP Meta SEO – WP Meta SEO is a plugin for WordPress to fill meta for content, images and main SEO info in a single view.
  • WP Performance Score Booster – Speed-up page load times and improve website scores in services like PageSpeed, YSlow, Pingdom and GTmetrix.
  • WP SEO HTML Sitemap – A responsive HTML sitemap that uses all of the settings for your XML sitemap in the WordPress SEO by Yoast Plugin.
  • WP-Optimize – WP-Optimize is WordPress’s #1 most installed optimisation plugin. With it, you can clean up your database easily and safely, without manual queries.
  • WP News and Scrolling Widgets Pro – WP News Pro plugin with six different types of shortcode and seven different types of widgets. Display News posts with various designs.
  • Yoast SEO – The first true all-in-one SEO solution for WordPress, including on-page content analysis, XML sitemaps and much more.
  • YouTube – YouTube Embed and YouTube Gallery WordPress Plugin. Embed a responsive video, YouTube channel, playlist gallery, or live stream

How I use these plugins to speed up my site.

  • I use EWWW Image Optimizer plugin to auto-compress my images and to provide a CDN for media asset deliver (pre-Cloudflare). Learn more about ExactDN and EWWW.io here.
  • I use Autoptimize plugin to optimise HTML/CSS/JS and ensure select assets are on my EWWW CDN. This plugin also removes WordPress Emojis, removed the use of Google Fonts, allows you to define pre-configured domains, Async Javascript-files etc.
  • I use BJ Lazy Load to prevent all images in a post from loading on load (and only as the user scrolls down the page).
  • GTmetrix for WordPress and Cloudflare plugins are for information only?
  • I use WP-Optimize to ensure my database is healthy and to disable comments/trackbacks and pingbacks.

Let’s Test UpCloud’s Disk IO in Chicago

Looks good to me, Read IO is a little bit lower than UpCloud’s Singapore data centre but still, it’s faster than Vultr.  I can’t wait for more data centres to become available around the world.

Why is UpCloud Disk IO so good?

I asked UpCloud on Twitter why the Disk IO was so good.

  • “MaxIOPS is UpCloud’s proprietary block-storage technology. MaxIOPS is physically redundant storage technology where all customer’s data is located in two separate physical devices at all times. UpCloud uses InfiniBand (!) network to connect storage backends to compute nodes, where customers’ cloud servers are running. All disks are enterprise-grade SSD’s. And using separate storage backends, it allows us to live migrate our customers’ cloud servers freely inside our infrastructure between compute nodes – whether it be due to hardware malfunction (compute node) or backend software updates (example CPU vulnerability and immediate patching).“

My Answers to Questions to support

Q1) What’s the difference between backups and snapshots (a Twitter user said Snapshots were a thing)

A1) Backups and snapshots are the same things with our infrastructure.

Q2) What are charges for backup of a 50GB drive?

A2) We charge $0.06 / GB of the disk being captured. But capture the whole disk, not just what was used. So for a 50GB drive, we charge $0.06 * 50 = $3/month. Even if 1GB were only used.

  • Support confirmed that each backup is charged (so 5 times manual backups are charged 5 times). Setting up a daily auto backup schedule for 2 weeks would create 14 billable backup charges.
  • I guess a 25GB server will be $1.50 a month

Q3) What are data charges if I go over my 2TB quota?

A3) Outgoing data charges are $0.056/GB after the pre-configured allowance.

Q4) What happens if my balance hits $0?

A4) You will get notification of low account balance 2 weeks in advance based on your current daily spend. When your balance reaches zero, your servers will be shut down. But they will still be charged for. You can automatically top-up if you want to assign a payment type from your Control Panel. You deposit into your balance when you want. We use a prepay model of payment, so you need to top up before using, not billing you after usage. We give you lots of chances to top-up.

Support Tips

  • One thing to note, when deleting servers (CPU, RAM) instances, you get the option to delete the storages separately via a pop-up window. Choose to delete permanently to delete the disk, to save credit. Any disk storage lying around even unattached to servers will be billed.
  • Charges are in USD.

I think it’s time to delete my domain from Vultr in Sydney.

Deleted my Vultr domain

I deleted my Vultr domain.

Delete Vultr Server

Done.

Check out my new guide on Nixstats for awesome monitoring

What I would like

  1. Ability to name individual manual backups (tag with why I backed up).
  2. Ability to push user defined data from my VM to the dashboard
  3. Cheaper scheduled backups
  4. Sydney data centres (one day)

Update: Post UpCloud Launch Tweaks (Awesome)

I had a look at https://www.webpagetest.org/ results to see where else I can optimise webpage delivery.

Optimisation Options

HTTP2 Push

I added http2 to my listening servers I tested a http2 push page by defining this in /etc/nginx/sites-available/default 

Once I tested that push (demo here) was working I then defined two files to push that were being sent from my server

2FA Authentication at login

I recently checked out YubiCo YubiKeys and I have secured my Linux servers with 2FA prompts at login. Read the guide here. I secured my WordPress aswel.

Performance

I used the WordPress Plugin Autoptimize to remove Google font usage (this removed a number of files being loaded when my page loads).

I used the WordPress Plugin WP-Optimize plugin into to remove comments and disable pingbacks and trackbacks.

Results

Conclusion

I love UpCloud’s fast servers, give them a go (use my link and get $25 free credit).

I love Cloudflare for providing a fast CDN.

I love ewww.io’s automatic Image Compression and Resizing plugin that automatically handles image optimisations and pre Cloudflare/first hit CDN caching.

Read my post about server monitoring with Nixstats here.

Let the results speak for themselves (sub <1 second load times).

More Reading on UpCloud

https://www.upcloud.com/documentation/faq/

UpCloud Server Status

http://status.upcloud.com

I hope this guide helps someone.

Free Credit

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

2020 Update. I have stopped using Putty and WinSCP. I now use MobaXterm (a tabbed SSH client for Windows) as it is way faster than WinSCP and better than Putty. Read my review post of MobaXTerm here.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v2.2 Converting to Blocks

v2.1 Newer GTMetrix scores

v2.0 New UpCloud UI Update and links to new guides.

v1.9 Spelling and grammar

v1.8 Trial mode gotcha (deposit money ASAP)

v1.7 Added RSA Private key info

v1.7 – Added new firewall rules info.

v1.6 – Added more bloat to the site, still good.

v1.5 Improving Accessibility

v1.4 Added Firewall Price

v1.3 Added wp-config and plugin usage descriptions.

v1.2 Added GTMetrix historical chart.

v1.1 Fixed free typos and added final conclusion images.

v1.0 Added final results

v0.9 added more tweaks (http2 push, removing unwanted files etc)

v0.81 Draft  – Added memory usage chart and added MaxIOPS info from UpCloud.

v0.8 Draft post.

Setup a dedicated Debian subdomain (VM), Install MySQL 14 and connect to it from a WordPress on a different VM

by

This is how I set up a dedicated Debian subdomain (VM), Installed MySQL 14 and connected to it from a WordPress installation on a different VM

Aside

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving fearby.com from Vultr to UpCloud.

Buy a domain name here

Domain names for just 88 cents!

Now on with the post.

Fearby.com

I will be honest, fearby.com is my play server where I can code, learn about InfoSec and share (It’s also my stroke rehab blog).

There is no faster way to learn than actually doing. The problem is my “doing” usually breaks the live site from time to time (sorry).

I really need to set up a testing environment (DEV-TEST-LIVE or GREEN-BLUE) server(s). GREEN-BLUE has advantages as I can always have a hot spare ready. All I need to do is toggle DNS and I can set the GREEN or BLUE server as the live server.

But first  I need to separate my database from my current fearby.com server and setup up a new web server. Having a Green and Blue server that uses one database server will help with near real-time production website switches.

Dedicated Database Server

I read the following ( Should MySQL and Web Server share the same server? ) at Percona Database Performance Blog. Having a separate database server should not negatively impact performance (It may even help improve speeds).

Deploy a Debian VM (not Ubuntu)

I decided to set up a Debian server instead of Ubuntu (mostly because of the good focus on stability and focus on security within Debian).

I logged into the UpCloud dashboard on my mobile phone and deployed a Debian server in 5 mins.  I will be using my existing how to setup Ubuntu on UpCloud guide (even though this is Debian).

TIP: Sign up to UpCloud using this link to get $25 free UpCloud VM credit.

Deploy Debian Sevrer

Deploy a Debian server setup steps:

  1. Login to UpCloud and go to Create server.
  2. Name your Server (use a fully qualified domain name)
  3. Add a description.
  4. Choose your data centre (Chicago for me)
  5. Choose the server specs (1x CPU, 50GB Disk, 2GB Memory, 2TB Traffic for me)
  6. Name the Primary disk
  7. Choose an operating system (Debian for me)
  8. Select an SSH Key
  9. Choose misc settings
  10. Click Deploy server

After 5 mins your server should be deployed.

After Deploy

Setup DNS

Login to your DNS provider and create DNS records to the new IP’s (IPv4 and IPv6) provided by UpCloud. It took DNS 12 hours to replicate to my in Australia.

Add a DNS record with your domain registra A NAMe = IPV4 and AAAA Name = IPv6

Setup a Firewall (at UpCloud)

I would recommend you set up a firewall at UpCloud as soon as possible (don’t forget to add the recommended UpCloud DNS IP’s and any whitelisted IP’s your firewall).

Block everything and only allow

  • Port 22: Allow known IP(s) of your ISP or VPN.
  • Port 53: Allow known UpCloud DNS servers
  • Port 80 (ALL)
  • Port 443 (ALL)
  • Port 3306 Allow your WordPress site and known IP(s) of your ISP or VPN.

Read my post on setting up a whitelisted IP on an UpCloud VM… as it is a good idea.

UpCloud thankfully has a copy firewall feature that is very handy.

Copy Firewall rules option at UpCloud

After I set up the firewall I SSH’ed into my server (I use vSSH on OSX buy you could use PUTTY).

I updated the Debian system with the following  command

Get the MySQL Package

Visit http://repo.mysql.com/ and get the URL of the latest apt-config repo deb file (e.g “mysql-apt-config_0.8.9-1_all.deb”). Make a temp folder.

Download the MySQL deb Package

Install the package

Update the system again

Install MySQL on Debian

Secure MySQL

You can secure the MySQL server deployment (set options as needed)

Install NGINX

I installed NGINX to allow Adminer MySQL GUI to be used

I ran these commands to install NGINX.

I edited my NGINX configuration as required.

  • Set a web server root
  • Set desired headers
  • Optimized NGINX (see past guides here, here and here)

I reloaded NGINX

Install PHP

I followed this guide to install PHP on Debian.

Install PHP FPM

Increase Upload Limits

You may need to temporarily increase upload limits in NGINX and PHP before you can restore a WordPress database. My feabry.com blog is about 87MB.

Add “client_max_body_size 100M;” to “/etc/nginx/nginx.conf”

Add the following to “/etc/php/7.2/fpm/php.ini”

  • post_max_size = 100M
  • upload_max_filesize = 100M

Restore a backup of my MySQL database in MySQL

You can now use Adminer to restore your blog to MySQL. Read my post here on Adminer here. I used Adminer to move my WordPress site from CPanel to a self-managed server a year ago.

First login to your source server and export your desired database then login to the target server and import the database.

Firewall Check

Don’t forget to allow your WordPress site’s 2x Public IP’s and 1x Private IP to access port 3306 in your UpCloud Firewall.

How to check open ports on your current server

Set MySQL Permissions

Open MySQL

I ran these statements to grant the user logging in on the nominate IP’s access to MySQL.

Reload permissions in MySQL

Allow access to the Debian machine from known IP’s

Edit “/etc/host.allow”

Additions (known safe IP’s that need access to this MySQL remotely).

Tell MySQL to listen on

Edit “/etc/mysql/my.cnf”

Added..

I guess you could change the port to something random???

Restart MySQL

Install a second local firewall on Debian

Install ufw

Do add the IP of your desired server or VPN to access SSH

Do add the IP of your desired server or VPN to access WWW

Now add the known IP’s (e.g any web servers public (IPv4/IPv6) or Private IP’s) that you wish to grant access to MySQL (e.g the source website that used to have MySQL)

Do add UpCloud DNS Servers to your firewall

Add all other rules as needed (if you stuff up and lock your self out you can login to the server with the Console on UpCloud)

Restart the ufw firewall

Prevent MySQL starting on the source server

Now we can shut down MySQL on the source server (leave it there just in case).

Edit “/etc/init/mysql.conf”

Comment out the line that contains “start on ” and save the file

and run

Reboot

Stop and Disable NGINX on the new DB server

We don’t need NGINX running now the database has been imported with Adminer.

Stop and prevent NGINX from starting up on startup.

Check to see if MySQL is Disabled

Yep

Test access to the database server in PHP code

Add to dbtest.php

<em>SELECT guid FROM wp_posts</em>()<br />
<ul><?php

//External IP (charged after quota hit)
//$servername = 'db.yourserver.com';

//Private IP (free)
//$servername = '10.x.x.x';

$username = 'username';
$password = '********your*password*********';
$dbname = 'database';

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);
// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
}

$sql = 'SELECT guid FROM wp_posts';
$result = $conn->query($sql);

if ($result->num_rows > 0) {
    // output data of each row
    while($row = $result->fetch_assoc()) {
        echo $row["guid"] . "<br>";
    }
} else {
    echo "0 results";
}
$conn->close();
?></ul>
Done

Check for open ports.

You can install nmap on another server and scan for open ports

Install nmap

Scan a server for open ports with nmap

You should see this on a server that has access to see port 3306 (port 3306 should not be visible by non-whitelisted IP’s).  Port 3shouldoudl not be seen via everyone.

You should see something like this on a server that has access to see port 80/443 (a web server)

I’d recommend you use a service like https://pentest-tools.com/network-vulnerability-scanning/tcp-port-scanner-online-nmap# to check for open ports.  https://hackertarget.com/tcp-port-scan/ is a great tool too.

https://www.infobyip.com/tcpportchecker.php is also a free port checker that you can use to verify individual closed ports.

Screeshot of https://www.infobyip.com/tcpportchecker.php

Hardening MySQL and Debian

Read: https://www.debian.org/doc/manuals/securing-debian-howto/ap-checklist.en.html

Configuring WordPress use the dedicated Debian VM

On the source server that used to have MySQL edit your wp-config.php file for WordPress.

Remove

define('DB_HOST', 'localhost');

add (read the update below, I changed the DNS IP to the Private IP to have free traffic)

//Oriinal localhost
//define('DB_HOST', 'localhost');

//New external host via DNS (Charged after quota hit)
//define('DB_HOST', 'db.fearby.com');

//New external host via Private IP (Free)
define('DB_HOST','10.x.x.x');

Restart NGINX

Restart PHP-FPM

Conclusion

Nice, I seem to have shaved off 0.3 seconds in load times (25% improvement)

1sec gtmtrix load time

Update: Using a Private IP or Public IP between WordPress and MySQL servers

After I released this blog post (version 1.0 with no help from UpCloud) UpCloud contacted me and said the following.

I will have updated my references in this post and replace the public IP address (that is linked to DNS record for db.fearby.com) and instead use the private ip address (e.g 10.x.x.x), your servers private IP address is listed against the public IPv$ and IPv6 address.

I checked that the local ufw firewall did indeed allow the private IP access to MySQL.

On my new Debian MySQL server, I edited the file /etc/mysql/my.cnf and changed the IP to the private IP and not the public IP.

Now it looked like

(10.x.x.x  my Debian servers private IP)

On my WordPress instance, I edited the file  /www-root/wp-config.php

I added the new private host

//Oriinal localhost
//define('DB_HOST', 'localhost');

//New external host via DNS (Charged after quota hit)
//define('DB_HOST', 'db.fearby.com');

//New external host via Private IP (Free)
define('DB_HOST','10.x.x.x');

(10.x.x.x  my Debian servers private IP)

Alos on Debian/MySQL ensure you have granted access to the private IP of the WordPress server

Edit /etc/host.allow

Add

Restart MySQL

TIP: Enable UpCloud Backups

Do setup automatic backups (and or take manual backups). Backups are an extra charge but are essential IMHO.

UpCloud backups

Troubleshooting

If you can’t access MySQL log back into MySQL

and run

Reboot

Lower Upload Limits

Don’t forget to lower file upload sizes in NGINX and PHP (e.g 2M) now that the database has been restored.

I hope this guide helps someone.

TIP: Sign up to UpCloud using this link to get $25 free UpCloud VM credit.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.6 Changed Public IP use to private IP use to ensure we are not charged when the serves sage goes over the quota

v1.5 Fixed 03 type (should have been 0.3)

v1.4 added disable nginx info

v1.3 added https://www.infobyip.com/tcpportchecker.php

v1.1 added https://hackertarget.com/tcp-port-scan/

v1.0 Initial Post

Set up a whitelisted IP on an UpCloud VM and WordPress using a VPN to get a static IP address

by

This is how I set up a whitelisted IP on an UpCloud VM and WordPress using a VPN to get a static IP address

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Before you begin

Take a backup of WordPress files + database and take a snapshot of your VM (see my UpCloud VM guide here).

Having a ready backup IS a good idea.

Screenshot of https://my.upcloud.com/server/details/

Why Whitelist

Whitelisting is not bulletproof but it is an important link in the security chain. Security is only as good or bad as the strength of your weakest link.

Using updated software, applying patches, using HTTPS, using a reliable host in a reliable location, using good passwords are equally important as IP filtering. Whitelisting IP’s goes a long way to ensuring you have least access privileges on connections.

Remember to scan your site with OWASP Zap, Qualys and Kali Linux too.

What IP’s are you going to Whitelist?

Q1) Does your ISP offer a static IP address (or a dynamic IP)?

My ISP does NOT provide a static IP by default (I can pay $20 a month for one (that’s too expensive)).

You can check your public IP by loading http://icanhazip.com/ (this will return your public IPV4 address).

Load https://ipv6.icanhazip.com/ to view your IPV6 IP (if you have one)

Q2) Do you need to whitelist IP addresses while on the go (Mobile)? If so I would recommend you whitelist a VPN’s IP or IP range.

Recently I had Apache web server auto-install and knock out my NGINX web server and I needed to login on a mobile device to investigate,  Luckily I whitelist my VPN’s IP and logged in from my mobile device and resolved the issue.

Use a VPN to get a static IP

If you don’t have a static IP or you want to connect to your site on the go (Mobile) you can set up a VPN and use their static IP

I was using http://cyberghostvpn.com/ to have a static IP but a server failure in Sydney caused my defined whitelisted IP to disappear so I change to https://protonvpn.com/ (as Cybergost were unable to provide known IP’s of VPN servers).

TIP: Don’t just whitelist one server, whitelist a few as you never know when a server will go down.

Here is a screenshot of the 1st VPN I tried (Cyberghost), Cyberghost VPN is connected to a specified server (Dallas).

Cyberghost VN screenshot connected to Dallas

I switched to ProtonVPN.

Here is a screenshot of ProtonVPN connected to a Switzerland server. Read more about Proton VPN here.

Screnshot of Protonvpn

I set Proton VPN to auto-start and connect to my desired server

Screenshot of Proton VNS startup settings

Proton VPN offered me a 7-day PLUS trial (All Countries, 5 devices, highest speed, secure core etc) after I started using the free version (3 countries, 1 device, speed low). I assume everyone gets the same PLUS trail offer.

You can view Proton plans and pricing here.

Ok, now that we know how to get a static IP, let’s configure some firewalls.

Network Firewall at UpCloud

I use the awesome UpCloud to hold my domains (read more about UpCloud performance here). You can log in to your UpCloud Dashboard and load the server list, click your server and then click Firewall and define firewalls.

Firewall: Open IPv4/IPv6 ports for:

  1. ICMP
  2. 53 (DNS)
  3. 80 (HTTP)
  4. 443 (HTTPS)

Only allow access to port 22 from whitelisted IP’s (or IP ranges)

Screenshot of UpCloud firewall screen at https://my.upcloud.com/server/details/

I like to set separate firewall rules for IPV4 and IPV6, for TCP or UDP and I limit rules to certain IP range and port.

Ubuntu Firewall

I also like to run a ufw firewall (more information on ufw) on my Ubuntu server (read this guide on securing Ubuntu in the cloud and running a Lynis audit).

Manually setup firewall rules in ufw.

Don’t forget t restart your firewall

Run a  local nmap scan to find open ports

Don’t be concerned if you see open ports from a local nmap scan (e.g port 22 or 3306), these are locally open.  We need to scan externally to see if these ports are opened.

Scan your site with an external nmap tool like pen-test-tools or here.

Screenshot of a public nmap scanYou should not have non-web based service ports freely open externally (web-based ports e.g 80 and 443 are ok)

Port 22 access should be whitelisted to select IP’s only. You should not have any database ports open externally.

Whitelisting WordPress Access

Download WordFence plugin for WordPress from https://www.wordfence.com/

Read more on downloading WordPress plugins from the command line here. Read my past Wordfence post here.

Once Wordfence is installed open the WordFence All Options screen  (/wp-admin/admin.php?page=WordfenceOptions).

Now you can add your static IP (or IP ranges) to the WordFence whitelist.

Picture of WordFence whitelist

Setup auto block for any non whitelisted Itryingng to login to /wp-login.php

I permanently ban any IP accessing my login page (there are many).

What to do with rejected IP connections?

Wordfence will block connections to WordPress. I’d suggest you setup fail2ban to block other unwanted connections at network level too.

Conclusion

You should now have a VM that will allow port 22 access by whitelisted IP’s and a WordPress that only allows logins from whitelisted IP’s.

Cons

  • If you forget to start your VPN you can’t log in to your VM via port 22 or log in to WordPress (excellent, this is by design).

Pros

  • Secure (need I say more)

I hope this guide helps someone.

Please consider using my UpCloud referral code and get $25 UpCloud VM credit for free when you signup to create a new VM.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 Added Proton plans link

v1.1 Added auto block WordFence option

v1.0 Initial Post

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4

Read Part 1, Part 2, Part 3 or Part 4

I ran the MySQL benchmark preparation command again (no problem this time).

sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=###################### prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Creating table 'sbtest'...
Creating 1000000 records in table 'sbtest'...

Test table and records created

Test Records Created

Now I can benchmark MySQL on my main server.

sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=################################# --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run

RAW Output

sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.

OLTP test statistics:
    queries performed:
        read:                            336210
        write:                           0
        other:                           48030
        total:                           384240
    transactions:                        24015  (400.09 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 336210 (5601.24 per sec.)
    other operations:                    48030  (800.18 per sec.)

Test execution summary:
    total time:                          60.0242s
    total number of events:              24015
    total time taken by event execution: 480.0242
    per-request statistics:
         min:                                  1.79ms
         avg:                                 19.99ms
         max:                                141.00ms
         approx.  95 percentile:              37.49ms

Threads fairness:
    events (avg/stddev):           3001.8750/27.36
    execution time (avg/stddev):   60.0030/0.01

Results

queries performed (in 60 seconds):

  • read: 336210
  • other: 48030
  • total: 384240

I decided to add an index to see if I can speed this query up (read the MySQL index page here). I added an index (in Adminer) on the columns “Id” and “pad” for the sbtest table in the test database

I restarted the MySQL process

mysql restart
[ ok ] Restarting mysql (via systemctl): mysql.service.

I ran the same benchmark again.

Raw Output

sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=########################## --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.

OLTP test statistics:
    queries performed:
        read:                            426538
        write:                           0
        other:                           60934
        total:                           487472
    transactions:                        30467  (507.69 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 426538 (7107.67 per sec.)
    other operations:                    60934  (1015.38 per sec.)

Test execution summary:
    total time:                          60.0110s
    total number of events:              30467
    total time taken by event execution: 479.9124
    per-request statistics:
         min:                                  5.75ms
         avg:                                 15.75ms
         max:                                138.57ms
         approx.  95 percentile:              25.10ms

Threads fairness:
    events (avg/stddev):           3808.3750/8.70
    execution time (avg/stddev):   59.9891/0.00

Results

The quick index added 20% extra throughput on queries 🙂

Mysql before and after an index

Don’t forget to delete your test database

DROP DATABASE `test`;

Viewing MySQL Index Usage (on the “test” database)

Query to show Index stats for a table ‘test’

SELECT
 OBJECT_SCHEMA as 'Database', OBJECT_NAME as 'Table', 
 INDEX_NAME as 'Index', 
 COUNT_STAR, 
 SUM_TIMER_WAIT,  MIN_TIMER_WAIT, AVG_TIMER_WAIT, MAX_TIMER_WAIT, 
 COUNT_READ, 
 SUM_TIMER_READ, MIN_TIMER_READ, AVG_TIMER_READ, MAX_TIMER_READ,  
 COUNT_FETCH, SUM_TIMER_FETCH, MIN_TIMER_FETCH, AVG_TIMER_FETCH, MAX_TIMER_FETCH
FROM 
 performance_schema.table_io_waits_summary_by_index_usage
WHERE 
 object_schema = 'test'

I can see the MySQL PRIMARY index is getting used 🙂

Index Summary

Read more in viewable query stats (columns) here.

Other System Information Tools

Show processor information

cat /proc/cpuinfo

Output

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 61
model name      : Virtual CPU a7769a6388d5
stepping        : 2
microcode       : 0x1
cpu MHz         : 2394.454
cache size      : 16384 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single kaiser fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 4788.90
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:

Memory Information

You can assign 512MB, 1GB, 2GB or more memory to a server on Vultr, Read my guide here on upgrading resources for Vultr VM’s here.

Only upgrade your server’s memory when server processes demand it, there is no need to pay for extra idle memory. Read my older guides on upgrading Digital Ocean and AWS servers.

I use the htop utility to monitor memory and processes. The memory usage will depend on how you have configured your server to use connection pools in code, MySQL or services.  Also what memory demands do you get in pean bandwidth times?

HTOP

You can check your server memory details on Ubuntu with this command

cat /proc/meminfo

Output

MemTotal:        2048104 kB
MemFree:           96176 kB
MemAvailable:     693072 kB
Buffers:          183476 kB
Cached:           526124 kB
SwapCached:            0 kB
Active:          1467220 kB
Inactive:         243228 kB
Active(anon):    1070464 kB
Inactive(anon):    27004 kB
Active(file):     396756 kB
Inactive(file):   216224 kB
Unevictable:        3652 kB
Mlocked:            3652 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                64 kB
Writeback:             0 kB
AnonPages:       1004504 kB
Mapped:           114664 kB
Shmem:             94192 kB
Slab:             192692 kB
SReclaimable:     171892 kB
SUnreclaim:        20800 kB
KernelStack:        3072 kB
PageTables:        20528 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1024052 kB
Committed_AS:    2424332 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    247808 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       67440 kB
DirectMap2M:     2029568 kB

Use Memory or Disk (Swap)

You can configure the use of Memory over Disk by configuring your/etc/sysctl.conf file (setting value “vm.swappiness”)

You can check your swap file settings by running the following command

cat /proc/sys/vm/swappiness
1

Or By running

sysctl vm.swappiness
vm.swappiness = 1

Set a new swap file value by editing /etc/sysctl.conf

sudo nano /etc/sysctl.conf

Set the following to use more ram over the swap disk.

vm.swappiness = 1

Read about swappiness values here: https://en.wikipedia.org/wiki/Swappiness

Service Performance

Performance (and allocated resources) depends on the demands of your operating system and installed software

What operating system do you have?

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:        16.04
Codename:       xenial

View NGINX Status, how much memory does it use?

/etc/init.d/nginx status
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-05-25 21:28:25 AEST; 1 weeks 3 days ago
     Docs: man:nginx(8)
 Main PID: #### (nginx)
    Tasks: 3
   Memory: 58.9M
      CPU: 33min 11.515s
   CGroup: /system.slice/nginx.service
           ├─#### nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─#### nginx: worker process
           └─#### nginx: cache manager process

PHP (and Child Worker) status how much memory does it use and how many child workers do you have? Read my add PHP child workers post here (and update to PHP 7.2 here)

sudo service php7.2-fpm status
● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-05-25 21:28:26 AEST; 1 weeks 3 days ago
     Docs: man:php-fpm7.2(8)
 Main PID: #### (php-fpm7.2)
   Status: "Processes active: 0, idle: 20, Requests: 75911, slow: 0, Traffic: 0.1req/sec"
    Tasks: 21
   Memory: 694.2M
      CPU: 20h 49min 45.132s
   CGroup: /system.slice/php7.2-fpm.service
           ├─ #### php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf)
           ├─ #### php-fpm: pool www-acc
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-acc
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           └─ #### php-fpm: pool www-usr

MySQL Status

sudo service mysql status
● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-05-25 21:28:27 AEST; 1 weeks 3 days ago
 Main PID: ##### (mysqld)
    Tasks: 35
   Memory: 405.9M
      CPU: 2h 17min 31.822s
   CGroup: /system.slice/mysql.service
           └─#### /usr/sbin/mysqld

Shared VM Hosts

One of the biggest impacts (after server latency) for your server is not the disk performance but the number of hosts/websites on the server who are also using the disk and server resources.

Reverse IP Lookup

I have 80 other web servers on my server (based on a reverse lookup).

I may move to a dedicated box when I can afford it.

Security

Above all else ensure that security is number 1 priority and make performance second priority.

Scan your site with Zap, Qualys and Kali Linux. Performance means nothing if you are hacked.

website-report

Simulated concurrent users

You can use Siege to test the maximum concurrent users accessing your site before the server starts to drop connections.

FYI: If you use Cloudflare (you should) this may not work as it will block connections.

Install Siege

sudo apt-get install siege

Test  your server with 10 concurrent serves for 1 minute

siege -t1m c10 'https://yourserver.com/'

Results

siege -t1m c10 'https://yourserver.com/'
** SIEGE 3.0.8
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege...      done.

Transactions:                    417 hits
Availability:                 100.00 %
Elapsed time:                  59.01 secs
Data transferred:               8.24 MB
Response time:                  1.62 secs
Transaction rate:               7.07 trans/sec
Throughput:                     0.14 MB/sec
Concurrency:                   11.46
Successful transactions:         417
Failed transactions:               0
Longest transaction:            2.26
Shortest transaction:           1.49

Keep upping the connections (from 10 above) to a limit where connections start dropping.

I tried 25 then 50 concurrent users hitting a server on Digital Ocean and it did not fail.

Conclusion

  • Choose a server near your customers
  • Change hosts if one is faster and cheaper
  • Measure or benchmark your server (and compare over time).
  • Use Cloudflare

Create your own server today

  • Create your own server on Vultr here.
  • Create your own server on Digital Ocean here.
  • Create your own server on UpCloud here.

And remember you can install the Runcloud server management dashboard here.

I hope this guide helps someone.

< Previous

Read Part 1, Part 2, Part 3 or Part 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

Read Part 1, Part 2, Part 3 or Part 4

Measure Disk Performance with Bonnie++

Installing Bonnie++ on Ubuntu

apt-get install bonnie++

Read this. post on using Bonnie++

Benchmark disk IO with DD and Bonnie++

Starting Bonnie++

bonnie++ -d /tmp -r 2048 -u username

Bonnie++ Readme.

Disk io with bonnie++ on Vultr/Sydney

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 656 99 308954 68 113706 33 1200 92 188671 30 10237 251
Latency 26067us 119ms 179ms 29139us 26069us 16118us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1463us 703us 880us 263us 119us 593us
1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us

Disk io with bonnie++ on Digital Ocean/London

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 699 99 778636 74 610414 60 1556 99 1405337 59 +++++ +++
Latency 17678us 10099us 17014us 7027us 3067us 2366us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1243us 376us 611us 108us 59us 181us
1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us

Disk io with bonnie++ on UpCloud/Singapore

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 1014 99 407179 24 366622 32 2137 99 451886 17 +++++ +++
Latency 11297us 54232us 16443us 4949us 44883us 1595us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 264us 340us 561us 138us 66us 327us
1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us

Now read this site on how to make sense of this data

< Previous – Next >

Read Part 1, Part 2, Part 3 or Part 4

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4

by

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4. Update: I moved my domain to UpCloud.

Update (June 2018): I moved my domain to UpCloud (they are that awesome). Use this link to signup and get $25 free credit. Read the steps I took to move my domain to UpCloud here.

Upcloud Site Speed in GTMetrix

Comparing Digital Ocean/Vultr and UpCloud Disk IO

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean (all in the search of extra performance) but how do you know when a server performance is ok apart from running GT Metrix and other external site benchmarking tools.

This post is split up as it was too long.

Read Part 1, Part 2, Part 3 or Part 4

Spoiler: It all depends on where your server is located and what you do with it (Tweaks will improve the performance).

P.S This is NOT a paid endorsement or conclusive test (just a quick benchmark/review).

What does your server do?

You need to know what your server does 24/7 and what resources the services need.

I use htop to view real-time and historical usage data for each process.

htop

Tweaking Advice

A friend gave me good advice re-tweaking a cheap host to get good performance

I am not a fan of just throwing more money at a host and expecting better performance. Host have unique features and cons., there is no shortage of hosts or host cons.

How can you run synthetic benchmarks to determine comparable performance metrics?

WARNING: Comparing synthetic benchmarks can be far removed from real-world speeds. Benchmark results below were from 3 different servers I have on 3 different hosts in three different locations (the only thing the same was the use of Ubuntu 16.04 $5/m servers). These results are not scientific and should not be used to compare host providers. Benchmark runs were one-off (not averages over multiple timezones/days).

Disk Performance

Speaking of disk performance I noticed this the other day on the RunCloud blog. Faster than SSD (UpCloud)?

UpCloud Faster-than-SSD Cloud Hosting Server (Promo Code Inside)

Runcloud is a server management console that can interface with your domains (read my old review here).  I don’t use Runcloud but it is great for those who need a GUI to help manage VM via a dashboard. However, I prefer to know what is going on under the hood. I have investigated webmin in the past though.

Let’s do a quick IO benchmark test between UpCloud, Digital Ocean and Vultr on similarly low end $5/m servers,

Good advice on command line benchmarking tools from a friend.


Installing iozone to test disk performance

I searched for a post on using iozone (Thanks thegeekstuff).  I will be reviewing the “Writer report” and “Reader report”. Read more about iozone here.

View the iozone page for how to break down results.

iozone results breakdown

(image snip from http://www.iozone.org/)

Install iozone on Ubuntu

sudo apt-get install iozone3

Run an iozone disk test and output the results to a spreadsheet.

iozone -a -b iozone.xls

Now let’s run a Read/Write test on Vultr/Digital Ocean and UpCloud. Multiple runs were not performed, this is not a scientific test (just a simple benchmark test (as is, ignoring sever load and local infrastructure/timezone load)).

iozone Benchmark results for Vultr “Read” (Sydney)

 “4”  “8”  “16”  “32”  “64”  “128”  “256”  “512”  “1024”  “2048”  “4096”  “8192”  “16384”
64 2133730 3363612 4274062 4564786 6421025
128 2248149 3536566 4135958 7082197 4135958 11720614
256 1884399 2699161 3879045 3667079 5971678 5687020 5687020
512 3140488 3736016 3684733 4262523 4610256 2638816 5067142 5684095
1024 1617808 1939207 3411938 3999762 4048778 4614246 3083680 5885083 6609617
2048 1926510 2569678 4423683 4997618 3937075 459605 2896324 3542524 4971585 4707314
4096 1701683 2151300 4209920 5001700 4751325 4869845 5389246 3647681 4928521 6207035 4347346
8192 2063424 2329346 3203763 2937280 3221485 3232699 3626431 3650706 3789200 4110603 3715045 4350542
16384 1738553 2778362 3397613 3679205 3693442 3171501 3524291 3393586 3004024 3552531 3456574 2693845 2488861
32768 0 0 0 0 2952894 3537153 3574875 3768155 4719613 3890280 3394995 2735222 2542914
65536 0 0 0 0 4057489 3610789 3619967 3800078 3275327 3591212 3607188 1770426 2826659
131072 0 0 0 0 3552270 1890742 5275167 3727339 3527607 1753893 3234736 2341111 1378601
262144 0 0 0 0 3798586 1302021 1491429 3712825 3228816 3757963 3715510 2592485 2481061
524288 0 0 0 0 2758756 2487923 3705741 1807328 2118309 3675988 3196367 3394330 2396842

iozone Benchmark results for Digital Ocean “Read” (London)

4  “8”  “16”  “32”  “64”  “128”  “256”   “512”   “1024”   “2048”  “4096”   “8192”   “16384”
64 4564786 7100397 9006179 10402178 12902017
128 4717434 7082197 8548124 9795896 10567140 10779307
256 4840911 7073132 8271916 9868433 10148242 10651598 1E+07
512 4742616 6909408 8140399 9304292 9638369 10044089 1E+07 10044089
1024 4249053 5917516 6208343 7537599 9300377 10454984 7E+06 7113161 9946527
2048 3885431 6967792 6603549 6845629 10401883 9808036 9E+06 7903836 9308497 7817519
4096 2506983 5953231 6263611 6953144 7774379 6225028 6E+06 8081580 7683972 8081580 8240513
8192 3665114 4850463 5479317 6141364 6277120 6108608 6E+06 6569983 5732541 7166033 6633402 5479317
16384 3673501 4828584 5416182 6187150 6614761 6298872 6E+06 6430310 5984033 6402750 6046159 4791883 3405527
32768 0 0 0 0 4692542 6140929 6E+06 6295642 5231224 6545707 5781108 4513475 3702577
65536 0 0 0 0 6315430 5830131 6E+06 6444695 6219125 6473838 5338595 4248118 3679324
131072 0 0 0 0 6130002 6461496 6E+06 5958068 5983423 6387547 6138078 3994888 3602079
262144 0 0 0 0 6456746 6323727 6E+06 6504146 6390176 6486151 6433963 3955165 3654188
524288 0 0 0 0 1667337 6381456 6E+06 6445708 6448714 6421071 5981200 4155185 3770740

iozone Benchmark results for UpCloud “Read” (Singapore)

 “4”  “8”  “16”  “32”  “64”  “128”  “256”  “512”  “1024”  “2048”  “4096”  “8192”  “16384”
64 6421025 6421025 10821524 12902017 15972885
128 4889281 6406138 9129573 10779307 14200794 14200794
256 5320671 3879045 10758322 8815202 10245071 12812277 12228612
512 4305250 5115422 8844453 8234036 7091952 8394979 7540170 10235583
1024 4339202 4762630 5821271 6163794 6819511 4674510 6479979 8183918 10230845
2048 4204968 5319484 5800851 5816563 6243566 6378005 5953632 6851089 7940367 8229438
4096 4526013 5556581 4817948 5404504 7301864 5759634 5810280 6007355 6919538 8620945 6281934
8192 4298295 5019093 5927357 6036702 6781341 6082655 5855636 6527546 6553692 6792065 6466126 4437634
16384 4282172 5849558 6313919 6635840 6741958 6657054 6423097 5536622 6558575 6442970 4527032 3784777 3901898
32768 0 0 0 0 5825460 5423408 6504198 6665385 6365329 6426343 5263076 3718605 3705971
65536 0 0 0 0 6908075 6623116 6493259 6609738 6311805 6483610 5489674 4035982 3561526
131072 0 0 0 0 5650180 5718949 2465429 5391253 3495911 5784844 5367408 3733490 3582175
262144 0 0 0 0 6814627 6691250 6189661 5906786 6081645 5799913 5247919 4121250 3637601
524288 0 0 0 0 6404764 6309263 5673979 5751609 6288245 6305103 5978680 3911984 3767116

iozone Benchmark results for Vultr “Write” (Sydney)

 “4”  “8”  “16”  “32”  “64”  “128”  “256”  “512”  “1024”  “2048”  “4096”  “8192”  “16384”
64 289322 532815 507625 429630 566551
128 398921 465304 434078 417212 669577 821147
256 530031 613985 820398 474937 891956 815414 370025
512 387576 754083 709019 819085 702295 609421 924123 496091
1024 297233 448522 716089 923488 854073 817340 1203137 1072453 601636
2048 408697 634655 695383 1358134 549657 1295458 821154 797520 964207 258493
4096 236150 433804 1215774 1245025 820832 809958 1371339 914269 921083 1004682 1481431
8192 611113 666677 806286 715219 779825 824294 875947 870091 1046378 791192 1023592 453248
16384 435454 706149 718313 845499 893495 888068 812778 842885 820591 941120 839610 862672 406590
32768 0 0 0 0 465196 786067 938881 627294 890917 968147 872369 871329 842843
65536 0 0 0 0 515057 790172 937568 915601 897235 867197 907562 852002 743856
131072 0 0 0 0 501091 480492 813147 870886 880239 805333 684630 1117578 633185
262144 0 0 0 0 387126 323185 323656 473258 405744 369599 422554 468992 453563
524288 0 0 0 0 325588 380450 392965 451608 303255 355148 386250 432054 416512

iozone Benchmark results for Digital Ocean “Write” (London)

 “4”  “8”  “16”  “32”  “64”  “128”  “256”  “512”  “1024”  “2048”  “4096”  “8192”  “16384”
64 831569 566551 1279447 1363961 1392258
128 652488 1319723 1421023 990891 1663139 1561553
256 1185399 1152323 1534342 1598292 1826695 1707589 1514860
512 1166599 1296159 1399189 1620980 1620980 1361920 1589779 1672748
1024 1079190 1321200 1584972 1917562 1592612 1701108 1718120 1462960 1643814
2048 1210394 1470172 1621719 1550584 1796378 1643753 1713598 1759581 1649117 1488257
4096 916513 1287575 1574718 1406594 1742237 1734148 1652418 1583280 1599346 1661045 1533532
8192 1109745 1318748 1178567 1544201 1502340 1371492 1466747 1499521 1479759 1564878 1291292 1347609
16384 1106205 1282084 1374037 1503649 1429398 1461407 1496119 1578132 1547289 1333431 1203371 1198815 1501316
32768 0 0 0 0 1270914 1406589 1513114 1468226 1558303 1552038 1516336 1443280 1440360
65536 0 0 0 0 1319322 1327984 1311504 1411955 1266988 1359645 1386446 1347092 1368295
131072 0 0 0 0 1100658 1229326 1227197 1318631 1265552 1233306 1227747 1237896 1233502
262144 0 0 0 0 1167160 1064078 1155828 1185185 1086152 1193673 1080872 1062611 1141960
524288 0 0 0 0 977835 1124816 1052757 1219183 1128972 1140177 1091954 1141635 1132063

iozone Benchmark results for UpCloud “Write” (Singapore)

 “8”  “16”  “32”  “64”  “128”  “256”  “512”  “1024”  “2048”  “4096”  “8192”  “16384”
64 1143223 1255511 1562436 1452528 1279447
128 1451764 1406136 1543594 1504659 1852520 1749872
256 1642294 1829808 1970871 1855098 1802167 1952947 2000242
512 1537424 1854787 1801873 2294796 1983258 2124526 1895721 1417662
1024 1434138 1553442 1609925 1931359 2098375 2044438 1872419 1768345 1892218
2048 1562145 1901771 1817281 1848169 1967097 1296240 2267786 2081497 1915768 2007554
4096 1625372 1966378 1924741 1342092 1950306 2078175 1914873 1459656 1995152 2102849 1326855
8192 1444062 1808330 1956503 1924397 2127300 2042328 2135630 1986478 2062557 2061319 1337016 1812049
16384 1667066 1820248 1898495 2051339 2012530 2111080 2119806 1491217 2060875 1974254 1934789 1815823 1921911
32768 0 0 0 0 2057506 1454537 2075621 2070899 1869795 2052896 1892347 1855382 1873440
65536 0 0 0 0 2067127 2077673 2088994 2179809 2087471 2099108 1904723 1642505 1832204
131072 0 0 0 0 1234663 1824959 1304340 1775514 1287481 1560379 1631992 1085609 1675467
262144 0 0 0 0 685774 808487 823824 662524 681762 548308 814946 645663 732176
524288 0 0 0 0 547296 517384 503422 521173 538714 518429 528950 529593 512944

Here is my quick unscientific take on a one-pass benchmark results above.

Vultr (Read) Vultr (Write) Digital Ocean (Write) UpCloud (Read) UpCloud (Write)

These results need some decoding.

Next >>

Read Part 1, Part 2, Part 3 or Part 4

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Update (June 2018): I moved my domain to UpCloud (they are that awesome). Use this link to signup and get $25 free credit. Read the steps I took to move my domain to UpCloud here.

Upcloud Site Speed in GTMetrix

Revision History

v1.2 added the fact that I Moved to UpCloud.

v1.1 Re ran iozone -a -b iozone.xls on all servers.

v1.0 Initial post

Moving an Ubuntu 16.04 VM on Vultr from one data centre to another via snapshots

by

This guide will show how you can move an Ubuntu VM server domain between Vultr data centres via snapshots.

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Sometimes you need to move a sever between locations and/or upgrade the server (to have more memory t install WordPress).

Moving an existing Vultr server

If you don’t have an Ubuntu server click here (follow this guide).

Login to Vultr and specify a source server, click Snapshots and click Take Snapshot.

Make snapshot

Wait for the snapshot to finish (It may take 1 hour).

Snapshot Started

Great, the snapshot is done.

Snapshot Ready

Now I can create a new server (in a different data centre).

Add

Deploy New Instance

Choose a location (Australia is at capacity, so I’ll deploy to Silicon Valley then move again in a few weeks), choose the snapshot to restore, choose a plan, I enabled IPV6/Auto Backups and Private Networking.

TIP: The password for the server will be the same as the source server so write it down.

Deploy

Click Deploy Now

Deploy

After a few minutes, you can see the new servers IP address, you can log in to your domain name provider (in my case Namecheap) and update the target IPV4 and IPV6 address.

You can find IPV4 and IPV6 addresses by opening your server, clicking settings then IPxV4 or IPV6.

ip

You will need to update Vultr DNS settings (login to Vultr, Click Servers, Click DNS then edit your existing Domain DNS entry).  Add you’re new serves IP addresses.

Vultr DNS

Update: I added an IPV6/AAAA record too.

Wait for DNS Replication

Goto https://www.whatsmydns.net/ and check the global DNS propagation for your new domain’s server.

DNS Propigation

If you are happy that the server has been migrated (snapshot restored) and that the domain DNS is pointing to your new server you can delete the old server in the Vultr server list.

Servers

Post-Migrate Actions

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Vultr Link

v1.0 Initial post

How to purchase your own domain name and set up a web server from $5 a month

by

This guide will show technically minded people how you can purchase your own domain name, set up a web server on Vultr with an online store using WordPress/WooCommerce from $5 a month. Warning this post is technical (if you have never used SSH, Ubuntu, Linux Command Line, hate risk or are not patient then this is NOT the guide you are after).

I personally recommend (not a paid endorsement) the free WooCommerce plugin for the free WordPress.org CMS on the free Ubuntu Operating system with the free NGINX web server and the free MYSQL database engine and free SSL certificates from Lets Encrypt.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Sorry for using the word free a lot but I like free things.  One of the benefits of a using a self-managed server is you get the option to install free software and configure the server how you want and secure it how you want. Truth be told managed ho (e.g CPanel, etc) are in the business of making money via monthly feed, expensive SSL certificates, taxing your transactions or pushing you to higher-priced tiers.

Legend:

  • Self Managed Server = A server that you create, you configure patch and support (all the reward and risk is owned by you and costs are low).
  • Hosted Server = A server you have partial control of and the hosts manage the server and support (You hand away all risk and most of the control and pay for support/features).

I moved to a self-managed server after I was paying $25/m for a poorly performing website and $150/y for a poor quality  SSL certificate and a slice of a server that seemed to always say “Usage Limit Exceeded”. Why pay for an insecure website that my visitors could not view because the usage limit was exceeded.

Bad CPanel SSL Certificate

fyi: Fearby.com costs me $10 a month for a server and $5/m for CDN abilities.

CPanel hosts are an option when you don’t want to self-manage a service and take on the hassle but be prepared for server limitations (The image below was taken on an older CPanel based hosts before I moved to a self-managed Vultr server)

cpenal_usage_exceeded

I recently discovered a well known and established website hosting service (that I used to use) and a friend is still using is insecure. My friend’s site has a static website on it but the server underneath was very old and insecure. Having a secure web server should be at the top of your list with any self-managed or hosted website (this will help search engine optimization and prevent risks to your website visitors).

Static Website

Sites like Virus Total, SSL Labs and Alexa Site Info , Qualys are good ways to review a site’s credibility.

fyi: The awesome https://seositecheckup.com/ is awesome for evaluating our sites SEO score.

Before we set up a server with WordPress on your own server let’s quickly look at the alternative commercial ready to go website builders.

Alternative (paid) DIY Website Builders

The following leading commercial sites will allow you to build a site online.

In my opinion, five things matter with setting up site online website.

  • Setup Costs, Monthly Cost and Commissions (what are the hidden charges)
  • Security (having a food SSL Certificate is key to having a good organic traffic from search engines)
  • Site Speed (Having a slow site will impact search engine optimization and drive visitors away)
  • Accessibility (if your site is not WCAG accessible it will not rank high on search engines).
  • Control (will you be able to do everything you want too, nothing worse than going so far and being limited)

Ok, let’s see how much it will cost to set up a simple business site on the sites above.

Wix 

Setup: Goto https://www.wix.com/, Login to Wix, click Create Site, click Business, Click Choose a Template, Edit the page, Click Save, Click “Connect your own customized domain“, Click “Connect a domain you already own“.

I was redirected to a Wix plan pricing page where I need to choose a plan to continue. From what I researched you cant control HTML on Wix so can’t add a MailChimp newsletter signup form so you would have to go with the $24.5/m option to enable Email Campaigns.

Wix Plans Chooser

I could not see information about included SSL certificates, SEO or other chargers.  SSL is free after you pay right?

The Wix editor appears OK (it may take a bit of learning though).

Wix Editoer

I clicked publish and the site was live

Wix site published

A quick check of the SSL, Accessibility and SEO and no obvious deal breakers here apart from the price and platform lock-in.

Wix Checkup

I performed a security check on the site with https://freescan.qualys.com (passed)

Conclusion: I hear Wix templates are hard to change so choose your template wisely, A large collection of apps are available that you can add to the site.

Although Wix was nice and it does include a full-featured look at the engine it is not for me ($24/m USD is too expensive).

Squarespace 

Squarespace basic websites cost $16/$25 a month or $34/52 for online stores: https://www.squarespace.com/pricing/

SquareSpace Pricing

Setup a Squarespace website: Goto https://www.squarespace.com/, Click Start a Free Trial, Choose a Template, Create an Account (a quick read of the terms of service and privacy policy, #scary), SpareSpace sites are pre-published?

Square Space Build

Loading the webpage on a non-logged-in (with SquareSpace login) browser displays a trial warning.  Trial pages are essentially restricted (unlike Wix).

Login Challenge

The mobile view does not match the template?  I guess the chosen template is more of a vibe and not a template.

Mobile view

Setting up a Squarespace website may take some time. Squarespace does have some nifty advance options in a slide-out menu though.

Squarespace Settings

Because the public view of the page is restricted I cannot scan it with WCAG accessibility tools. Scanning the site performance speeds with gtmetrix also fails.

Performance Rejected

Squarespace is well known to be difficult to set up a website when compared to other drag and drop editors (but Squarespace sites do look nice).

I am not paying $54/m for a website so let’s move on.

Shopify

Shopify Setup: Goto https://shopify.com and click Create, Sign up and enter your store name. Complete the wizard. 

Shopify

Choose a Shopify Plan

Shopify Plan

Scalping transactions, no thanks. let’s move on.

Weebly

Weebly Setup: Goto https://www.weebly.com/au and click Get Started under Create Store. Enter your account details and click Create Your Site, enter the name of the store, Click I’m just trying Weebly, click the type of product you will be selling.

Weebly Site Setup

Weebly Setup

Theme Selection

Theme Select

Choose a Domain

Domain Select

Publish the site

Publish

Clicking publish appears to be a dead end.

Verify Weebly

“Please contact Weebly Support to verify your account”, No Thanks, let’s move on.

One candidate remains and that is WordPress hosted (wordpress.com not wordpress.org).

WordPress.com

WordPress.com offer hosted plans for WordPress in the cloud.

Setup a WordPress site, the only one that removes WordPress branding and allows third-party plugins to be installed it the Business plans for $33 a month.

WordPress Plans

Setup Basics

Wordpress

Choose a WordPress theme.

Choose Theme

Assign a Domain

WordPress Domain

In order to buy a domain, you need to log in (top right) with an account

My working WordPress account (is no longer working), it was in my password manager.

wordpresscomerror

I seem to be stuck in a signup loop

Wordpress

Time to move on. Time to set up my own server on Vultr and setup WordPress and  WooCommerce,

But, before we do, let’s ensure our name is secure online.

Search for your Name/Brand

Do search for your website (or thing) in search engines to see if your name is already taken, don’t buy a domain that is owned or has IP or trademark presence. It is a  good idea to use sites like https://namechk.com/ to see if your site or social media is already taken.

https://namechk.com/

namechk.com will allow you to search for name availability online.  The name “mything” is not fully available online.

https://namechk.com/ 2

You will want to see all green squares (name available) below before buying a domain name. This looks better.

Namechk ok

I would recommend you create your social media accounts before or right after buying your domain. Sites like Twitter will insist on short usernames names so get your social media sites first.

Trademark and Brand Search

Also, perform a trademark and IP search.

Australian Trademark Search: https://search.ipaustralia.gov.au/trademarks/search/quick

United States Trademark Database: https://www.uspto.gov/trademarks-application-process/search-trademark-database

Global brand Search: http://www.wipo.int/branddb/en/

etc

Self Managed Warning

I tend to go the “self-managed server route” and install the free WordPress CMS because:

There are many reasons why you would not want to “self-manage” your own server

  • Technical Requirements (and time to support).
  • Higher Risk.
  • Applying Updates and Patches.
  • etc.

Being technically minded and choosing a “self-managed web servers” can take away time from the fun stuff like SEO, Site Design, customer needs, branding etc.

Self Managed Costs

For $5 a month you can buy a server with enough memory to install WordPress (cheaper if you don’t need WordPress)

Vultr is great. Vultr does have ready to go servers that you can deploy that have WordPress all set up.

wordpress-template

The Vultr template above does use the Centos OS (read my guide setting up Centos on a different service provider here) but I prefer to manually setup a server with Ubuntu 16.04 OS on Vultr.

With $5 server you can do what you want with it.  I have blogged before about setting up your own Server. e.g Installing Centos and Ubuntu server on Digital Ocean.  Digital Ocean does not have data centres in Australia and this kills scalability. AWS is good but 4x the price of Vultr. I have blogged about setting up and AWS server here (and upgrading an AWS instance). I tried to check out Alibaba Cloud but the verification process was broken so I decided to check our Vultr.

Manual Setup of Vultr on an Ubuntu 16.04 server

  • Deploy a Vultr Server – Guide here  ($2.5/m to New Jersey or Florida or $5/m to Sydney,  I would recommend you opt-in for the auto backup for $0.50c/m and $1/m respectively).
  • Setup NGINX.
  • Setup PHP and PHP-FPM (see guide above), consider adding PHP child workers.
  • Setup and secure MySQL (see guide above), create a database for WordPress to use.
  • Instal Adminder MySQL GUI (guide here).
  • Setup a free Lets Encrypt SSL certificate (guide here).
  • Install WordPress (and Jetpack plugin).
  • Install WordPress CLI.
  • Instal the WooCommerce Storefront WordPress Theme.
  • Install WooCommerce Plugin.
  • Secure Ubuntu.
  • Also consider linking your domain to Cloudflare to boost performance, scanning your site with Qualys Freescan and OWASP ZAP).
  • Consider setting up a WordPress image compressor and CDN plugin. like EWWW.io

Manual WooCommerce Plugin Setup

Once you setup Woocommerce you can set up the store defaults. Go to the WordPress dashboard and click WooCommerce Settings

Woo Commerce Settings

Settings – General

General

  • Set Address, City and State and Postcode
  • Set allowed countries to sell in (e.g Australia)
  • Set allowed countries to ship items to (e.g Australia)
  • Set Enable Taxes
  • Set Currency
  • etc

Settings – Products

Products

  • Set Weight
  • Set Dimensions
  • Enable Product Reviews
  • Enable Star Ratings on Reviews
  • etc

Settings – Shipping

Shipping

  • Enable Shipping Calculator
  • Add Shipping Classes
  • Shipping Zones
  • etc.

Settings – Checkout

Settings checkout

  • Force Secure Checkup
  • Create a Terms and Conditions page (and set).
  • etc

Settings – Account

Accounts

  • Set Account Options
  • etc

Settings – Emails

Emails

  • Set Email Preferences
  • Set Email Header Image
  • Set Email Colour
  • Set Footer Text
  • etc

Settings – API

API

  • API can be disabled if you don’t need it.

Optional Actions

  • Setup Yoast Plugin
  • Setup other plugins

Instaling a Woo Commerce Child Theme

Go to https://woocommerce.com/product-category/themes/storefront-child-theme-themes/ and choose a theme.

Themes

Purchase and Install the desired child theme (I uploaded it to my /wp-content/themes/ folder with forklift). I chose a free deli theme.

Goto your WordPress then themes folder and activate your new child theme.

Activate Theme

Post Site Setup

Just because your site is live does not mean you can rest.

SEO Optimization

Do use sites like https://seositecheckup.com/ and follow recommended actions to improve your SEO like updating meta tags.

More Reading

Attaching an email to your domain

You can pay $5 a month and link a G Suite email to your domain.

  • Dedicated professional Google G Suite email account for $5 a month with 30GB storage (If you don’t want ot to buy a G Suite email and link it to your domain then you don’t need this).

Once you have a G Suite account you can link other domains (and domain emails) to it. You can login to your G Suite emails via G Mail and send emails from apps or the command line.

Why Vultr

I use the server host Vultr as they have data centres all around the world and the support of great, Digital Ocean is good too but they don’t have data centres in my country (Australia). Vultr allows you to deploy all over the world upgrade servers, move servers, add storage and restore servers.

Alternatively, you can buy a $2.5/m server and generate  a static website

I use the Platforma Web HTML generator to build mobile and WCAG compliant websites.

Buying a domain,  I buy my domains from https://www.namecheap.com/  it is a good idea to look for coupons first at https://www.namecheap.com/promos/coupons.aspx before buying a domain.

Once you buy a domain you can point it to a Vultr server and upload your website.

I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v.1.2 WordPress WooCommerce

v1.1 SEO

v1.0 Initial Draft

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

by

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Buy a Domain 

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Cloudflare Benefits (Free Plan)

  • DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
  • Global CDN
  • Shared SSL certificate (I disabled this and opted to use my own)
  • Access to audit logs
  • 3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Cloudflare Global Network

Setup

You will need to sign up at cloudflare.com

Cloudflare

After you create an account you will be prompted to add a siteAdd SiteCloudflare will pull your public DNS records to import.

Query DNS

You will be prompted to select a plan (I selected free)

Plan Select

Verify DNS settings to import.

DNS Import

You will now be asked to change your DNS nameservers with your domain reseller

DNS Nameservers

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Strict SSL

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

The Cloudflare CTO responded.  🙂

Confirm Cloudflare link to a domain from the OSX Comand line

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

Page Rules

“fearby.com.com/wp-*” Cache level: Bypass

“fearby.com.com/wp-admin/post.php*” Cache level: Bypass

“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours.  having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before).  I just need to wait for caching and minification to kick in.

DNS Improved

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

  • Load Time: 1.80s
  • First Byte 0.176s
  • Start Render 1.200s

webpagetest

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Under Attack Flow

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here.  A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

cloudflare-attack-001

Thanks, Cloudflare.

Cloudflare Pros

  • Enabling Attack mode was simple.
  • Soaked up an attack.
  • Free Tier
  • Many Reports
  • Option to force HTTPS over HTTP
  • Option to ban/challenge suspicious IP’s and set challenge timeframes.
  • Ability to setup IP firewall rules and Application Firewalls.
  • User-agent blocking
  • Lockdown URL’s to IP’s (pro feature)
  • Option to minify Javascript, CSS and HTML
  • Option to accelerate mobile links
  • Brotli compression on assets served.
  • Optio to enable BETA Rocket loader for Javascript performance tweaks.
  • Run Javascript service workers from the 120+ CDN’s
  • Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
  • HTTP/2 on, IPV6 ON
  • Option to setup load balancing/failover
  • CTO of Cloudflare responded in Twitter 🙂
  • Option to enable rate limiting (charged at 10,000 hits for $0.05c)
  • Option to block countries (pro feature)
  • Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

  • No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
  • Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
  • Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
  • WordPress draft posts are being cached even though page riles block wp-admin page caching.
  • Would be nice to have ad automatic Under Attack mode
  • Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post