IoT, Code, Security and Server Stuff

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

UpCloud

I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.

by

I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance. Here is what I did to set up a complete Ubuntu 18.04 system (NGINX, PHP, MySQL, WordPress etc). This is not a paid review (just me documenting my steps over 2 days).

Background (CPanel hosts)

In 1999 I hosted my first domain (www.fearby.com) on a host in Seattle (for $10 USD a month), the host used CPanel and all was good.  After a decade I was using the domain more for online development and the website was now too slow (I think I was on dial-up or ADSL 1 at the time). I moved my domain to an Australian host (for $25 a month).

After 8 years the domain host was sold and performance remained mediocre. After another year the new host was sold again and performance was terrible.

Advertisement:



Page load times were near 30 seconds.

I started receiving Resource Limit Is Reached warnings (basically this was a plot by the new CPanel host to say “Pay us more and this message will go away”).

This is a screenshot of my old host giving a Usage Limit Exceeded error

The straw that broke the camel’s back was their demand of $150/year for a dodgy SSL certificate.

This is a screenshot of a crappy C rated SSL certificate (measured at ssl labs)

I needed to move to a self-managed server where I was in control.

Advertisement:



Buying a Domain Name

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Self Managed Server

I found a good web IDE ( http://www.c9.io/ ) that allowed me to connect to a cloud VM.  C9 allowed me to open many files and terminal windows and reconnect to them later. Don’t get excited, though, as AWS have purchased C9 and it’s not the same.

C9 IDE

C9 IDE

I spun up a Digital Ocean Server at the closest data centre in Singapore. Here was my setup guide creating a Digital Ocean VM, connecting to it with C9 and configuring it. I moved my email to G Suite and moved my WordPress to Digital Ocean (other guides here and here).

I was happy since I could now send emails via CLI/code, set up free SSL certs, add second domain email to G Suite and Secure G Suite. No more usage limit errors either.

Self-managing servers requires more work but it is more rewarding (flexible, faster and cheaper).  Page load times were now near 20 seconds (10-second improvement).

Latency Issue

Over 6 months, performance on Digital Ocean (in Singapore) from Australia started to drop (mentioned here).  I tried upgrading the memory but that did not help (latency was king).

Moved the website to Australia

I moved my domain to Vultr in Australia (guide here and here). All was good for a year until traffic growth started to increase.

Blog Growth

I tried upgrading the memory on Vultr and I setup PHP child workers, setup Cloudflare.

GT Metrix scores were about a “B” and Google Page Speed Scores were in the lower 40’s. Page loads were about 14 seconds (5-second improvement).

Tweaking WordPress

I set up an image compression plugin in WordPress then set up a cloud image compression and CDN Plugin from the same vendor.  Page Speed info here.

GT Metrix scores were now occasionally an “A” and Page Speed Scores were in the lower 20’s. Page loads were about 3-5 seconds (10-second improvement).

Mixed bag from Vultr (more optimisation and performance improvements were needed).

This screenshot is showing poor www.gtmetrix.com scores , pool google page speed index scores and upgrading from 1GB to 2GB memory on my server.

Google Chrome Developer Console Audit Results on Vultr hosted website were not very good (I stopped checking as nothing helped).

This is a screenshot showing poor site performance (screenshot taken in Google Dev tools audit feature)

The problem was the Vultr server (400km away in Sydney) was offline (my issue) and everything above (adding more memory, adding 2x CDN’s (EWWW and Cloudflare), adding PHP Child workers etc) did not seem to help???

Enter UpCloud…

Recently, a friend sent a link to a blog article about a host called “UpCloud” who promised “Faster than SSD” performance.  This can’t be right: “Faster than SSD”? I was intrigued. I wanted to check it out as I thought nothing was faster than SSD (well, maybe RAM).

I signed up for a trial and ran a disk IO test (read the review here) and I was shocked. It’s fast. Very fast.

Summary: UpCloud was twice as fast (Disk IO and CPU) as Vultr (+ an optional $4/m firewall and $3/m for 1x backup).

This is a screenshot showing Vultr.com servers getting half the read and write disk io performance compared to upcloud.com.

fyi: Labels above are K Bytes per second. iozone loops through all file sizes from 4 KB to 16,348 KB and measures through the reads per second. To be honest, the meaning of the numbers doesn’t interest me, I just want to compare apples to apples.

This is am image showing iozone results breakdown chart (kbytes per sec on vertical axis, file size in horizontal axis and transfer size on third access)

(image snip from http://www.iozone.org/ which explains the numbers)

I might have to copy my website on UpCloud and see how fast it is.

Where to Deploy and Pricing

UpCloud Pricing: https://www.upcloud.com/pricing/

UpCloud Pricing

UpCloud does not have a data centre in Australia yet so why choose UpCloud?

Most of my site’s visitors are based in the US and UpCloud have disk IO twice as fast as Vultr (win-win?).  I could deploy to Chicago?

This image sows most of my visitors are in the US

My site’s traffic is growing and I need to ensure the site is fast enough in the future.

This image shows that most of my sites visitors are hitting my site on week days.

Advertisement:



Creating an UpCloud VM

I used a friend’s referral code and signed up to create my first VM.

FYI: use my Referral code and get $25 free credit.  Sign up only takes 2 minutes.

https://www.upcloud.com/register/?promo=D84793

When you click the link above you will receive 25$ to try out serves for 3 days. You can exit his trail and deposit $10 into UpCloud.

Trial Limitations

The trial mode restrictions are as following:

* Cloud servers can only be accessed using SSH, RDP, HTTP or HTTPS protocols
* Cloud servers are not allowed to send outgoing e-mails or to create outbound SSH/RDP connections
* The internet connection is restricted to 100 Mbps (compared to 500 Mbps for non-trial accounts)
* After your 72 hours free trial, your services will be deleted unless you make a one-time deposit of $10

UpCloud Links

The UpCloud support page is located here: https://www.upcloud.com/support/

More UpCloud links to read:

Signing up to UpCloud

Navigate to https://upcloud.com/signup and add your username, password and email address and click signup.

New UpCloud Signup Page

Add your address and payment details and click proceed (you don’t need to pay anything ($1 may be charged and instantly refunded to verify the card)

Add address and payment details

That’s it, check yout email.

Signup Done

Look for the UpCloud email and click https://my.upcloud.com/

Check Email

Now login

Login to UpCloud

Now I can see a dashboard 🙂

UpCloud Dashboard

I was happy to see 24/7 support is available.

This image shows the www.upcloud.com live chat

I opted in for the new dashboard

UpCloud new new dashboard

Deploy My First UpCloud Server

This is how I deployed a server.

Note: If you are going to deploy a server consider using my referral code and get $25 credit for free.

Under the “deploy a server” widget I named the server and chose a location (I think I was supposed to use an FQDN name -e.g., “fearby.com”). The deployment worked though. I clicked continue, then more options were made available:

  1. Enter a short server description.
  2. Choose a location (Frankfurt, Helsinki, Amsterdam, Singapore, London and Chicago)
  3. Choose the number of CPU’s and amount of memory
  4. Specify disk number/names and type (MaxIOPS or HDD).
  5. Choose an Operating System
  6. Select a Timezone
  7. Define SSH Keys for access
  8. Allowed login methods
  9. Choose hardware adapter types
  10. Where the send the login password

Deploy Server

FYI: How to generate a new SSH Key (on OSX or Ubuntu)

Output

Did the key export? (yes)

> /temp# ls /temp/ -al
> drwxr-xr-x 2 root root 4096 Jun 9 15:33 .
> drwxr-xr-x 27 root root 4096 Jun 8 14:25 ..
> -rw——- 1 user user 1766 Jun 9 15:33 example_rsa
> -rw-r–r– 1 user user 396 Jun 9 15:33 example_rsa.pub

“example_rsa” is the private key and “example_rsa.pub “is the public key.

  • The public key needs to be added to the server to allow access.
  • The private key needs to be added to any local ssh program used for remote access.

Initialisation script (after deployment)

I was pleased to see an initialization script section that calls actions after the server is deployed. I configured the initialisation script to pull down a few GB of backups from my Vultr website in Sydney (files now removed).

This was my Initialisation script:

Confirm and Deploy

I clicked “Confirm and deploy” but I had an alert that said trial mode can only deploy servers up to 1024MB of memory.

This image shows I cant deploy servers with 2/GB in trial modeExiting UpCloud Trial Mode

I opened the dashboard and clicked My Account then Billing, I could see the $25 referral credit but I guess I can’t use that in Trial.

I exited trial mode by depositing $10 (USD).

View Billing Details

Make a manual 1-time deposit of $10 to exit trial mode.

Deposit $10 to exit the trial

FYI: Server prices are listed below (or view prices here).

UpCloud Pricing

Now I can go back and deploy the server with the same settings above (1x CPU, 2GB Memory, Ubuntu 18.04, MaxIOPS Storage etc)

Advertisement:



Deployment takes a few minutes and depending on how you specified a password may be emailed to you.

UpCloud Server Deployed

The server is now deployed; now I can connect to it with my SSH program (vSSH).  Simply add the server’s IP, username, password and the SSH private key (generated above) to your ssh program of choice.

fyi: The public key contents start with “ssh-rsa”.

This image shows me connecting to my sever via ssh

I noticed that the initialisation script downloaded my 2+GB of files already. Nice.

UpCloud Billing Breakdown

I can now see on the UpCloud billing page in my dashboard that credit is deducted daily (68c); at this rate, I have 49 days credit left?

Billing Breakdown

I can manually deposit funds or set up automatic payments at any time 🙂

UpCloud Backup Options

You do not need to setup backups but in case you want to roll back (if things stuff up), it is a good idea. Backups are an additional charge.

I have set up automatic daily backups with an auto deletion after 2 days

To view backup scheduled click on your deployed server then click backup

List of UpCloud Backups

Note: Backups are charged at $0.056 for every GB stored – so $5.60 for every 100GB per month (half that for 50GB etc)

You can take manual backups at any time (and only be charged for the hour)

UpCloud Firewall Options

I set up a firewall at UpCloud to only allow the minimum number of ports (UpCloud DNS, HTTP, HTTPS and My IP to port 22).  The firewall feature is charged at $0.0056 an hour ($4.03 a month)

I love the ability to set firewall rules on incoming, destination and outgoing ports.

To view your firewall click on your deployed server then click firewall

UpCloud firewall

Update: I modified my firewall to allow inbound ICMP (IPv4/IPv6) and UDP (IPv4/IPv6) packets.

(Note: Old firewall screenshot)

Firewall Rules Allow port 80, 443 and DNS

Because my internet provider has a dynamic IP, I set up a VPN with a static IP and whitelisted it for backdoor access.

Local Ubuntu ufw Firewall

I duplicated the rules in my local ufw (2nd level) firewall (and blocked mail)

UpCloud Download Speeds

I pulled down a 1.8GB Ubuntu 18.08 Desktop ISO 3 times from gigenet.com and the file downloaded in 32 seconds (57MB/sec). Nice.

Install Common Ubuntu Packages

I installed common Ubuntu packages.

Timezone

I checked the server’s time (I thought this was auto set before I deployed)?

I reset the time to Australia/Sydney.

Now the timezone is set 🙂

Shell History

I increased the shell history.

SSH Login

I created a ~/.ssh/authorized_keys file and added my SSH public key to allow password-less logins.

I added my pubic ssh key, then exited the ssh session and logged back in. I can now log in without a password.

Install NGINX

nginx/1.14.0 is now installed.

A quick GT Metrix test.

This image shows awesome static nginx performance ratings of of 99%

Install MySQL

Run these commands to install and secure MySQL.

Securing the MySQL server deployment.
> Would you like to setup VALIDATE PASSWORD plugin?: n
> New password: **********************************************
> Re-enter new password: **********************************************
> Remove anonymous users? (Press y|Y for Yes, any other key for No) : y
> Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y
> Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y
> Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y
> Success.

I disabled the validate password plugin because I hate it.

MySQL Ver 14.14 Distrib 5.7.22 is now installed.

Set MySQL root login password type

Set MySQL root user to authenticate via “mysql_native_password”. Run the “mysql” command.

Now let’s set the root password authentication method to “mysql_native_password”

Check authentication method.

Now we need to flush permissions.

Done.

Advertisement:



Install PHP

Install PHP 7.2

PHP 7.2.5, Zend Engine v3.2.0 with Zend OPcache v7.2.5-1 is now installed. Do update PHP frequently.

I made the following changes in /etc/php/7.2/fpm/php.ini

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 20M
> post_max_size = 20M

Install PHP Modules

sudo apt-get install php-pear php7.2-curl php7.2-dev php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml

Install PHP FPM

Configure PHP FPM config.

Edit /etc/php/7.2/fpm/php.ini

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 20M
> post_max_size = 20M

Reload php sudo service.

Install PHP Modules

Configuring NGINX

If you are not comfortable editing NGINX config files read here, here and here.

I made a new “www root” folder, set permissions and created a default html file.

I edited the “root” key in “/etc/nginx/sites-enabled/default” file and set the root a new location (e.g., “/www-root”)

I added these performance tweaks to /etc/nginx/nginx.conf

> worker_cpu_affinity auto;
> worker_rlimit_nofile 100000

I add the following lines to “http {” section in /etc/nginx/nginx.conf

Check NGINX Status

Install Open SSL that supports TLS 1.3

This is a work in progress. The steps work just fine for me on Ubuntu 16.04. but not Ubuntu 18.04.?

Installing Adminer MySQL GUI

I will use the PHP based Adminer MySQL GUI to export and import my blog from one server to another. All I needed to do is install it on both servers (simple 1 file download)

Use Adminer to Export My Blog (on Vultr)

On the original server open Adminer (http) and..

  1. Login with the MySQL root account
  2. Open your database
  3. Choose “Save” as the output
  4. Click on Export

This image shows the export of the wordpress adminer page

Save the “.sql” file.

I used Adminer on the UpCloud server to Import My Blog

FYI: Depending on the size of your database backup you may need to temporarily increase your upload and post sizes limits in PHP and NGINX before you can import your database.

Edit /etc/php/7.2/fpm/php.ini
> max_file_uploads = 100M
> post_max_size =100M

And Edit: /etc/nginx/nginx.conf
> client_max_body_size 100M;

Don’t forget to reload NGINX config and restart NGINX and PHP. Take note of the maximum allowed file size in the screenshot below. I temporarily increased my upload limits to 100MB in order to restore my 87MB blog.

Now I could open Adminer on my UpCloud server.

  1. Create a new database
  2. Click on the database and click Import
  3. Choose the SQL file
  4. Click Execute to import it

Import MuSQL backup with Adminer

Don’t forget to create a user and assign permissions (as required – check your wp-config.php file).

Import MySQL Database

Tip: Don’t forget to lower the maximum upload file size and max post size after you import your database,

Cloudflare DNS

I use Cloudflare to manage DNS, so I need to tell it about my new server.

You can get your server’s IP details from the UpCloud dashboard.

Find IP

At Cloudflare update your DNS details to point to the server’s new IPv4 (“A Record”) and IPv6 (“AAAA Record”).

Cloudflare DNS

Advertisement:



Domain Error

I waited an hour and my website was suddenly unavailable.  At first, I thought this was Cloudflare forcing the redirection of my domain to HTTP (that was not yet set up).

DNS Not Replicated Yet

I chatted with UpCloud chat on their webpage and they kindly assisted me to diagnose all the common issues like DNS values, DNS replication, Cloudflare settings and the error was pinpointed to my NGINX installation.  All NGINX config settings were ok from what we could see?  I uninstalled NGINX and reinstalled it (and that fixed it). Thanks UpCloud Support 🙂

Reinstalled NGINX

I reinstalled NGINX and reconfigured /etc/nginx/nginx.conf (I downloaded my SSL cert from my old server just in case).

Here is my /etc/nginx/nginx.conf file.

Here is my /etc/nginx/sites-available/default file (fyi, I have not fully re-setup TLS 1.3 yet so I commented out the settings)

SSL Labs SSL Certificate Check

All good thanks to the config above.

SSL Labs

Install WP-CLI

I don’t like setting up FTP to auto-update WordPress plugins. I use the WP-CLI tool to manage WordPress installations by the command line. Read my blog here on using WP-CLI.

Download WP-CLI

Move WP-CLI to the bin folder as “wp”

Test wp

Update WordPress Plugins

Now I can run “wp plugin update” to update all WordPress plugins

Update WordPress Core

WordPress core file can be updated with “wp core update”

Troubleshooting: Use the flag “–allow-root “if wp needs higher access (unsafe action though).

Install PHP Child Workers

I edited the following file to setup PHP child workers /etc/php/7.2/fpm/pool.d/www.conf

Changes

> pm = dynamic
> pm.max_children = 40
> pm.start_servers = 15
> pm.min_spare_servers = 5
> pm.max_spare_servers = 15
> pm.process_idle_timeout = 30s;
> pm.max_requests = 500;
> php_admin_value[error_log] = /var/log/www-fpm-php.www.log
> php_admin_value[memory_limit] = 512M

Advertisement:



Restart PHP

Test NGINX config, reload NGINX config and restart NGINX

Output (14 workers are ready)

Memory Tweak (set at your own risk)

vm.swappiness = 1

Setting swappiness to a value of 1 all but disables the swap file and tells the Operating System to aggressively use ram, a value of 10 is safer. Only set this if you have enough memory available (and free).

Possible swappiness settings:

> vm.swappiness = 0 Swap is disabled. In earlier versions, this meant that the kernel would swap only to avoid an out of memory condition when free memory will be below vm.min_free_kbytes limit, but in later versions, this is achieved by setting to 1.[2]> vm.swappiness = 1 Kernel version 3.5 and over, as well as Red Hat kernel version 2.6.32-303 and over: Minimum amount of swapping without disabling it entirely.
> vm.swappiness = 10 This value is sometimes recommended to improve performance when sufficient memory exists in a system.[3]
> vm.swappiness = 60 The default value.
> vm.swappiness = 100 The kernel will swap aggressively.

The “htop” tool is a handy memory monitoring tool to “top”

Also you can use good old “watch” command to show near-live memory usage (auto-refreshes every 2 seconds)

Script to auto-clear the memory/cache

As a habit, I am setting up a cronjob to check when free memory falls below 100MB, then cache is automatically cleared (freeing memory).

Script Contents: clearcache.sh

I set the cronjob to run every 15 mins, I added this to my cronjob.

Sample log output

I will check the log (/scripts/clearcache.log) in a few days and view the memory trends.

After 1/2 a day Ubuntu 18.04 is handling memory just fine, no externally triggered cache clears have happened 🙂

Free memory over time

I used https://crontab.guru/every-hour to set the right schedule in crontab.

I rebooted the VM.

Update: I now use Nixstats monitoring

Swap File

FYI: Here is a handy guide on viewing swap file usage here. I’m not using swap files so it is only an aside.

After the system rebooted I checked if the swappiness setting was active.

Yes, swappiness is set.

File System Tweaks – Write Back Cache (set at your own risk)

First, check your disk name and file system

Take note of your disk name (e.g vda1)

I used TuneFS to enable writing data to the disk before writing to the journal. tunefs is a great tool for setting file system parameters.

Warning (snip from here): “I set the mode to journal_data_writeback. This basically means that data may be written to the disk before the journal. The data consistency guarantees are the same as the ext3 file system. The downside is that if your system crashes before the journal gets written then you may loose new data — the old data may magically reappear.”

Warning this can corrupt your data. More information here.

I ran this command.

I edited my fstab to append the “writeback,noatime,nodiratime” flags for my volume after a reboot.

Edit FS Tab:

I added “writeback,noatime,nodiratime” flags to my disk options.

Updating Ubuntu Packages

Show updatable packages.

Update Packages.

Unattended Security Updates

Read more on Ubuntu 18.04 Unattended upgrades here, here and here.

Install Unattended Upgrades

Enable Unattended Upgrades.

Now I configure what packages not to auto update.

Edit /etc/apt/apt.conf.d/50unattended-upgrades

Find “Unattended-Upgrade::Package-Blacklist” and add packages that you don’t want automatically updated, you may want to manually update these (and monitor updates).

I prefer not to auto-update critical system apps (I will do this myself).

FYI: You can find installed packages by running this command:

Enable automatic updates by editing /etc/apt/apt.conf.d/20auto-upgrades

Edit the number at the end (the number is how many days to wait before updating) of each line.

> APT::Periodic::Update-Package-Lists “1”;
> APT::Periodic::Download-Upgradeable-Packages “1”;
> APT::Periodic::AutocleanInterval “7”;
> APT::Periodic::Unattended-Upgrade “1”;

Set to “0” to disable automatic updates.

The results of unattended-upgrades will be logged to /var/log/unattended-upgrades

Update packages now.

Almost done.

I Rebooted

Advertisement:



GT Metrix Score

I almost fell off my chair. It’s an amazing feeling hitting refresh in GT Metrix and getting sub-2-second score consistently (and that is with 17 assets loading and 361KB of HTML content)

0.9sec load times

WebPageTest.org Test Score

Nice. I am not sure why the effective use of CDN has an X rating as I have the EWWW CDN and Cloudflare. First Byte time is now a respectable “B”, This was always bad.

Update: I found out the longer you set cache delays in Cloudflare the higher the score.

Web Page Test

GT Metrix has a nice historical breakdown of load times (night and day).

Upcloud Site Speed in GTMetrix

Google Page Speed Insight Desktop Score

I benchmarked with https://developers.google.com/speed/pagespeed/insights/

This will help with future SEO rankings. It is well known that Google is pushing fast servers.

100% Desktop page speed score

Google Chrome 70 Dev Console Audit (Desktop)

100% Chrome Audit Score

This is amazing, I never expected to get this high score.  I know Google like (and are pushing) sub-1-second scores.

My site is loading so well it is time I restored some old features that were too slow on other servers

  • I disabled Lazy loading of images (this was not working on some Android devices)
  • I re-added the News Widget and news images.

GTMetrix and WebpageTest sores are still good (even after adding bloat)

Benchmarks are still good

My WordPress site is not really that small either

Large website

FYI: WordPress Plugins I use.

These are the plugins I use.

How I use these plugins to speed up my site.

  • I use EWWW Image Optimizer plugin to auto-compress my images and to provide a CDN for media asset deliver (pre-Cloudflare). Learn more about ExactDN and EWWW.io here.
  • I use Autoptimize plugin to optimise HTML/CSS/JS and ensure select assets are on my EWWW CDN. This plugin also removes WordPress Emojis, removed the use of Google Fonts, allows you to define pre-configured domains, Async Javascript-files etc.
  • I use BJ Lazy Load to prevent all images in a post from loading on load (and only as the user scrolls down the page).
  • GTmetrix for WordPress and Cloudflare plugins are for information only?
  • I use WP-Optimize to ensure my database is healthy and to disable comments/trackbacks and pingbacks.

Let’s Test UpCloud’s Disk IO in Chicago

Looks good to me, Read IO is a little bit lower than UpCloud’s Singapore data centre but still, it’s faster than Vultr.  I can’t wait for more data centres to become available around the world.

Why is UpCloud Disk IO so good?

I asked UpCloud on Twitter why the Disk IO was so good.

  • “MaxIOPS is UpCloud’s proprietary block-storage technology. MaxIOPS is physically redundant storage technology where all customer’s data is located in two separate physical devices at all times. UpCloud uses InfiniBand (!) network to connect storage backends to compute nodes, where customers’ cloud servers are running. All disks are enterprise-grade SSD’s. And using separate storage backends, it allows us to live migrate our customers’ cloud servers freely inside our infrastructure between compute nodes – whether it be due to hardware malfunction (compute node) or backend software updates (example CPU vulnerability and immediate patching).“

My Answers to Questions to support

Q1) What’s the difference between backups and snapshots (a Twitter user said Snapshots were a thing)

A1) Backups and snapshots are the same things with our infrastructure.

Q2) What are charges for backup of a 50GB drive?

A2) We charge $0.06 / GB of the disk being captured. But capture the whole disk, not just what was used. So for a 50GB drive, we charge $0.06 * 50 = $3/month. Even if 1GB were only used.

  • Support confirmed that each backup is charged (so 5 times manual backups are charged 5 times). Setting up a daily auto backup schedule for 2 weeks would create 14 billable backup charges.
  • I guess a 25GB server will be $1.50 a month

Q3) What are data charges if I go over my 2TB quota?

A3) Outgoing data charges are $0.056/GB after the pre-configured allowance.

Q4) What happens if my balance hits $0?

A4) You will get notification of low account balance 2 weeks in advance based on your current daily spend. When your balance reaches zero, your servers will be shut down. But they will still be charged for. You can automatically top-up if you want to assign a payment type from your Control Panel. You deposit into your balance when you want. We use a prepay model of payment, so you need to top up before using, not billing you after usage. We give you lots of chances to top-up.

Support Tips

  • One thing to note, when deleting servers (CPU, RAM) instances, you get the option to delete the storages separately via a pop-up window. Choose to delete permanently to delete the disk, to save credit. Any disk storage lying around even unattached to servers will be billed.
  • Charges are in USD.

I think it’s time to delete my domain from Vultr in Sydney.

Deleted my Vultr domain

I deleted my Vultr domain.

Delete Vultr Server

Done.

More Reading on UpCloud

https://www.upcloud.com/documentation/faq/

UpCloud Server Status

http://status.upcloud.com

Check out my new guide on Nixstats for awesome monitoring

What I would like

  1. Ability to name individual manual backups (tag with why I backed up).
  2. Ability to push user defined data from my VM to the dashboard
  3. Cheaper scheduled backups
  4. Sydney data centres (one day)

Advertisement:



Update: Post UpCloud Launch Tweaks (Awesome)

I had a look at https://www.webpagetest.org/ results to see where else I can optimise webpage delivery.

Optimisation Options

Disable dasjhicons.min.css (for unauthenticated WordPress users).

Find functions.php in the www root

Edit functions.php

Add the following

// Remove dashicons in frontend for unauthenticated users
add_action( 'wp_enqueue_scripts', 'bs_dequeue_dashicons' );
function bs_dequeue_dashicons() {
    if ( ! is_user_logged_in() ) {
        wp_deregister_style( 'dashicons' );
    }
}

HTTP2 Push

I added http2 to my listening servers

I tested a http2 push page by defining this in /etc/nginx/sites-available/default 

Once I tested that push (demo here) was working I then defined two files to push that were being sent from my server

I used the WordPress Plugin Autoptimize to remove Google font usage (this removed a number of files being loaded when my page loads).

I used the WordPress Plugin WP-Optimize plugin into to remove comments and disable pingbacks and trackbacks.

WordPress wp-config.php tweaks

Add 2FA Authentication to server logins.

I recently checked out YubiCo YubiKeys and I have secured my Linux servers with 2FA prompts at login. Read the guide here. I secured my WordPress too.

Tweaks Todo

  • Compress placeholder BJ Lazy Load Image (plugin is broken)
  • Solve 2x Google Analytics tracker redirects (done, switched to Matomo)

Conclusion

I love UpCloud’s fast servers, give them a go (use my link and get $25 free credit).

I love Cloudflare for providing a fast CDN.

I love ewww.io’s automatic Image Compression and Resizing plugin that automatically handles image optimisations and pre Cloudflare/first hit CDN caching.

Read my post about server monitoring with Nixstats here.

Let the results speak for themselves (sub <1 second load times).

Results

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v2.1 Newer GTMetrix scores

v2.0 New UpCloud UI Update and links to new guides.

v1.9 Spelling and grammar

v1.8 Trial mode gotcha (deposit money ASAP)

v1.7 Added RSA Private key info

v1.7 – Added new firewall rules info.

v1.6 – Added more bloat to the site, still good.

v1.5 Improving Accessibility

v1.4 Added Firewall Price

v1.3 Added wp-config and plugin usage descriptions.

v1.2 Added GTMetrix historical chart.

v1.1 Fixed free typos and added final conclusion images.

v1.0 Added final results

v0.9 added more tweaks (http2 push, removing unwanted files etc)

v0.81 Draft  – Added memory usage chart and added MaxIOPS info from UpCloud.

v0.8 Draft post.

I set the cronjob to run every 15 mins, I added this to my cronjob.

Sample log output
I will check the log (/scripts/clearcache.log) in a few days and view the memory trends.
After 1/2 a day Ubuntu 18.04 is handling memory just fine, no externally triggered cache clears have happened 🙂
Free memory over time
I used https://crontab.guru/every-hour to set the right schedule in crontab.
I rebooted the VM.
Update: I now use Nixstats monitoring
Swap File
FYI: Here is a handy guide on viewing swap file usage here. I’m not using swap files so it is only an aside.
After the system rebooted I checked if the swappiness setting was active.
Yes, swappiness is set.
File System Tweaks – Write Back Cache (set at your own risk)
First, check your disk name and file system
Take note of your disk name (e.g vda1)
I used TuneFS to enable writing data to the disk before writing to the journal. tunefs is a great tool for setting file system parameters.
Warning (snip from here): “I set the mode to journal_data_writeback. This basically means that data may be written to the disk before the journal. The data consistency guarantees are the same as the ext3 file system. The downside is that if your system crashes before the journal gets written then you may loose new data — the old data may magically reappear.”
Warning this can corrupt your data. More information here.
I ran this command.
I edited my fstab to append the “writeback,noatime,nodiratime” flags for my volume after a reboot.
Edit FS Tab:
I added “writeback,noatime,nodiratime” flags to my disk options.
Updating Ubuntu Packages
Show updatable packages.
Update Packages.
Unattended Security Updates
Read more on Ubuntu 18.04 Unattended upgrades here, here and here.
Install Unattended Upgrades
Enable Unattended Upgrades.
Now I configure what packages not to auto update.
Edit /etc/apt/apt.conf.d/50unattended-upgrades
Find “Unattended-Upgrade::Package-Blacklist” and add packages that you don’t want automatically updated, you may want to manually update these (and monitor updates).
I prefer not to auto-update critical system apps (I will do this myself).
FYI: You can find installed packages by running this command:
Enable automatic updates by editing /etc/apt/apt.conf.d/20auto-upgrades
Edit the number at the end (the number is how many days to wait before updating) of each line.
> APT::Periodic::Update-Package-Lists “1”;
 > APT::Periodic::Download-Upgradeable-Packages “1”;
 > APT::Periodic::AutocleanInterval “7”;
 > APT::Periodic::Unattended-Upgrade “1”;
Set to “0” to disable automatic updates.
The results of unattended-upgrades will be logged to /var/log/unattended-upgrades
Update packages now.
Almost done.
I Rebooted
Advertisement:

 
 
GT Metrix Score
I almost fell off my chair. It’s an amazing feeling hitting refresh in GT Metrix and getting sub-2-second score consistently (and that is with 17 assets loading and 361KB of HTML content)
0.9sec load times
WebPageTest.org Test Score
Nice. I am not sure why the effective use of CDN has an X rating as I have the EWWW CDN and Cloudflare. First Byte time is now a respectable “B”, This was always bad.
Update: I found out the longer you set cache delays in Cloudflare the higher the score.
Web Page Test
GT Metrix has a nice historical breakdown of load times (night and day).
Upcloud Site Speed in GTMetrix
Google Page Speed Insight Desktop Score
I benchmarked with https://developers.google.com/speed/pagespeed/insights/
This will help with future SEO rankings. It is well known that Google is pushing fast servers.
100% Desktop page speed score
Google Chrome 70 Dev Console Audit (Desktop)
100% Chrome Audit Score
This is amazing, I never expected to get this high score.  I know Google like (and are pushing) sub-1-second scores.
My site is loading so well it is time I restored some old features that were too slow on other servers
  • I disabled Lazy loading of images (this was not working on some Android devices)
  • I re-added the News Widget and news images.
GTMetrix and WebpageTest sores are still good (even after adding bloat)
Benchmarks are still good
My WordPress site is not really that small either
Large website
FYI: WordPress Plugins I use.
These are the plugins I use.
How I use these plugins to speed up my site.
  • I use EWWW Image Optimizer plugin to auto-compress my images and to provide a CDN for media asset deliver (pre-Cloudflare). Learn more about ExactDN and EWWW.io here.
  • I use Autoptimize plugin to optimise HTML/CSS/JS and ensure select assets are on my EWWW CDN. This plugin also removes WordPress Emojis, removed the use of Google Fonts, allows you to define pre-configured domains, Async Javascript-files etc.
  • I use BJ Lazy Load to prevent all images in a post from loading on load (and only as the user scrolls down the page).
  • GTmetrix for WordPress and Cloudflare plugins are for information only?
  • I use WP-Optimize to ensure my database is healthy and to disable comments/trackbacks and pingbacks.
Let’s Test UpCloud’s Disk IO in Chicago
Looks good to me, Read IO is a little bit lower than UpCloud’s Singapore data centre but still, it’s faster than Vultr.  I can’t wait for more data centres to become available around the world.
Why is UpCloud Disk IO so good?
I asked UpCloud on Twitter why the Disk IO was so good.
  • “MaxIOPS is UpCloud’s proprietary block-storage technology. MaxIOPS is physically redundant storage technology where all customer’s data is located in two separate physical devices at all times. UpCloud uses InfiniBand (!) network to connect storage backends to compute nodes, where customers’ cloud servers are running. All disks are enterprise-grade SSD’s. And using separate storage backends, it allows us to live migrate our customers’ cloud servers freely inside our infrastructure between compute nodes – whether it be due to hardware malfunction (compute node) or backend software updates (example CPU vulnerability and immediate patching).“
My Answers to Questions to support
Q1) What’s the difference between backups and snapshots (a Twitter user said Snapshots were a thing)
A1) Backups and snapshots are the same things with our infrastructure.
Q2) What are charges for backup of a 50GB drive?
A2) We charge $0.06 / GB of the disk being captured. But capture the whole disk, not just what was used. So for a 50GB drive, we charge $0.06 * 50 = $3/month. Even if 1GB were only used.
  • Support confirmed that each backup is charged (so 5 times manual backups are charged 5 times). Setting up a daily auto backup schedule for 2 weeks would create 14 billable backup charges.
  • I guess a 25GB server will be $1.50 a month
Q3) What are data charges if I go over my 2TB quota?
A3) Outgoing data charges are $0.056/GB after the pre-configured allowance.
Q4) What happens if my balance hits $0?
A4) You will get notification of low account balance 2 weeks in advance based on your current daily spend. When your balance reaches zero, your servers will be shut down. But they will still be charged for. You can automatically top-up if you want to assign a payment type from your Control Panel. You deposit into your balance when you want. We use a prepay model of payment, so you need to top up before using, not billing you after usage. We give you lots of chances to top-up.
Support Tips
  • One thing to note, when deleting servers (CPU, RAM) instances, you get the option to delete the storages separately via a pop-up window. Choose to delete permanently to delete the disk, to save credit. Any disk storage lying around even unattached to servers will be billed.
  • Charges are in USD.
I think it’s time to delete my domain from Vultr in Sydney.
Deleted my Vultr domain
I deleted my Vultr domain.
Delete Vultr Server
Done.
More Reading on UpCloud
https://www.upcloud.com/documentation/faq/
UpCloud Server Status
http://status.upcloud.com
Check out my new guide on Nixstats for awesome monitoring
What I would like
  1. Ability to name individual manual backups (tag with why I backed up).
  2. Ability to push user defined data from my VM to the dashboard
  3. Cheaper scheduled backups
  4. Sydney data centres (one day)
Advertisement:

 
 
Update: Post UpCloud Launch Tweaks (Awesome)
I had a look at https://www.webpagetest.org/ results to see where else I can optimise webpage delivery.
Optimisation Options
Disable dasjhicons.min.css (for unauthenticated WordPress users).
Find functions.php in the www root
Edit functions.php
Add the following
HTTP2 Push
I added http2 to my listening servers
I tested a http2 push page by defining this in /etc/nginx/sites-available/default 
Once I tested that push (demo here) was working I then defined two files to push that were being sent from my server
I used the WordPress Plugin Autoptimize to remove Google font usage (this removed a number of files being loaded when my page loads).
I used the WordPress Plugin WP-Optimize plugin into to remove comments and disable pingbacks and trackbacks.
WordPress wp-config.php tweaks
Add 2FA Authentication to server logins.
I recently checked out YubiCo YubiKeys and I have secured my Linux servers with 2FA prompts at login. Read the guide here. I secured my WordPress too.
Tweaks Todo
  • Compress placeholder BJ Lazy Load Image (plugin is broken)
  • Solve 2x Google Analytics tracker redirects (done, switched to Matomo)
Conclusion
I love UpCloud’s fast servers, give them a go (use my link and get $25 free credit).
I love Cloudflare for providing a fast CDN.
I love ewww.io’s automatic Image Compression and Resizing plugin that automatically handles image optimisations and pre Cloudflare/first hit CDN caching.
Read my post about server monitoring with Nixstats here.
Let the results speak for themselves (sub <1 second load times).
Results
I hope this guide helps someone.
Please consider using my referral code and get $25 credit for free.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
 [contact-form-7 404 "Not Found"]
Revision History
v2.1 Newer GTMetrix scores
v2.0 New UpCloud UI Update and links to new guides.
v1.9 Spelling and grammar
v1.8 Trial mode gotcha (deposit money ASAP)
v1.7 Added RSA Private key info
v1.7 – Added new firewall rules info.
v1.6 – Added more bloat to the site, still good.
v1.5 Improving Accessibility
v1.4 Added Firewall Price
v1.3 Added wp-config and plugin usage descriptions.
v1.2 Added GTMetrix historical chart.
v1.1 Fixed free typos and added final conclusion images.
v1.0 Added final results
v0.9 added more tweaks (http2 push, removing unwanted files etc)
v0.81 Draft  – Added memory usage chart and added MaxIOPS info from UpCloud.
v0.8 Draft post.
Connecting to a server via SSH with Putty
 by 
This post aims to show how you can connect to a remote VM server using  Telnet/SSH Secure shell with a free program called Putty on Windows. This not an advanced guide, I hope you find it useful.
You will learn how to connect (via Windows) to a remote computer (Linux) over the Telnet protocol using SSH (Secure Shell). Once you login you can remotely edit web pages, learn to code, install programs or do just about anything.
Advertisement:


Common Terms (Glossary)
  • Putty: Putty is a free program that allows you to connect to a server via Telnet. Putty can be downloaded from here.
  • Port: A port is a number given to a virtual lane on the internet (a port is similar to a frequency in radio waves but all ports share the same transport layer frequency on the internet). Older unencrypted webpages work on Port (lane 80), older mail worked on Port 25, encrypted web pages work on Port 443. Telnet (that SSH Secure Shell uses) used Port 22. Read about port numbers here.
  • SSH: SSH is a standard that allows you to securely connect to a server over the telnet protocol. Read more here.
  • Shell: Shell or Unix Shell is the name given to the interactive command line interface to Linux. Read more about the shell here.
  • Telnet: Telnet is a standard on the TCP/IP protocol that allows two-way communication between computers (all communicatin issent as characters and not graphics).  Read more on telnet here and read about the TCP protocols here and here.
  • VM: VM stands for Virtual Machine and is a name given to a server you can buy (but it is owned by someone else). Read more here.
Read about other common glossary terms used on the Inetre here: 
https://en.wikipedia.org/wiki/Glossary_of_Internet-related_terms
Background
If you want a webpage on the internet (or just a server to learn how to program) it’s easier to rent a VM for a few dollars a month and manage it yourself (with Telnet/SSH Secure Shell) than it is to buy a $5,000 server, place it in a data centre and pay for electricity and drive in every few days and update it. Remote management of VM servers via SSH/Secure Shell is the way for small to medium solutions.
  • A simple web hosting site may cost < $5 a month but is very limited.
  • A self-managed VM costs about $5 a month
  • A website service like Wix, Squarespace, Shopify or WordPress will cost about $30~99 a month.
  • A self-owned server will cost hundreds to thousands upfront.
There are pros and cons to all solutions above (e.g cost, security, scalability, performance, risk) but these are outside this post’s topic. I have deployed VMs on provides like AWS, Digital Ocean, Vultr and UpCloud for years. If you need to buy a VM you can use this link and get $25 free credit.
I used to use the OSX Operating System on Apple computers. I was used to using the VSSH software program to connect to servers deployed on UpCloud (using this method). With the demise of my old Apple Mac book (due to heat) I have moved back to using Windows (I am never using Apple hardware again until they solve the heat issues).
Also, I prefer to use Linux servers in the cloud (over say Windows) because I believe they are cheaper, faster and more secure.
Enough talking lets configure a connection.
Public and Private Keys?
Whenever you want to connect to a remote server via Telnet/SSH Secure Shell you will need a public and private key to encrypt communications between you and the remote server.
The public key is configured on your server (on Linux you add the public key to this file ~/.ssh/authorized_keys).
The private key is used by programs (usually on your local computer) to connect to the remote server.

How to create a Public and Private Key on Linux
I usually run this command on Ubuntu or Debian Linux to generate a public and private SSH key.
sudo ssh-keygen -t rsa -b 4096
The key below was generated for this post and is not used online. Keys are like physical keys, people who have them and know where to use them can use them.
Output:
Generating public/private rsa key pair.
Enter file in which to save the key (/username/.ssh/id_rsa): ./server
Enter passphrase (empty for no passphrase): ********
Enter same passphrase again: ********
Your identification has been saved in ./server.
Your public key has been saved in ./server.pub.
The key fingerprint is:
SHA256:sxfcyn4oHQ1ugAdIEGwetd5YhxB8wsVFxANRaBUpJF4 [email protected]
The key's randomart image is:
+---[RSA 4096]----+
| .oB**[email protected]       |
|  +.==B.+        |
| o .o+o+..       |
|  .. +..o...     |
|    o ..Sooo.    |
|         ++o.    |
|        .o+o     |
|        .oo .    |
|         ...     |
+----[SHA256]-----+

Advertisement:


The two files were created
server
server.pub
  • “server” is the private key
  • “server.pub“is the public key
Public/Private Key Contents
Public Key Contents (“server.pub”)
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC7Xo9bOCXJ7gVjP8tKOxHVId3KTo5I0VRU/kSRK3+mGd5VbDbQABo3tdWzYhzkjODzRS9TeL2dcLAQNNQKshi9IW5IGDS1NocSCLFQId5BFr9s3E79fkWqcZkKmwocepXOOZ91EDKgIFxviOzZKe99sdxxMoZzi1nxgVyXl4TnaelyiQxeKYniVs1iqDfYWQCxkKsmYit8TvGtOwrhLvKNh9362/y5ebpXVdFlRuB83eF7k8RHNYCQyOJJVx4cwnTIsAN0GMOwjuaOZbp7rR1d6k7RZmaApNRTbaWOXy32UiBST5TV/jXF2UL/4IBnn+yvCrM0v79e/3omgjlVVKfWByFzMv/YlBKCAX3xxtJQ9RkzTqseKupXmmJU0rik6Xuz31N2oyw4M7yJofSUGVCN0pnpKEvnKxqolfD9egdQy2XDaNioY7cvOO1qRegCKE0sDh1m5MzJWMhbDs7macSMyd6+0O5qWc/ByHy0G/mVbd8kO4jIuEzEs4IFkPCToEZp7KfkY7KRkOhccLbQ4ApCesUfBtGGAN1f33NnXCHae3Cx46nSd23fvgDZUVnjI47tNJH5Z8FNVlW/fp5Rgeu/aPUephnDX2IBxwIKQOmSTDY+nxU4V+c93H1gSOJfvqYbVKIAXKyN9Yh6LC44ZvLrL4q0TC0QlH2+kxMLnuj7zw== [email protected]
Private Key Contents (“server”), always keep the private key safe and never publish it.
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,D34670C40CE3778974BEF97094010597

b4oecyqLsWt9n+G12ldVNlaQxSKF1wSrlBPg6FGiHRauTCyreUwoI2dMOAkwnGmN
8fcy51fH7D3Kg0G9fWWNPd+oUDwZmrpB8Mv6Ndk4bLYZEbkNOFgvPwNre7edTBOD
JGZRdWqb+yrywgvz3iTXPNjNK5REU3u3JmD69jInFNo92j765QQKA4sFgEyD/8g+
zg8yefIQAhEsVELC5LXPPyuTfA+x0Q+040PqCJ+FCISJI1CeZjLwk7Fbe453Vj81
zaDsurl5X5gaRUlVjB2asr6etWdMLWcalX4Nbyj2A10L3J4ONjKq3Wc2muJ0Q6ES
oNqBaU2iHPlK8yK0TGj/ERfjaG1qdlhBcow0pSapRqGopXBuVBLVuyc2NHe5CCTk
Ezq+LZGsVYmiOIIY4QRJdEN/DVLFHRGK/xA9A7unm484zXIEO6wznE0DuCTtyZs0
luJ3bKLRcack3K1Dphq0LjSG4YxQlkHewa9k9AKpDPTqeeKKckySakiDCGPT6htk
VqaCKrApAt6GQ2hLVXZ0BFVN5A3WUJ5s+HpFvTUzHTNZcdsVS4PgxhuCtnSO/BdS
/G+ODc4aZJNYQD9QQfWUnxkgnQJCWJ+aBZtKF7eDPRYY7qD9jWxubDzrFplBkmAi
O+aX5N8dpU3lEty4INjyh5LpgZWSswjUhEKWi/c1k+Qd1gCWzYzwAq2BfpWcF8Z+
c+y9lQUKbq2yDlxReCIsfb/hda5k1HjgaUlhKbjWIITSlGqf/NE9i+vj0rQEMQXQ
mxBoilfLUPd5A1ttG5XvqC2ex5HBmjzCazZ13Z/2c/PkwicHBmrf5bKYHZp49niV
44n8tZRamCUv6HaJUaKR22MigOG/qGppGPodGeLNj1DFLYAEQ78SYcVhEqIICBo1
t1yaIemUq8MWXSZz1K3cP4FEXQcEziQxFLU/0DCE0P0mIU3MExUmjB/nVE8vxb5l
p3ej3yrRGe+P2neco2gttgaTEi6l/S+0TIiZNstnVPG48BPW71mwVg9XR1d+avO7
OpXt0UgocX0xp7zBgK2up8Ai6v66WwjoNgyvFe02aK4/+fSC+aJ5D6N7JVNxd/bn
Py4W8oLKnrE1PKtIfBw/aE+rgudaMIyuxCaLllRKyDxVPPiJFp2iFcH/Y+k+0vDa
xE9Jpdd0zOWkZyebAxrS8zAUUNNaTQ+rWkj/zORjE4ptHpdwdazzHoQwIs+1kjsv
e/+JEmoskH7XozLnxClVhhWMXWfgQsPWBqPnGzieW0tv9SeIAU/BLJCHJRhBMAT1
ugBtcda1VMlAPVroYtVyUdCxkYZqGfIDbKqtOvvuBgUIUe/HnC3ExQQycC9F05BH
RJibaM/11MLTcZSO7KOK65Dg2v3VBhe6rfDl4tTR0yOySPXCacb9aMt2pMPTEe0/
wU49wCefchfD2bsR3kXPpUqm+HbkHORpIwsMZfQO/8dooXYdiYUdzV9roXG6OGVQ
SsV/xR2lE3XrR71TBegfRnQirI8tj4psSor+yCj3qV936Oh31D96Z6P4glshibsG
ffWAO/TSdu5ZV+UVahh6bTozs+g+odUu/S48TeI1fk7lPlqwZdjoSHXUI2v1FAQ2
jSSywuZQxHlGhg6OeI052cxx3zcVyVVLFHhIrfvufNc3c3+KYhtyiSzBNYN1BrJi
xNXwlDS1jYWgRHkf9zbNBU0MLTYHjZZvO9Jpl/UhKKBdIvJFwmGmXS2lgU6slunJ
Ojp4tY1tbI520KOskV/OoqEfmhXh5fTlIconzoK1aLqxk1d0d65ONcxqVbAG79RN
b0Q5PgewSOgFlcZ7tEIZKAWsWVhjlFTSGRujdZVM1vZB9fCJesemai7HU0e4J+Do
tqvss8I2n6TPxlTYFzQ4w12pIiOzx/8cFLX78NLN8wQFElhhczeuW5HDAnmPxYhQ
eLY0HgDCFSvVAvGXo0j1gcBUcOr/LzZSsJhxsB7FKyrUjlmD/7Y45WoKJj41bKL+
y4+iDhXyLBiqVClRijsguwiCkmPFiR7Bng2pglS0oIWPWu1UbTJWVJPfuUTOBC+M
4/2fBtgFjUz8iUISs9ncEKkERlxodBIu+ekgLJZAigSMvUKfGE1YB1AA9x96VLjd
VJSjjWvnhMEoSwNzlNQ9+dhoD5Cg9zicgIIKnHnovYGOu8g9ZWfvhJFrKZgkfLRv
r2KgkWiHWpf0swiyGUOlGJDe39nMMkoxib7XE/J3VI3na1ZUOIf8kl9kdHXJ0R3C
2IjdbfiFHEDOrakp5oeVf8BbLK7RB8OlxgJAS47Byh8j97U7f13A5ZYlK3bkZ7E4
h7mCJQozgWP81ut0d9WUlcKp5M8yg2ctZ7h4oeG4Js4ceHqd19Z4P+1xWKwXcdmV
+uhiTftevTu3/UhYQVV4ck98C9pursJJYL5hTnIIpTSWIR+jSahhtzUy/upjugPp
cKi6eGlOkcHdKNRtiu7/IZqni85fC8PAwPZ93SICdiq6BpGaGWFh046weIJuflSK
Pd76+M70YRd+pkaRjJyFJ3hLyg7W5mlOb1+yBIlXKzpbch9B5E4dRHCcOsg4+v/9
exRgAnvUIhR/GpSySDDwgKHg8rAyjjoGeZFH3TJIemAAimyaR608a9tCn7SxVobs
UQlZ9WwC0dQIEv7mSvSige3imbybPtCoBHJAqsJqKCFJEDWbIF5l2VYZcfJUYaEI
oZAJHYGnZm33yQ6eSOusXJ2SnnGZ+ZsGO4bDVSwN20FkSt11gN8Wjrki9CxeVQp7
dWbKX1r/lZw74yUB4cYN23hgLJsdqvM7THzwlBkVtgV74RGY0qv59ecBUSQedlSK
dkOnkmoCiGRSNyf+ebijQaygnfK0ArG5wiRF/RQWiPFj7S6DHRxIOrXqcmvhJ7Ly
NApn9pPYyoZEAbk82MAXkapZ5+YLIKLjdNsYuKq5xVty+mc+FfxLWmZGX+QQinra
Z9DfY9KQw4rxJ/ju4ILnDrygm/QBsNFXBojOuzOIULt7c26s3d/47T+IXA4SIX4v
cPqYa6S3PU/Yoe5/Ya3tFxXmBXgEgVLZuujMs7dyCOAqLEyBEHYqIclp+TElWQLR
V660fczVXeedfd2tNBy1IBj1vhGa9j5mZLbFwTczykwCFfihLIrxSEc1MQA4CaSX
-----END RSA PRIVATE KEY-----

The Public and Private keys is used to encrypt all Telnet/SSH connections and traffic to your server. Keep these key’s private.
Advertisement:


fyi: Putty can create SSH Keys too
If you do not have a Linux computer or Linux server to generate keys the Putty generator can create keys too.
Puttygen generating a key based on the randomness of mouse movements.
I did not know Putty can create keys.
Do save the public and private key(s) that were generated in Puttygen (tip: PPK files are what we are after along with the public key later in this post).
Public keys are added to your server when you deploy them. On Linux, you can add new Pulic keys after deployment by adding them to this file “~/.ssh/authorized_keys” to allow people to log in.
Puttygen does format the keys differently than how Ubuntu generates them. Read more here. I’ll keep generating keys in Linux over Puttygen.
Output of the public and PPK files from Puttygen

Putty SSH Client on Windows
Putty is a free windows program that you can use to connect to serves via SSH. Download and install the Putty program.
Open Putty
Putty Icon
Default Putty User Interface.
Screenshot of the Putty Program
To create a connection add an exiting IP address (server name) and SSH port (22) to Putty.
Screenshot of an IP and port entered into putty
In Putty (note the tree view to the left of the image), You can set the auto login name to use to log into the remote server under the Connection the Data in the tree view item
Screenshot showing the SSH usename being added to putty under Connection then Data menu,

You can also set the username under the Connection then Rlogin section of Putty.
Set the usernmae undser rlogin area of putty
OK, lets add the private SSH Key to Putty.
Putty Screehshot showing no support for standard SSH keys (only PPK files)
It looks like Putty only supports PPK private key files not ones generated by Linux. I used to be able to use the private key in the VSSH program on OSX and add the private key to connect to the server over SSH.  Putty does not allow you to use Linux generated Private keys directly.
Convert your (Linux generated) private key to (Putty) PPK format with Puttygen
Putty comes with a Key Generator/Converter, you can open your existing RSA private key and convert it (or generate a new one).
TIP: If you generate a key in Puttygen don;t forget to ad’d it to your authorized host file in your remote server.
Open Puttygen
Puttygen icon
Click Conversions than Import Key and choose the private key you generated in Linux
Screenshot showing import RSA key to convert
The private key will be opened
Screenshot of imported RSA key
You can then save the private key as a PPK file.
Save the private key as a PPK file
Advertisement:


“server.ppk” Key contents
PuTTY-User-Key-File-2: ssh-rsa
Encryption: aes256-cbc
Comment: imported-openssh-key
Public-Lines: 12
AAAAB3NzaC1yc2EAAAADAQABAAACAQC7Xo9bOCXJ7gVjP8tKOxHVId3KTo5I0VRU
/kSRK3+mGd5VbDbQABo3tdWzYhzkjODzRS9TeL2dcLAQNNQKshi9IW5IGDS1NocS
CLFQId5BFr9s3E79fkWqcZkKmwocepXOOZ91EDKgIFxviOzZKe99sdxxMoZzi1nx
gVyXl4TnaelyiQxeKYniVs1iqDfYWQCxkKsmYit8TvGtOwrhLvKNh9362/y5ebpX
VdFlRuB83eF7k8RHNYCQyOJJVx4cwnTIsAN0GMOwjuaOZbp7rR1d6k7RZmaApNRT
baWOXy32UiBST5TV/jXF2UL/4IBnn+yvCrM0v79e/3omgjlVVKfWByFzMv/YlBKC
AX3xxtJQ9RkzTqseKupXmmJU0rik6Xuz31N2oyw4M7yJofSUGVCN0pnpKEvnKxqo
lfD9egdQy2XDaNioY7cvOO1qRegCKE0sDh1m5MzJWMhbDs7macSMyd6+0O5qWc/B
yHy0G/mVbd8kO4jIuEzEs4IFkPCToEZp7KfkY7KRkOhccLbQ4ApCesUfBtGGAN1f
33NnXCHae3Cx46nSd23fvgDZUVnjI47tNJH5Z8FNVlW/fp5Rgeu/aPUephnDX2IB
xwIKQOmSTDY+nxU4V+c93H1gSOJfvqYbVKIAXKyN9Yh6LC44ZvLrL4q0TC0QlH2+
kxMLnuj7zw==
Private-Lines: 28
DkpbM78GgGBSgfs9MsmZwDJj6HFXdoe+fCP1rLnwbE99mvU6Fbs23hXd+FsVdQbb
VR5tKTocV7tEwGjtLCHSTSF6gap0l4ww0Ecuvr/Dra2CJ2BsntyssBrWnlUT7OlA
M9zKQAzywAy4AHkph0YvH4l7BcJ5V1pUltm2JDTU6+iFqXDsstUUEDcQ4u0EalWU
EEsW+quNSwO0HBHvWY6N7tbiuEN9L+cFYIdsJEDfqM4hNi+7Ym+SQq5FOPyA6gXa
vhujsjPQAWI3TFxh7EIvsPDMCXxWHL6qaDvOMmPPTZDbEvm4nQ5Kax9jWacPILn7
ezc7ZAiZdDiFbkF3TLyuHx71mjChZgLoZLWYfXR3MBEEYnkNO/7oSMRUwDzEyWKW
ZgqdUtGg0cR+qWvaxQTDQsN/DjB7jGgnlreF92S8xSsbk5GgpZnTQ1V0cm3oecB+
+JP90K4Fi979gPWnwTfg6ZvmLUiVz3uBbvegkT9CVZhhZXSKq53H+SjTZfKBPrM9
NHGLkYr1WjToGR39LMrh4X3KChGewMFyuxtpkEQV60eCnHZBHgTco2A0yriRprOP
Ks4qJXOtZnsMYMesUDX9W5wLc4HcRvRh2UBPw/8bPz6mNrBk8j5SPIwBrPMIBejd
4IPoYezaEFKPg2bP7dn+Nftz5CGagcV2g+zhE615dsWzX1P7yu/1dTmz9LXaMmN6
d+zJE8TtjeaoW5NE1HVFlj9rknzJW7xQfokhS5hMkOg6J0AA6Pk13tupu8WHMkVB
x7nVu876f8tIbT8GzXCGgSl+zS7IJO3pt9T9QHIYa+T3oTIUqfBfK1WffUZwHMRn
Xn/VKUtIIIPiVfCtQuxSrTiJzQcoJ/yvfv62YAGv2LsDlBoHfXRdf6h3TCCCOVxT
WE45sbj3gJ1Cgjt1SEd/8A3hkstn2U2NKBI9gkB9H5BbDJoAXq6/4CkwaQvSEzs7
LK5btRlWop+7gqkyMPpgxv9li9IEDJ999ufMqxkFgOBkmkR5Si71elXRnwiKrjfU
Ce14iy7Dd7lb7IU9OEBjWFZlSigVEnc8klhGHDuxnojiW1ld7pUDIkAAbdTMOFON
abcpfNwcg5Y3l+1KwIQHuewAUuA9472jV4V9EAn7pJ7wgmYHbzMzg9Z9dM8h/3UI
axBzAW+cJM80gN+nZMbmDC9FkXV16GSuqC2iQUVGb2TIheAS7oCR+JFZFQNv0ytF
rGQ9K1wIGbMI4oDPcAid7DzrEXVl3d2x8MtwF/WzfHehVJD1h1uNwezLf1gBKyas
9GBfDOYwd8zgaL2H99GYD1Ba7TePJY81mx7m10eYdwDj1vCpboKE3cE6AyL7ki+4
Ix7GSzQs9NBckF9+8eVXe2T4Cc450hIoN0BWcxVUUdGCA1skZ1PczPs1z/ae4lxd
l5WmPy8Gyh7cnZpyqzvwAPSFDadkNP60eekfkRHyo4QyLhj7QZtO0kOgWhT3CHma
FjZ5jJu59U/4gc0TpQ8ra3vgKQKudloExsg027+34nR98dN+zzUj4S2C/J34W98C
DEEu/SO7nfW/a2UARXBKWCbS+3j24zHc9dbgX2tZoAoInUvRGiSOsLVsMhDiBoyb
wWoNxrKPR3Fi5zZ+GfDUgUGpZoW/b54KnFouIHBYbI41Gkh4vj6lxOGh/sb3SPHd
Wg6EN/0z/mer3bG0a2/ZHKYA5KGWRXWYvYLz4Je8fb/egBrSU6BztwSNeilzA9lI
J4BO7pzXECnWYutB14UxHw==
Private-MAC: 12298fa865ac574da81898252e83b812200cba59

Now the PPK key can be added to Putty for any server connection that uses the public key. Use the right key for the right server though.
Add the private key to a Putty server by clicking Connection, SSH, AUTH section and browing to the PPK file.
Screenshot showing the PPK key file added to Putty
Advertisement:


Now we need to save the connection, click back on the Session note at the top of the treeview, type a server name and click Save
Save Putty connection.
Connecting to your sever via Telnet/SSH wiht Putty.
Once you have added a server name, port, usernames and private key to Putty you can double click the server list item to connect to your server.
You will see a message about accepting the public key from the server. Click Yes. This fingerprint will be the same fingerprint that was shown when you generated the keys (if not maybe someone is hacking in the middle of your local computer and server)
Putty messgae box asking to to remember the public key
Hopefully, you will now have full access to your server with the account you logged in with.
Screenshot of an Ubuntu screen after login
Happy Coding.
Alternatives to self-managed VM’s
I will always run self-managed server (and configure it myself) as its the most economical way to build a fast and secure server in my humble opinion.
I have blogged about alternatives but these solutions always sacrifice something and costs are usually higher and performance can be slower.
I am also lucky enough I can do this as a hobby and its not my day job. when you self manage a VM you will have endless tasks or securing your server and tweaking but its fun.
More Reading
Read some useful Linux commands here and read my past guides here. If you want to buy a domain name click here.
If you are bored and want to learn more about SSH Secure shell read this.
Related Blog Posts
Advertisement:


Version: Draft 1.0 Initial Draft
Set up a whitelisted IP on an UpCloud VM and WordPress using a VPN to get a static IP address
 by 
This is how I set up a whitelisted IP on an UpCloud VM and WordPress using a VPN to get a static IP address
Advertisement:

 
 
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
Buy a domain name from Namecheap here.
Domain names for just 88 cents!
Before you begin
Take a backup of WordPress files + database and take a snapshot of your VM (see my UpCloud VM guide here).
Having a ready backup IS a good idea.
Screenshot of https://my.upcloud.com/server/details/
Why Whitelist
Whitelisting is not bulletproof but it is an important link in the security chain. Security is only as good or bad as the strength of your weakest link.
Using updated software, applying patches, using HTTPS, using a reliable host in a reliable location, using good passwords are equally important as IP filtering. Whitelisting IP’s goes a long way to ensuring you have least access privileges on connections.
Remember to scan your site with OWASP Zap, Qualys and Kali Linux too.
What IP’s are you going to Whitelist?
Q1) Does your ISP offer a static IP address (or a dynamic IP)?
My ISP does NOT provide a static IP by default (I can pay $20 a month for one (that’s too expensive)).
You can check your public IP by loading http://icanhazip.com/ (this will return your public IPV4 address).
Load https://ipv6.icanhazip.com/ to view your IPV6 IP (if you have one)
Q2) Do you need to whitelist IP addresses while on the go (Mobile)? If so I would recommend you whitelist a VPN’s IP or IP range.
Advertisement:

 
 
Recently I had Apache web server auto install and knock out my NGINX web server and I needed to login on a mobile device to investigate,  Luckily I whitelist my VPN’s IP and logged in from my mobile device and resolved the issue.
Use a VPN to get a static IP
If you don’t have a static IP or you want to connect to your site on the go (Mobile) you can setup a VPN and use their static IP
I was using http://cyberghostvpn.com/ to have a static IP but a server failure in Sydney caused my defined whitelisted IP to disappear so I change to https://protonvpn.com/ (as Cybergost were unable to provide known IP’s of VPN servers).
TIP: Don’t just whitelist one server, whitelist a few as you never know when a server will go down.
Here is a screenshot of the 1st VPN I tried (Cyberghost), Cyberghost VPN is connected to a specified server (Dallas).
Cyberghost VN screenshot connected to Dallas
I swithced to ProtonVPN.
Here is a screenshot of ProtonVPN connected to a Switzerland server. Read more about Proton VPN here.
Screnshot of Protonvpn
I set Proton VPN to auto start and connect to my desired server
Screenshot of Proton VNS startup settings
Proton VPN offered me a 7-day PLUS trial (All Countries, 5 devices, highest speed, secure core etc) after I started using the free version (3 countries, 1 device, speed low). I assume everyone gets the same PLUS trail offer.
You can view Proton plans and pricing here.
Ok, now that we know how to get a static IP, let’s configure some firewalls.
Network Firewall at UpCloud
I use the awesome UpCloud to hold my domains (read more about UpCloud performance here). You can log in to your UpCloud Dashboard and load the server list, click your server and then click Firewall and define firewalls.
Firewall: Open IPv4/IPv6 ports for:
  1. ICMP
  2. 53 (DNS)
  3. 80 (HTTP)
  4. 443 (HTTPS)
Only allow access to port 22 from whitelisted IP’s (or IP ranges)
Screenshot of UpCloud firewall screen at https://my.upcloud.com/server/details/
I like to set separate firewall rules for IPV4 and IPV6, for TCP or UDP and I limit rules to certain IP range and port.
Ubuntu Firewall
I also like to run a ufw firewall (more information on ufw) on my Ubuntu server (read this guide on securing Ubuntu in the cloud and running a Lynis audit).
Manually setup firewall rules in ufw.
Don’t forget t restart your firewall
Run a  local nmap scan to find open ports
Don’t be concerned if you see open ports from a local nmap scan (e.g port 22 or 3306), these are locally open.  We need to scan externally to see if these ports are opened.
Scan your site with an external nmap tool like pen-test-tools or here.
Screenshot of a public nmap scanYou should not have non-web based service ports freely open externally (web-based ports e.g 80 and 443 are ok)
Port 22 access should be whitelisted to select IP’s only. You should not have any database ports open externally.
Whitelisting WordPress Access
Download WordFence plugin for WordPress from https://www.wordfence.com/
Read more on downloading WordPress plugins from the command line here. Read my past Wordfence post here.
Once Wordfence is installed open the WordFence All Options screen  (/wp-admin/admin.php?page=WordfenceOptions).
Now you can add your static IP (or IP ranges) to the WordFence whitelist.
Picture of WordFence whitelist
Setup auto block for any non whitelisted Itryingng to login to /wp-login.php
I permanently ban any IP accessing my login page (there are many).
What to do with rejected IP connections?
Wordfence will block connections to WordPress. I’d suggest you setup fail2ban to block other unwanted connections at network level too.
Advertisement:

 
 
Conclusion
You should now have a VM that will allow port 22 access by whitelisted IP’s and a WordPress that only allows logins from whitelisted IP’s.
Cons
  • If you forget to start your VPN you can’t log in to your VM via port 22 or log in to WordPress (excellent, this is by design).
Pros
  • Secure (need I say more)
 
I hope this guide helps someone.
Please consider using my UpCloud referral code and get $25 UpCloud VM credit for free when you signup to create a new VM.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
[contact-form-7 404 "Not Found"]
Revision History
v1.2 Added Proton plans link
v1.1 Added auto block WordFence option
v1.0 Initial Post
Adding two sub domains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04
 by 
Here is how I added two subdomains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04
Advertisement:

 
 
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
UpCloud performance is great.
Upcloud Site Speed in GTMetrix
Buy a domain name from Namecheap here.
Domain names for just 88 cents!
Goal(s)
Setup 2x subdomains on https://fearby.com
– Sub Domain #1: https://test.fearby.com (pointing to a dedicated UpCloud VM in Singapore for testing).
– Sub Domain #2: https://audit.fearby.com (pointing to a sub-website on the NGINX/VM that runs https://fearby.com )
Let’s set up the first Sub Domain (dedicated VM) and SSL
Backup
Do backup your server first.
VM
I created a second server ($5 month or $0.07c hour 1,024MB Memory, 25GB Disk, 1024 GB Month Data Transfer) at UpCloud. If you don’t already have an account at UpCloud use this link to signup and get $25 free credit ( https://www.upcloud.com/register/?promo=D84793 ). Read my blog post on why UpCloud is awesome and how I moved my domain to UpCloud.
Once I spun up a second server I obtained the IPv4 and IPv6 IP addresses of the new “test” VM from the UpCloud dashboard.
DNS
These DNS records were already in place with my DNS provider (Cloudflare).
I added these DNS records for the subdomains.
I added a new A NAME record for the new shared NGINX subdomain (for https://audit.fearby.com), this subdomain will be a sub-website that is running off the same server as https://fearby.com
I added another set of records for the new dedicated VM  subdomain (for https://test.fearby.com)
I waited for DNS to replicate around the globe by watching https://www.whatsmydns.net/
Setup a Firewall
On the new dedicated https://test.fearby.com VM, I installed the ufw firewall.
I configured the firewall to allow minimum ports (and added whitelisted IP for port 22 and added UpCloud DNS servers). I will lock this down some more later.
TIP: If your ISP does not offer a dedicated IP try a VPN. I use https://cyberghostvpn.com on OSX and Android.
Firewall rules.
I enabled the firewall.
Install NGINX (on https://test.fearby.com)
On the new dedicated https://test.fearby.com VM I…
Created a new www root
Set permissions
Installed NGINX
I created a placeholder webpage
Configured the root value in /etc/nginx/sites-available/default
Created a symbolic link of the nginx config
Lets Encrypt SSL
I have previously setup Lets encrypt on Ubuntu 16.04 but not 18.04. Certbot had info on setting up Lets Encrypt for 14.x 16.x and 17.x but not 18.x
Full credit for the SSL steps goes to @Linuxize ( tips on setting up Lets Encrypt on Ubuntu 18.04 ). Check out https://linuxize.com/
I installed Lets Encrypt certbot
I created a new Diffie–Hellman key
Map requests to http://test.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).
Create a /etc/nginx/snippets/letsencrypt.conf on http://test.fearby.com and enforce the redirect.
Create a /etc/nginx/snippets/ssl.conf file on http://test.fearby.com
Let’s get a certificate
Certificates have been created 🙂
Now lets edit “/etc/nginx/sites-available/default” on https://test.fearby.com VM and add the cert paths.
Advertisement:

 
 
Reload NGINX
or
Now let’s setup the second subdomain (subsite off https://fearby.com) and SSL
VM
I already have NGINX on https://fearby.com set up a second site.
DNS
We have already setup a DNS record for https://audit.fearby.com (above)
Firewall
Already configured at https://fearby.com
SSL
Because I had an existing Comodo certificate on https://fearby.com I am going to repeat the steps above to generate a new certificate but save the NGINX config to /etc/nginx/sites-available/audit.fearby.com (this activates the second site)
TIP: Follow the Linuxize guide here (for creating ssl.conf, letsencrypt.conf etc config files), Do a backup and restore if need be.
I created a new Diffie–Hellman key
Let’s get a certificate
Configure NGINX
Map requests to http://audit.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).
I created a new NGINX site ( /etc/nginx/sites-available/audit.fearby.com )
I created a symbolic link of the config file
Reload NGINX
or
How to test the certificate renewal
Automate the renewal in crontab (every 12 hours)
I set this crontab entry up on https://fearby.com and https://test.fearby.com
Conclusion
Yes, I haVe 2 subdomains (1x dedicated VM and the other is a sub-website off an existing server) with SSL certificates.
Ping Results
Webpage Results
Screenshow showing the main site and 2 subdomains in a web browser
Troubleshooting
If you are having troubles generating the initial certificate check that you have not blocked port 80 and don’t have “Strict-Transport-Security” heavers enabled.
I re-ran the certbot command but pointed to the real /www-root (not/var/lib/letsencrypt/)
Create a new
mkdir /www-root/.well-known/
mkdir /www-root/.well-known/acme-challenge/
sudo certbot certonly --agree-tos --email [email protected] --webroot -w /www-root -d yoursubdomain.domain.com
 
 
Advertisement:

 
 
I hope this guide helps someone.
Please consider using my referral code and get $25 credit for free.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
[contact-form-7 404 "Not Found"]
Revision History
v1.1 Troubleshooting
v1.0 Initial Post
Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space
 by 
Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space
Advertisement:

 
 
If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).
Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
Spoiler: UpCloud performance is great.
Upcloud Site Speed in GTMetrix
Why Upgrade
I have 1 CPU, 1 GB memory and 50GB storage and it is running well?  I have PHP child workers set up and have set up the preferred use of memory over swap file usage.
View of htop querying processes on Ubuntu
Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here).  To further increase the speed of the WordPress I have configured the OS to use memory over the disk.  About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).
Graph of memory falling below 100MB every day
I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.
How to Upgrade
This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.
UpCloud Pricing: https://www.upcloud.com/pricing/
Pricing table form https://www.upcloud.com/pricing/
Upgrade an UpCloud VM
I shut down my existing VM. Read this guide to setup a VM.
Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.
Upgrade the Server, shut it down the server and choose upgrade
I confirmed the upgrade options (2x CPU, 4096 MB Memory).
Confirm Upgrade Options
Click Update
Upgrade Applied
After 10 seconds you can start your server from the UpCloud server.
I confirmed the CPU upgrade was visible in the VM
Software Tweaks Post Upgrade.
I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.
I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase the memory usage and child workers. I doubled child workers and max memeory limit.
I restarted NGINX and PHP
I tweaked WordPress max memory limits
MySQL Tweaks: I logged into MySQL
I ran “SHOW GLOBAL STATUS” to view stats
mysql> SHOW GLOBAL STATUS;
+-----------------------------------------------+--------------------------------------------------+
| Variable_name                                 | Value                                            |
+-----------------------------------------------+--------------------------------------------------+
| Aborted_clients                               | 0                                                |
| Aborted_connects                              | 0                                                |
| Binlog_cache_disk_use                         | 0                                                |
| Binlog_cache_use                              | 0                                                |
| Binlog_stmt_cache_disk_use                    | 0                                                |
| Binlog_stmt_cache_use                         | 0                                                |
| Bytes_received                                | 3179986                                          |
| Bytes_sent                                    | 223872114                                        |
| Com_admin_commands                            | 0                                                |
| Com_assign_to_keycache                        | 0                                                |
| Com_alter_db                                  | 0                                                |
| Com_alter_db_upgrade                          | 0                                                |
| Com_alter_event                               | 0                                                |
| Com_alter_function                            | 0                                                |
| Com_alter_instance                            | 0                                                |
| Com_alter_procedure                           | 0                                                |
| Com_alter_server                              | 0                                                |
| Com_alter_table                               | 0                                                |
| Com_alter_tablespace                          | 0                                                |
| Com_alter_user                                | 0                                                |
| Com_analyze                                   | 0                                                |
| Com_begin                                     | 0                                                |
| Com_binlog                                    | 0                                                |
| Com_call_procedure                            | 0                                                |
| Com_change_db                                 | 284                                              |
| Com_change_master                             | 0                                                |
| Com_change_repl_filter                        | 0                                                |
| Com_check                                     | 0                                                |
| Com_checksum                                  | 0                                                |
| Com_commit                                    | 0                                                |
| Com_create_db                                 | 0                                                |
| Com_create_event                              | 0                                                |
| Com_create_function                           | 0                                                |
| Com_create_index                              | 0                                                |
| Com_create_procedure                          | 0                                                |
| Com_create_server                             | 0                                                |
| Com_create_table                              | 0                                                |
| Com_create_trigger                            | 0                                                |
| Com_create_udf                                | 0                                                |
| Com_create_user                               | 0                                                |
| Com_create_view                               | 0                                                |
| Com_dealloc_sql                               | 0                                                |
| Com_delete                                    | 18                                               |
| Com_delete_multi                              | 0                                                |
| Com_do                                        | 0                                                |
| Com_drop_db                                   | 0                                                |
| Com_drop_event                                | 0                                                |
| Com_drop_function                             | 0                                                |
| Com_drop_index                                | 0                                                |
| Com_drop_procedure                            | 0                                                |
| Com_drop_server                               | 0                                                |
| Com_drop_table                                | 0                                                |
| Com_drop_trigger                              | 0                                                |
| Com_drop_user                                 | 0                                                |
| Com_drop_view                                 | 0                                                |
| Com_empty_query                               | 0                                                |
| Com_execute_sql                               | 0                                                |
| Com_explain_other                             | 0                                                |
| Com_flush                                     | 0                                                |
| Com_get_diagnostics                           | 0                                                |
| Com_grant                                     | 0                                                |
| Com_ha_close                                  | 0                                                |
| Com_ha_open                                   | 0                                                |
| Com_ha_read                                   | 0                                                |
| Com_help                                      | 0                                                |
| Com_insert                                    | 342                                              |
| Com_insert_select                             | 0                                                |
| Com_install_plugin                            | 0                                                |
| Com_kill                                      | 0                                                |
| Com_load                                      | 0                                                |
| Com_lock_tables                               | 0                                                |
| Com_optimize                                  | 0                                                |
| Com_preload_keys                              | 0                                                |
| Com_prepare_sql                               | 0                                                |
| Com_purge                                     | 0                                                |
| Com_purge_before_date                         | 0                                                |
| Com_release_savepoint                         | 0                                                |
| Com_rename_table                              | 0                                                |
| Com_rename_user                               | 0                                                |
| Com_repair                                    | 0                                                |
| Com_replace                                   | 0                                                |
| Com_replace_select                            | 0                                                |
| Com_reset                                     | 0                                                |
| Com_resignal                                  | 0                                                |
| Com_revoke                                    | 0                                                |
| Com_revoke_all                                | 0                                                |
| Com_rollback                                  | 0                                                |
| Com_rollback_to_savepoint                     | 0                                                |
| Com_savepoint                                 | 0                                                |
| Com_select                                    | 16358                                            |
| Com_set_option                                | 849                                              |
| Com_signal                                    | 0                                                |
| Com_show_binlog_events                        | 0                                                |
| Com_show_binlogs                              | 0                                                |
| Com_show_charsets                             | 0                                                |
| Com_show_collations                           | 0                                                |
| Com_show_create_db                            | 0                                                |
| Com_show_create_event                         | 0                                                |
| Com_show_create_func                          | 0                                                |
| Com_show_create_proc                          | 0                                                |
| Com_show_create_table                         | 0                                                |
| Com_show_create_trigger                       | 0                                                |
| Com_show_databases                            | 3                                                |
| Com_show_engine_logs                          | 0                                                |
| Com_show_engine_mutex                         | 0                                                |
| Com_show_engine_status                        | 0                                                |
| Com_show_events                               | 0                                                |
| Com_show_errors                               | 0                                                |
| Com_show_fields                               | 921                                              |
| Com_show_function_code                        | 0                                                |
| Com_show_function_status                      | 0                                                |
| Com_show_grants                               | 0                                                |
| Com_show_keys                                 | 1                                                |
| Com_show_master_status                        | 0                                                |
| Com_show_open_tables                          | 0                                                |
| Com_show_plugins                              | 0                                                |
| Com_show_privileges                           | 0                                                |
| Com_show_procedure_code                       | 0                                                |
| Com_show_procedure_status                     | 0                                                |
| Com_show_processlist                          | 0                                                |
| Com_show_profile                              | 0                                                |
| Com_show_profiles                             | 0                                                |
| Com_show_relaylog_events                      | 0                                                |
| Com_show_slave_hosts                          | 0                                                |
| Com_show_slave_status                         | 0                                                |
| Com_show_status                               | 6                                                |
| Com_show_storage_engines                      | 0                                                |
| Com_show_table_status                         | 0                                                |
| Com_show_tables                               | 2                                                |
| Com_show_triggers                             | 0                                                |
| Com_show_variables                            | 6                                                |
| Com_show_warnings                             | 1                                                |
| Com_show_create_user                          | 0                                                |
| Com_shutdown                                  | 0                                                |
| Com_slave_start                               | 0                                                |
| Com_slave_stop                                | 0                                                |
| Com_group_replication_start                   | 0                                                |
| Com_group_replication_stop                    | 0                                                |
| Com_stmt_execute                              | 4                                                |
| Com_stmt_close                                | 4                                                |
| Com_stmt_fetch                                | 0                                                |
| Com_stmt_prepare                              | 4                                                |
| Com_stmt_reset                                | 0                                                |
| Com_stmt_send_long_data                       | 4                                                |
| Com_truncate                                  | 2                                                |
| Com_uninstall_plugin                          | 0                                                |
| Com_unlock_tables                             | 0                                                |
| Com_update                                    | 70                                               |
| Com_update_multi                              | 0                                                |
| Com_xa_commit                                 | 0                                                |
| Com_xa_end                                    | 0                                                |
| Com_xa_prepare                                | 0                                                |
| Com_xa_recover                                | 0                                                |
| Com_xa_rollback                               | 0                                                |
| Com_xa_start                                  | 0                                                |
| Com_stmt_reprepare                            | 0                                                |
| Connection_errors_accept                      | 0                                                |
| Connection_errors_internal                    | 0                                                |
| Connection_errors_max_connections             | 0                                                |
| Connection_errors_peer_address                | 0                                                |
| Connection_errors_select                      | 0                                                |
| Connection_errors_tcpwrap                     | 0                                                |
| Connections                                   | 292                                              |
| Created_tmp_disk_tables                       | 1124                                             |
| Created_tmp_files                             | 30                                               |
| Created_tmp_tables                            | 1369                                             |
| Delayed_errors                                | 0                                                |
| Delayed_insert_threads                        | 0                                                |
| Delayed_writes                                | 0                                                |
| Flush_commands                                | 1                                                |
| Handler_commit                                | 6094                                             |
| Handler_delete                                | 33                                               |
| Handler_discover                              | 0                                                |
| Handler_external_lock                         | 38571                                            |
| Handler_mrr_init                              | 0                                                |
| Handler_prepare                               | 0                                                |
| Handler_read_first                            | 2299                                             |
| Handler_read_key                              | 134761                                           |
| Handler_read_last                             | 237                                              |
| Handler_read_next                             | 310119                                           |
| Handler_read_prev                             | 2733                                             |
| Handler_read_rnd                              | 222350                                           |
| Handler_read_rnd_next                         | 472820                                           |
| Handler_rollback                              | 0                                                |
| Handler_savepoint                             | 0                                                |
| Handler_savepoint_rollback                    | 0                                                |
| Handler_update                                | 15605                                            |
| Handler_write                                 | 17310                                            |
| Innodb_buffer_pool_dump_status                | Dumping of buffer pool not started               |
| Innodb_buffer_pool_load_status                | Buffer pool(s) load completed at 180624 23:38:01 |
| Innodb_buffer_pool_resize_status              |                                                  |
| Innodb_buffer_pool_pages_data                 | 1035                                             |
| Innodb_buffer_pool_bytes_data                 | 16957440                                         |
| Innodb_buffer_pool_pages_dirty                | 0                                                |
| Innodb_buffer_pool_bytes_dirty                | 0                                                |
| Innodb_buffer_pool_pages_flushed              | 1936                                             |
| Innodb_buffer_pool_pages_free                 | 7144                                             |
| Innodb_buffer_pool_pages_misc                 | 13                                               |
| Innodb_buffer_pool_pages_total                | 8192                                             |
| Innodb_buffer_pool_read_ahead_rnd             | 0                                                |
| Innodb_buffer_pool_read_ahead                 | 0                                                |
| Innodb_buffer_pool_read_ahead_evicted         | 0                                                |
| Innodb_buffer_pool_read_requests              | 306665                                           |
| Innodb_buffer_pool_reads                      | 950                                              |
| Innodb_buffer_pool_wait_free                  | 0                                                |
| Innodb_buffer_pool_write_requests             | 26509                                            |
| Innodb_data_fsyncs                            | 1229                                             |
| Innodb_data_pending_fsyncs                    | 0                                                |
| Innodb_data_pending_reads                     | 0                                                |
| Innodb_data_pending_writes                    | 0                                                |
| Innodb_data_read                              | 16273920                                         |
| Innodb_data_reads                             | 1078                                             |
| Innodb_data_writes                            | 2857                                             |
| Innodb_data_written                           | 53379584                                         |
| Innodb_dblwr_pages_written                    | 1275                                             |
| Innodb_dblwr_writes                           | 109                                              |
| Innodb_log_waits                              | 0                                                |
| Innodb_log_write_requests                     | 450                                              |
| Innodb_log_writes                             | 585                                              |
| Innodb_os_log_fsyncs                          | 793                                              |
| Innodb_os_log_pending_fsyncs                  | 0                                                |
| Innodb_os_log_pending_writes                  | 0                                                |
| Innodb_os_log_written                         | 664064                                           |
| Innodb_page_size                              | 16384                                            |
| Innodb_pages_created                          | 56                                               |
| Innodb_pages_read                             | 988                                              |
| Innodb_pages_written                          | 1936                                             |
| Innodb_row_lock_current_waits                 | 0                                                |
| Innodb_row_lock_time                          | 0                                                |
| Innodb_row_lock_time_avg                      | 0                                                |
| Innodb_row_lock_time_max                      | 0                                                |
| Innodb_row_lock_waits                         | 0                                                |
| Innodb_rows_deleted                           | 2                                                |
| Innodb_rows_inserted                          | 19219                                            |
| Innodb_rows_read                              | 249102                                           |
| Innodb_rows_updated                           | 77                                               |
| Innodb_num_open_files                         | 81                                               |
| Innodb_truncated_status_writes                | 0                                                |
| Innodb_available_undo_logs                    | 128                                              |
| Key_blocks_not_flushed                        | 0                                                |
| Key_blocks_unused                             | 12751                                            |
| Key_blocks_used                               | 645                                              |
| Key_read_requests                             | 321877                                           |
| Key_reads                                     | 648                                              |
| Key_write_requests                            | 196                                              |
| Key_writes                                    | 150                                              |
| Locked_connects                               | 0                                                |
| Max_execution_time_exceeded                   | 0                                                |
| Max_execution_time_set                        | 0                                                |
| Max_execution_time_set_failed                 | 0                                                |
| Max_used_connections                          | 3                                                |
| Max_used_connections_time                     | 2018-06-24 23:43:48                              |
| Not_flushed_delayed_rows                      | 0                                                |
| Ongoing_anonymous_transaction_count           | 0                                                |
| Open_files                                    | 229                                              |
| Open_streams                                  | 0                                                |
| Open_table_definitions                        | 206                                              |
| Open_tables                                   | 786                                              |
| Opened_files                                  | 502                                              |
| Opened_table_definitions                      | 208                                              |
| Opened_tables                                 | 817                                              |
| Performance_schema_accounts_lost              | 0                                                |
| Performance_schema_cond_classes_lost          | 0                                                |
| Performance_schema_cond_instances_lost        | 0                                                |
| Performance_schema_digest_lost                | 0                                                |
| Performance_schema_file_classes_lost          | 0                                                |
| Performance_schema_file_handles_lost          | 0                                                |
| Performance_schema_file_instances_lost        | 0                                                |
| Performance_schema_hosts_lost                 | 0                                                |
| Performance_schema_index_stat_lost            | 0                                                |
| Performance_schema_locker_lost                | 0                                                |
| Performance_schema_memory_classes_lost        | 0                                                |
| Performance_schema_metadata_lock_lost         | 0                                                |
| Performance_schema_mutex_classes_lost         | 0                                                |
| Performance_schema_mutex_instances_lost       | 0                                                |
| Performance_schema_nested_statement_lost      | 0                                                |
| Performance_schema_prepared_statements_lost   | 0                                                |
| Performance_schema_program_lost               | 0                                                |
| Performance_schema_rwlock_classes_lost        | 0                                                |
| Performance_schema_rwlock_instances_lost      | 0                                                |
| Performance_schema_session_connect_attrs_lost | 0                                                |
| Performance_schema_socket_classes_lost        | 0                                                |
| Performance_schema_socket_instances_lost      | 0                                                |
| Performance_schema_stage_classes_lost         | 0                                                |
| Performance_schema_statement_classes_lost     | 0                                                |
| Performance_schema_table_handles_lost         | 0                                                |
| Performance_schema_table_instances_lost       | 0                                                |
| Performance_schema_table_lock_stat_lost       | 0                                                |
| Performance_schema_thread_classes_lost        | 0                                                |
| Performance_schema_thread_instances_lost      | 0                                                |
| Performance_schema_users_lost                 | 0                                                |
| Prepared_stmt_count                           | 0                                                |
| Qcache_free_blocks                            | 1                                                |
| Qcache_free_memory                            | 16760152                                         |
| Qcache_hits                                   | 0                                                |
| Qcache_inserts                                | 0                                                |
| Qcache_lowmem_prunes                          | 0                                                |
| Qcache_not_cached                             | 16355                                            |
| Qcache_queries_in_cache                       | 0                                                |
| Qcache_total_blocks                           | 1                                                |
| Queries                                       | 19164                                            |
| Questions                                     | 19155                                            |
| Select_full_join                              | 0                                                |
| Select_full_range_join                        | 0                                                |
| Select_range                                  | 2677                                             |
| Select_range_check                            | 0                                                |
| Select_scan                                   | 2098                                             |
| Slave_open_temp_tables                        | 0                                                |
| Slow_launch_threads                           | 0                                                |
| Slow_queries                                  | 0                                                |
| Sort_merge_passes                             | 12                                               |
| Sort_range                                    | 4859                                             |
| Sort_rows                                     | 244452                                           |
| Sort_scan                                     | 854                                              |
| Ssl_accept_renegotiates                       | 0                                                |
| Ssl_accepts                                   | 0                                                |
| Ssl_callback_cache_hits                       | 0                                                |
| Ssl_cipher                                    |                                                  |
| Ssl_cipher_list                               |                                                  |
| Ssl_client_connects                           | 0                                                |
| Ssl_connect_renegotiates                      | 0                                                |
| Ssl_ctx_verify_depth                          | 0                                                |
| Ssl_ctx_verify_mode                           | 0                                                |
| Ssl_default_timeout                           | 0                                                |
| Ssl_finished_accepts                          | 0                                                |
| Ssl_finished_connects                         | 0                                                |
| Ssl_server_not_after                          |                                                  |
| Ssl_server_not_before                         |                                                  |
| Ssl_session_cache_hits                        | 0                                                |
| Ssl_session_cache_misses                      | 0                                                |
| Ssl_session_cache_mode                        | NONE                                             |
| Ssl_session_cache_overflows                   | 0                                                |
| Ssl_session_cache_size                        | 0                                                |
| Ssl_session_cache_timeouts                    | 0                                                |
| Ssl_sessions_reused                           | 0                                                |
| Ssl_used_session_cache_entries                | 0                                                |
| Ssl_verify_depth                              | 0                                                |
| Ssl_verify_mode                               | 0                                                |
| Ssl_version                                   |                                                  |
| Table_locks_immediate                         | 11962                                            |
| Table_locks_waited                            | 0                                                |
| Table_open_cache_hits                         | 19395                                            |
| Table_open_cache_misses                       | 817                                              |
| Table_open_cache_overflows                    | 12                                               |
| Tc_log_max_pages_used                         | 0                                                |
| Tc_log_page_size                              | 0                                                |
| Tc_log_page_waits                             | 0                                                |
| Threads_cached                                | 2                                                |
| Threads_connected                             | 1                                                |
| Threads_created                               | 3                                                |
| Threads_running                               | 1                                                |
| Uptime                                        | 2944                                             |
| Uptime_since_flush_status                     | 2944                                             |
+-----------------------------------------------+--------------------------------------------------+
353 rows in set (0.00 sec)
Read more on SHOW GLOBAL STATUS here. Read more on the values here.
I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.
Configure Ubuntu System Memory Usage
Edit /etc/sysctl.conf
Add the following to allow things to sit in ram longer
Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt
This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are."
Read these pages here and here regarding setting MySQL memory.
Reboot
Resize the disk
The 2x CPU, 4GB memory plan comes with 80GB storage allowance.  My disk at present is 50GB and I will update the size soon following this guide soon.
Upgrade disk from 50gb to 80GB soon
Quick Benchmark
I used loader.io to load 500 users to access my site in 1 minute.
HTOP showing 2x busy CPU's running at 60%
The benchmark worked with no errors.
Loader.io Success with 500 concurrent users
This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.
Conclusion
After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.
No Low memory low triggers
This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.
I hope this guide helps someone.
Please consider using my referral code and get $25 credit for free.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
[contact-form-7 404 "Not Found"]
Revision History
1.0 Initial Post
How to use the UpCloud API to manage your UpCloud servers
 by 
How to use the UpCloud API to manage your UpCloud servers.
Advertisement:

 
 
If you have not read my previous posts I have now moved my blog etc to the awesome UpCloud host. Sign up using this link to get $25 free credit.
I recently compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here).
Here is my blog post on moving from Vultr to UpCloud.
Spoiler: UpCloud performance is great.
Upcloud Site Speed in GTMetrix
I have never had an UpCloud page load take longer than 2 seconds since moving.
UpCloud API
UpCloud has an API that we can opt into to using where we can manage servers. Read the official UpCloud API documentation here.
The API allows you to control:
  • Accounts
  • Pricing
  • Zones
  • Timezones
  • Plans
  • Servers
  • Storages
  • IP-Addresses
  • Firewall
  • Tags
  • etc
Create a sub-account to query the API 
You should create a new user account (in the UpCloud dasbboard) just for API access. I created two accounts for use on my server and on my home laptop and my server (and set a limiting IP(s) that can access it).
Create a Sub Account for API Access
Login to your UpCloud account (create an account here and get $25 free credit),
  1. Click My Accounts,
  2. Click User Accounts,
  3. Click Change on your user and enable API connections.
  4. TIP: Set up an IP rule to limit access to your API for security (I set up a VPN to get a static IP on my dynamic IP Internet host at home)).
  5. Save the changes
Enable API Connections
TIP: Lock down the account to have the minimum permissions required.
e.g
  • Disable access to the control panel (Untick).
  • Allow API Connections (Tick) and specify an IP
  • Disable access to billing contact (Untick).
  • Disable access to billing section in the control panel (Untick).
  • Disable allowing of emails to billing contact (Untick).
  • Allow or Remove access to all server (or manually add access to desired servers)
  • Allow or Remove access to modify storage (or manually allow or remove access to desired storage)
  • etc
Lock down the account to the minimum needed
Save the account.
Now let’s make our first API call
I use OSX and I use the awesome Paw API testing tool from https://paw.cloud (This is not a plug, they are awesome). Postman is a popular API testing tool too. Any good programing language or CLI will allow you to send API requests.
First, let’s prepare the authorization string (this is a Base64 encoded combination of your username and password) read more here.
  1. Head over to https://www.base64encode.org/
  2. Click the Encode tab
  3. Add your “username:password” (without the quotes).
  4. Click Encode
A Base64 string will be outputted 🙂
e.g > eW91cmFwaXVzZXJuYW1lOnlvdXJzdXBlcnNlY3VyZXBhc3N3b3Jk
fyi
You can encode also Encode and Decode Base64 from the Ubuntu Command line
Encode Base64 from the CLI Sample
Decode Base64 from the CLI Sample
Now we can add an “Authorization Basic” token to the API request in Paw.
Authorization Header added with my base64 token.
A quick test of the UpCloud Prices API endpoint https://api.upcloud.com/1.2/price reveals the API is working.
Add Authorization Token
I can now see a full breakdown of my service prices in JSON 🙂
Query My Account
OK, Let’s see how much credit I have left by querying the https://api.upcloud.com/1.2/account, I duplicated the item in Paw and changed the URL to https://api.upcloud.com/1.2/account but no data returned?
I had to enable “Access to Billing section in Control Panel” for the user before this data returned from the API (make sense).
> HTTP/1.1 200 OK
Query (GET)
GET /1.2/account HTTP/1.1
Host: api.upcloud.com
User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23
Authorization: Basic *******************************************
Output
HTTP/1.1 200 OK
Date: Sun, 17 Jun 2018 04:23:32 GMT
Content-Type: application/json; charset=UTF-8
Connection: close
Content-Length: 91
Server: Apache

{
   "account" : {
      "credits" : 2500.00,
      "username" : "yourapiusername"
   }
}
“2500.00” = cents ($25)
Query All of Your Servers
Ok, Let’s get server information by querying https://api.upcloud.com/1.2/server
Query (GET)
GET /1.2/server HTTP/1.1
Host: api.upcloud.com
User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23
Authorization: Basic ##############base64hash##############
Output
HTTP/1.1 200 OK
Date: Sun, 17 Jun 2018 04:32:22 GMT
Content-Type: application/json; charset=UTF-8
Connection: close
Content-Length: 1154
Server: Apache

{
   "servers" : {
      "server" : [
         {
            "core_number" : "1",
            "hostname" : "server1nameredacted.com",
            "license" : 0,
            "memory_amount" : "2048",
            "plan" : "1xCPU-2GB",
            "plan_ipv4_bytes" : "3472464313",
            "plan_ipv6_bytes" : "166293599",
            "state" : "started",
            "tags" : {
               "tag" : [
                  "tag1"
               ]
            },
            "title" : "server1nameredacted.com",
            "uuid" : "########-####-####-####-############",
            "zone" : "us-chi1"
         },
         {
            "core_number" : "1",
            "hostname" : "server2nameredacted.com",
            "license" : 0,
            "memory_amount" : "1024",
            "plan" : "1xCPU-1GB",
            "plan_ipv4_bytes" : "198911",
            "plan_ipv6_bytes" : "19742",
            "state" : "started",
            "tags" : {
               "tag" : [
                  "tag2"
               ]
            },
            "title" : "server1nameredacted.com",
            "uuid" : "########-####-####-####-############",
            "zone" : "us-chi1"
         }
      ]
   }
}
Query Server Information
I have redated the UUID’s for my servers but once you know them you can query them by hitting https://api.upcloud.com/1.2/server/########-####-####-####-############
Query (GET)
Output
HTTP/1.1 200 OK
Date: Sun, 17 Jun 2018 04:45:14 GMT
Content-Type: application/json; charset=UTF-8
Connection: close
Content-Length: 1656
Server: Apache

{
   "server" : {
      "boot_order" : "cdrom,disk",
      "core_number" : "1",
      "firewall" : "on",
      "host" : redacted,
      "hostname" : "server1nameredacted.com",
      "ip_addresses" : {
         "ip_address" : [
            {
               "access" : "private",
               "address" : "##.#.#.###",
               "family" : "IPv4"
            },
            {
               "access" : "public",
               "address" : "###.###.###.###",
               "family" : "IPv4",
               "part_of_plan" : "yes"
            },
            {
               "access" : "public",
               "address" : "####:####:####:####:####:####:########",
               "family" : "IPv6"
            }
         ]
      },
      "license" : 0,
      "memory_amount" : "2048",
      "nic_model" : "virtio",
      "plan" : "1xCPU-2GB",
      "plan_ipv4_bytes" : "3519033266",
      "plan_ipv6_bytes" : "168200052",
      "state" : "started",
      "storage_devices" : {
         "storage_device" : [
            {
               "address" : "virtio:0",
               "boot_disk" : "0",
               "part_of_plan" : "yes",
               "storage" : "########-####-####-####-############",
               "storage_size" : 50,
               "storage_title" : "system",
               "type" : "disk"
            }
         ]
      },
      "tags" : {
         "tag" : [
            "fearby"
         ]
      },
      "timezone" : "Australia/Sydney",
      "title" : "server1nameredacted.com",
      "uuid" : "########-####-####-####-############",
      "video_model" : "cirrus",
      "vnc" : "off",
      "vnc_password" : "#########################",
      "zone" : "us-chi1"
   }
}
The servers name, IPv4 and IPV6 network adapters are listed, CPU(s), Memory, Disk Sized and UUID’s are all visible 🙂
Surprisingly the VNC password is visible (enabling access to the root console).
TIP: Ensure your API account is safe and secure.
Query Storage Information
Now, Let’s query the storage with the GUID from above by querying https://api.upcloud.com/1.2/storage/########-####-####-####-############
Query (GET)
GET /1.2/storage/########-####-####-####-############ HTTP/1.1
Host: api.upcloud.com
User-Agent: Paw/3.1.7 (Macintosh; OS X/10.13.5) NSURLConnection/1452.23
Authorization: Basic  ##############base64hash##############
Output
HTTP/1.1 200 OK
Date: Sun, 17 Jun 2018 04:53:36 GMT
Content-Type: application/json; charset=UTF-8
Connection: close
Content-Length: 559
Server: Apache

{
   "storage" : {
      "access" : "private",
      "backup_rule" : {},
      "backups" : {
         "backup" : [
            "########-####-####-####-############"
         ]
      },
      "license" : 0,
      "part_of_plan" : "yes",
      "servers" : {
         "server" : [
            "########-####-####-####-############"
         ]
      },
      "size" : 50,
      "state" : "online",
      "tier" : "maxiops",
      "title" : "system",
      "type" : "normal",
      "uuid" : "########-####-####-####-############",
      "zone" : "us-chi1"
   }
}
I can see information about the storage’s assigned server and backups 🙂
Query Backup Information
Backup storage can be queried with the same storge API endpoint https://api.upcloud.com/1.2/storage/########-####-####-####-############
Query (GET)
Output
HTTP/1.1 200 OK
Date: Sun, 17 Jun 2018 05:01:11 GMT
Content-Type: application/json; charset=UTF-8
Connection: close
Content-Length: 412
Server: Apache

{
   "storage" : {
      "access" : "private",
      "created" : "2018-06-16T04:47:56Z",
      "license" : 0,
      "origin" : "########-####-####-####-############",
      "servers" : {
         "server" : []
      },
      "size" : 50,
      "state" : "online",
      "title" : "On-Demand Backup",
      "type" : "backup",
      "uuid" : "########-####-####-####-############",
      "zone" : "us-chi1"
   }
}
Rename Backup
One thing that I would like to be able to do is to rename on-demand backups in the UpCloud dashboard (this is not a feature yet) but I can rename manual backup’s in the API though 🙂
Boring “On-Demand Backup” label.
Rename Backups Not possible in the GUI
I tried sending JSON to https://api.upcloud.com/1.2/storage/########-####-####-####-############ to rename a backup but kept getting an error?
JSON
{
 > “storage”: {
 > “title”: “Latest manual backup , Working NGINX, PHP, MySQL w Tweaks”,
 > “size”: “50”
 > }
 > }
Result
> “error_code” : “CONTENT_TYPE_INVALID”,
 > “error_message” : “The Content-Type header has an invalid value.”
I googled and found an old manual for UpClouds API (official support here).
I added these missing content type headers (108 was the length in chars of the payload)
> Content-Type: application/json; Charset=UTF-8'
> Content-Length: 108
Still no go?
I think the content length value is wrong, more here.
I fixed it, it turned out I had a semicolon in the Content-Type value. The JSON RFC always assumes that Content-Type is UTF8 encoded (more here).
This Fails
This Works
Content-Type: application/json
Now I can rename my Backup (storage). I manually calculated the length of the JSON payload and added a “Content-Length” header and value.
Query (PUT)
Output
HTTP/1.1 202 ACCEPTED
Date: Sun, 17 Jun 2018 05:47:02 GMT
Content-Type: application/json; charset=UTF-8
Connection: close
Content-Length: 453
Server: Apache

{
   "storage" : {
      "access" : "private",
      "created" : "2018-06-16T04:47:56Z",
      "license" : 0,
      "origin" : "########-####-####-####-############",
      "servers" : {
         "server" : []
      },
      "size" : 50,
      "state" : "online",
      "title" : "Latest manual backup , Working NGINX, PHP, MySQL w Tweaks",
      "type" : "backup",
      "uuid" : "########-####-####-####-############",
      "zone" : "us-chi1"
   }
}
Success 🙂
Backup Renamed
Create a Backup
Backups can be performed with a “/backup” added to the end of the query string.
Query (POST)
Output
Success (UpCloud GUI)
Conclusion
UpCloud does have great API docs.
I can easily integrate this into bash scripts to manage my servers via API and a future Java app for managing servers.
Paw does give CURL output to allow me to copy working API’s for use in BASH 🙂
More to come
  1. BASH Script to Deploy and configure a server on UpCloud via Initalization scripts (or manual) (1 week)
  2. JAVA App to manage your server (3 months)
If you are signing up for UpCloud please consider using my referral code and get $25 credit for free.
Read my setup guide here.
https://www.upcloud.com/register/?promo=D84793
I hope this guide helps someone.
Ask a question or recommend an article
[contact-form-7 404 "Not Found"]
Revision History
V1.1 updated typo
v1.0 Initial Post.
Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4
 by 
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 4 of 4
Advertisement:

 
 
Read Part 1, Part 2, Part 3 or Part 4
I ran the MySQL benchmark preparation command again (no problem this time).
sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=###################### prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Creating table 'sbtest'...
Creating 1000000 records in table 'sbtest'...
Test table and records created
Test Records Created
Now I can benchmark MySQL on my main server.
sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=################################# --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run
RAW Output
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.

OLTP test statistics:
    queries performed:
        read:                            336210
        write:                           0
        other:                           48030
        total:                           384240
    transactions:                        24015  (400.09 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 336210 (5601.24 per sec.)
    other operations:                    48030  (800.18 per sec.)

Test execution summary:
    total time:                          60.0242s
    total number of events:              24015
    total time taken by event execution: 480.0242
    per-request statistics:
         min:                                  1.79ms
         avg:                                 19.99ms
         max:                                141.00ms
         approx.  95 percentile:              37.49ms

Threads fairness:
    events (avg/stddev):           3001.8750/27.36
    execution time (avg/stddev):   60.0030/0.01
Results
queries performed (in 60 seconds):
  • read: 336210
  • other: 48030
  • total: 384240
I decided to add an index to see if I can speed this query up (read the MySQL index page here). I added an index (in Adminer) on the columns “Id” and “pad” for the sbtest table in the test database
I restarted the MySQL process
mysql restart
[ ok ] Restarting mysql (via systemctl): mysql.service.
I ran the same benchmark again.
Raw Output
sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=########################## --max-time=60 --oltp-read-only=on --max-requests=0 --num-threads=8 run
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 8

Doing OLTP test.
Running mixed OLTP test
Doing read-only test
Using Special distribution (12 iterations,  1 pct of values are returned in 75 pct cases)
Using "BEGIN" for starting transactions
Using auto_inc on the id column
Threads started!
Time limit exceeded, exiting...
(last message repeated 7 times)
Done.

OLTP test statistics:
    queries performed:
        read:                            426538
        write:                           0
        other:                           60934
        total:                           487472
    transactions:                        30467  (507.69 per sec.)
    deadlocks:                           0      (0.00 per sec.)
    read/write requests:                 426538 (7107.67 per sec.)
    other operations:                    60934  (1015.38 per sec.)

Test execution summary:
    total time:                          60.0110s
    total number of events:              30467
    total time taken by event execution: 479.9124
    per-request statistics:
         min:                                  5.75ms
         avg:                                 15.75ms
         max:                                138.57ms
         approx.  95 percentile:              25.10ms

Threads fairness:
    events (avg/stddev):           3808.3750/8.70
    execution time (avg/stddev):   59.9891/0.00
Results
The quick index added 20% extra throughput on queries 🙂
Mysql before and after an index
Don’t forget to delete your test database
DROP DATABASE `test`;
Viewing MySQL Index Usage (on the “test” database)
Query to show Index stats for a table ‘test’
SELECT
 OBJECT_SCHEMA as 'Database', OBJECT_NAME as 'Table', 
 INDEX_NAME as 'Index', 
 COUNT_STAR, 
 SUM_TIMER_WAIT,  MIN_TIMER_WAIT, AVG_TIMER_WAIT, MAX_TIMER_WAIT, 
 COUNT_READ, 
 SUM_TIMER_READ, MIN_TIMER_READ, AVG_TIMER_READ, MAX_TIMER_READ,  
 COUNT_FETCH, SUM_TIMER_FETCH, MIN_TIMER_FETCH, AVG_TIMER_FETCH, MAX_TIMER_FETCH
FROM 
 performance_schema.table_io_waits_summary_by_index_usage
WHERE 
 object_schema = 'test'
I can see the MySQL PRIMARY index is getting used 🙂
Index Summary
Read more in viewable query stats (columns) here.
Other System Information Tools
Show processor information
cat /proc/cpuinfo
Output
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 61
model name      : Virtual CPU a7769a6388d5
stepping        : 2
microcode       : 0x1
cpu MHz         : 2394.454
cache size      : 16384 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm constant_tsc rep_good nopl xtopology eagerfpu pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm invpcid_single kaiser fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt arat
bugs            : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass
bogomips        : 4788.90
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:
Memory Information
You can assign 512MB, 1GB, 2GB or more memory to a server on Vultr, Read my guide here on upgrading resources for Vultr VM’s here.
Only upgrade your server’s memory when server processes demand it, there is no need to pay for extra idle memory. Read my older guides on upgrading Digital Ocean and AWS servers.
I use the htop utility to monitor memory and processes. The memory usage will depend on how you have configured your server to use connection pools in code, MySQL or services.  Also what memory demands do you get in pean bandwidth times?
HTOP
You can check your server memory details on Ubuntu with this command
cat /proc/meminfo
Output
MemTotal:        2048104 kB
MemFree:           96176 kB
MemAvailable:     693072 kB
Buffers:          183476 kB
Cached:           526124 kB
SwapCached:            0 kB
Active:          1467220 kB
Inactive:         243228 kB
Active(anon):    1070464 kB
Inactive(anon):    27004 kB
Active(file):     396756 kB
Inactive(file):   216224 kB
Unevictable:        3652 kB
Mlocked:            3652 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Dirty:                64 kB
Writeback:             0 kB
AnonPages:       1004504 kB
Mapped:           114664 kB
Shmem:             94192 kB
Slab:             192692 kB
SReclaimable:     171892 kB
SUnreclaim:        20800 kB
KernelStack:        3072 kB
PageTables:        20528 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:     1024052 kB
Committed_AS:    2424332 kB
VmallocTotal:   34359738367 kB
VmallocUsed:           0 kB
VmallocChunk:          0 kB
HardwareCorrupted:     0 kB
AnonHugePages:    247808 kB
CmaTotal:              0 kB
CmaFree:               0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
DirectMap4k:       67440 kB
DirectMap2M:     2029568 kB
Use Memory or Disk (Swap)
You can configure the use of Memory over Disk by configuring your/etc/sysctl.conf file (setting value “vm.swappiness”)
You can check your swap file settings by running the following command
cat /proc/sys/vm/swappiness
1
Or By running
sysctl vm.swappiness
vm.swappiness = 1
Set a new swap file value by editing /etc/sysctl.conf
sudo nano /etc/sysctl.conf
Set the following to use more ram over the swap disk.
vm.swappiness = 1
Read about swappiness values here: https://en.wikipedia.org/wiki/Swappiness
Service Performance
Performance (and allocated resources) depends on the demands of your operating system and installed software
What operating system do you have?
lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 16.04.4 LTS
Release:        16.04
Codename:       xenial
View NGINX Status, how much memory does it use?
/etc/init.d/nginx status
● nginx.service - A high performance web server and a reverse proxy server
   Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-05-25 21:28:25 AEST; 1 weeks 3 days ago
     Docs: man:nginx(8)
 Main PID: #### (nginx)
    Tasks: 3
   Memory: 58.9M
      CPU: 33min 11.515s
   CGroup: /system.slice/nginx.service
           ├─#### nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
           ├─#### nginx: worker process
           └─#### nginx: cache manager process
PHP (and Child Worker) status how much memory does it use and how many child workers do you have? Read my add PHP child workers post here (and update to PHP 7.2 here)
sudo service php7.2-fpm status
● php7.2-fpm.service - The PHP 7.2 FastCGI Process Manager
   Loaded: loaded (/lib/systemd/system/php7.2-fpm.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-05-25 21:28:26 AEST; 1 weeks 3 days ago
     Docs: man:php-fpm7.2(8)
 Main PID: #### (php-fpm7.2)
   Status: "Processes active: 0, idle: 20, Requests: 75911, slow: 0, Traffic: 0.1req/sec"
    Tasks: 21
   Memory: 694.2M
      CPU: 20h 49min 45.132s
   CGroup: /system.slice/php7.2-fpm.service
           ├─ #### php-fpm: master process (/etc/php/7.2/fpm/php-fpm.conf)
           ├─ #### php-fpm: pool www-acc
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-acc
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           ├─ #### php-fpm: pool www-usr
           └─ #### php-fpm: pool www-usr
MySQL Status
sudo service mysql status
● mysql.service - MySQL Community Server
   Loaded: loaded (/lib/systemd/system/mysql.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2018-05-25 21:28:27 AEST; 1 weeks 3 days ago
 Main PID: ##### (mysqld)
    Tasks: 35
   Memory: 405.9M
      CPU: 2h 17min 31.822s
   CGroup: /system.slice/mysql.service
           └─#### /usr/sbin/mysqld
Shared VM Hosts
One of the biggest impacts (after server latency) for your server is not the disk performance but the number of hosts/websites on the server who are also using the disk and server resources.
Reverse IP Lookup
I have 80 other web servers on my server (based on a reverse lookup).
I may move to a dedicated box when I can afford it.
Security
Above all else ensure that security is number 1 priority and make performance second priority.
Scan your site with Zap, Qualys and Kali Linux. Performance means nothing if you are hacked.
website-report
Simulated concurrent users
You can use Siege to test the maximum concurrent users accessing your site before the server starts to drop connections.
FYI: If you use Cloudflare (you should) this may not work as it will block connections.
Install Siege
sudo apt-get install siege
Test  your server with 10 concurrent serves for 1 minute
siege -t1m c10 'https://yourserver.com/'
Results
siege -t1m c10 'https://yourserver.com/'
** SIEGE 3.0.8
** Preparing 15 concurrent users for battle.
The server is now under siege...
Lifting the server siege...      done.

Transactions:                    417 hits
Availability:                 100.00 %
Elapsed time:                  59.01 secs
Data transferred:               8.24 MB
Response time:                  1.62 secs
Transaction rate:               7.07 trans/sec
Throughput:                     0.14 MB/sec
Concurrency:                   11.46
Successful transactions:         417
Failed transactions:               0
Longest transaction:            2.26
Shortest transaction:           1.49
Keep upping the connections (from 10 above) to a limit where connections start dropping.
I tried 25 then 50 concurrent users hitting a server on Digital Ocean and it did not fail.
Conclusion
  • Choose a server near your customers
  • Change hosts if one is faster and cheaper
  • Measure or benchmark your server (and compare over time).
  • Use Cloudflare
Create your own server today
  • Create your own server on Vultr here.
  • Create your own server on Digital Ocean here.
  • Create your own server on UpCloud here.
And remember you can install the Runcloud server management dashboard here.
I hope this guide helps someone.
 
< Previous
Read Part 1, Part 2, Part 3 or Part 4
Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4
 by 
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 3 of 4
Advertisement:

 
 
Read Part 1, Part 2, Part 3 or Part 4
I used these commands to generate bonnie++ reports from the data in part 2
echo "<h1>Bonnie Results</h1>" > /www-data/bonnie.html
echo "<h2>Vultr (Sydney)</h2>" >> /www-data/bonnie.html
echo "1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us" | bon_csv2html >> /www-data/bonnie.html
echo "<h2>Digital Ocean (London)</h2>" >> /www-data/bonnie.html
echo "1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us" | bon_csv2html >> /www-data/bonnie.html
echo "<h2>UpCloud (Singapore)</h2>" >> /www-data/bonnie.html
echo "1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us" | bon_csv2html >> /www-data/bonnie.html
Image of results here
Bonnie Results
Network Performace
IMHO Network Latency is the biggest impact on server performance, Read my old post on scalability on a budget here. I am in Australia an having a server in Singapore was too far away and latency was terrible.
Here is a non-scientific example of pinging a Vultr, Digital Ocean and UpCloud server in three different locations (and Google).
Ping Test
Test Ping Results
Latency matters, run a https://www.webpagetest.org/ scan over your site to see why.
Adding https added almost 0.7 seconds to https communications in the past on Digital Ocean (a few thousand kilometres away). The longer the latency the longer HTTPS handshakes take.
SSL
Deploying a server to Singapore (in my experience) is bad if your visitors are in Australia. But deploying to other regions may be lower in cost though. It’s a trade off.
Server Location
Deploy servers as close as you can to your customers is the best tip for performance.
Deploy serves close to your customers
Also, consider setting up Image Optimization and Image CDN plugins (guide here) in WordPress and using Cloudflare (guide here)
Benchmarking with SysBench
Install CPU Benchmark
sudo apt-get install sysbench
CPU Benchmark (Vultr/Sydney)
Result
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          39.1700s
    total number of events:              10000
    total time taken by event execution: 39.1586
    per-request statistics:
         min:                                  2.90ms
         avg:                                  3.92ms
         max:                                 20.44ms
         approx.  95 percentile:               7.43ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   39.1586/0.00
39.15 seconds
CPU Benchmark (Digital Ocean/London)
sysbench --test=cpu --cpu-max-prime=20000 run
Result
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1

Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          33.4382s
    total number of events:              10000
    total time taken by event execution: 33.4352
    per-request statistics:
         min:                                  3.24ms
         avg:                                  3.34ms
         max:                                  6.45ms
         approx.  95 percentile:               3.45ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   33.4352/0.00
33.43 sec
CPU Benchmark (UpCloud/Singapore)
sysbench --test=cpu --cpu-max-prime=20000 run
Result
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1



Doing CPU performance benchmark

Threads started!
Done.

Maximum prime number checked in CPU test: 20000


Test execution summary:
    total time:                          23.7809s
    total number of events:              10000
    total time taken by event execution: 23.7780
    per-request statistics:
         min:                                  2.35ms
         avg:                                  2.38ms
         max:                                  6.92ms
         approx.  95 percentile:               2.46ms

Threads fairness:
    events (avg/stddev):           10000.0000/0.00
    execution time (avg/stddev):   23.7780/0.00
23.77 sec
Surprisingly, 1st place in prime generation goes to UpCloud, then Digital Ocean then Vultr.  UpCloud has some good processors.
Processors:
  • UpCLoud (Singapore): Intel(R) Xeon(R) CPU E5-2687W v4 @ 3.00GHz
  • Digital Ocean (London): Intel(R) Xeon(R) CPU E5-2630L v2 @ 2.40GHz
  • Vultr (Sydney): Virtual CPU a7769a6388d5 (Masked/Hidden CPU @ 2.40GHz)
(Lower is better)
prime benchmark results
(oops, typo in the chart should say Vultr)
Benchmark the file IO
Confirm free space
df -h /
Install Sysbench
sudo apt-get install sysbench
I had 10GB free on all servers (Vultr, Digitial Ocean and UpCloud) so I created a 10GB test file.
sysbench --test=fileio --file-total-size=10G prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

128 files, 81920Kb each, 10240Mb total
Creating files for the test...
Now I can run the benchmark and use the pre-created text file.
sysbench --test=fileio --file-total-size=10G --file-test-mode=rndrw --init-rng=on --max-time=300 --max-requests=0 run
SysBench description from the Ubuntu manpage.
“SysBench is a modular, cross-platform and multi-threaded benchmark tool for evaluating OS parameters that are important for a system running a database under intensive load. The idea of this benchmark suite is to quickly get an impression about system performance without setting up complex database benchmarks or even without installing a database at all.”
SysBench Results (Vultr/Sydney)
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1
Initializing random number generator from timer.


Extra file open flags: 0
128 files, 80Mb each
10Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed:  385920 Read, 257280 Write, 823266 Other = 1466466 Total
Read 5.8887Gb  Written 3.9258Gb  Total transferred 9.8145Gb  (33.5Mb/sec)
 2143.98 Requests/sec executed

Test execution summary:
    total time:                          300.0026s
    total number of events:              643200
    total time taken by event execution: 182.4249
    per-request statistics:
         min:                                  0.01ms
         avg:                                  0.28ms
         max:                                 18.12ms
         approx.  95 percentile:               0.55ms

Threads fairness:
    events (avg/stddev):           643200.0000/0.00
    execution time (avg/stddev):   182.4249/0.00
SysBench Results (Digital Ocean/London)
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1
Initializing random number generator from timer.


Extra file open flags: 0
128 files, 80Mb each
10Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed:  944280 Read, 629520 Write, 2014432 Other = 3588232 Total
Read 14.409Gb  Written 9.6057Gb  Total transferred 24.014Gb  (81.968Mb/sec)
 5245.96 Requests/sec executed

Test execution summary:
    total time:                          300.0024s
    total number of events:              1573800
    total time taken by event execution: 160.5558
    per-request statistics:
         min:                                  0.00ms
         avg:                                  0.10ms
         max:                                 18.62ms
         approx.  95 percentile:               0.34ms

Threads fairness:
    events (avg/stddev):           1573800.0000/0.00
    execution time (avg/stddev):   160.5558/0.00
SysBench Results (UpCloud/Singapore)
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Running the test with following options:
Number of threads: 1
Initializing random number generator from timer.


Extra file open flags: 0
128 files, 80Mb each
10Gb total file size
Block size 16Kb
Number of random requests for random IO: 0
Read/Write ratio for combined random IO test: 1.50
Periodic FSYNC enabled, calling fsync() each 100 requests.
Calling fsync() at the end of test, Enabled.
Using synchronous I/O mode
Doing random r/w test
Threads started!
Time limit exceeded, exiting...
Done.

Operations performed:  994320 Read, 662880 Write, 2121090 Other = 3778290 Total
Read 15.172Gb  Written 10.115Gb  Total transferred 25.287Gb  (86.312Mb/sec)
 5523.97 Requests/sec executed

Test execution summary:
    total time:                          300.0016s
    total number of events:              1657200
    total time taken by event execution: 107.4434
    per-request statistics:
         min:                                  0.00ms
         avg:                                  0.06ms
         max:                                 15.43ms
         approx.  95 percentile:               0.13ms

Threads fairness:
    events (avg/stddev):           1657200.0000/0.00
    execution time (avg/stddev):   107.4434/0.00
Comparison
Sysbench Results table
sysbench fileio results (text)
Read
  • Vultr (Sydney): 385,920
  • Digital Ocean (London): 944,280
  • UpCloud (Singapore): 944,320
Write
  • Vultr (Sydney): 823,266
  • Digital Ocean (London): 629,520
  • UpCloud (Singapore): 662,880
Other
  • Vultr (Sydney): 1,466,466
  • Digital Ocean (London): 3,588,232
  • UpCloud (Singapore): 2,121,090
Total Read Gb
  • Vultr (Sydney): 5.8887 Gb
  • Digital Ocean (London): 14.409 Gb
  • UpCloud (Singapore): 15.172 Gb
Total Written Gb
  • Vultr (Sydney): 3.9258 Gb
  • Digital Ocean (London): 9.6057 Gb
  • UpCloud (Singapore): 10.115 Gb
Total Transferred Gb
  • Vultr (Sydney): 9.8145 Gb
  • Digital Ocean (London): 24.014 Gb
  • UpCloud (Singapore): 25.287 Gb
Now I can remove test file io benchmark file
sysbench --test=fileio --file-total-size=2=10G cleanup
sysbench 0.4.12:  multi-threaded system evaluation benchmark

Removing test files...
Confirm the test file has been deleted
df -h /
Filesystem      Size  Used Avail Use% Mounted on
/dev/vda1        40G   16G   23G  41% /
Bonus: Benchmark MySQL (on my main server (Vultr) not on Digital Ocean and UpCLoud)
I tried to run a command
sysbench --test=oltp --oltp-table-size=1000000 --db-driver=mysql --mysql-db=test --mysql-user=root --mysql-password=#################################### prepare
sysbench 0.4.12:  multi-threaded system evaluation benchmark

FATAL: unable to connect to MySQL server, aborting...
FATAL: error 1049: Unknown database 'test'
FATAL: failed to connect to database server!
To fix the error I created a test table with Adminer (guide here).
Create Test Table
< Previous – Next >
Read Part 1, Part 2, Part 3 or Part 4
Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4
 by 
How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4
Advertisement:

 
 
Read Part 1, Part 2, Part 3 or Part 4
Measure Disk Performance with Bonnie++
Installing Bonnie++ on Ubuntu
apt-get install bonnie++
Read this. post on using Bonnie++
Benchmark disk IO with DD and Bonnie++
Starting Bonnie++
bonnie++ -d /tmp -r 2048 -u username
Bonnie++ Readme.
Disk io with bonnie++ on Vultr/Sydney
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 656 99 308954 68 113706 33 1200 92 188671 30 10237 251
Latency 26067us 119ms 179ms 29139us 26069us 16118us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1463us 703us 880us 263us 119us 593us
1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us
Disk io with bonnie++ on Digital Ocean/London
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 699 99 778636 74 610414 60 1556 99 1405337 59 +++++ +++
Latency 17678us 10099us 17014us 7027us 3067us 2366us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1243us 376us 611us 108us 59us 181us
1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us
Disk io with bonnie++ on UpCloud/Singapore
Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 1014 99 407179 24 366622 32 2137 99 451886 17 +++++ +++
Latency 11297us 54232us 16443us 4949us 44883us 1595us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 264us 340us 561us 138us 66us 327us
1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us
Now read this site on how to make sense of this data
 
< Previous – Next >
Read Part 1, Part 2, Part 3 or Part 4