IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

Ubuntu

I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.

by

I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance. Here is what I did to set up a complete Ubuntu 18.04 system (NGINX, PHP, MySQL, WordPress etc). This is not a paid review (just me documenting my steps over 2 days).

Background (CPanel hosts)

In 1999 I hosted my first domain (www.fearby.com) on a host in Seattle (for $10 USD a month), the host used CPanel and all was good.  After a decade I was using the domain more for online development and the website was now too slow (I think I was on dial-up or ADSL 1 at the time). I moved my domain to an Australian host (for $25 a month).

After 8 years the domain host was sold and performance remained mediocre. After another year the new host was sold again and performance was terrible.

I started receiving Resource Limit Is Reached warnings (basically this was a plot by the new CPanel host to say “Pay us more and this message will go away”).

Page load times were near 30 seconds.

cpenal_usage_exceeded

The straw that broke the camel’s back was their demand of $150/year for a dodgy SSL certificate.

I needed to move to a self-managed server where I was in control.

Buying a Domain Name

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Self Managed Server

I found a good web IDE ( http://www.c9.io/ ) that allowed me to connect to a cloud VM.  C9 allowed me to open many files and terminal windows and reconnect to them later. Don’t get excited, though, as AWS has purchased C9 and it’s not the same.

C9 IDE

C9 IDE

I spun up a Digital Ocean Server at the closest data centre in Singapore. Here was my setup guide creating a Digital Ocean VM, connecting to it with C9 and configuring it. I moved my email to G Suite and moved my WordPress to Digital Ocean (other guides here and here).

I was happy since I could now send emails via CLI/code, set up free SSL certs, add second domain email to G Suite and Secure G Suite. No more usage limit errors either.

Self-managing servers require more work but it is more rewarding (flexible, faster and cheaper).  Page load times were now near 20 seconds (10-second improvement).

Latency Issue

Over 6 months, performance on Digital Ocean (in Singapore) from Australia started to drop (mentioned here).  I tried upgrading the memory but that did not help (latency was king).

Moved the website to Australia

I moved my domain to Vultr in Australia (guide here and here). All was good for a year until traffic growth started to increase.

Blog Growth

I tried upgrading the memory on Vultr and I setup PHP child workers, set up Cloudflare.

GT Metrix scores were about a “B” and Google Page Speed Scores were in the lower 40’s. Page loads were about 14 seconds (5-second improvement).

Tweaking WordPress

I set up an image compression plugin in WordPress then set up a cloud image compression and CDN Plugin from the same vendor.  Page Speed info here.

GT Metrix scores were now occasionally an “A” and Page Speed scores were in the lower 20’s. Page loads were about 3-5 seconds (10-second improvement).

A mixed bag from Vultr (more optimisation and performance improvements were needed).

This screenshot is showing poor www.gtmetrix.com scores , pool google page speed index scores and upgrading from 1GB to 2GB memory on my server.

Google Chrome Developer Console Audit Results on Vultr hosted website were not very good (I stopped checking as nothing helped).

This is a screenshot showing poor site performance (screenshot taken in Google Dev tools audit feature)

The problem was the Vultr server (400km away in Sydney) was offline (my issue) and everything above (adding more memory, adding 2x CDN’s (EWWW and Cloudflare), adding PHP Child workers etc) did not seem to help???

Enter UpCloud…

Recently, a friend sent a link to a blog article about a host called “UpCloud” who promised “Faster than SSD” performance.  This can’t be right: “Faster than SSD”? I was intrigued. I wanted to check it out as I thought nothing was faster than SSD (well, maybe RAM).

I signed up for a trial and ran a disk IO test (read the review here) and I was shocked. It’s fast. Very fast.

Summary: UpCloud was twice as fast (Disk IO and CPU) as Vultr (+ an optional $4/m firewall and $3/m for 1x backup).

This is a screenshot showing Vultr.com servers getting half the read and write disk io performance compared to upcloud.com.

fyi: Labels above are K Bytes per second. iozone loops through all file size from 4 KB to 16,348 KB and measures through the reads per second. To be honest, the meaning of the numbers doesn’t interest me, I just want to compare apples to apples.

This is am image showing iozone results breakdown chart (kbytes per sec on vertical axis, file size in horizontal axis and transfer size on third access)

(image snip from http://www.iozone.org/ which explains the numbers)

I might have to copy my website on UpCloud and see how fast it is.

Where to Deploy and Pricing

UpCloud Pricing: https://www.upcloud.com/pricing/

UpCloud Pricing

UpCloud does not have a data centre in Australia yet so why choose UpCloud?

Most of my site’s visitors are based in the US and UpCloud have disk IO twice as fast as Vultr (win-win?).  I could deploy to Chicago?

This image sows most of my visitors are in the US

My site’s traffic is growing and I need to ensure the site is fast enough in the future.

This image shows that most of my sites visitors are hitting my site on week days.

Creating an UpCloud VM

I used a friend’s referral code and signed up to create my first VM.

FYI: use my Referral code and get $25 free credit.  Sign up only takes 2 minutes.

https://www.upcloud.com/register/?promo=D84793

When you click the link above you will receive 25$ to try out serves for 3 days. You can exit his trail and deposit $10 into UpCloud.

Trial Limitations

The trial mode restrictions are as following:

* Cloud servers can only be accessed using SSH, RDP, HTTP or HTTPS protocols
* Cloud servers are not allowed to send outgoing e-mails or to create outbound SSH/RDP connections
* The internet connection is restricted to 100 Mbps (compared to 500 Mbps for non-trial accounts)
* After your 72 hours free trial, your services will be deleted unless you make a one-time deposit of $10

UpCloud Links

The UpCloud support page is located here: https://www.upcloud.com/support/

More UpCloud links to read:

Signing up to UpCloud

Navigate to https://upcloud.com/signup and add your username, password and email address and click signup.

New UpCloud Signup Page

Add your address and payment details and click proceed (you don’t need to pay anything ($1 may be charged and instantly refunded to verify the card)

Add address and payment details

That’s it, check yout email.

Signup Done

Look for the UpCloud email and click https://my.upcloud.com/

Check Email

Now login

Login to UpCloud

Now I can see a dashboard 🙂

UpCloud Dashboard

I was happy to see 24/7 support is available.

This image shows the www.upcloud.com live chat

I opted in for the new dashboard

UpCloud new new dashboard

Deploy My First UpCloud Server

This is how I deployed a server.

Note: If you are going to deploy a server consider using my referral code and get $25 credit for free.

Under the “deploy a server” widget I named the server and chose a location (I think I was supposed to use an FQDN name -e.g., “fearby.com”). The deployment worked though. I clicked continue, then more options were made available:

  1. Enter a short server description.
  2. Choose a location (Frankfurt, Helsinki, Amsterdam, Singapore, London and Chicago)
  3. Choose the number of CPU’s and amount of memory
  4. Specify disk number/names and type (MaxIOPS or HDD).
  5. Choose an Operating System
  6. Select a Timezone
  7. Define SSH Keys for access
  8. Allowed login methods
  9. Choose hardware adapter types
  10. Where the send the login password

Deploy Server

FYI: How to generate a new SSH Key (on OSX or Ubuntu)

Output

Did the key export? (yes)

> /temp# ls /temp/ -al
> drwxr-xr-x 2 root root 4096 Jun 9 15:33 .
> drwxr-xr-x 27 root root 4096 Jun 8 14:25 ..
> -rw——- 1 user user 1766 Jun 9 15:33 example_rsa
> -rw-r–r– 1 user user 396 Jun 9 15:33 example_rsa.pub

“example_rsa” is the private key and “example_rsa.pub “is the public key.

  • The public key needs to be added to the server to allow access.
  • The private key needs to be added to any local ssh program used for remote access.

Initialisation script (after deployment)

I was pleased to see an initialization script section that calls actions after the server is deployed. I configured the initialisation script to pull down a few GB of backups from my Vultr website in Sydney (files now removed).

This was my Initialisation script:

Confirm and Deploy

I clicked “Confirm and deploy” but I had an alert that said trial mode can only deploy servers up to 1024MB of memory.

This image shows I cant deploy servers with 2/GB in trial modeExiting UpCloud Trial Mode

I opened the dashboard and clicked My Account then Billing, I could see the $25 referral credit but I guess I can’t use that in Trial.

I exited trial mode by depositing $10 (USD).

View Billing Details

Make a manual 1-time deposit of $10 to exit trial mode.

Deposit $10 to exit the trial

FYI: Server prices are listed below (or view prices here).

UpCloud Pricing

Now I can go back and deploy the server with the same settings above (1x CPU, 2GB Memory, Ubuntu 18.04, MaxIOPS Storage etc)

Deployment takes a few minutes and depending on how you specified a password may be emailed to you.

UpCloud Server Deployed

The server is now deployed; now I can connect to it with my SSH program (vSSH).  Simply add the server’s IP, username, password and the SSH private key (generated above) to your ssh program of choice.

fyi: The public key contents start with “ssh-rsa”.

This image shows me connecting to my sever via ssh

I noticed that the initialisation script downloaded my 2+GB of files already. Nice.

UpCloud Billing Breakdown

I can now see on the UpCloud billing page in my dashboard that credit is deducted daily (68c); at this rate, I have 49 days credit left?

Billing Breakdown

I can manually deposit funds or set up automatic payments at any time 🙂

UpCloud Backup Options

You do not need to setup backups but in case you want to roll back (if things stuff up), it is a good idea. Backups are an additional charge.

I have set up automatic daily backups with an auto deletion after 2 days

To view backup scheduled click on your deployed server then click backup

List of UpCloud Backups

Note: Backups are charged at $0.056 for every GB stored – so $5.60 for every 100GB per month (half that for 50GB etc)

You can take manual backups at any time (and only be charged for the hour)

UpCloud Firewall Options

I set up a firewall at UpCloud to only allow the minimum number of ports (UpCloud DNS, HTTP, HTTPS and My IP to port 22).  The firewall feature is charged at $0.0056 an hour ($4.03 a month)

I love the ability to set firewall rules on incoming, destination and outgoing ports.

To view your firewall click on your deployed server then click firewall

UpCloud firewall

Update: I modified my firewall to allow inbound ICMP (IPv4/IPv6) and UDP (IPv4/IPv6) packets.

(Note: Old firewall screenshot)

Firewall Rules Allow port 80, 443 and DNS

Because my internet provider has a dynamic IP, I set up a VPN with a static IP and whitelisted it for backdoor access.

Local Ubuntu ufw Firewall

I duplicated the rules in my local ufw (2nd level) firewall (and blocked mail)

UpCloud Download Speeds

I pulled down a 1.8GB Ubuntu 18.08 Desktop ISO 3 times from gigenet.com and the file downloaded in 32 seconds (57MB/sec). Nice.

Install Common Ubuntu Packages

I installed common Ubuntu packages.

Timezone

I checked the server’s time (I thought this was auto set before I deployed)?

I reset the time to Australia/Sydney.

Now the timezone is set 🙂

Shell History

I increased the shell history.

SSH Login

I created a ~/.ssh/authorized_keys file and added my SSH public key to allow password-less logins.

I added my pubic ssh key, then exited the ssh session and logged back in. I can now log in without a password.

Install NGINX

nginx/1.14.0 is now installed.

A quick GT Metrix test.

This image shows awesome static nginx performance ratings of of 99%

Install MySQL

Run these commands to install and secure MySQL.

Securing the MySQL server deployment.
> Would you like to setup VALIDATE PASSWORD plugin?: n
> New password: **********************************************
> Re-enter new password: **********************************************
> Remove anonymous users? (Press y|Y for Yes, any other key for No) : y
> Disallow root login remotely? (Press y|Y for Yes, any other key for No) : y
> Remove test database and access to it? (Press y|Y for Yes, any other key for No) : y
> Reload privilege tables now? (Press y|Y for Yes, any other key for No) : y
> Success.

I disabled the validate password plugin because I hate it.

MySQL Ver 14.14 Distrib 5.7.22 is now installed.

Set MySQL root login password type

Set MySQL root user to authenticate via “mysql_native_password”. Run the “mysql” command.

Now let’s set the root password authentication method to “mysql_native_password”

Check authentication method.

Now we need to flush permissions.

Done.

Install PHP

Install PHP 7.2

PHP 7.2.5, Zend Engine v3.2.0 with Zend OPcache v7.2.5-1 is now installed. Do update PHP frequently.

I made the following changes in /etc/php/7.2/fpm/php.ini

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 20M
> post_max_size = 20M

Install PHP Modules

sudo apt-get install php-pear php7.2-curl php7.2-dev php7.2-mbstring php7.2-zip php7.2-mysql php7.2-xml

Install PHP FPM

Configure PHP FPM config.

Edit /etc/php/7.2/fpm/php.ini

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 20M
> post_max_size = 20M

Reload php sudo service.

Install PHP Modules

Configuring NGINX

If you are not comfortable editing NGINX config files read here, here and here.

I made a new “www root” folder, set permissions and created a default html file.

I edited the “root” key in “/etc/nginx/sites-enabled/default” file and set the root a new location (e.g., “/www-root”)

I added these performance tweaks to /etc/nginx/nginx.conf

> worker_cpu_affinity auto;
> worker_rlimit_nofile 100000

I add the following lines to “http {” section in /etc/nginx/nginx.conf

Check NGINX Status

Install Open SSL that supports TLS 1.3

This is a work in progress. The steps work just fine for me on Ubuntu 16.04. but not Ubuntu 18.04.?

Installing Adminer MySQL GUI

I will use the PHP based Adminer MySQL GUI to export and import my blog from one server to another. All I needed to do is install it on both servers (simple 1 file download)

Use Adminer to Export My Blog (on Vultr)

On the original server open Adminer (http) and..

  1. Login with the MySQL root account
  2. Open your database
  3. Choose “Save” as the output
  4. Click on Export

This image shows the export of the wordpress adminer page

Save the “.sql” file.

I used Adminer on the UpCloud server to Import My Blog

FYI: Depending on the size of your database backup you may need to temporarily increase your upload and post sizes limits in PHP and NGINX before you can import your database.

Edit /etc/php/7.2/fpm/php.ini
> max_file_uploads = 100M
> post_max_size =100M

And Edit: /etc/nginx/nginx.conf
> client_max_body_size 100M;

Don’t forget to reload NGINX config and restart NGINX and PHP. Take note of the maximum allowed file size in the screenshot below. I temporarily increased my upload limits to 100MB in order to restore my 87MB blog.

Now I could open Adminer on my UpCloud server.

  1. Create a new database
  2. Click on the database and click Import
  3. Choose the SQL file
  4. Click Execute to import it

Import MuSQL backup with Adminer

Don’t forget to create a user and assign permissions (as required – check your wp-config.php file).

Import MySQL Database

Tip: Don’t forget to lower the maximum upload file size and max post size after you import your database,

Cloudflare DNS

I use Cloudflare to manage DNS, so I need to tell it about my new server.

You can get your server’s IP details from the UpCloud dashboard.

Find IP

At Cloudflare update your DNS details to point to the server’s new IPv4 (“A Record”) and IPv6 (“AAAA Record”).

Cloudflare DNS

Domain Error

I waited an hour and my website was suddenly unavailable.  At first, I thought this was Cloudflare forcing the redirection of my domain to HTTP (that was not yet set up).

DNS Not Replicated Yet

I chatted with UpCloud chat on their webpage and they kindly assisted me to diagnose all the common issues like DNS values, DNS replication, Cloudflare settings and the error was pinpointed to my NGINX installation.  All NGINX config settings were ok from what we could see?  I uninstalled NGINX and reinstalled it (and that fixed it). Thanks UpCloud Support 🙂

Reinstalled NGINX

I reinstalled NGINX and reconfigured /etc/nginx/nginx.conf (I downloaded my SSL cert from my old server just in case).

Here is my /etc/nginx/nginx.conf file.

Here is my /etc/nginx/sites-available/default file (fyi, I have not fully re-setup TLS 1.3 yet so I commented out the settings)

SSL Labs SSL Certificate Check

All good thanks to the config above.

SSL Labs

Install WP-CLI

I don’t like setting up FTP to auto-update WordPress plugins. I use the WP-CLI tool to manage WordPress installations by the command line. Read my blog here on using WP-CLI.

Download WP-CLI

Move WP-CLI to the bin folder as “wp”

Test wp

Update WordPress Plugins

Now I can run “wp plugin update” to update all WordPress plugins

Update WordPress Core

WordPress core file can be updated with “wp core update“

Troubleshooting: Use the flag “–allow-root “if wp needs higher access (unsafe action though).

Install PHP Child Workers

I edited the following file to setup PHP child workers /etc/php/7.2/fpm/pool.d/www.conf

Changes

> pm = dynamic
> pm.max_children = 40
> pm.start_servers = 15
> pm.min_spare_servers = 5
> pm.max_spare_servers = 15
> pm.process_idle_timeout = 30s;
> pm.max_requests = 500;
> php_admin_value[error_log] = /var/log/www-fpm-php.www.log
> php_admin_value[memory_limit] = 512M

Restart PHP

Test NGINX config, reload NGINX config and restart NGINX

Output (14 workers are ready)

Memory Tweak (set at your own risk)

vm.swappiness = 1

Setting swappiness to a value of 1 all but disables the swap file and tells the Operating System to aggressively use ram, a value of 10 is safer. Only set this if you have enough memory available (and free).

Possible swappiness settings:

> vm.swappiness = 0 Swap is disabled. In earlier versions, this meant that the kernel would swap only to avoid an out of memory condition when free memory will be below vm.min_free_kbytes limit, but in later versions, this is achieved by setting to 1.[2]> vm.swappiness = 1 Kernel version 3.5 and over, as well as Red Hat kernel version 2.6.32-303 and over: Minimum amount of swapping without disabling it entirely.
> vm.swappiness = 10 This value is sometimes recommended to improve performance when sufficient memory exists in a system.[3]
> vm.swappiness = 60 The default value.
> vm.swappiness = 100 The kernel will swap aggressively.

The “htop” tool is a handy memory monitoring tool to “top”

Also, you can use good old “watch” command to show near-live memory usage (auto-refreshes every 2 seconds)

Script to auto-clear the memory/cache

As a habit, I am setting up a cronjob to check when free memory falls below 100MB, then the cache is automatically cleared (freeing memory).

Script Contents: clearcache.sh

I set the cronjob to run every 15 mins, I added this to my cronjob.

Sample log output

I will check the log (/scripts/clearcache.log) in a few days and view the memory trends.

After 1/2 a day Ubuntu 18.04 is handling memory just fine, no externally triggered cache clears have happened 🙂

Free memory over time

I used https://crontab.guru/every-hour to set the right schedule in crontab.

I rebooted the VM.

Update: I now use Nixstats monitoring

Swap File

FYI: Here is a handy guide on viewing swap file usage here. I’m not using swap files so it is only an aside.

After the system rebooted I checked if the swappiness setting was active.

Yes, swappiness is set.

File System Tweaks – Write Back Cache (set at your own risk)

First, check your disk name and file system

Take note of your disk name (e.g vda1)

I used TuneFS to enable writing data to the disk before writing to the journal. tunefs is a great tool for setting file system parameters.

Warning (snip from here): “I set the mode to journal_data_writeback. This basically means that data may be written to the disk before the journal. The data consistency guarantees are the same as the ext3 file system. The downside is that if your system crashes before the journal gets written then you may lose new data — the old data may magically reappear.“

Warning this can corrupt your data. More information here.

I ran this command.

I edited my fstab to append the “writeback,noatime,nodiratime” flags for my volume after a reboot.

Edit FS Tab:

I added “writeback,noatime,nodiratime” flags to my disk options.

Updating Ubuntu Packages

Show updatable packages.

Update Packages.

Unattended Security Updates

Read more on Ubuntu 18.04 Unattended upgrades here, here and here.

Install Unattended Upgrades

Enable Unattended Upgrades.

Now I configure what packages not to auto-update.

Edit /etc/apt/apt.conf.d/50unattended-upgrades

Find “Unattended-Upgrade::Package-Blacklist” and add packages that you don’t want automatically updated, you may want to manually update these (and monitor updates).

I prefer not to auto-update critical system apps (I will do this myself).

FYI: You can find installed packages by running this command:

Enable automatic updates by editing /etc/apt/apt.conf.d/20auto-upgrades

Edit the number at the end (the number is how many days to wait before updating) of each line.

> APT::Periodic::Update-Package-Lists “1”;
> APT::Periodic::Download-Upgradeable-Packages “1”;
> APT::Periodic::AutocleanInterval “7”;
> APT::Periodic::Unattended-Upgrade “1”;

Set to “0” to disable automatic updates.

The results of unattended-upgrades will be logged to /var/log/unattended-upgrades

Update packages now.

Almost done.

I Rebooted

GT Metrix Score

I almost fell off my chair. It’s an amazing feeling hitting refresh in GT Metrix and getting sub-2-second score consistently (and that is with 17 assets loading and 361KB of HTML content)

0.9sec load times

WebPageTest.org Test Score

Nice. I am not sure why the effective use of CDN has an X rating as I have the EWWW CDN and Cloudflare. First Byte time is now a respectable “B”, This was always bad.

Update: I found out the longer you set cache delays in Cloudflare the higher the score.

Web Page Test

GT Metrix has a nice historical breakdown of load times (night and day).

Upcloud Site Speed in GTMetrix

Google Page Speed Insight Desktop Score

I benchmarked with https://developers.google.com/speed/pagespeed/insights/

This will help with future SEO rankings. It is well known that Google is pushing fast servers.

100% Desktop page speed score

Google Chrome 70 Dev Console Audit (Desktop)

100% Chrome Audit Score

This is amazing, I never expected to get this high score.  I know Google like (and are pushing) sub-1-second scores.

My site is loading so well it is time I restored some old features that were too slow on other servers

  • I disabled Lazy loading of images (this was not working on some Android devices)
  • I re-added the News Widget and news images.

GTMetrix and WebpageTest sores are still good (even after adding bloat)

Benchmarks are still good

My WordPress site is not really that small either

Large website

FYI: WordPress Plugins I use.

These are the plugins I use.

How I use these plugins to speed up my site.

  • I use EWWW Image Optimizer plugin to auto-compress my images and to provide a CDN for media asset deliver (pre-Cloudflare). Learn more about ExactDN and EWWW.io here.
  • I use Autoptimize plugin to optimise HTML/CSS/JS and ensure select assets are on my EWWW CDN. This plugin also removes WordPress Emojis, removed the use of Google Fonts, allows you to define pre-configured domains, Async Javascript-files etc.
  • I use BJ Lazy Load to prevent all images in a post from loading on load (and only as the user scrolls down the page).
  • GTmetrix for WordPress and Cloudflare plugins are for information only?
  • I use WP-Optimize to ensure my database is healthy and to disable comments/trackbacks and pingbacks.

Let’s Test UpCloud’s Disk IO in Chicago

Looks good to me, Read IO is a little bit lower than UpCloud’s Singapore data centre but still, it’s faster than Vultr.  I can’t wait for more data centres to become available around the world.

Why is UpCloud Disk IO so good?

I asked UpCloud on Twitter why the Disk IO was so good.

  • “MaxIOPS is UpCloud’s proprietary block-storage technology. MaxIOPS is physically redundant storage technology where all customer’s data is located in two separate physical devices at all times. UpCloud uses InfiniBand (!) network to connect storage backends to compute nodes, where customers’ cloud servers are running. All disks are enterprise-grade SSD’s. And using separate storage backends, it allows us to live migrate our customers’ cloud servers freely inside our infrastructure between compute nodes – whether it be due to hardware malfunction (compute node) or backend software updates (example CPU vulnerability and immediate patching).“

My Answers to Questions to support

Q1) What’s the difference between backups and snapshots (a Twitter user said Snapshots were a thing)

A1) Backups and snapshots are the same things with our infrastructure.

Q2) What are charges for backup of a 50GB drive?

A2) We charge $0.06 / GB of the disk being captured. But capture the whole disk, not just what was used. So for a 50GB drive, we charge $0.06 * 50 = $3/month. Even if 1GB were only used.

  • Support confirmed that each backup is charged (so 5 times manual backups are charged 5 times). Setting up a daily auto backup schedule for 2 weeks would create 14 billable backup charges.
  • I guess a 25GB server will be $1.50 a month

Q3) What are data charges if I go over my 2TB quota?

A3) Outgoing data charges are $0.056/GB after the pre-configured allowance.

Q4) What happens if my balance hits $0?

A4) You will get notification of low account balance 2 weeks in advance based on your current daily spend. When your balance reaches zero, your servers will be shut down. But they will still be charged for. You can automatically top-up if you want to assign a payment type from your Control Panel. You deposit into your balance when you want. We use a prepaid model of payment, so you need to top up before using, not billing you after usage. We give you lots of chances to top-up.

Support Tips

  • One thing to note, when deleting servers (CPU, RAM) instances, you get the option to delete the storages separately via a pop-up window. Choose to delete permanently to delete the disk, to save credit. Any disk storage lying around even unattached to servers will be billed.
  • Charges are in USD.

I think it’s time to delete my domain from Vultr in Sydney.

Deleted my Vultr domain

I deleted my Vultr domain.

Delete Vultr Server

Done.

More Reading on UpCloud

https://www.upcloud.com/documentation/faq/

UpCloud Server Status

http://status.upcloud.com

Check out my new guide on Nixstats for awesome monitoring

What I would like

  1. Ability to name individual manual backups (tag with why I backed up).
  2. Ability to push user-defined data from my VM to the dashboard
  3. Cheaper scheduled backups
  4. Sydney data centres (one day)

Update: Post UpCloud Launch Tweaks (Awesome)

I had a look at https://www.webpagetest.org/ results to see where else I can optimise webpage delivery.

Optimisation Options

Disable dasjhicons.min.css (for unauthenticated WordPress users).

Find functions.php in the www root

Edit functions.php

Add the following

// Remove dashicons in frontend for unauthenticated users
add_action( 'wp_enqueue_scripts', 'bs_dequeue_dashicons' );
function bs_dequeue_dashicons() {
    if ( ! is_user_logged_in() ) {
        wp_deregister_style( 'dashicons' );
    }
}

HTTP2 Push

I added http2 to my listening servers

I tested a http2 push page by defining this in /etc/nginx/sites-available/default 

Once I tested that push (demo here) was working I then defined two files to push that were being sent from my server

I used the WordPress Plugin Autoptimize to remove Google font usage (this removed a number of files being loaded when my page loads).

I used the WordPress Plugin WP-Optimize plugin into to remove comments and disable pingbacks and trackbacks.

WordPress wp-config.php tweaks

Add 2FA Authentication to server logins.

I recently checked out YubiCo YubiKeys and I have secured my Linux servers with 2FA prompts at login. Read the guide here. I secured my WordPress too.

Tweaks Todo

  • Compress placeholder BJ Lazy Load Image (plugin is broken)
  • Solve 2x Google Analytics tracker redirects (done, switched to Matomo)

Conclusion

I love UpCloud’s fast servers, give them a go (use my link and get $25 free credit).

I love Cloudflare for providing a fast CDN.

I love ewww.io’s automatic Image Compression and Resizing plugin that automatically handles image optimisations and pre Cloudflare/first hit CDN caching.

Read my post about server monitoring with Nixstats here.

Let the results speak for themselves (sub <1 second load times).

Results

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

2020 Update. I have stopped using Putty and WinSCP. I now use MobaXterm (a tabbed SSH client for Windows) as it is way faster than WinSCP and better than Putty. Read my review post of MobaXTerm here.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v2.1 Newer GTMetrix scores

v2.0 New UpCloud UI Update and links to new guides.

v1.9 Spelling and grammar

v1.8 Trial mode gotcha (deposit money ASAP)

v1.7 Added RSA Private key info

v1.7 – Added new firewall rules info.

v1.6 – Added more bloat to the site, still good.

v1.5 Improving Accessibility

v1.4 Added Firewall Price

v1.3 Added wp-config and plugin usage descriptions.

v1.2 Added GTMetrix historical chart.

v1.1 Fixed free typos and added final conclusion images.

v1.0 Added final results

v0.9 added more tweaks (http2 push, removing unwanted files etc)

v0.81 Draft  – Added memory usage chart and added MaxIOPS info from UpCloud.

v0.8 Draft post.

I set the cronjob to run every 15 mins, I added this to my cronjob.

 

Sample log output

 

I will check the log (/scripts/clearcache.log) in a few days and view the memory trends.

After 1/2 a day Ubuntu 18.04 is handling memory just fine, no externally triggered cache clears have happened 🙂

Free memory over time

I used https://crontab.guru/every-hour to set the right schedule in crontab.

I rebooted the VM.

Update: I now use Nixstats monitoring

Swap File

FYI: Here is a handy guide on viewing swap file usage here. I’m not using swap files so it is only an aside.

After the system rebooted I checked if the swappiness setting was active.

 

Yes, swappiness is set.

File System Tweaks – Write Back Cache (set at your own risk)

First, check your disk name and file system

 

Take note of your disk name (e.g vda1)

I used TuneFS to enable writing data to the disk before writing to the journal. tunefs is a great tool for setting file system parameters.

Warning (snip from here): “I set the mode to journal_data_writeback. This basically means that data may be written to the disk before the journal. The data consistency guarantees are the same as the ext3 file system. The downside is that if your system crashes before the journal gets written then you may loose new data — the old data may magically reappear.“

Warning this can corrupt your data. More information here.

I ran this command.

 

I edited my fstab to append the “writeback,noatime,nodiratime” flags for my volume after a reboot.

Edit FS Tab:

 

I added “writeback,noatime,nodiratime” flags to my disk options.

 

Updating Ubuntu Packages

Show updatable packages.

 

Update Packages.

 

Unattended Security Updates

Read more on Ubuntu 18.04 Unattended upgrades here, here and here.

Install Unattended Upgrades

 

Enable Unattended Upgrades.

 

Now I configure what packages not to auto update.

Edit /etc/apt/apt.conf.d/50unattended-upgrades

Find “Unattended-Upgrade::Package-Blacklist” and add packages that you don’t want automatically updated, you may want to manually update these (and monitor updates).

I prefer not to auto-update critical system apps (I will do this myself).

 

FYI: You can find installed packages by running this command:

 

Enable automatic updates by editing /etc/apt/apt.conf.d/20auto-upgrades

Edit the number at the end (the number is how many days to wait before updating) of each line.

> APT::Periodic::Update-Package-Lists “1”;
> APT::Periodic::Download-Upgradeable-Packages “1”;
> APT::Periodic::AutocleanInterval “7”;
> APT::Periodic::Unattended-Upgrade “1”;

Set to “0” to disable automatic updates.

The results of unattended-upgrades will be logged to /var/log/unattended-upgrades

Update packages now.

 

Almost done.

I Rebooted

GT Metrix Score

I almost fell off my chair. It’s an amazing feeling hitting refresh in GT Metrix and getting sub-2-second score consistently (and that is with 17 assets loading and 361KB of HTML content)

0.9sec load times

WebPageTest.org Test Score

Nice. I am not sure why the effective use of CDN has an X rating as I have the EWWW CDN and Cloudflare. First Byte time is now a respectable “B”, This was always bad.

Update: I found out the longer you set cache delays in Cloudflare the higher the score.

Web Page Test

GT Metrix has a nice historical breakdown of load times (night and day).

Upcloud Site Speed in GTMetrix

Google Page Speed Insight Desktop Score

I benchmarked with https://developers.google.com/speed/pagespeed/insights/

This will help with future SEO rankings. It is well known that Google is pushing fast servers.

100% Desktop page speed score

Google Chrome 70 Dev Console Audit (Desktop)

100% Chrome Audit Score

This is amazing, I never expected to get this high score.  I know Google like (and are pushing) sub-1-second scores.

My site is loading so well it is time I restored some old features that were too slow on other servers

  • I disabled Lazy loading of images (this was not working on some Android devices)
  • I re-added the News Widget and news images.

GTMetrix and WebpageTest sores are still good (even after adding bloat)

Benchmarks are still good

My WordPress site is not really that small either

Large website

FYI: WordPress Plugins I use.

These are the plugins I use.

How I use these plugins to speed up my site.

  • I use EWWW Image Optimizer plugin to auto-compress my images and to provide a CDN for media asset deliver (pre-Cloudflare). Learn more about ExactDN and EWWW.io here.
  • I use Autoptimize plugin to optimise HTML/CSS/JS and ensure select assets are on my EWWW CDN. This plugin also removes WordPress Emojis, removed the use of Google Fonts, allows you to define pre-configured domains, Async Javascript-files etc.
  • I use BJ Lazy Load to prevent all images in a post from loading on load (and only as the user scrolls down the page).
  • GTmetrix for WordPress and Cloudflare plugins are for information only?
  • I use WP-Optimize to ensure my database is healthy and to disable comments/trackbacks and pingbacks.

Let’s Test UpCloud’s Disk IO in Chicago

Looks good to me, Read IO is a little bit lower than UpCloud’s Singapore data centre but still, it’s faster than Vultr.  I can’t wait for more data centres to become available around the world.

Why is UpCloud Disk IO so good?

I asked UpCloud on Twitter why the Disk IO was so good.

  • “MaxIOPS is UpCloud’s proprietary block-storage technology. MaxIOPS is physically redundant storage technology where all customer’s data is located in two separate physical devices at all times. UpCloud uses InfiniBand (!) network to connect storage backends to compute nodes, where customers’ cloud servers are running. All disks are enterprise-grade SSD’s. And using separate storage backends, it allows us to live migrate our customers’ cloud servers freely inside our infrastructure between compute nodes – whether it be due to hardware malfunction (compute node) or backend software updates (example CPU vulnerability and immediate patching).“

My Answers to Questions to support

Q1) What’s the difference between backups and snapshots (a Twitter user said Snapshots were a thing)

A1) Backups and snapshots are the same things with our infrastructure.

Q2) What are charges for backup of a 50GB drive?

A2) We charge $0.06 / GB of the disk being captured. But capture the whole disk, not just what was used. So for a 50GB drive, we charge $0.06 * 50 = $3/month. Even if 1GB were only used.

  • Support confirmed that each backup is charged (so 5 times manual backups are charged 5 times). Setting up a daily auto backup schedule for 2 weeks would create 14 billable backup charges.
  • I guess a 25GB server will be $1.50 a month

Q3) What are data charges if I go over my 2TB quota?

A3) Outgoing data charges are $0.056/GB after the pre-configured allowance.

Q4) What happens if my balance hits $0?

A4) You will get notification of low account balance 2 weeks in advance based on your current daily spend. When your balance reaches zero, your servers will be shut down. But they will still be charged for. You can automatically top-up if you want to assign a payment type from your Control Panel. You deposit into your balance when you want. We use a prepay model of payment, so you need to top up before using, not billing you after usage. We give you lots of chances to top-up.

Support Tips

  • One thing to note, when deleting servers (CPU, RAM) instances, you get the option to delete the storages separately via a pop-up window. Choose to delete permanently to delete the disk, to save credit. Any disk storage lying around even unattached to servers will be billed.
  • Charges are in USD.

I think it’s time to delete my domain from Vultr in Sydney.

Deleted my Vultr domain

I deleted my Vultr domain.

Delete Vultr Server

Done.

Check out my new guide on Nixstats for awesome monitoring

What I would like

  1. Ability to name individual manual backups (tag with why I backed up).
  2. Ability to push user defined data from my VM to the dashboard
  3. Cheaper scheduled backups
  4. Sydney data centres (one day)

Update: Post UpCloud Launch Tweaks (Awesome)

I had a look at https://www.webpagetest.org/ results to see where else I can optimise webpage delivery.

Optimisation Options

HTTP2 Push

I added http2 to my listening servers I tested a http2 push page by defining this in /etc/nginx/sites-available/default 

Once I tested that push (demo here) was working I then defined two files to push that were being sent from my server

2FA Authentication at login

I recently checked out YubiCo YubiKeys and I have secured my Linux servers with 2FA prompts at login. Read the guide here. I secured my WordPress aswel.

Performance

I used the WordPress Plugin Autoptimize to remove Google font usage (this removed a number of files being loaded when my page loads).

I used the WordPress Plugin WP-Optimize plugin into to remove comments and disable pingbacks and trackbacks.

Results

Conclusion

I love UpCloud’s fast servers, give them a go (use my link and get $25 free credit).

I love Cloudflare for providing a fast CDN.

I love ewww.io’s automatic Image Compression and Resizing plugin that automatically handles image optimisations and pre Cloudflare/first hit CDN caching.

Read my post about server monitoring with Nixstats here.

Let the results speak for themselves (sub <1 second load times).

More Reading on UpCloud

https://www.upcloud.com/documentation/faq/

UpCloud Server Status

http://status.upcloud.com

I hope this guide helps someone.

Free Credit

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

2020 Update. I have stopped using Putty and WinSCP. I now use MobaXterm (a tabbed SSH client for Windows) as it is way faster than WinSCP and better than Putty. Read my review post of MobaXTerm here.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v2.2 Converting to Blocks

v2.1 Newer GTMetrix scores

v2.0 New UpCloud UI Update and links to new guides.

v1.9 Spelling and grammar

v1.8 Trial mode gotcha (deposit money ASAP)

v1.7 Added RSA Private key info

v1.7 – Added new firewall rules info.

v1.6 – Added more bloat to the site, still good.

v1.5 Improving Accessibility

v1.4 Added Firewall Price

v1.3 Added wp-config and plugin usage descriptions.

v1.2 Added GTMetrix historical chart.

v1.1 Fixed free typos and added final conclusion images.

v1.0 Added final results

v0.9 added more tweaks (http2 push, removing unwanted files etc)

v0.81 Draft  – Added memory usage chart and added MaxIOPS info from UpCloud.

v0.8 Draft post.

I thought my website was hacked. Here is how I hardened my Linux servers security with Lynis Enterprise

by

Disclaimer

I have waited a year before posting this, and I have tried my best to hide the bank’s identity as I never got a good explanation back from them about they the were whitelisting my website.

Background

I was casually reading Twitter one evening and found references to an awesome service (https://publicwww.com/) that allows you to find string references in CSS, JS, CSP etc files on websites.

See how people are using the publicwww service on Twitter.

I searched https://publicwww.com/ for “https://fearby.com“. I was expecting to only see only resources that were loading from my site.

I was shocked to see a bank in Asia was whistling my website and my websites CDN (hosted via ewww.io) in it’s Content Security Policy.

Screenshot of publicwww.com scan of "fearby.com

I was not hosting content for a bank and they should not be whitelisting my site?

Were they hacked? Was I hacked and delivering malware to their customers? Setting up a Content Security Policy (CSP) is not a trivial thing to do and I would suggest you check out https://report-uri.com/products/content_security_policy (by Scott Helme) for more information on setting up a good Content Security Policy (CSP).

Were we both hacked or was I serving malicious content?

Hacked Koala meme

I have written a few blog posts on creating Content Security Policies, and maybe they did copy my starter Content Security Policy and added it to their site?

I do have a lot of blog readers from their country.

Analytics map of Asia

I went to https://www.securityheaders.com and scanned their site and yes they have whitelisted my website and CDN. This was being sent in a header from their server to any connecting client.

I quickly double-checked the banks Content Security Policy (CSP) with https://cspvalidator.org/ and they too confirmed the bank was telling their customers that my website was ok to load files from.

I would not be worried if a florist’s website had white-listed my website but a bank that has 250 physical branches, 2,500 employees in a country that has 29 million people.

Below is the banks Content Security Policy.

https://cspvalidator.org/ screenshot of the banks csp

I thought I had been hacked into so I downloaded my Nginx log files (with MobaXTerm,) and scanned them for hits to my site from their website.

Screenshot of a years nginx logs.

After I scanned the logs I could see that I had zero traffic from their website

I sent a direct message to Scott Helme on Twitter (CSP Guru) and he replied with advice on the CSP.

Blocking Traffic

As a precaution, I edited my /etc/nginx/sites-available/default file and added this to block all traffic from their site.

if ($http_referer ~* "##########\.com") {
        return 404;
}

I tested and reloaded my Nginx config and restarted my web server

nginx -t
nginx -s reload
/etc/init.d/nginx restart

I also emailed my website CDN’s admin at https://ewww.io/ and asked them to block traffic from the bank as a precaution. They responded quickly as said this was done and they enabled extra logging in case more information was needed data.

If you need a good and fast WordPress Content Delivery Network (CDN) check out https://ewww.io/. They are awesome. Read my old review of ewww.io here.

I contacted the Bank

I searched the bank’s website for a way to contact them, their website was slow, their contact page was limited, they have a chat feature but I needed to log in with FaceBook (I don’t use FaceBook)

I viewed their contact us web page and they had zero dedicated security contacts listed. The CIO was only contactable via phone only.

They did not have a security.txt file on their website.

http://www.bankdomain.com/.well-known/security.txt file not found

TIP: If you run a website, please consider creating a security.txt file, information here.

I then viewed their contact us page and emailed everyone I could.

I asked if they could..

  • Check their logs for malicious files loaded from my site
  • Please remove the references to my website and CDN from their CSP.
  • Hinted they may want to review your CI/CD logs to see why this happened

My Server Hardening (to date)

My website was already hardened but was my site compromised?

Hardening actions to date..

  • Using a VPS firewall, Linux firewall 2x software firewalls
  • I have used the free Lynis Scan
  • Whitelisting access to port 22 to limited IP’s
  • Using hardware 2FA keys on SSH and WordPress Logins
  • Using the WordFence Security Plugin
  • Locked down unwanted ports.
  • I had a strong HTTPS certificate and website configuration (test here)
  • I have set up appropriate security headers (test here). I did need to re-setup a Content Security Policy (keep reading)
  • Performed many actions (some blogged a while ago) here: https://fearby.com/article/securing-ubuntu-cloud/
  • etc

I had used the free version of Lynis before but now is the time to use the Lynis Enterprise.

A free version of Lynis can be installed from Github here: https://github.com/CISOfy/lynis/

What is Lynis Enterprise?

Lynis Enterprise software is commercial auditing, system hardening, compliance testing tool for AIX, FreeBSD, HP-UX, Linux, macOS, NetBSD, OpenBSD, Solaris etc. The Enterprise version is a paid version (with web portal). Lynis Enterprise has more features over the free version.

Snip from here: “Lynis is a battle-tested security tool for systems running Linux, macOS, or Unix-based operating system. It performs an extensive health scan of your systems to support system hardening and compliance testing. The project is open-source software with the GPL license and available since 2007.”

Visit the Lynis Enterprise site here: https://cisofy.com/solutions/#lynis-enterprise.

I created a Lynis Enterprise Trial

I have used the free version of Lynis in the past (read here), but the Enterprise version offers a lot of extra features (read here).

Screenshot of https://cisofy.com/lynis-enterprise/why-upgrade/

View the main Lynis Enterprise site here and the pricing page here

View a tour of features here: https://cisofy.com/lynis-enterprise/

Create a Cisofy Trial Account

You can request a trial of Lynis Enterprise here: https://cisofy.com/demo/

Request a Lynis Enterprise trial screenshot

After the trial account was set up I logged in here. Upon login, I was prompted to add a system to my account (also my licence key was visible)

Lynis portal main screen

Install Lynis (Clone GIT Repo/latest features)

I am given 3 options to install Lynis from the add system page here.

  1. Add the software repository and install the client (The suggested and easiest way to install Lynis and keep it up-to-date).
  2. Clone the repository from Github (The latest development version, containing the most recent changes)
  3. Manually install or activate an already installed Lynis.

I will clone a fresh install from Github as I prefer seeing the latest issues, latest changes from GitHub notifications. I like getting notifications about security.

I logged into my server via SSH and ran the following command(s).

sudo apt-get instal git
mkdir /thefolder
cd /thefolder
git clone https://github.com/CISOfy/lynis

Cloning into 'lynis'...
remote: Enumerating objects: 7, done.
remote: Counting objects: 100% (7/7), done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 10054 (delta 0), reused 1 (delta 0), pack-reused 10047
Receiving objects: 100% (10054/10054), 4.91 MiB | 26.60 MiB/s, done.
Resolving deltas: 100% (7387/7387), done.

I logged into https://portal.cisofy.com/ and clicked ‘Add’ system to find my API key

I noted my licence key.

I then changed to my Lynis folder

cd lynis

I then created a “custom.prf” file

touch custom.prf

I ran this command to activate my licence (I have replaced my licence with ########’s).

View the documentation here.

./lynis configure settings license-key=########-####-####-####-############:upload-server=portal.cisofy.com

Output:

Configuring setting 'license-key'
Setting changed
Configuring setting 'upload-server'
Setting changed

I performed my first scan and uploaded the report.

TIP: Make sure you have curl installed

./lynis audit system --upload

After the scan is complete, make sure you see the following.

Data upload status (portal.cisofy.com) [ OK ]

I logged into https://portal.cisofy.com/enterprise/systems/ and I could view my systems report.

You can read the basic Lynis documentation here: https://cisofy.com/documentation/lynis/

Manual Lynis Scans

I can run a manual scan at any time

cd /thefolder/lynis/
sudo ./lynis audit system --upload

To view results I can login to https://portal.cisofy.com/

Automated Lynis Scans

I have created a bash script that updates Lynis (basically running ‘sudo /usr/bin/git pull origin master’ in the lynis folder)

#!/bin/bash

sendemail -f [email protected] -t [email protected] -u "CRON: Updating Lynis (yourserver.com) START" -m "/folder/runlynis.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp ***my*google*gsuite*email*app*password***

echo "Changing Directory to /folder/lynis"
cd /folder/lynis

echo "Updating Lynis"
sudo /usr/bin/git pull origin master

sendemail -f [email protected] -t [email protected] -u "CRON: Updated Lynis (yourserver.com) END" -m "/folder/runlynis.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp ***my*google*gsuite*email*app*password***

This is my bash script that runs Lynis scans and emails the report

#!/bin/bash

sendemail -f [email protected] -t [email protected] -u "CRON: Run Lynis (yourserver.com) START" -m "/folder/runlynis.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp ***my*google*gsuite*email*app*password***

echo "Running Lynis Scan"
cd /utils/lynis/
sudo /utils/lynis/lynis audit system --upload > /folder/lynis/lynis.txt

sendemail -f [email protected] -t [email protected] -u "CRON: Run Lynis (yourserver.com) END" -m "/folder/runlynis.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp ***my*google*gsuite*email*app*password***  -a /folder/lynis/lynis.txt

I set up two cron jobs to update Lynis (from Git) and to scan with Lynis every day.

#Lynis Update 11:55PM
55 21 * * * /bin/bash /folder/runlynis.sh && curl -fsS --retry 3 https://hc-ping.com/########-####-####-####-############ > /dev/null

#Lynis Scan 2AM
0 2 * * * /bin/bash /folder/runlynis.sh && curl -fsS --retry 3 https://hc-ping.com/########-####-####-####-############ > /dev/null

Thanks to sendemail I get daily emails

I have set up cronjob motoring and emails at the start and end of the bash scripts.

The attachment is not a pretty text file report but a least I can see the output of the scan (without logging into the portal).

Maybe I add the following file also

/var/log/lynis.log

Lynis Enterprise (portal.cisofy.com)

Best of all Lynis Enterprise comes with a great online dashboard available at
https://portal.cisofy.com/enterprise/dashboard/.

Lynis Enterprise Portal

Dashboard (portal.cisofy.com)

Clicking the ‘Dashboard‘ button in the toolbar at the top of the portal reveals a summary of your added systems, alerts, compliance, system integrity, Events and statistics.

Dashboard button

The dashboard has three levels

  • Business (less information)
  • Operational
  • Technical (more information)

Read about the differences here.

three dashboard breadcrumbs

Each dashboard has a limited number of elements, but the technical dashboard has all the elements.

Technical Dashboard

Lynis Enterprise Dashboard https://portal.cisofy.com/enterprise/dashboard/

From here you can click and open server scan results (see below)

Server Details

If you click on a server name you can see detailed information. I created 2 test servers (I am using the awesome UpCloud host)

A second menu appears when you click on a server

Linus Menu

Test Server 01: Ubuntu 18.04 default Scan Results (66/100)

Ubuntu Server Score 66/100

Test Server 02: Debian 9.9 default Scan Results (65/100)

Server

It is interesting to see Debian is 1 point below Ubuntu.

The server page will give a basic summary and highlights like the current and previous hardening score, open ports, firewall status, installed packages, users.

When I click the server name to load the report I can click to see ‘Warnings’ or ‘Suggestions’ to resolve

Suggested System Hardening Actions

I had 47 system hardening recommendations on one system

Lynis identified quick wins.

Some of the security hardening actions included the following.

e.g

  • Audit daemon is enabled with an empty ruleset. Disable the daemon or define rules
  • Incorrect permissions for file /root/.ssh
  • A reboot of the system is most likely needed
  • Found some information disclosure in SMTP banner (OS or software name)
  • Configure maximum password age in /etc/login.defs
  • Default umask in /etc/login.defs could be more strict like 027
  • Add a legal banner to /etc/issue.net, to warn unauthorized users
  • Check available certificates for expiration
  • To decrease the impact of a full /home file system, place /home on a separate partition
  • Install a file integrity tool to monitor changes to critical and sensitive files
  • Check iptables rules to see which rules are currently not used
  • Harden compilers like restricting access to root user only
  • Disable the ‘VRFY’ command
  • Add the IP name and FQDN to /etc/hosts for proper name resolving
  • Purge old/removed packages (59 found) with aptitude purge or dpkg –purge command. This will clean up old configuration files, cron jobs and startup scripts.
  • Remove any unneeded kernel packages
  • Determine if automation tools are present for system management
  • etc

Hardening Suggestion (Ignore or Solve)

If you click ‘Solve‘ Cisofy will provide a link to detailed information to help you solve issues.

Suggested fix: ACCT-9630 Audit daemon is enabled with an empty ruleset. Disable the daemon or define rules

I will not list every suggested problem and fix but here are some fixes below.

ACCT-9630 Audit daemon is enabled with an empty ruleset. Disable the daemon or define rules (fixed)

TIP: If you don’t have auditd installed run this command below to install it

sudo apt-get install auditd
/etc/init.d/auditd start
/etc/init.d/auditd status

I added the following to ‘/etc/audit/rules.d/audit.rules‘ (thanks to the solution recommendations on the Cisofy portal.

# This is an example configuration suitable for most systems
# Before running with this configuration:
# - Remove or comment items which are not applicable
# - Check paths of binaries and files

###################
# Remove any existing rules
###################

-D

###################
# Buffer Size
###################
# Might need to be increased, depending on the load of your system.
-b 8192

###################
# Failure Mode
###################
# 0=Silent
# 1=printk, print failure message
# 2=panic, halt system
-f 1

###################
# Audit the audit logs.
###################
-w /var/log/audit/ -k auditlog

###################
## Auditd configuration
###################
## Modifications to audit configuration that occur while the audit (check your paths)
-w /etc/audit/ -p wa -k auditconfig
-w /etc/libaudit.conf -p wa -k auditconfig
-w /etc/audisp/ -p wa -k audispconfig

###################
# Monitor for use of audit management tools
###################
# Check your paths
-w /sbin/auditctl -p x -k audittools
-w /sbin/auditd -p x -k audittools

###################
# Special files
###################
-a exit,always -F arch=b32 -S mknod -S mknodat -k specialfiles
-a exit,always -F arch=b64 -S mknod -S mknodat -k specialfiles

###################
# Mount operations
###################
-a exit,always -F arch=b32 -S mount -S umount -S umount2 -k mount
-a exit,always -F arch=b64 -S mount -S umount2 -k mount

###################
# Changes to the time
###################
-a exit,always -F arch=b32 -S adjtimex -S settimeofday -S stime -S clock_settime -k time
-a exit,always -F arch=b64 -S adjtimex -S settimeofday -S clock_settime -k time
-w /etc/localtime -p wa -k localtime

###################
# Use of stunnel
###################
-w /usr/sbin/stunnel -p x -k stunnel

###################
# Schedule jobs
###################
-w /etc/cron.allow -p wa -k cron
-w /etc/cron.deny -p wa -k cron
-w /etc/cron.d/ -p wa -k cron
-w /etc/cron.daily/ -p wa -k cron
-w /etc/cron.hourly/ -p wa -k cron
-w /etc/cron.monthly/ -p wa -k cron
-w /etc/cron.weekly/ -p wa -k cron
-w /etc/crontab -p wa -k cron
-w /var/spool/cron/crontabs/ -k cron

## user, group, password databases
-w /etc/group -p wa -k etcgroup
-w /etc/passwd -p wa -k etcpasswd
-w /etc/gshadow -k etcgroup
-w /etc/shadow -k etcpasswd
-w /etc/security/opasswd -k opasswd

###################
# Monitor usage of passwd command
###################
-w /usr/bin/passwd -p x -k passwd_modification

###################
# Monitor user/group tools
###################
-w /usr/sbin/groupadd -p x -k group_modification
-w /usr/sbin/groupmod -p x -k group_modification
-w /usr/sbin/addgroup -p x -k group_modification
-w /usr/sbin/useradd -p x -k user_modification
-w /usr/sbin/usermod -p x -k user_modification
-w /usr/sbin/adduser -p x -k user_modification

###################
# Login configuration and stored info
###################
-w /etc/login.defs -p wa -k login
-w /etc/securetty -p wa -k login
-w /var/log/faillog -p wa -k login
-w /var/log/lastlog -p wa -k login
-w /var/log/tallylog -p wa -k login

###################
# Network configuration
###################
-w /etc/hosts -p wa -k hosts
-w /etc/network/ -p wa -k network

###################
## system startup scripts
###################
-w /etc/inittab -p wa -k init
-w /etc/init.d/ -p wa -k init
-w /etc/init/ -p wa -k init

###################
# Library search paths
###################
-w /etc/ld.so.conf -p wa -k libpath

###################
# Kernel parameters and modules
###################
-w /etc/sysctl.conf -p wa -k sysctl
-w /etc/modprobe.conf -p wa -k modprobe
###################

###################
# PAM configuration
###################
-w /etc/pam.d/ -p wa -k pam
-w /etc/security/limits.conf -p wa -k pam
-w /etc/security/pam_env.conf -p wa -k pam
-w /etc/security/namespace.conf -p wa -k pam
-w /etc/security/namespace.init -p wa -k pam

###################
# Puppet (SSL)
###################
#-w /etc/puppet/ssl -p wa -k puppet_ssl

###################
# Postfix configuration
###################
#-w /etc/aliases -p wa -k mail
#-w /etc/postfix/ -p wa -k mail
###################

###################
# SSH configuration
###################
-w /etc/ssh/sshd_config -k sshd

###################
# Hostname
###################
-a exit,always -F arch=b32 -S sethostname -k hostname
-a exit,always -F arch=b64 -S sethostname -k hostname

###################
# Changes to issue
###################
-w /etc/issue -p wa -k etcissue
-w /etc/issue.net -p wa -k etcissue

###################
# Log all commands executed by root
###################
-a exit,always -F arch=b64 -F euid=0 -S execve -k rootcmd
-a exit,always -F arch=b32 -F euid=0 -S execve -k rootcmd

###################
## Capture all failures to access on critical elements
###################
-a exit,always -F arch=b64 -S open -F dir=/etc -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/home -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/sbin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/srv -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/local/bin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/usr/sbin -F success=0 -k unauthedfileacess
-a exit,always -F arch=b64 -S open -F dir=/var -F success=0 -k unauthedfileacess

###################
## su/sudo
###################
-w /bin/su -p x -k priv_esc
-w /usr/bin/sudo -p x -k priv_esc
-w /etc/sudoers -p rw -k priv_esc

###################
# Poweroff/reboot tools
###################
-w /sbin/halt -p x -k power
-w /sbin/poweroff -p x -k power
-w /sbin/reboot -p x -k power
-w /sbin/shutdown -p x -k power

###################
# Make the configuration immutable
###################
-e 2

# EOF

I reloaded my audit daemon config

auditctl -R /etc/audit/rules.d/audit.rules

Further configuration can be added (read this), read the auditd man page here or read logs you can use the ‘auditsearch‘ tool (read the Ubuntu Man Page here)

Here is a great guide on viewing audit events.

Because we have this rule ( ‘-w /etc/passwd -p wa -k etcpasswd ) to monitor the passwords file, If I read the contents of \etc\passwd it will show up in the audit logs.

We can verify the access of this file by running this command

ausearch -f /etc/passwd

Output

ausearch -f /etc/passwd
----
time->Mon Jun 10 16:58:13 2019
type=PROCTITLE msg=audit(##########.897:3639): proctitle=##########################
type=PATH msg=audit(##########.897:3639): item=1 name="/etc/passwd" inode=1303 dev=fc:01 mode=0100644 ouid=0 ogid=0 rdev=00:00 nametype=NORMAL cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
type=PATH msg=audit(##########.897:3639): item=0 name="/etc/" inode=12 dev=fc:01 mode=040755 ouid=0 ogid=0 rdev=00:00 nametype=PARENT cap_fp=0000000000000000 cap_fi=0000000000000000 cap_fe=0 cap_fver=0
type=CWD msg=audit(##########.897:3639): cwd="/root"
type=SYSCALL msg=audit(##########.897:3639): arch=c000003e syscall=257 success=yes exit=3 a0=ffffff9c a1=556241ea9650 a2=441 a3=1b6 items=2 ppid=1571 pid=1572 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=446 comm="nano" exe="/bin/nano" key="etcpasswd"

I might write a list of handy ausearech commands and blog about this in the future

SSH Permissions (fixed)

to fish the ssh permissions warning I ran the command to show the issue on my server

./lynis show details FILE-7524
2019-05-25 23:00:04 Performing test ID FILE-7524 (Perform file permissions check)
2019-05-25 23:00:04 Test: Checking file permissions
2019-05-25 23:00:04 Using profile /utils/lynis/default.prf for baseline.
2019-05-25 23:00:04 Checking /etc/lilo.conf
2019-05-25 23:00:04   Expected permissions:
2019-05-25 23:00:04   Actual permissions:
2019-05-25 23:00:04   Result: FILE_NOT_FOUND
2019-05-25 23:00:04 Checking /root/.ssh
2019-05-25 23:00:04   Expected permissions: rwx------
2019-05-25 23:00:04   Actual permissions: rwxr-xr-x
2019-05-25 23:00:04   Result: BAD
2019-05-25 23:00:04 Warning: Incorrect permissions for file /root/.ssh [test:FILE-7524] [details:-] [solution:-]
2019-05-25 23:00:04 Using profile /utils/lynis/custom.prf for baseline.
2019-05-25 23:00:04 Checking permissions of /utils/lynis/include/tests_homedirs
2019-05-25 23:00:04 File permissions are OK
2019-05-25 23:00:04 ===---------------------------------------------------------------===

I tightened permissions on the /root/.ssh folder with this command

chmod 700 /root/.ssh

Configure minimum/maximum password age in /etc/login.defs (fixed)

I set a maximum and minimum password age in ‘/etc/login.defs‘

Defaults

PASS_MAX_DAYS   99999
PASS_MIN_DAYS   0
PASS_WARN_AGE   7

Add a legal banner to /etc/issue, to warn unauthorized users (fixed)

I edited ‘/etc/issue’ on Ubuntu and Linux

Ubuntu 18.04 default

Ubuntu 18.04.2 LTS \n \l

Debian Default

Debian GNU/Linux 9 \n \l

Cisofy said this “Define a banner text to inform both authorized and unauthorized users about the machine and service they are about to access. The purpose is to share your policy before an access attempt is being made. Users should know that there privacy might be invaded, due to monitoring of the system and its resources, to protect the integrity of the system. Also unauthorized users should be deterred from trying to access it in the first place.“

Done

Default umask in /etc/login.defs could be more strict like 027 (fixed)

Related files..

  • /etc/profile
  • /etc/login.defs
  • /etc/passwd

I edited ‘/etc/login.defs’ and set 

UMASK           027

I ran

umask 027 /etc/profile
umask 027 /etc/login.defs
umask 027 /etc/passwd

Check iptables rules to see which rules are currently not used (fixed)

I ran the following command to review my firewall settings

iptables --list --numeric --verbose

TIP: Scan for open ports with ‘nmap’

Watch this handy video if you are not sure how to use nmap

Install nmap

sudo apt-get install nmap

I do set firewall rules in ufw (guide here) and ufw is a front end for iptables.

Scan for open ports with nmap

nmap -v -sT localhost

Starting Nmap 7.60 ( https://nmap.org ) at 2019-06-12 22:09 AEST
Initiating Connect Scan at 22:09
Scanning localhost (127.0.0.1) [1000 ports]
Discovered open port 443/tcp on 127.0.0.1
Discovered open port 22/tcp on 127.0.0.1
Discovered open port 8080/tcp on 127.0.0.1
Discovered open port 25/tcp on 127.0.0.1
Discovered open port 80/tcp on 127.0.0.1
Completed Connect Scan at 22:09, 0.02s elapsed (1000 total ports)
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00012s latency).
Not shown: 994 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
25/tcp   open  smtp
80/tcp   open  http
443/tcp  open  https
8080/tcp open  http-proxy

Everything looked good.

Harden compilers like restricting access to root user only (fixed)

Cicofy said

Compilers turn source code into binary executable code. For a production system a compiler is usually not needed, unless package upgrades are performed by means of their source code (like FreeBSD ports collection). If a compiler is found, execution should be limited to authorized users only (e.g. root user).

To solve this finding, remove any unneeded compiler or change the file permissions. Usually chmod 700 or chmod 750 will be enough to prevent normal users from using a compiler. Related compilers are as, cc, ld, gcc, go etc. To determine what files are affected, check the Lynis log file, then chmod these files.

I ran

chmod 700 /usr/bin/as
chmod 700 /usr/bin/gcc

Turn off PHP information exposure (fixed)

Cisofy siad

Disable the display of version information by setting the expose_php option to 'Off' in php.ini. As several instances of PHP might be installed, ensure that all related php.ini files have this setting turned off, otherwise this control will show up again.

This was already turned off but a unused php.ini may have been detected.

I searched for all php.ini files

find / -name php.ini

Output

/etc/php/7.3/apache2/php.ini
/etc/php/7.3/fpm/php.ini
/etc/php/7.3/cli/php.ini

yep, the cli version of php.ini had the following

expose_php = On

I set this to Off

Purge old/removed packages (59 found) with aptitude purge or dpkg –purge command. This will cleanup old configuration files, cron jobs and startup scripts. (fixed)

Cisofy said

While not directly a security concern, unpurged packages are not installed but still have remains left on the system (e.g. configuration files). In case software is reinstalled, an old configuration might be applied. Proper cleanups are therefore advised.

To remove the unneeded packages, select the ones marked with the 'rc' status. This means the package is removed, but the configuration files are still there.

I ran the following recommended command

dpkg -l | grep "^rc" | cut -d " " -f 3 | xargs dpkg --purge

Done

Install debsums utility for the verification of packages with known good database. (fixed)Cisofy said 

Install the debsums utility to do more in-depth auditing of your packages.

I ran the following suggested command

apt-get install debsums

I googled and found this handy page

I scanned packages and asked ‘debsums” to only show errors with this command

sudo debsums -s

The only error was..

debsums: missing file /usr/bin/pip (from python-pip package)

I did not need pip so I removed it

apt-get remove --purge python-pip

Install a PAM module for password strength testing like pam_cracklib or pam_passwdqc (fixed)

I ignore this as I do not allow logins via password and only I have an account (it’s not a multi user system).

I white list logins to IP’s.

I only allow ssh access with a private key and long passphrase.

I have 2FA OTP enabled at logins.

I have cloudflare over my domain.

I setup fail2ban to auto block logins using this guide

Reboot (fixed)

I restated the server

shutdown -r now

Done

Check available certificates for expiration (fixed)

I tested my SSL certificate with https://dev.ssllabs.com

https://dev.ssllabs.com/ scan of my site

Add legal banner to /etc/issue.net, to warn unauthorized users (fixed)

Cisofy said…

Define a banner text to inform both authorized and unauthorized users about the machine and service they are about to access. The purpose is to share your policy before an access attempt is being made. Users should know that there privacy might be invaded, due to monitoring of the system and its resources, to protect the integrity of the system. Also unauthorized users should be deterred from trying to access it in the first place.

Do not reveal sensitive information, like the specific goal of the machine, or what can be found on it. Consult with your legal department, to determine appropriate text.

I edited the file ‘/etc/issue.net’ and added a default pre login message (same as ‘/etc/issue’).

Install Apache mod_evasive to guard webserver against DoS/brute force attempts (ignored)

I ignored this message and I don’t use the Apache (I use the Nginx web server). I have added Apache to be blocked from installing.

I clicked Ignore in the Cisofy portal.

Ignore Button

Install Apache modsecurity to guard webserver against web application attacks (ignored)

I clicked Ignore for this one too

Ignore Button

Check your Nginx access log for proper functioning (reviewed)

Cisofy said…

Disabled logging:
Check in the Lynis log for entries which are disabled, or in the nginx configuration (access_log off).

Missing logging:
Check for missing log files. They are references in the configuration of nginx, but not on disk. The Lynis log will reveal to what specific files this applies.

I checked my Nginx config (‘/etc/nginx/nginx.conf‘) for all log references and ensured the logs were writing to disk (OK).

I checked my ‘/etc/nginx/sites-available/default‘ config and I did have 2 settings of ‘access_log off ‘ (this was added during the setup for two sub reporting subfolders for the Nixstats agent.

I restarted Nginx

nginx -t
nginx -s reload
/etc/init.d/nginx restart

Check what deleted files are still in use and why. (fixed)

Cisofy said..

Why it matters
Deleted files may sometimes be in use by applications. Normally this should not happen, as an application should delete a file and release the file handle. This test might discover malicious software, trying to hide its presence on the system. Investigate the related files by determining which application keeps it open and the related reason.

Details
The following details have been found as part of the scan.

/lib/systemd/systemd-logind(systemd-l)
/tmp/ib1ekCtf(mysqld)
/tmp/ibhuK1At(mysqld)
/tmp/ibmTO5F5(mysqld)
/tmp/ibR0dkxD(mysqld)
/tmp/ibvf69KH(mysqld)
/tmp/.ZendSem.gq3mnz(php-fpm7.)
/usr/bin/python3.6(networkd-)
/usr/bin/python3.6(unattende)
/var/log/mysql/error.log.1(mysqld)

I ran the following command to show deleted files in use

lsof | grep deleted

I noticed on my database server a php-fpm service was using files. I don’t have a webserver enabled on this server, so I uninstalled the web-based services.

I have separate web and database servers.

sudo apt-get remove apache*
sudo apt-get remove -y --purge nginx*
sudo apt-get remove -y --purge php7*
sudo apt autoremove

Check DNS configuration for the dns domain name (fixed)

Cisofy said..

Some software can work incorrectly when the system can't resolve itself. 
Add the IP name and fully qualified domain name (FQDN) to /etc/hosts. Usually this is done with an entry of 127.0.0.1, or 127.0.1.1 (to leave the localhost entry alone).

I edited my ‘/etc/hosts’ file

I added a domain name to the end of the localhost entry and added a new line with my server(s) IP and domain name

Disable the ‘VRFY’ command (fixed)

I was advised to run this command

postconf -e disable_vrfy_command=yes

(Debian) Enable sysstat to collect accounting (no results) (fixed)

Cisofy said..

The sysstat is collection of utilities to provide system information insights. While one should aim for the least amount of packages, the sysstat utilities can be a good addition to help recording system details. They can provide insights for performance monitoring, or guide in discovering unexpected events (like a spam run). If you already use extensive system monitoring, you can safely ignore this control.

I ran the suggested commands

apt-get install sysstat
sed -i 's/^ENABLED="false"/ENABLED="true"/' /etc/default/sysstat

More info on sysstat here.

Consider running ARP monitoring software (arpwatch,arpon) (fixed)

Cisofy said

Networks are very dynamic, often with devices come and go as they please. For sensitive machines and network zones, you might want to know what happens on the network itself. An utility like arpwatch can help tracking changes, like new devices showing up, or others leaving the network.

I read this page to setup and configure arpwatch

sudo apt-get install arpwatch
/etc/init.d/arpwatch start

I will add more on how to use arpwatch soon

Disable drivers like USB storage when not used, to prevent unauthorized storage or data theft (fixed)

Cosofy siad..

Disable drivers like USB storage when not used. This helps preventing unauthorized storage, data copies, or data theft.

I ran the suggested fix

echo "# Block USB storage" >> /etc/modprobe.d/disable-usb-storage.conf
echo "install usb-storage /bin/false" >> /etc/modprobe.d/disable-usb-storage.conf

Determine if automation tools are present for system management (ignored)

I ignored this one

Ignore Button

One or more sysctl values differ from the scan profile and could be tweaked

Cisofy said..

By means of sysctl values we can adjust kernel related parameters. Many of them are related to hardening of the network stack, how the kernel deals with processes or files. This control is a generic test with several sysctl variables (configured by the scan profile).

I was advised to adjust these settings

  • net.ipv4.conf.all.send_redirects=0
  • net.ipv4.conf.default.accept_source_route=0
  • kernel.sysrq=0
  • net.ipv4.conf.all.log_martians=1
  • net.ipv4.conf.default.log_martians=1
  • kernel.core_uses_pid=1
  • kernel.kptr_restrict=2
  • fs.suid_dumpable=0
  • kernel.dmesg_restrict=1

I edited ‘/etc/sysctl.conf‘ and made the advised changes along with these (I Googled each item first)

Install a file integrity tool to monitor changes to critical and sensitive files (fixed)

Cisofy said..

To monitor for unauthorized changes, a file integrity tool can help with the detection of such event. Each time the contents or the properties of a file change, it will have a different checksum. With regular checks of the related integrity database, discovering changes becomes easy. Install a tool like AIDE, Samhain or Tripwire to monitor important system and data files. Additionally configure the tool to alert system or security personnel on events.

It also gave a solution

# Step 1: Install package with appropriate command apt-get install aide yum install aide # Step 2: Initialise database aide --init # If this fails: try aideinit # Step 3: Copy newly created database (/var/lib/aide) cp /var/lib/aide/aide.db.new.gz /var/lib/aide/aide.db.gz # Step 4: aide --check

I installed ‘aide’ (read the guide here).

TIP: Long story but the steps above were not exactly correct. Thanks to this post for I was able to set up aide. without seeing this error.

Couldn't open file /var/lib/aide/please-dont-call-aide-without-parameters/aide.db.new for writing

This is how I installed aide

apt-get install aide
apt-get install aide-common

I initialised aide.

aideinit

This was the important part (I was stuck for hours on this one)

aide.wrapper --check

I can run the following to see what files have changed.

I could see many files have changed since the initial scan (e.g mysql, log files nano search history).

Nice

Now lets schedule daily checks and create a cron job.

cat /folder/runaide.sh
#!/bin/bash

sendemail -f [email protected] -t [email protected] -u "CRON: AIDE Run (yourserver.com) START" -m "/folder/runaide.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp ***my*google*gsuite*email*app*password***

MYDATE=`date +%Y-%m-%d`
MYFILENAME="Aide-"$MYDATE.txt
/bin/echo "Aide check !! `date`" > /tmp/$MYFILENAME
/usr/bin/aide.wrapper --check > /tmp/myAide.txt
/bin/cat /tmp/myAide.txt|/bin/grep -v failed >> /tmp/$MYFILENAME
/bin/echo "**************************************" >> /tmp/$MYFILENAME
/usr/bin/tail -100 /tmp/myAide.txt >> /tmp/$MYFILENAME
/bin/echo "****************DONE******************" >> /tmp/$MYFILENAME

#/usr/bin/mail -s"$MYFILENAME `date`" [email protected] < /tmp/$MYFILENAME

sendemail -f [email protected] -t [email protected] -u "CRON: AIDE Run (yourserver.com) END" -m "/folder/runaide.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp ***my*google*gsuite*email*app*password*** -a /tmp/$MYFILENAME -a /tmp/myAide.txt

Above thanks to this post

I setup a cron job to run this daily

#Run AIDE
0 6 * * * /folder/runaide.sh && curl -fsS --retry 3 https://hc-ping.com/######-####-####-####-############> /dev/null

ACCT-9622 – Enable process accounting. (fixed)

Solution:

Install “acct” process and login accounting.

sudo apt-get install acct

Start the “acct” service

/etc/init.d/acct start
touch /var/log/pacct
chown root /var/log/pacct
chmod 0644 /var/log/pacct
accton /var/log/pacct

Check the status

/etc/init.d/acct status
* acct.service - LSB: process and login accounting
   Loaded: loaded (/etc/init.d/acct; generated)
   Active: active (exited) since Sun 2019-05-26 19:42:15 AEST; 4min 42s ago
     Docs: man:systemd-sysv-generator(8)
    Tasks: 0 (limit: 4660)
   CGroup: /system.slice/acct.service

May 26 19:42:15 servername systemd[1]: Starting LSB: process and login accounting...
May 26 19:42:15 servername acct[27419]: Turning on process accounting, file set to '/var/log/account/pacct'.
May 26 19:42:15 servername systemd[1]: Started LSB: process and login accounting.
May 26 19:42:15 servername acct[27419]:  * Done.

Run CISOfy recommended commands

touch /var/log/pacct
chown root /var/log/pacct
chmod 0644 /var/log/pacct
accton /var/log/pacct

Manual Scan of Lynis

I re-ran an audit of the system (and uploaded the report to the portal) so I can see how I am progressing.

./lynis audit system --upload

I then checked the error status and the warnings were resolved.

Progress?

I rechecked my servers and all warnings are solved, now I just need to work on information level issues

Warning level errors fixed, and informational to go

Cisofy Portal Overview

Quick breakdown of the Cisofy Portal

Overview Tab (portal.cisofy.com)

The Overview lab displays any messages, change log, API information, add a new system link, settings etc.

Lynis Overview tab

Dashboard Tab (portal.cisofy.com)

The dashboard tab will display compliant systems any outdated systems, alerts and events.

Lynis Dashboard screenshot https://portal.cisofy.com/enterprise/dashboard/

TIP: If you have a system that reports “Outdated” run the following command.

./lynis audit system --upload

Systems Tab (portal.cisofy.com)

The systems tab shows all systems, OS version, warnings, information counts, the date the system’s client last uploaded a report and the client version.

Systems tab shows all systems, OS version, warnings, information counts, date client last uploaded a report update and client version

If you are making many changes and manual Lynis scans keep an eye on your upload credits, You can see by the above and below image, I have lowered my suggested actions to harden my servers (red text).

Lynis scans reached

Clicking a host name reveals a summary of the system.

Clicking a system reveals a summary of the system.

Remaining information level issues are listed.

I can click Solve and see more information about the issue to resolve.

TIP: I thought it would be a good idea to copy this list to a spreadsheet for detailed tracking.

Spreadsheet listing issues to complete and done

I had another issue appear a few days later.

Compliance Tab (portal.cisofy.com)

A lot of information is listed here.

Compliance Tab

Best practice guides are available

best practice ghttps://portal.cisofy.com/compliance/udes

I could go on an on but https://cisofy.com/ is awesome.

TIP: Manually updating Lynis

from the command line I can view the Linus version with this command

./lynis --version
2.7.4

To update the Lynis git repository from the Lynis folder run this command

git pull
Already up to date.

Automatically updating and running Lynis scans

I added the following commands to my crontab to update then scan and report Lynis results to the portal.

TIP: Use https://crontab.guru/ to choose the right time to run commands (I chose 5 mins past 1 AM every day to update and 5 mins past 2 AM to run a scan.


#Lynis Update
5 1 * * * root -s /bin/bash -c 'cd /utils/lynis && /usr/bin/git pull origin master'

#Lynis Scan
5 2 * * * root -s /bin/bash -c '/utils/lynis/lynis audit system --upload'

Troubleshooting

fyi: Lynis Log file location: /var/log/lynis.log

Cisofy Enterprise Conclusion

Pros:

  • I can learn so much about securing Linux just from the Cisofy Fix recommendations.
  • I have secured my server beyond what I thought possible.
  • Very active development on Github: https://github.com/CISOfy/lynis/
  • Cisofy has a very good inteface and updates often.
  • New security issues are synced down and included in new scans (if you update)

Cons:

  • I am unable to pay for this for my servers here in Australia (European legal issues).
  • Needs Hardware 2FA

Tips

Make sure you have curl installed to allow reports to upload. I had this error on Debian 9.4.

View the latest repository version information here.

I added my Lynis folder to the Linux $PATH variable

export PATH=$PATH:/folder/lynis

Fatal: can’t find curl binary. Please install the related package or put the binary in the PATH. Quitting..

Lynis Enterprise API

View the Lynis Enterprise API documentation here

Lynis Enterprise Support

Support can be found here, email support [email protected].

Getting started guide is found here.

Bonus: Setting Up Content Security Policy and reporting violations to https://report-uri.com/

I have a few older posts on Content Security Policies (CSP) but they are a bit dated.

Wikipedia Definition of a Content Security Policy

Content Security Policy (CSP) is a computer security standard introduced to prevent cross-site scripting (XSS), clickjacking and other code injection attacks resulting from execution of malicious content in the trusted web page context.[1] It is a Candidate Recommendation of the W3C working group on Web Application Security,[2] widely supported by modern web browsers.[3] CSP provides a standard method for website owners to declare approved origins of content that browsers should be allowed to load on that website—covered types are JavaScript, CSS, HTML frames, web workers, fonts, images, embeddable objects such as Java applets, ActiveX, audio and video files, and other HTML5 features.

If you want to learn about to setup CSP’s head over to https://report-uri.com/products/content_security_policy or https://report-uri.com/home/tools and read more.

I did have Content Security Policies (CSP) set up a few years back, but I had issues with broken resources. A lack of time on my behalf to investigate the issues forced me to disable the Content Security Policy (CSP). I should have changed the “Content-Security-Policy” header to “Content-Security-Policy-Report-Only.”

I will re-add the Content Security Policy (CSP) to my site but this time I will not disable it and will report to https://report-uri.com/, and if need be I will change the header from “content-security-policy” to “content-security-policy-report-only”. That way a broken policy won’t take down my site in future.

If you want to set up a Content Security Policy header and with good reporting of any violations of your CSP policy simply head over to https://report-uri.com/ and create a new account.

Read the official Report URI help documents here: https://docs.report-uri.com/.

Create a Content Security Policy

The hardest part of creating a Content Security Policy is knowing what to add where.

You could generate your own Content Security Policy by heading here (https://report-uri.com/home/generate) but that will take a while.

Create a CSP

TIP: Don’t make your policy live straight away by using the “Content-Security-Policy” header, instead use the “Content-Security-Policy-Report-Only” header.

To create a content Security Policy faster I would recommend you to use this Firefox plugin to generate a starter Content Security Policy.

Screenshot of https://addons.mozilla.org/en-US/firefox/addon/laboratory-by-mozilla/

Install this plugin to Firefox, enable it and click the Plugins icon and ensure “Record this site…” is ticked.

Laboratory plugin inFirefix

Then simply browse to your site (browse as many pages as possible) and a Content Security Policy will be generated based on the content on the page(s) loaded.

TIP: Always review the generated CSP, it allows everything needed to display your site.

Export the CSP from the Firefox plugin to the clipboard

This is the policy that was generated for me in 5 minutes browsing 20 pages.

default-src 'none'; connect-src 'self' https://onesignal.com/api/v1/apps/772f27ad-0d58-494f-9f06-e89f72fd650b/icon https://onesignal.com/api/v1/notifications https://onesignal.com/api/v1/players/67a2f360-687f-4513-83e8-f477da085b26 https://onesignal.com/api/v1/players/67a2f360-687f-4513-83e8-f477da085b26/on_session https://yoast.com/feed/widget/; font-src 'self' data: https://fearby-com.exactdn.com https://fonts.gstatic.com; form-action 'self' https://fearby.com https://syndication.twitter.com https://www.paypal.com; frame-src 'self' https://en-au.wordpress.org https://fearby.com https://googleads.g.doubleclick.net https://onesignal.com https://platform.twitter.com https://syndication.twitter.com https://www.youtube.com; img-src 'self' data: https://a.impactradius-go.com https://abs.twimg.com https://fearby-com.exactdn.com https://healthchecks.io https://pagead2.googlesyndication.com https://pbs.twimg.com https://platform.twitter.com https://secure.gravatar.com https://syndication.twitter.com https://ton.twimg.com https://www.paypalobjects.com; script-src 'self' 'unsafe-eval' 'unsafe-inline' https://adservice.google.com.au/adsid/integrator.js https://adservice.google.com/adsid/integrator.js https://cdn.onesignal.com/sdks/OneSignalPageSDKES6.js https://cdn.onesignal.com/sdks/OneSignalSDK.js https://cdn.syndication.twimg.com/tweets.json https://fearby-com.exactdn.com/wp-content/cache/fvm/1553589606/out/footer-45a3439e.min.js https://fearby-com.exactdn.com/wp-content/cache/fvm/1553589606/out/footer-e6604f67.min.js https://fearby-com.exactdn.com/wp-content/cache/fvm/1553589606/out/footer-f4213fd6.min.js https://fearby-com.exactdn.com/wp-content/cache/fvm/1553589606/out/header-1583146a.min.js https://fearby-com.exactdn.com/wp-content/cache/fvm/1553589606/out/header-823c0a0e.min.js https://fearby-com.exactdn.com/wp-content/piwik.js https://onesignal.com/api/v1/sync/772f27ad-0d58-494f-9f06-e89f72fd650b/web https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js https://pagead2.googlesyndication.com/pagead/js/r20190610/r20190131/show_ads_impl.js https://pagead2.googlesyndication.com/pub-config/r20160913/ca-pub-9241521190070921.js https://platform.twitter.com/js/moment~timeline~tweet.a20574004ea824b1c047f200045ffa1e.js https://platform.twitter.com/js/tweet.73b7ab8a56ad3263cad8d36ba66467fc.js https://platform.twitter.com/widgets.js https://s.ytimg.com/yts/jsbin/www-widgetapi-vfll-F3yY/www-widgetapi.js https://www.googletagservices.com/activeview/js/current/osd.js https://www.youtube.com/iframe_api; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com/ https://onesignal.com/sdks/ https://platform.twitter.com/css/ https://ton.twimg.com/tfw/css/; worker-src 'self'

I can truncate starter Content Security Polity and remove some elements. Remove duplicated entries to separate files on a remote server add a wildcard (if I trust the server).

I truncated the policy with the help of the sublime text editor and Report URI CSP Generator.

I added this to the file ‘/etc/nginx/sites-available/default’

add_header "Content-Security-Policy-Report-Only" "default-src 'self' https://fearby.com/; script-src 'self' 'unsafe-inline' 'unsafe-eval' https://adservice.google.com.au https://adservice.google.com https://cdn.onesignal.com https://cdn.syndication.twimg.com https://fearby-com.exactdn.com https://onesignal.com https://pagead2.googlesyndication.com https://platform.twitter.com https://s.ytimg.com https://www.googletagservices.com https://www.youtube.com; style-src 'self' 'unsafe-inline' https://fonts.googleapis.com https://onesignal.com https://platform.twitter.com https://ton.twimg.com; img-src 'self' data: https://a.impactradius-go.com https://abs.twimg.com https://fearby-com.exactdn.com https://healthchecks.io https://pagead2.googlesyndication.com https://pbs.twimg.com https://platform.twitter.com https://secure.gravatar.com https://syndication.twitter.com https://ton.twimg.com https://www.paypalobjects.com; font-src 'self' data: https://fearby-com.exactdn.com https://fonts.gstatic.com; connect-src 'self' https://onesignal.com https://yoast.com; object-src https://fearby.com/; frame-src 'self' https://en-au.wordpress.org https://fearby.com https://googleads.g.doubleclick.net https://onesignal.com https://platform.twitter.com https://syndication.twitter.com https://www.youtube.com; worker-src 'self'; form-action 'self' https://fearby.com https://syndication.twitter.com https://www.paypal.com; report-uri https://fearby.report-uri.com/r/d/csp/reportOnly";

I added the following to the file ‘/etc/nginx/sites-available/default‘ (inside the server node).

Any issues with the Content Security policy will be reported to my web browsers development console and to https://report-uri.com/.

My Chrome development console reports an issue with a graphic not loading from Namecheap.

Namecleap icon not loading

The event was also reported to the Report URI server.

Screenshot of reports at https://report-uri.com/account/reports/csp/

Don’t forget to check the reports often. When you have no more issues left you can make the Policy live by renaming the “Content-Security-Policy-Report-Only” header to “Content-Security-Policy”.

FYI: I had directive reports of ‘script-src-elem’ and it looks like they are new directives added to Chrome 75.

Don’t forget to visit the Report URI setup page and get a URL for where live reports get sent to.

Screenshot of https://report-uri.com/account/setup/

If you go to the Generate CSP page and import your website’s policy you can quickly add new exclusions to your policy

After a few months of testing and tweaking the policy, I can make it live (‘Content-Security-Policy’).

Lynis Enterprise

I have learned so much by using Lynis Enterprise from https://cisofy.com/

I am subscribed to issues notifications at https://github.com/CISOfy/lynis/issues/ and observe about 20 notifications a day in this GitHub community. Maybe one day I will contribute to this project?

Finally, Did the Bank reply?

Yes but it was not very informative.

Dear Simon, Thank you very much for the information and we have completely removed the reference that you have raised concern. We are extremely sorry and apology for the inconvenience caused due to this mistake. We are thankful for the information and support you have extended.

I tried to inquire how this happened and each time the answer was vague.

Thank you for your support. This was mistakenly used during the testing and we have warned the vendor as well. I like to request you to close the ticket for this as we have already removed this. We like to assure such things won’t happen in future.

It looks like the bank used my blog post to create their CSP.

Oh well at least I have a secured my servers.

Thanks for reading.

 

 

Version:

v1.1 – Changed the URL, Removed Ads and added a Lynis Enterprise Conclusion

v1.01 – Fixed the URL

v1.0 – Initial Version

Connecting to a server via SSH with Putty

by

This post aims to show how you can connect to a remote VM server using Telnet/SSH Secure shell with a free program called Putty on Windows. This not an advanced guide, I hope you find it useful.

2020 Update. I have stopped using Putty and WinSCP. I now use MobaXterm (a tabbed SSH client for Windows) as it is way faster than WinSCP and better than Putty. Read my review post of MobaXTerm here.

You will learn how to connect (via Windows) to a remote computer (Linux) over the Telnet protocol using SSH (Secure Shell). Once you login you can remotely edit web pages, learn to code, install programs or do just about anything.

Common Terms (Glossary)

  • Putty: Putty is a free program that allows you to connect to a server via Telnet. Putty can be downloaded from here.
  • Port: A port is a number given to a virtual lane on the internet (a port is similar to a frequency in radio waves but all ports share the same transport layer frequency on the internet). Older unencrypted webpages work on Port (lane 80), older mail worked on Port 25, encrypted web pages work on Port 443. Telnet (that SSH Secure Shell uses) used Port 22. Read about port numbers here.
  • SSH: SSH is a standard that allows you to securely connect to a server over the telnet protocol. Read more here.
  • Shell: Shell or Unix Shell is the name given to the interactive command line interface to Linux. Read more about the shell here.
  • Telnet: Telnet is a standard on the TCP/IP protocol that allows two-way communication between computers (all communicatin issent as characters and not graphics). Read more on telnet here and read about the TCP protocols here and here.
  • VM: VM stands for Virtual Machine and is a name given to a server you can buy (but it is owned by someone else). Read more here.

Read about other common glossary terms used on the Inetre here:
https://en.wikipedia.org/wiki/Glossary_of_Internet-related_terms

Background

If you want a webpage on the internet (or just a server to learn how to program) it’s easier to rent a VM for a few dollars a month and manage it yourself (with Telnet/SSH Secure Shell) than it is to buy a $5,000 server, place it in a data centre and pay for electricity and drive in every few days and update it. Remote management of VM servers via SSH/Secure Shell is the way for small to medium solutions.

  • A simple web hosting site may cost < $5 a month but is very limited.
  • A self-managed VM costs about $5 a month
  • A website service like Wix, Squarespace, Shopify or WordPress will cost about $30~99 a month.
  • A self-owned server will cost hundreds to thousands upfront.

There are pros and cons to all solutions above (e.g cost, security, scalability, performance, risk) but these are outside this post’s topic. I have deployed VMs on provides like AWS, Digital Ocean, Vultr and UpCloud for years. If you need to buy a VM you can use this link and get $25 free credit.

I used to use the OSX Operating System on Apple computers. I was used to using the VSSH software program to connect to servers deployed on UpCloud (using this method). With the demise of my old Apple Mac book (due to heat) I have moved back to using Windows (I am never using Apple hardware again until they solve the heat issues).

Also, I prefer to use Linux servers in the cloud (over say Windows) because I believe they are cheaper, faster and more secure.

Enough talking lets configure a connection.

Public and Private Keys?

Whenever you want to connect to a remote server via Telnet/SSH Secure Shell you will need a public and private key to encrypt communications between you and the remote server.

The public key is configured on your server (on Linux you add the public key to this file ~/.ssh/authorized_keys).

The private key is used by programs (usually on your local computer) to connect to the remote server.


How to create a Public and Private Key on Linux

I usually run this command on Ubuntu or Debian Linux to generate a public and private SSH key. 

sudo ssh-keygen -t rsa -b 4096

The key below was generated for this post and is not used online. Keys are like physical keys, people who have them and know where to use them can use them.

Output:

Generating public/private rsa key pair.
Enter file in which to save the key (/username/.ssh/id_rsa): ./server
Enter passphrase (empty for no passphrase): ********
Enter same passphrase again: ********
Your identification has been saved in ./server.
Your public key has been saved in ./server.pub.
The key fingerprint is:
SHA256:sxfcyn4oHQ1ugAdIEGwetd5YhxB8wsVFxANRaBUpJF4 [email protected]
The key's randomart image is:
+---[RSA 4096]----+
| .oB**[email protected]       |
|  +.==B.+        |
| o .o+o+..       |
|  .. +..o...     |
|    o ..Sooo.    |
|         ++o.    |
|        .o+o     |
|        .oo .    |
|         ...     |
+----[SHA256]-----+

The two files were created

server
server.pub
  • “server” is the private key
  • “server.pub“is the public key

Public/Private Key Contents

Public Key Contents (“server.pub”)

ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC7Xo9bOCXJ7gVjP8tKOxHVId3KTo5I0VRU/kSRK3+mGd5VbDbQABo3tdWzYhzkjODzRS9TeL2dcLAQNNQKshi9IW5IGDS1NocSCLFQId5BFr9s3E79fkWqcZkKmwocepXOOZ91EDKgIFxviOzZKe99sdxxMoZzi1nxgVyXl4TnaelyiQxeKYniVs1iqDfYWQCxkKsmYit8TvGtOwrhLvKNh9362/y5ebpXVdFlRuB83eF7k8RHNYCQyOJJVx4cw3TIsAN0GMOwjuaOZbp7rR1d6k7RZmaApNRTbaWOXy32UiBST5TV/jXF2UL/4IBnn+yvCrM0v79e/3omgjlVVKfWByFzMv/YlBKCAX3xxtJQ9RkzTqseKupXmmJU0rik6Xuz31N2oyw4M7yJofSUGVCN0pnpKEvnKxqolfD9egdQy2XDaNioY7cvOO1qRegCKE0sDh1m5MzJWMhbDs7macSMyd6+0O5qWc/ByHy0G/mVbd8kO4jIuEzEs4IFkPCToEZp7KfkY7KRkOhccLbQ4ApCesUfBtGGAN1f33NnXCHae3Cx46nSd23fvgDZUVnjI47tNJH5Z8FNVlW/fp5Rgeu/aPUephnDX2IBxwIKQOmSTDY+nxU4V+c93H1gSOJfvqYbVKIAXKyN9Yh6LC44ZvLrL4q0TC0QlH2+kxMLnuj7zw== [email protected]

Private Key Contents (“server”), always keep the private key safe and never publish it.

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: AES-128-CBC,D34670C40CE3778974BEF97094010597

b4oecyqLsWt9n+G12ldVNlaQxSKF1wSrlBPg6FGiHRauTCyreUwoI2dMOAkwnGmN
8fcy51fH7D3Kg0G9fWWNPd+oUDwZmrpB8Mv6Ndk4bLYZEbkNOFgvPwNre7edTBOD
JGZRdWqb+yrywgvz3iTXPNjNK5REU3u3JmD69jInFNo92j765QQKA4sFgEyD/8g+
zg8yefIQAhEsVELC5LXPPyuTfA+x0Q+040PqCJ+FCISJI1CeZjLwk7Fbe453Vj81
zaDsurl5X5gaRUlVjB2asr6etWdMLWcalX4Nbyj2A10L3J4ONjKq3Wc2muJ0Q6ES
oNqBaU2iHPlK8yK0TGj/ERfjaG1qdlhBcow0pSapRqGopXBuVBLVuyc2NHe5CCTk
Ezq+LZGsVYmiOIIY4QRJdEN/DVLFHRGK/xA9A7unm484zXIEO6wznE0DuCTtyZs0
luJ3bKLRcack3K1Dphq0LjSG4YxQlkHewa9k9AKpDPTqeeKKckySakiDCGPT6htk
VqaCKrApAt6GQ2hLVXZ0BFVN5A3WUJ5s+HpFvTUzHTNZcdsVS4PgxhuCtnSO/BdS
/G+ODc4aZJNYQD9QQfWUnxkgnQJCWJ+aBZtKF7eDPRYY7qD9jWxubDzrFplBkmAi
O+aX5N8dpU3lEty4INjyh5LpgZW3swjUhEKWi/c1k+Qd1gCWzYzwAq2BfpWcF8Z+
c+y9lQUKbq2yDlxReCIsfb/hda5k1HjgaUlhKbjWIITSlGqf/NE9i+vj0rQEMQXQ
mxBoilfLUPd5A1ttG5XvqC2ex5HBmjzCazZ13Z/2c/PkwicHBmrf5bKYHZp49niV
44n8tZRamCUv6HaJUaKR22MigOG/qGppGPodGeLNj1DFLYAEQ78SYcVhEqIICBo1
t1yaIemUq8MWXSZz1K3cP4FEXQcEziQxFLU/0DCE0P0mIU3MExUmjB/nVE8vxb5l
p3ej3yrRGe+P2neco2gttgaTEi6l/S+0TIiZNstnVPG48BPW71mwVg9XR1d+avO7
OpXt0UgocX0xp7zBgK2up8Ai6v66WwjoNgyvFe02aK4/+fSC+aJ5D6N7JVNxd/bn
Py4W8oLKnrE1PKtIfBw/aE+rgudaMIyuxCaLllRKyDxVPPiJFp2iFcH/Y+k+0vDa
xE9Jpdd0zOWkZyebAxrS8zAUUNNaTQ+rWkj/zORjE4ptHpdwdazzHoQwIs+1kjsv
e/+JEmoskH7XozLnxClVhhWMXWfgQsPWBqPnGzieW0tv9SeIAU/BLJCHJRhBMAT1
ugBtcda1VMlAPVroYtVyUdCxkYZqGfIDbKqtOvvuBgUIUe/HnC3ExQQycC9F05BH
RJibaM/11MLTcZSO7KOK65Dg2v3VBhe6rfDl4tTR0yOySPXCacb9aMt2pMPTEe0/
wU49wCefchfD2bsR3kXPpUqm+HbkHORpIwsMZfQO/8dooXYdiYUdzV9roXG6OGVQ
SsV/xR2lE3XrR71TBegfRnQirI8tj4psSor+yCj3qV936Oh31D96Z6P4glshibsG
ffWAO/TSdu5ZV+UVahh6bTozs+g+odUu/S48TeI1fk7lPlqwZdjoSHXUI2v1FAQ2
jSSywuZQxHlGhg6OeI052cxx3zcVyVVLFHhIrfvufNc3c3+KYhtyiSzBNYN1BrJi
xNXwlDS1jYWgRHkf9zbNBU0MLTYHjZZvO9Jpl/UhKKBdIvJFwmGmXS2lgU6slunJ
Ojp4tY1tbI520KOskV/OoqEfmhXh5fTlI3onzoK1aLqxk1d0d65ONcxqVbAG79RN
b0Q5PgewSOgFlcZ7tEIZKAWsWVhjlFTSGRujdZVM1vZB9fCJesemai7HU0e4J+Do
tqvss8I2n6TPxlTYFzQ4w12pIiOzx/8cFLX78NLN8wQFElhhczeuW5HDAnmPxYhQ
eLY0HgDCFSvVAvGXo0j1gcBUcOr/LzZSsJhxsB7FKyrUjlmD/7Y45WoKJj41bKL+
y4+iDhXyLBiqVClRijsguwiCkmPFiR7Bng2pglS0oIWPWu1UbTJWVJPfuUTOBC+M
4/2fBtgFjUz8iUISs9ncEKkERlxodBIu+ekgLJZAigSMvUKfGE1YB1AA9x96VLjd
VJSjjWvnhMEoSwNzlNQ9+dhoD5Cg9zicgIIKnHnovYGOu8g9ZWfvhJFrKZgkfLRv
r2KgkWiHWpf0swiyGUOlGJDe39nMMkoxib7XE/J3VI3na1ZUOIf8kl9kdHXJ0R3C
2IjdbfiFHEDOrakp5oeVf8BbLK7RB8OlxgJAS47Byh8j97U7f13A5ZYlK3bkZ7E4
h7mCJQozgWP81ut0d9WUlcKp5M8yg2ctZ7h4oeG4Js4ceHqd19Z4P+1xWKwXcdmV
+uhiTftevTu3/UhYQVV4ck98C9pursJJYL5hTnIIpTSWIR+jSahhtzUy/upjugPp
cKi6eGlOkcHdKNRtiu7/IZqni85fC8PAwPZ93SICdiq6BpGaGWFh046weIJuflSK
Pd76+M70YRd+pkaRjJyFJ3hLyg7W5mlOb1+yBIlXKzpbch9B5E4dRHCcOsg4+v/9
exRgAnvUIhR/GpSySDDwgKHg8rAyjjoGeZFH3TJIemAAimyaR608a9tCn7SxVobs
UQlZ9WwC0dQIEv7mSvSige3imbybPtCoBHJAqsJqKCFJEDWbIF5l2VYZcfJUYaEI
oZAJHYGnZm33yQ6eSOusXJ2SnnGZ+ZsGO4bDVSwN20FkSt11gN8Wjrki9CxeVQp7
dWbKX1r/lZw74yUB4cYN23hgLJsdqvM7THzwlBkVtgV74RGY0qv59ecBUSQedlSK
dkOnkmoCiGRSNyf+ebijQaygnfK0ArG5wiRF/RQWiPFj7S6DHRxIOrXqcmvhJ7Ly
NApn9pPYyoZEAbk82MAXkapZ5+YLIKLjdNsYuKq5xVty+mc+FfxLWmZGX+QQinra
Z9DfY9KQw4rxJ/ju4ILnDrygm/QBsNFXBojOuzOIULt7c26s3d/47T+IXA4SIX4v
cPqYa6S3PU/Yoe5/Ya3tFxXmBXgEgVLZuujMs7dyCOAqLEyBEHYqIclp+TElWQLR
V660fczVXeedfd2tNBy1IBj1vhGa9j5mZLbFwTczykwCFfihLIrxSEc1MQA4CaSX
-----END RSA PRIVATE KEY-----

The Public and Private keys is used to encrypt all Telnet/SSH connections and traffic to your server. Keep these key’s private.

fyi: Putty can create SSH Keys too

If you do not have a Linux computer or Linux server to generate keys the Putty generator can create keys too.

Puttygen generating a key based on the randomness of mouse movements.

I did not know Putty can create keys.

Do save the public and private key(s) that were generated in Puttygen (tip: PPK files are what we are after along with the public key later in this post).

Public keys are added to your server when you deploy them. On Linux, you can add new Pulic keys after deployment by adding them to this file “~/.ssh/authorized_keys” to allow people to log in.

Puttygen does format the keys differently than how Ubuntu generates them. Read more here. I’ll keep generating keys in Linux over Puttygen.

Output of the public and PPK files from Puttygen

Putty SSH Client on Windows

Putty is a free windows program that you can use to connect to serves via SSH. Download and install the Putty program.

Open Putty

Putty Icon

Default Putty User Interface.

Screenshot of the Putty Program

To create a connection add an exiting IP address (server name) and SSH port (22) to Putty.

Screenshot of an IP and port entered into putty

In Putty (note the tree view to the left of the image), You can set the auto login name to use to log into the remote server under the Connection the Data in the tree view item

Screenshot showing the SSH usename being added to putty under Connection then Data menu,

You can also set the username under the Connection then Rlogin section of Putty.

Set the usernmae undser rlogin area of putty

OK, lets add the private SSH Key to Putty.

Putty Screehshot showing no support for standard SSH keys (only PPK files)

It looks like Putty only supports PPK private key files not ones generated by Linux. I used to be able to use the private key in the VSSH program on OSX and add the private key to connect to the server over SSH. Putty does not allow you to use Linux generated Private keys directly.

Convert your (Linux generated) private key to (Putty) PPK format with Puttygen

Putty comes with a Key Generator/Converter, you can open your existing RSA private key and convert it (or generate a new one).

TIP: If you generate a key in Puttygen don;t forget to ad’d it to your authorized host file in your remote server.

Open Puttygen

Puttygen icon

Click Conversions than Import Key and choose the private key you generated in Linux

Screenshot showing import RSA key to convert

The private key will be opened

Screenshot of imported RSA key

You can then save the private key as a PPK file.

Save the private key as a PPK file
“server.ppk” Key contents (sample key) 
PuTTY-User-Key-File-2: ssh-rsa
Encryption: aes256-cbc
Comment: imported-openssh-key
Public-Lines: 12
AAAAB3NzaC1yc2EAAAADAQABAAACAQC7Xo9bOCXJ7gVjP8tKOxHVId3KTo5I0VRU
/kSRK3+mGd5VbDbQABo3tdWzYhzkjODzRS9TeL2dcLAQNNQKshi9IW5IGDS1NocS
CLFQId5BFr9s3E79fkWqcZkKmwocepXOOZ91EDKgIFxviOzZKe99sdxxMoZzi1nx
gVyXl4TnaelyiQxeKYniVs1iqDfYWQCxkKsmYit8TvGtOwrhLvKNh9362/y5ebpX
VdFlRuB83eF7k8RHNYCQyOJJVx4cwnTIsAN0GMOwjuaOZbp7rR1d6k7RZmaApNRT
baWOXy32UiBST5TV/jXF2UL/4IBnn+yvCrM0v79e/3omgjlVVKfWByFzMv/YlBKC
AX3xxtJQ9RkzTqseKupXmmJU0rik6Xuz31N2oyw4M7yJofSUGVCN0pnpKEvnKxqo
lfD9egdQy2XDaNioY7cvOO1qRegCKE0sDh1m5MzJWMhbDs7macSMyd6+0O5qWc/B
yHy0G/mVbd8kO4jIuEzEs4IFkPCToEZp7KfkY7KRkOhccLbQ4ApCesUfBtGGAN1f
33NnXCHae3Cx46nSd23fvgDZUVnjI47tNJH5Z8FNVlW/fp5Rgeu/aPUephnDX2IB
xwIKQOmSTDY+nxU4V+c93H1gSOJfvqYbVKIAXKyN9Yh6LC44ZvLrL4q0TC0QlH2+
kxMLnuj7zw==
Private-Lines: 28
DkpbM78GgGBSgfs9MsmZwDJj6HFXdoe+fCP1rLnwbE99mvU6Fbs23hXd+FsVdQbb
VR5tKTocV7tEwGjtLCHSTSF6gap0l4ww0Ecuvr/Dra2CJ2BsntyssBrWnlUT7OlA
M9zKQAzywAy4AHkph0YvH4l7BcJ5V1pUltm2JDTU6+iFqXDsstUUEDcQ4u0EalWU
EEsW+quNSwO0HBHvWY6N7tbiuEN9L+cFYIdsJEDfqM4hNi+7Ym+SQq5FOPyA6gXa
vhujsjPQAWI3TFxh7EIvsPDMCXxWHL6qaDvOMmPPTZDbEvm4nQ5Kax9jWacPILn7
ezc7ZAiZdDiFbkF3TLyuHx71mjChZgLoZLWYfXR3MBEEYnkNO/7oSMRUwDzEyWKW
ZgqdUtGg0cR+qWvaxQTDQsN/DjB7jGgnlreF92S8xSsbk5GgpZnTQ1V0cm3oecB+
+JP90K4Fi979gPWnwTfg6ZvmLUiVz3uBbvegkT9CVZhhZXSKq53H+SjTZfKBPrM9
NHGLkYr1WjToGR39LMrh4X3KChGewMFyuxtpkEQV60eCnHZBHgTco2A0yriRprOP
Ks4qJXOtZnsMYMesUDX9W5wLc4HcRvRh2UBPw/8bPz6mNrBk8j5SPIwBrPMIBejd
4IPoYezaEFKPg2bP7dn+Nftz5CGagcV2g+zhE615dsWzX1P7yu/1dTmz9LXaMmN6
d+zJE8TtjeaoW5NE1H3Flj9rknzJW7xQfokhS5hMkOg6J0AA6Pk13tupu8WHMkVB
x7nVu876f8tIbT8GzXCGgSl+zS7IJO3pt9T9QHIYa+T3oTIUqfBfK1WffUZwHMRn
Xn/VKUtIIIPiVfCtQuxSrTiJzQcoJ/yvfv62YAGv2LsDlBoHfXRdf6h3TCCCOVxT
WE45sbj3gJ1Cgjt1SEd/8A3hkstn2U2NKBI9gkB9H5BbDJoAXq6/4CkwaQvSEzs7
LK5btRlWop+7gqkyMPpgxv9li9IEDJ999ufMqxkFgOBkmkR5Si71elXRnwiKrjfU
Ce14iy7Dd7lb7IU9OEBjWFZlSigVEnc8klhGHDuxnojiW1ld7pUDIkAAbdTMOFON
abcpfNwcg5Y3l+1KwIQHuewAUuA9472jV4V9EAn7pJ7wgmYHbzMzg9Z9dM8h/3UI
axBzAW+cJM80gN+nZMbmDC9FkXV16GSuqC2iQUVGb2TIheAS7oCR+JFZFQNv0ytF
rGQ9K1wIGbMI4oDPcAid7DzrEXVl3d2x8MtwF/WzfHehVJD1h1uNwezLf1gBKyas
9GBfDOYwd8zgaL2H99GYD1Ba7TePJY81mx7m10eYdwDj1vCpboKE3cE6AyL7ki+4
Ix7GSzQs9NBckF9+8eVXe2T4Cc450hIoN0BWcxVUUdGCA1skZ1PczPs1z/ae4lxd
l5WmPy8Gyh7cnZpyqzvwAPSFDadkNP60eekfkRHyo4QyLhj7QZtO0kOgWhT3CHma
FjZ5jJu59U/4gc0TpQ8ra3vgKQKudloExsg027+34nR98dN+zzUj4S2C/J34W98C
DEEu/SO7nfW/a2UARXBKWCbS+3j24zHc9dbgX2tZoAoInUvRGiSOsLVsMhDiBoyb
wWoNxrKPR3Fi5zZ+GfDUgUGpZoW/b54KnFouIHBYbI41Gkh4vj6lxOGh/sb3SPHd
Wg6EN/0z/mer3bG0a2/ZHKYA5KGWRXWYvYLz4Je8fb/egBrSU6BztwSNeilzA9lI
J4BO7pzXECnWYutB14UxHw==
Private-MAC: 12298fa865ac574da81898252e83b812200cba59

Now the PPK key can be added to Putty for any server connection that uses the public key. Use the right key for the right server though.

Add the private key to a Putty server by clicking Connection, SSH, AUTH section and browing to the PPK file.

Screenshot showing the PPK key file added to Putty

Now we need to save the connection, click back on the Session note at the top of the treeview, type a server name and click Save

Save Putty connection.

Connecting to your sever via Telnet/SSH wiht Putty.

Once you have added a server name, port, usernames and private key to Putty you can double click the server list item to connect to your server.

You will see a message about accepting the public key from the server. Click Yes. This fingerprint will be the same fingerprint that was shown when you generated the keys (if not maybe someone is hacking in the middle of your local computer and server)

Putty messgae box asking to to remember the public key

Hopefully, you will now have full access to your server with the account you logged in with.

Screenshot of an Ubuntu screen after login

Happy Coding.

Alternatives to self-managed VM’s

I will always run self-managed server (and configure it myself) as its the most economical way to build a fast and secure server in my humble opinion.

I have blogged about alternatives but these solutions always sacrifice something and costs are usually higher and performance can be slower.

I am also lucky enough I can do this as a hobby and its not my day job. when you self manage a VM you will have endless tasks or securing your server and tweaking but its fun.

More Reading

Read some useful Linux commands here and read my past guides here. If you want to buy a domain name click here.

If you are bored and want to learn more about SSH Secure shell read this.

Related Blog Posts

Version: 1.1 Added MobaXterm link

Updating NGINX to the development branch to get more frequent updates and features over the stable branch

by

Updating NGINX to the development branch (on Ubuntu) to get more frequent updates and features over the stable branch

Aside

I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. View all recent posts here https://fearby.com/all/

Now on with the post

Warning

Backup your Nginx and Server before making any changes. The Nginx development branch is quite stable but anything can happen. If your site is mission-critical then stay on the stable branch.

Nginx Branches

By default, you will most likely get the stable branch of Nginx when instaling and updating Nginx.  I have been running the stable version for the last few years but was made aware of a DDoS vulnerability in Nginx.

Here is a good write-up on development merges into the stable branch.

Nginx Updates

I was aware recently of a DDoS bug affecting Nginx and the recommendation was to update ot Nginx 1.15.6 development branch (or 1.14.1 stable branch).

A few days ago no 1.14.1 update was available but a 1.15.6 was, should I switch to the development branch to get updates earlier?

Recent Nginx Changes

Here are the recent changes to Nginx: http://nginx.org/en/CHANGES

Changes with nginx 1.15.6                                        06 Nov 2018

    *) Security: when using HTTP/2 a client might cause excessive memory
       consumption (CVE-2018-16843) and CPU usage (CVE-2018-16844).

    *) Security: processing of a specially crafted mp4 file with the
       ngx_http_mp4_module might result in worker process memory disclosure
       (CVE-2018-16845).

    *) Feature: the "proxy_socket_keepalive", "fastcgi_socket_keepalive",
       "grpc_socket_keepalive", "memcached_socket_keepalive",
       "scgi_socket_keepalive", and "uwsgi_socket_keepalive" directives.

    *) Bugfix: if nginx was built with OpenSSL 1.1.0 and used with OpenSSL
       1.1.1, the TLS 1.3 protocol was always enabled.

    *) Bugfix: working with gRPC backends might result in excessive memory
       consumption.


Changes with nginx 1.15.5                                        02 Oct 2018

    *) Bugfix: a segmentation fault might occur in a worker process when
       using OpenSSL 1.1.0h or newer; the bug had appeared in 1.15.4.

    *) Bugfix: of minor potential bugs.


Changes with nginx 1.15.4                                        25 Sep 2018

    *) Feature: now the "ssl_early_data" directive can be used with OpenSSL.

    *) Bugfix: in the ngx_http_uwsgi_module.
       Thanks to Chris Caputo.

    *) Bugfix: connections with some gRPC backends might not be cached when
       using the "keepalive" directive.

    *) Bugfix: a socket leak might occur when using the "error_page"
       directive to redirect early request processing errors, notably errors
       with code 400.

    *) Bugfix: the "return" directive did not change the response code when
       returning errors if the request was redirected by the "error_page"
       directive.

    *) Bugfix: standard error pages and responses of the
       ngx_http_autoindex_module module used the "bgcolor" attribute, and
       might be displayed incorrectly when using custom color settings in
       browsers.
       Thanks to Nova DasSarma.

    *) Change: the logging level of the "no suitable key share" and "no
       suitable signature algorithm" SSL errors has been lowered from "crit"
       to "info".


Changes with nginx 1.15.3                                        28 Aug 2018

    *) Feature: now TLSv1.3 can be used with BoringSSL.

    *) Feature: the "ssl_early_data" directive, currently available with
       BoringSSL.

    *) Feature: the "keepalive_timeout" and "keepalive_requests" directives
       in the "upstream" block.

    *) Bugfix: the ngx_http_dav_module did not truncate destination file
       when copying a file over an existing one with the COPY method.

    *) Bugfix: the ngx_http_dav_module used zero access rights on the
       destination file and did not preserve file modification time when
       moving a file between different file systems with the MOVE method.

    *) Bugfix: the ngx_http_dav_module used default access rights when
       copying a file with the COPY method.

    *) Workaround: some clients might not work when using HTTP/2; the bug
       had appeared in 1.13.5.

    *) Bugfix: nginx could not be built with LibreSSL 2.8.0.


Changes with nginx 1.15.2                                        24 Jul 2018

    *) Feature: the $ssl_preread_protocol variable in the
       ngx_stream_ssl_preread_module.

    *) Feature: now when using the "reset_timedout_connection" directive
       nginx will reset connections being closed with the 444 code.

    *) Change: a logging level of the "http request", "https proxy request",
       "unsupported protocol", and "version too low" SSL errors has been
       lowered from "crit" to "info".

    *) Bugfix: DNS requests were not resent if initial sending of a request
       failed.

    *) Bugfix: the "reuseport" parameter of the "listen" directive was
       ignored if the number of worker processes was specified after the
       "listen" directive.

    *) Bugfix: when using OpenSSL 1.1.0 or newer it was not possible to
       switch off "ssl_prefer_server_ciphers" in a virtual server if it was
       switched on in the default server.

    *) Bugfix: SSL session reuse with upstream servers did not work with the
       TLS 1.3 protocol.


Changes with nginx 1.15.1                                        03 Jul 2018

    *) Feature: the "random" directive inside the "upstream" block.

    *) Feature: improved performance when using the "hash" and "ip_hash"
       directives with the "zone" directive.

    *) Feature: the "reuseport" parameter of the "listen" directive now uses
       SO_REUSEPORT_LB on FreeBSD 12.

    *) Bugfix: HTTP/2 server push did not work if SSL was terminated by a
       proxy server in front of nginx.

    *) Bugfix: the "tcp_nopush" directive was always used on backend
       connections.

    *) Bugfix: sending a disk-buffered request body to a gRPC backend might
       fail.


Changes with nginx 1.15.0                                        05 Jun 2018

    *) Change: the "ssl" directive is deprecated; the "ssl" parameter of the
       "listen" directive should be used instead.

    *) Change: now nginx detects missing SSL certificates during
       configuration testing when using the "ssl" parameter of the "listen"
       directive.

    *) Feature: now the stream module can handle multiple incoming UDP
       datagrams from a client within a single session.

    *) Bugfix: it was possible to specify an incorrect response code in the
       "proxy_cache_valid" directive.

    *) Bugfix: nginx could not be built by gcc 8.1.

    *) Bugfix: logging to syslog stopped on local IP address changes.

    *) Bugfix: nginx could not be built by clang with CUDA SDK installed;
       the bug had appeared in 1.13.8.

    *) Bugfix: "getsockopt(TCP_FASTOPEN) ... failed" messages might appear
       in logs during binary upgrade when using unix domain listen sockets
       on FreeBSD.

    *) Bugfix: nginx could not be built on Fedora 28 Linux.

    *) Bugfix: request processing rate might exceed configured rate when
       using the "limit_req" directive.

    *) Bugfix: in handling of client addresses when using unix domain listen
       sockets to work with datagrams on Linux.

    *) Bugfix: in memory allocation error handling.

Development branch changes are made every few weeks and stable branch changes are made less often.

Updating Nginx

Normally you update Nginx bu running an update and upgrade

Restart Nginx for good measure

Checking NGINX Version

Changing your repository to the development branch

I changed ot the development branch by running

Update and upgrade Nginx

Restart Nginx for good measure

Checking NGINX Version

Removing the stable Nginx repository

Run this command to remove the stable branch of Nginx

Check to see if the development branch is listed

Good luck and I hope this guide helps someone

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial post

How to install PHP 7.2.latest on Ubuntu 16.04

by

How to install PHP 7.2.latest on Ubuntu 16.04/ Ubuntu 18.04/Debian etc/

I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. PHP is my programming language of choice.

PHP has a support page that declares the support date ranges and support types: http://php.net/supported-versions.php

PHP 7.0 going EOL

A version of PHP is either actively supported, security fix supported or end of life. Read this post to check WordPress for PHP compatibility.

From time to time vulnerabilities come up that require PHP updates to be applied.

Source Link here

Advertisement:



I have guides on setting up PHP 7 here on Digital Ocean, here on AWS and here on Vultr. I have tried upgrading to PHP 7.1 in the past with no luck (I forgot to change something and rolled back to 7.0).

FYI: I have a guide on setting up PHP child workers so the output from some commands below may be different than yours. Here are the steps I performed to install PHP 7.2 alongside 7.0 then switch. to 7.2.

Backup your system

Do perform a Snapshot or Backup before proceeding. Nothing beats a quick restore if things fail.

Note: Use this information at your own risk.

Updating php 7.2.12 to 7.2.12

Update your Ubuntu systems

Updating from an older php (e.g 5.x, 7.1, 7.1 to say 7.2.12)

Backup PHP

Install Helper

This software provides an abstraction of the used apt repositories. It allows you to easily manage your distribution and independent software vendor software sources. More Info

Add the main PHP repo (more information)

Update the package lists

“In a nutshell, apt-get update doesn’t actually install new versions of the software. Instead, it updates the package lists for upgrades for packages that need upgrading, as well as new packages that have just come to the repositories.” from here

List Installed Packages (optional)

Install PHP 7.2

Install common PHP modules

Install PHP FPM

Update all packages (may be needed to update from php 7.2.4 to 7.2.5)

Edit your NGINX sites-available config

Edit your NGINX sites-enabled config

I edited these lines

Edit your PHP config (and make desired changes)

Edit your PHP pool config file (as required). See this guide here.

e.g.

> cgi.fix_pathinfo=0
> max_input_vars = 1000
> memory_limit = 1024M
> max_file_uploads = 8M
> post_max_size = 8M

Make sure you set: listen = /run/php/php7.2-fpm.sock

Set PHP 7.2 as the default PHP

Check your PHP version

Reload PHP

Reload NGINX

Check the status of your PHP (and child workers)

Check your website.

Troubleshooting

Guides that helped me.

https://thishosting.rocks/install-php-on-ubuntu/

https://websiteforstudents.com/wordpress-supports-php-7-2-heres-how-to-install-with-nginx-and-mariadb-support/

Check your log files

Debug FPM Service

Remove PHP 7.0

PHP 7.0 Removed 🙂

Remove other unused packages

At the time of writing (November the 18th 2018) PHP 7.2.12 is the latest version of PHP and PHP 7.3 will be out at the end of the year.

Good luck and I hope this guide helps someone

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.4 Updated the post to mention PHP 7.0 EOL

v1.3 Updated to add PHP 7.2.12 information

v1.2 PHP 7.2.9 and PHP 7.2 updates

v1.1 Remove PHP 7.0 steps

v1.0 Initial post

Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx

by

This is a quick post that shows how I set up the “Feature-Policy”, “Referrer-Policy” and “Content Security Policy” headers in Nginx to tighter security and privacy.

Aside

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name here

Domain names for just 88 cents!

Now on with the post.

Add a Feature Policy Header

Upon visiting https://securityheaders.com/ I found references to a Feature-Policy header (WC3 internet standard) that allows you to define what browse features you webpage can use along with other headers.

Google mentions the Feature-Policy header here.

Browser features that we can enable or block with feature-policy headers.

  • geolocation
  • midi
  • notifications
  • push
  • sync-xhr
  • microphone
  • camera
  • magnetometer
  • gyroscope
  • speaker
  • vibrate
  • fullscreen
  • payment

Feature Policy Values

  • * = The feature is allowed in documents in top-level browsing contexts by default, and when allowed, is allowed by default to documents in nested browsing contexts.
  • self = The feature is allowed in documents in top-level browsing contexts by default, and when allowed, is allowed by default to same-origin domain documents in nested browsing contexts, but is disallowed by default in cross-origin documents in nested browsing contexts.
  • none = The feature is disallowed in documents in top-level browsing contexts by default and is also disallowed by default to documents in nested browsing contexts.

My Final Feature Policy Header

I added this header to Nginx

This essentially disables all browser features when visitors access my site

add_header Feature-Policy "geolocation none;midi none;notifications none;push none;sync-xhr none;microphone none;camera none;magnetometer none;gyroscope none;speaker self;vibrate none;fullscreen self;payment none;";

I reloaded Nginx config and restart Nginx

Feature-Policy Results

I verified my feature-policy header with https://securityheaders.com/

Feature Policy score from https://securityheaders.com/?q=fearby.com&followRedirects=on

Nice, Feature -Policy is now enabled.

Now I need to enable the following headers

Add a Referrer-Policy Header

I added this header configuration in Nginx to prevent referrers being leaked over insecure protocols.

Referrer-Policy Results

Again, I verified my referrer policy header with https://securityheaders.com/

Referrer Policy resu;ts from https://securityheaders.com/?q=fearby.com&followRedirects=on

Done, now I just need to setup Content Security Policy.

Add a Content Security Policy header

I read my old guide on Beyond SSL with Content Security Policy, Public Key Pinning etc before setting up a Content Security policy again (I had disabled it a while ago). Setting a fully working CSP is very complex and if you don’t want to review CSP errors and modify the CSP over time this may not be for you.

Read more about Content Security Policy here: https://content-security-policy.com/

I added my old CSP to Nginx

I then imported the CSP into https://report-uri.com/home/generate and enabled more recent CSP values.

I restarted Nginx

I loaded the Google Developer Console to see any CSP errors when loading my site.

CPS Errors

I enabled reporting of CSP errors to https://fearby.report-uri.com/r/d/csp/enforce

Fyi: Content Security Policy OWASP Cheat Sheet.

You can validate CSP with https://cspvalidator.org

Now I won’t have to check my Chrome Developer Console and visitors to my site will report errors. I can see my site’s visitors CSP errors at https://report-uri.com/

report-cri.com Report

Content Security Policy Results

I reviewed the reported errors and made some more CSP changes. I will continue to lock down my CSP and make more changes before making this CSP policy live.

I verified my header with https://securityheaders.com/

Security Headers report from https://securityheaders.com/?q=https%3A%2F%2Ffearby.com&followRedirects=on

Testing Policies

TIP: Use the header name of “Content-Security-Policy-Report-Only” instead of “Content-Security-Policy” to report errors before making CSP changes live.

I did not want to go live too soon, I had issues with some WordPress plugins not working in the WordPress admins screens.

Reviewing Errors

Do check your reported errors and update your CSP often, I had a post with a load of Twitter-related errors.

Do check report-uri errors.

I hope this guide helps someone.

Please consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V1.3 https://cspvalidator.org

v1.2 OWASP Cheat Sheet.

v1.1 added info on WordPress errors.

v1.0 Initial Post

Set up a whitelisted IP on an UpCloud VM and WordPress using a VPN to get a static IP address

by

This is how I set up a whitelisted IP on an UpCloud VM and WordPress using a VPN to get a static IP address

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Before you begin

Take a backup of WordPress files + database and take a snapshot of your VM (see my UpCloud VM guide here).

Having a ready backup IS a good idea.

Screenshot of https://my.upcloud.com/server/details/

Why Whitelist

Whitelisting is not bulletproof but it is an important link in the security chain. Security is only as good or bad as the strength of your weakest link.

Using updated software, applying patches, using HTTPS, using a reliable host in a reliable location, using good passwords are equally important as IP filtering. Whitelisting IP’s goes a long way to ensuring you have least access privileges on connections.

Remember to scan your site with OWASP Zap, Qualys and Kali Linux too.

What IP’s are you going to Whitelist?

Q1) Does your ISP offer a static IP address (or a dynamic IP)?

My ISP does NOT provide a static IP by default (I can pay $20 a month for one (that’s too expensive)).

You can check your public IP by loading http://icanhazip.com/ (this will return your public IPV4 address).

Load https://ipv6.icanhazip.com/ to view your IPV6 IP (if you have one)

Q2) Do you need to whitelist IP addresses while on the go (Mobile)? If so I would recommend you whitelist a VPN’s IP or IP range.

Recently I had Apache web server auto-install and knock out my NGINX web server and I needed to login on a mobile device to investigate,  Luckily I whitelist my VPN’s IP and logged in from my mobile device and resolved the issue.

Use a VPN to get a static IP

If you don’t have a static IP or you want to connect to your site on the go (Mobile) you can set up a VPN and use their static IP

I was using http://cyberghostvpn.com/ to have a static IP but a server failure in Sydney caused my defined whitelisted IP to disappear so I change to https://protonvpn.com/ (as Cybergost were unable to provide known IP’s of VPN servers).

TIP: Don’t just whitelist one server, whitelist a few as you never know when a server will go down.

Here is a screenshot of the 1st VPN I tried (Cyberghost), Cyberghost VPN is connected to a specified server (Dallas).

Cyberghost VN screenshot connected to Dallas

I switched to ProtonVPN.

Here is a screenshot of ProtonVPN connected to a Switzerland server. Read more about Proton VPN here.

Screnshot of Protonvpn

I set Proton VPN to auto-start and connect to my desired server

Screenshot of Proton VNS startup settings

Proton VPN offered me a 7-day PLUS trial (All Countries, 5 devices, highest speed, secure core etc) after I started using the free version (3 countries, 1 device, speed low). I assume everyone gets the same PLUS trail offer.

You can view Proton plans and pricing here.

Ok, now that we know how to get a static IP, let’s configure some firewalls.

Network Firewall at UpCloud

I use the awesome UpCloud to hold my domains (read more about UpCloud performance here). You can log in to your UpCloud Dashboard and load the server list, click your server and then click Firewall and define firewalls.

Firewall: Open IPv4/IPv6 ports for:

  1. ICMP
  2. 53 (DNS)
  3. 80 (HTTP)
  4. 443 (HTTPS)

Only allow access to port 22 from whitelisted IP’s (or IP ranges)

Screenshot of UpCloud firewall screen at https://my.upcloud.com/server/details/

I like to set separate firewall rules for IPV4 and IPV6, for TCP or UDP and I limit rules to certain IP range and port.

Ubuntu Firewall

I also like to run a ufw firewall (more information on ufw) on my Ubuntu server (read this guide on securing Ubuntu in the cloud and running a Lynis audit).

Manually setup firewall rules in ufw.

Don’t forget t restart your firewall

Run a  local nmap scan to find open ports

Don’t be concerned if you see open ports from a local nmap scan (e.g port 22 or 3306), these are locally open.  We need to scan externally to see if these ports are opened.

Scan your site with an external nmap tool like pen-test-tools or here.

Screenshot of a public nmap scanYou should not have non-web based service ports freely open externally (web-based ports e.g 80 and 443 are ok)

Port 22 access should be whitelisted to select IP’s only. You should not have any database ports open externally.

Whitelisting WordPress Access

Download WordFence plugin for WordPress from https://www.wordfence.com/

Read more on downloading WordPress plugins from the command line here. Read my past Wordfence post here.

Once Wordfence is installed open the WordFence All Options screen  (/wp-admin/admin.php?page=WordfenceOptions).

Now you can add your static IP (or IP ranges) to the WordFence whitelist.

Picture of WordFence whitelist

Setup auto block for any non whitelisted Itryingng to login to /wp-login.php

I permanently ban any IP accessing my login page (there are many).

What to do with rejected IP connections?

Wordfence will block connections to WordPress. I’d suggest you setup fail2ban to block other unwanted connections at network level too.

Conclusion

You should now have a VM that will allow port 22 access by whitelisted IP’s and a WordPress that only allows logins from whitelisted IP’s.

Cons

  • If you forget to start your VPN you can’t log in to your VM via port 22 or log in to WordPress (excellent, this is by design).

Pros

  • Secure (need I say more)

I hope this guide helps someone.

Please consider using my UpCloud referral code and get $25 UpCloud VM credit for free when you signup to create a new VM.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 Added Proton plans link

v1.1 Added auto block WordFence option

v1.0 Initial Post

Adding two sub domains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04

by

Here is how I added two subdomains (one pointing to a new UpCloud VM and the other pointing to an NGINX subsite) on Ubuntu 18.04

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

UpCloud performance is great.

Upcloud Site Speed in GTMetrix

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Goal(s)

Setup 2x subdomains on https://fearby.com

– Sub Domain #1: https://test.fearby.com (pointing to a dedicated UpCloud VM in Singapore for testing).

– Sub Domain #2: https://audit.fearby.com (pointing to a sub-website on the NGINX/VM that runs https://fearby.com )

Let’s set up the first Sub Domain (dedicated VM) and SSL

Backup

Do back up your server first.

VM

I created a second server ($5 month or $0.07c hour 1,024MB Memory, 25GB Disk, 1024 GB Month Data Transfer) at UpCloud. If you don’t already have an account at UpCloud use this link to signup and get $25 free credit ( https://www.upcloud.com/register/?promo=D84793 ). Read my blog post on why UpCloud is awesome and how I moved my domain to UpCloud.

Once I spun up a second server I obtained the IPv4 and IPv6 IP addresses of the new “test” VM from the UpCloud dashboard.

DNS

These DNS records were already in place with my DNS provider (Cloudflare).

I added these DNS records for the subdomains.

I added a new A NAME record for the new shared NGINX subdomain (for https://audit.fearby.com), this subdomain will be a sub-website that is running off the same server as https://fearby.com

I added another set of records for the new dedicated VM  subdomain (for https://test.fearby.com)

I waited for DNS to replicate around the globe by watching https://www.whatsmydns.net/

Setup a Firewall

On the new dedicated https://test.fearby.com VM, I installed the ufw firewall.

I configured the firewall to allow minimum ports (and added whitelisted IP for port 22 and added UpCloud DNS servers). I will lock this down some more later.

TIP: If your ISP does not offer a dedicated IP try a VPN. I use https://cyberghostvpn.com on OSX and Android.

Firewall rules.

I enabled the firewall.

Install NGINX (on https://test.fearby.com)

On the new dedicated https://test.fearby.com VM I…

Created a new www root

Set permissions

Installed NGINX

I created a placeholder webpage

Configured the root value in /etc/nginx/sites-available/default

Created a symbolic link of the nginx config

Lets Encrypt SSL

I have previously setup Lets encrypt on Ubuntu 16.04 but not 18.04. Certbot had info on setting up Lets Encrypt for 14.x 16.x and 17.x but not 18.x

Full credit for the SSL steps goes to @Linuxize ( tips on setting up Lets Encrypt on Ubuntu 18.04 ). Check out https://linuxize.com/

I installed Lets Encrypt certbot

I created a new Diffie–Hellman key

Map requests to http://test.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).

Create a /etc/nginx/snippets/letsencrypt.conf on http://test.fearby.com and enforce the redirect.

Create a /etc/nginx/snippets/ssl.conf file on http://test.fearby.com

Let’s get a certificate

Certificates have been created 🙂

Now lets edit “/etc/nginx/sites-available/default” on https://test.fearby.com VM and add the cert paths.

Reload NGINX

or

Now let’s setup the second subdomain (subsite off https://fearby.com) and SSL

VM

I already have NGINX on https://fearby.com set up a second site.

DNS

We have already set up a DNS record for https://audit.fearby.com (above)

Firewall

Already configured at https://fearby.com

SSL

Because I had an existing Comodo certificate on https://fearby.com I am going to repeat the steps above to generate a new certificate but save the NGINX config to /etc/nginx/sites-available/audit.fearby.com (this activates the second site)

TIP: Follow the Linuxize guide here (for creating ssl.conf, letsencrypt.conf etc config files), Do a backup and restore if need be.

I created a new Diffie–Hellman key

Let’s get a certificate

Configure NGINX

Map requests to http://audit.fearby.com/.well-known/acme-challenge to /var/lib/letsencrypt/.well-known ( Read the linuxize post for detailed steps ).

I created a new NGINX site ( /etc/nginx/sites-available/audit.fearby.com )

I created a symbolic link of the config file

Reload NGINX

or

How to test the certificate renewal

Automate the renewal in crontab (every 12 hours)

I set this crontab entry up on https://fearby.com and https://test.fearby.com

Conclusion

Yes, I haVe 2 subdomains (1x dedicated VM and the other is a sub-website off an existing server) with SSL certificates.

Ping Results

Webpage Results

Screenshow showing the main site and 2 subdomains in a web browser

Troubleshooting

If you are having troubles generating the initial certificate check that you have not blocked port 80 and don’t have “Strict-Transport-Security” heavers enabled.

I re-ran the certbot command but pointed to the real /www-root (not/var/lib/letsencrypt/)

Create a new

mkdir /www-root/.well-known/
mkdir /www-root/.well-known/acme-challenge/
sudo certbot certonly --agree-tos --email [email protected] --webroot -w /www-root -d yoursubdomain.domain.com

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Troubleshooting

v1.0 Initial Post

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

by

Upgrading an Ubuntu server on UpCloud to add more CPU, Memory and Disk Space

If you have not read my previous posts I have now moved my blog from Vultr to the awesome UpCloud host (signup using this link to get $25 free credit).

Recently I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Spoiler: UpCloud performance is great.

Upcloud Site Speed in GTMetrix

Why Upgrade

I have 1 CPU, 1 GB memory and 50GB storage and it is running well?  I have PHP child workers set up and have set up the preferred use of memory over swap file usage.

View of htop querying processes on Ubuntu

Before UpCloud, when I had 512MB ram on Vultr I had multiple NGINX crashed a day so I used a bash script and scheduled a cron job to clear memory cache when memory fell below 100MB (view the script here).  To further increase the speed of the WordPress I have configured the OS to use memory over the disk.  About once a day free memory falls below 100MB (this is not a problem as my script clears cached items automatically).

Graph of memory falling below 100MB every day

I’d like to add more memory as I am working on some things (watch this space) and I will use the extra memory. I’d prefer the server is set up now for the expected workload.

How to Upgrade

This is how I upgraded from 1xCPU/2GB Memory/50GB Storage/2TB Transferred Data to a 2 CPU/4GB Mmeory/80GB Storage/4TB Transferred Data server.

UpCloud Pricing: https://www.upcloud.com/pricing/

Pricing table form https://www.upcloud.com/pricing/

Upgrade an UpCloud VM

I shut down my existing VM. Read this guide to setup a VM.

Login to the UpCloud dashboard, select your server (confirm the server has shut down) and click General Settings, choose the upgrade and click Update.

Upgrade the Server, shut it down the server and choose upgrade

I confirmed the upgrade options (2x CPU, 4096 MB Memory).

Confirm Upgrade Options

Click Update

Upgrade Applied

After 10 seconds you can start your server from the UpCloud server.

I confirmed the CPU upgrade was visible in the VM

Software Tweaks Post Upgrade.

I added these settings to the top of /etc/nginx/nginx.conf to ensure the extra CPU was used.

I increased PHP FPM ( /etc/php/7.2/fpm/php.ini ) to increase memory usage and child workers. I doubled child workers and max memory limit.

I restarted NGINX and PHP

I tweaked WordPress max memory limits

MySQL Tweaks: I logged into MySQL

I ran “SHOW GLOBAL STATUS” to view stats

mysql> SHOW GLOBAL STATUS;
+-----------------------------------------------+--------------------------------------------------+
| Variable_name                                 | Value                                            |
+-----------------------------------------------+--------------------------------------------------+
| Aborted_clients                               | 0                                                |
| Aborted_connects                              | 0                                                |
| Binlog_cache_disk_use                         | 0                                                |
| Binlog_cache_use                              | 0                                                |
| Binlog_stmt_cache_disk_use                    | 0                                                |
| Binlog_stmt_cache_use                         | 0                                                |
| Bytes_received                                | 3179986                                          |
| Bytes_sent                                    | 223872114                                        |
| Com_admin_commands                            | 0                                                |
| Com_assign_to_keycache                        | 0                                                |
| Com_alter_db                                  | 0                                                |
| Com_alter_db_upgrade                          | 0                                                |
| Com_alter_event                               | 0                                                |
| Com_alter_function                            | 0                                                |
| Com_alter_instance                            | 0                                                |
| Com_alter_procedure                           | 0                                                |
| Com_alter_server                              | 0                                                |
| Com_alter_table                               | 0                                                |
| Com_alter_tablespace                          | 0                                                |
| Com_alter_user                                | 0                                                |
| Com_analyze                                   | 0                                                |
| Com_begin                                     | 0                                                |
| Com_binlog                                    | 0                                                |
| Com_call_procedure                            | 0                                                |
| Com_change_db                                 | 284                                              |
| Com_change_master                             | 0                                                |
| Com_change_repl_filter                        | 0                                                |
| Com_check                                     | 0                                                |
| Com_checksum                                  | 0                                                |
| Com_commit                                    | 0                                                |
| Com_create_db                                 | 0                                                |
| Com_create_event                              | 0                                                |
| Com_create_function                           | 0                                                |
| Com_create_index                              | 0                                                |
| Com_create_procedure                          | 0                                                |
| Com_create_server                             | 0                                                |
| Com_create_table                              | 0                                                |
| Com_create_trigger                            | 0                                                |
| Com_create_udf                                | 0                                                |
| Com_create_user                               | 0                                                |
| Com_create_view                               | 0                                                |
| Com_dealloc_sql                               | 0                                                |
| Com_delete                                    | 18                                               |
| Com_delete_multi                              | 0                                                |
| Com_do                                        | 0                                                |
| Com_drop_db                                   | 0                                                |
| Com_drop_event                                | 0                                                |
| Com_drop_function                             | 0                                                |
| Com_drop_index                                | 0                                                |
| Com_drop_procedure                            | 0                                                |
| Com_drop_server                               | 0                                                |
| Com_drop_table                                | 0                                                |
| Com_drop_trigger                              | 0                                                |
| Com_drop_user                                 | 0                                                |
| Com_drop_view                                 | 0                                                |
| Com_empty_query                               | 0                                                |
| Com_execute_sql                               | 0                                                |
| Com_explain_other                             | 0                                                |
| Com_flush                                     | 0                                                |
| Com_get_diagnostics                           | 0                                                |
| Com_grant                                     | 0                                                |
| Com_ha_close                                  | 0                                                |
| Com_ha_open                                   | 0                                                |
| Com_ha_read                                   | 0                                                |
| Com_help                                      | 0                                                |
| Com_insert                                    | 342                                              |
| Com_insert_select                             | 0                                                |
| Com_install_plugin                            | 0                                                |
| Com_kill                                      | 0                                                |
| Com_load                                      | 0                                                |
| Com_lock_tables                               | 0                                                |
| Com_optimize                                  | 0                                                |
| Com_preload_keys                              | 0                                                |
| Com_prepare_sql                               | 0                                                |
| Com_purge                                     | 0                                                |
| Com_purge_before_date                         | 0                                                |
| Com_release_savepoint                         | 0                                                |
| Com_rename_table                              | 0                                                |
| Com_rename_user                               | 0                                                |
| Com_repair                                    | 0                                                |
| Com_replace                                   | 0                                                |
| Com_replace_select                            | 0                                                |
| Com_reset                                     | 0                                                |
| Com_resignal                                  | 0                                                |
| Com_revoke                                    | 0                                                |
| Com_revoke_all                                | 0                                                |
| Com_rollback                                  | 0                                                |
| Com_rollback_to_savepoint                     | 0                                                |
| Com_savepoint                                 | 0                                                |
| Com_select                                    | 16358                                            |
| Com_set_option                                | 849                                              |
| Com_signal                                    | 0                                                |
| Com_show_binlog_events                        | 0                                                |
| Com_show_binlogs                              | 0                                                |
| Com_show_charsets                             | 0                                                |
| Com_show_collations                           | 0                                                |
| Com_show_create_db                            | 0                                                |
| Com_show_create_event                         | 0                                                |
| Com_show_create_func                          | 0                                                |
| Com_show_create_proc                          | 0                                                |
| Com_show_create_table                         | 0                                                |
| Com_show_create_trigger                       | 0                                                |
| Com_show_databases                            | 3                                                |
| Com_show_engine_logs                          | 0                                                |
| Com_show_engine_mutex                         | 0                                                |
| Com_show_engine_status                        | 0                                                |
| Com_show_events                               | 0                                                |
| Com_show_errors                               | 0                                                |
| Com_show_fields                               | 921                                              |
| Com_show_function_code                        | 0                                                |
| Com_show_function_status                      | 0                                                |
| Com_show_grants                               | 0                                                |
| Com_show_keys                                 | 1                                                |
| Com_show_master_status                        | 0                                                |
| Com_show_open_tables                          | 0                                                |
| Com_show_plugins                              | 0                                                |
| Com_show_privileges                           | 0                                                |
| Com_show_procedure_code                       | 0                                                |
| Com_show_procedure_status                     | 0                                                |
| Com_show_processlist                          | 0                                                |
| Com_show_profile                              | 0                                                |
| Com_show_profiles                             | 0                                                |
| Com_show_relaylog_events                      | 0                                                |
| Com_show_slave_hosts                          | 0                                                |
| Com_show_slave_status                         | 0                                                |
| Com_show_status                               | 6                                                |
| Com_show_storage_engines                      | 0                                                |
| Com_show_table_status                         | 0                                                |
| Com_show_tables                               | 2                                                |
| Com_show_triggers                             | 0                                                |
| Com_show_variables                            | 6                                                |
| Com_show_warnings                             | 1                                                |
| Com_show_create_user                          | 0                                                |
| Com_shutdown                                  | 0                                                |
| Com_slave_start                               | 0                                                |
| Com_slave_stop                                | 0                                                |
| Com_group_replication_start                   | 0                                                |
| Com_group_replication_stop                    | 0                                                |
| Com_stmt_execute                              | 4                                                |
| Com_stmt_close                                | 4                                                |
| Com_stmt_fetch                                | 0                                                |
| Com_stmt_prepare                              | 4                                                |
| Com_stmt_reset                                | 0                                                |
| Com_stmt_send_long_data                       | 4                                                |
| Com_truncate                                  | 2                                                |
| Com_uninstall_plugin                          | 0                                                |
| Com_unlock_tables                             | 0                                                |
| Com_update                                    | 70                                               |
| Com_update_multi                              | 0                                                |
| Com_xa_commit                                 | 0                                                |
| Com_xa_end                                    | 0                                                |
| Com_xa_prepare                                | 0                                                |
| Com_xa_recover                                | 0                                                |
| Com_xa_rollback                               | 0                                                |
| Com_xa_start                                  | 0                                                |
| Com_stmt_reprepare                            | 0                                                |
| Connection_errors_accept                      | 0                                                |
| Connection_errors_internal                    | 0                                                |
| Connection_errors_max_connections             | 0                                                |
| Connection_errors_peer_address                | 0                                                |
| Connection_errors_select                      | 0                                                |
| Connection_errors_tcpwrap                     | 0                                                |
| Connections                                   | 292                                              |
| Created_tmp_disk_tables                       | 1124                                             |
| Created_tmp_files                             | 30                                               |
| Created_tmp_tables                            | 1369                                             |
| Delayed_errors                                | 0                                                |
| Delayed_insert_threads                        | 0                                                |
| Delayed_writes                                | 0                                                |
| Flush_commands                                | 1                                                |
| Handler_commit                                | 6094                                             |
| Handler_delete                                | 33                                               |
| Handler_discover                              | 0                                                |
| Handler_external_lock                         | 38571                                            |
| Handler_mrr_init                              | 0                                                |
| Handler_prepare                               | 0                                                |
| Handler_read_first                            | 2299                                             |
| Handler_read_key                              | 134761                                           |
| Handler_read_last                             | 237                                              |
| Handler_read_next                             | 310119                                           |
| Handler_read_prev                             | 2733                                             |
| Handler_read_rnd                              | 222350                                           |
| Handler_read_rnd_next                         | 472820                                           |
| Handler_rollback                              | 0                                                |
| Handler_savepoint                             | 0                                                |
| Handler_savepoint_rollback                    | 0                                                |
| Handler_update                                | 15605                                            |
| Handler_write                                 | 17310                                            |
| Innodb_buffer_pool_dump_status                | Dumping of buffer pool not started               |
| Innodb_buffer_pool_load_status                | Buffer pool(s) load completed at 180624 23:38:01 |
| Innodb_buffer_pool_resize_status              |                                                  |
| Innodb_buffer_pool_pages_data                 | 1035                                             |
| Innodb_buffer_pool_bytes_data                 | 16957440                                         |
| Innodb_buffer_pool_pages_dirty                | 0                                                |
| Innodb_buffer_pool_bytes_dirty                | 0                                                |
| Innodb_buffer_pool_pages_flushed              | 1936                                             |
| Innodb_buffer_pool_pages_free                 | 7144                                             |
| Innodb_buffer_pool_pages_misc                 | 13                                               |
| Innodb_buffer_pool_pages_total                | 8192                                             |
| Innodb_buffer_pool_read_ahead_rnd             | 0                                                |
| Innodb_buffer_pool_read_ahead                 | 0                                                |
| Innodb_buffer_pool_read_ahead_evicted         | 0                                                |
| Innodb_buffer_pool_read_requests              | 306665                                           |
| Innodb_buffer_pool_reads                      | 950                                              |
| Innodb_buffer_pool_wait_free                  | 0                                                |
| Innodb_buffer_pool_write_requests             | 26509                                            |
| Innodb_data_fsyncs                            | 1229                                             |
| Innodb_data_pending_fsyncs                    | 0                                                |
| Innodb_data_pending_reads                     | 0                                                |
| Innodb_data_pending_writes                    | 0                                                |
| Innodb_data_read                              | 16273920                                         |
| Innodb_data_reads                             | 1078                                             |
| Innodb_data_writes                            | 2857                                             |
| Innodb_data_written                           | 53379584                                         |
| Innodb_dblwr_pages_written                    | 1275                                             |
| Innodb_dblwr_writes                           | 109                                              |
| Innodb_log_waits                              | 0                                                |
| Innodb_log_write_requests                     | 450                                              |
| Innodb_log_writes                             | 585                                              |
| Innodb_os_log_fsyncs                          | 793                                              |
| Innodb_os_log_pending_fsyncs                  | 0                                                |
| Innodb_os_log_pending_writes                  | 0                                                |
| Innodb_os_log_written                         | 664064                                           |
| Innodb_page_size                              | 16384                                            |
| Innodb_pages_created                          | 56                                               |
| Innodb_pages_read                             | 988                                              |
| Innodb_pages_written                          | 1936                                             |
| Innodb_row_lock_current_waits                 | 0                                                |
| Innodb_row_lock_time                          | 0                                                |
| Innodb_row_lock_time_avg                      | 0                                                |
| Innodb_row_lock_time_max                      | 0                                                |
| Innodb_row_lock_waits                         | 0                                                |
| Innodb_rows_deleted                           | 2                                                |
| Innodb_rows_inserted                          | 19219                                            |
| Innodb_rows_read                              | 249102                                           |
| Innodb_rows_updated                           | 77                                               |
| Innodb_num_open_files                         | 81                                               |
| Innodb_truncated_status_writes                | 0                                                |
| Innodb_available_undo_logs                    | 128                                              |
| Key_blocks_not_flushed                        | 0                                                |
| Key_blocks_unused                             | 12751                                            |
| Key_blocks_used                               | 645                                              |
| Key_read_requests                             | 321877                                           |
| Key_reads                                     | 648                                              |
| Key_write_requests                            | 196                                              |
| Key_writes                                    | 150                                              |
| Locked_connects                               | 0                                                |
| Max_execution_time_exceeded                   | 0                                                |
| Max_execution_time_set                        | 0                                                |
| Max_execution_time_set_failed                 | 0                                                |
| Max_used_connections                          | 3                                                |
| Max_used_connections_time                     | 2018-06-24 23:43:48                              |
| Not_flushed_delayed_rows                      | 0                                                |
| Ongoing_anonymous_transaction_count           | 0                                                |
| Open_files                                    | 229                                              |
| Open_streams                                  | 0                                                |
| Open_table_definitions                        | 206                                              |
| Open_tables                                   | 786                                              |
| Opened_files                                  | 502                                              |
| Opened_table_definitions                      | 208                                              |
| Opened_tables                                 | 817                                              |
| Performance_schema_accounts_lost              | 0                                                |
| Performance_schema_cond_classes_lost          | 0                                                |
| Performance_schema_cond_instances_lost        | 0                                                |
| Performance_schema_digest_lost                | 0                                                |
| Performance_schema_file_classes_lost          | 0                                                |
| Performance_schema_file_handles_lost          | 0                                                |
| Performance_schema_file_instances_lost        | 0                                                |
| Performance_schema_hosts_lost                 | 0                                                |
| Performance_schema_index_stat_lost            | 0                                                |
| Performance_schema_locker_lost                | 0                                                |
| Performance_schema_memory_classes_lost        | 0                                                |
| Performance_schema_metadata_lock_lost         | 0                                                |
| Performance_schema_mutex_classes_lost         | 0                                                |
| Performance_schema_mutex_instances_lost       | 0                                                |
| Performance_schema_nested_statement_lost      | 0                                                |
| Performance_schema_prepared_statements_lost   | 0                                                |
| Performance_schema_program_lost               | 0                                                |
| Performance_schema_rwlock_classes_lost        | 0                                                |
| Performance_schema_rwlock_instances_lost      | 0                                                |
| Performance_schema_session_connect_attrs_lost | 0                                                |
| Performance_schema_socket_classes_lost        | 0                                                |
| Performance_schema_socket_instances_lost      | 0                                                |
| Performance_schema_stage_classes_lost         | 0                                                |
| Performance_schema_statement_classes_lost     | 0                                                |
| Performance_schema_table_handles_lost         | 0                                                |
| Performance_schema_table_instances_lost       | 0                                                |
| Performance_schema_table_lock_stat_lost       | 0                                                |
| Performance_schema_thread_classes_lost        | 0                                                |
| Performance_schema_thread_instances_lost      | 0                                                |
| Performance_schema_users_lost                 | 0                                                |
| Prepared_stmt_count                           | 0                                                |
| Qcache_free_blocks                            | 1                                                |
| Qcache_free_memory                            | 16760152                                         |
| Qcache_hits                                   | 0                                                |
| Qcache_inserts                                | 0                                                |
| Qcache_lowmem_prunes                          | 0                                                |
| Qcache_not_cached                             | 16355                                            |
| Qcache_queries_in_cache                       | 0                                                |
| Qcache_total_blocks                           | 1                                                |
| Queries                                       | 19164                                            |
| Questions                                     | 19155                                            |
| Select_full_join                              | 0                                                |
| Select_full_range_join                        | 0                                                |
| Select_range                                  | 2677                                             |
| Select_range_check                            | 0                                                |
| Select_scan                                   | 2098                                             |
| Slave_open_temp_tables                        | 0                                                |
| Slow_launch_threads                           | 0                                                |
| Slow_queries                                  | 0                                                |
| Sort_merge_passes                             | 12                                               |
| Sort_range                                    | 4859                                             |
| Sort_rows                                     | 244452                                           |
| Sort_scan                                     | 854                                              |
| Ssl_accept_renegotiates                       | 0                                                |
| Ssl_accepts                                   | 0                                                |
| Ssl_callback_cache_hits                       | 0                                                |
| Ssl_cipher                                    |                                                  |
| Ssl_cipher_list                               |                                                  |
| Ssl_client_connects                           | 0                                                |
| Ssl_connect_renegotiates                      | 0                                                |
| Ssl_ctx_verify_depth                          | 0                                                |
| Ssl_ctx_verify_mode                           | 0                                                |
| Ssl_default_timeout                           | 0                                                |
| Ssl_finished_accepts                          | 0                                                |
| Ssl_finished_connects                         | 0                                                |
| Ssl_server_not_after                          |                                                  |
| Ssl_server_not_before                         |                                                  |
| Ssl_session_cache_hits                        | 0                                                |
| Ssl_session_cache_misses                      | 0                                                |
| Ssl_session_cache_mode                        | NONE                                             |
| Ssl_session_cache_overflows                   | 0                                                |
| Ssl_session_cache_size                        | 0                                                |
| Ssl_session_cache_timeouts                    | 0                                                |
| Ssl_sessions_reused                           | 0                                                |
| Ssl_used_session_cache_entries                | 0                                                |
| Ssl_verify_depth                              | 0                                                |
| Ssl_verify_mode                               | 0                                                |
| Ssl_version                                   |                                                  |
| Table_locks_immediate                         | 11962                                            |
| Table_locks_waited                            | 0                                                |
| Table_open_cache_hits                         | 19395                                            |
| Table_open_cache_misses                       | 817                                              |
| Table_open_cache_overflows                    | 12                                               |
| Tc_log_max_pages_used                         | 0                                                |
| Tc_log_page_size                              | 0                                                |
| Tc_log_page_waits                             | 0                                                |
| Threads_cached                                | 2                                                |
| Threads_connected                             | 1                                                |
| Threads_created                               | 3                                                |
| Threads_running                               | 1                                                |
| Uptime                                        | 2944                                             |
| Uptime_since_flush_status                     | 2944                                             |
+-----------------------------------------------+--------------------------------------------------+
353 rows in set (0.00 sec)

Read more on SHOW GLOBAL STATUS here. Read more on the values here.

I can see NO major errors here (possibly due to UpClouds awesome disk IO) so I won’t be making memory tweaks in MySQL. Sign Up using this link and get $25 credit free on UpCloud and see for yourself how fast they are.

Configure Ubuntu System Memory Usage

Edit /etc/sysctl.conf

Add the following to allow things to sit in ram longer

Snip from: https://www.kernel.org/doc/Documentation/sysctl/vm.txt

This percentage value controls the tendency of the kernel to reclaim the memory which is used for caching of directory and inode objects.

At the default value of vfs_cache_pressure=100 the kernel will attempt to reclaim dentries and inodes at a "fair" rate with respect to pagecache and swapcache reclaim.  Decreasing vfs_cache_pressure causes the kernel to prefer to retain dentry and inode caches. When vfs_cache_pressure=0, the kernel will never reclaim dentries and inodes due to memory pressure and this can easily lead to out-of-memory conditions. Increasing vfs_cache_pressure beyond 100 causes the kernel to prefer to reclaim dentries and inodes.

Increasing vfs_cache_pressure significantly beyond 100 may have negative performance impact. Reclaim code needs to take various locks to find freeable directory and inode objects. With vfs_cache_pressure=1000, it will look for ten times more freeable objects than there are."

Read these pages here and here regarding setting MySQL memory.

Reboot

Resize the disk

The 2x CPU, 4GB memory plan comes with 80GB storage allowance.  My disk at present is 50GB and I will update the size soon following this guide soon.

Upgrade disk from 50gb to 80GB soon

Quick Benchmark

I used loader.io to load 500 users to access my site in 1 minute.

HTOP showing 2x busy CPU's running at 60%

The benchmark worked with no errors.

Loader.io Success with 500 concurrent users

This benchmark was performed with no Cloudflare caching. I should get Cloudflare caching working again to lower the average response time. I loaded my website manually in Google Chrome while loader.io threw 500 users at my site and it loaded very fast.

Conclusion

After a few days, I checked my memory logs and there were no low memory triggers (just normal internal memory management triggers). Ubuntu was happier.

No Low memory low triggers

This graph was taken before I set “vm.vfs_cache_pressure” so I will update this graph in a few days.

I hope this guide helps someone.

Please consider using my referral code and get $25 credit for free.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

1.0 Initial Post