• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

the

Using the Qualys FreeScan Scanner to test your website for online vulnerabilities

March 23, 2018 by Simon

It is possible to deploy a server in minutes to hours but it can take days to secure.  What tools can you use to help identify what to secure on your website?

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line, installing a Free SSL certificate and setting up SSL security.

Security Tools

  • https://asafaweb.com/ is a good tool for quick scanning
  • Kali Linux has a number of security tools you can use.
  • You can run a system audit Lynis Audit.
  • Checking your site for vulnerabilities with Zap.
  • Run a Gravity Scan malware and supply chain scan
  • Use Qualys SSL scan to test your SSL certificate: https://www.ssllabs.com/ssltest/

Qualys

Qualys SSL Labs SSL Tester is the best tool for checking an SSL certificate strength

Most people don’t know Qualys also has another free (limited to 10 scans) vulnerability scanner for websites.

Goto https://freescan.qualys.com/ and click Start your free account.

Complete the signup form

Now check your email to login and confirm your email account

Login now from the email.

Create a password (why the 25 char max Qualys?)

Enter your website URL and click Scan

The scan can take hours

While the scan was being performed I noticed that Qualys offers alerts (I’ll check this out later): https://www.qualys.com/research/security-alerts/

Yes, the scan can take hours, take a walk or read other posts here.

The scan is almost complete

Yay, my latest scan revealed 0 High, 0 Medium and 0 Low-risk vulnerabilities.

It did report 23 informational alerts like “Firewall Detected“.

Threat Report Results

Patch Report Results

This report was empty (probably because I don’t run Windows)

Threat Report Results

The OWASP report contained partial scan results (maybe the full report is available to pro users)

Previous Scan Results

The Qualys dashboard will show all past scans.

My first scan showed a Low priority issue with the /wp-login.php page as the input fields did not have “autocomplete=”off””, I fixed this by adding “autocomplete=”off”” the removing the page (safer).

The second scan found two issues with cookies (possibly ad banner cookies) and 2 subfolders that I created in past development exercises. I deleted the two sub-folders that were not needed.

The third scan was clean.

Here is a scan of a static website of a friends server (static can be less secure if the server underneath is old or unpatched).

Static Website

Happy scanning. I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Static Web Server Scan

v1.0 Initial post

Filed Under: Firewall, LetsEncrypt, Linux, Malware, Security, Server, Ubuntu, Vulnerabilities, Vulnerability, WP Security Tagged With: for, FreeScan, online, Qualys, Scanner, test, the, to, Using, Vulnerabilities, website, your

Backing up OSX or an Ubuntu server with Backblaze B2 Cloud Storage from the Command Line

March 14, 2018 by Simon

This computer will show you can back up computer or server with Backblaze B2 Cloud Storage from the Command Line n OSX and Ubuntu.

This post is still being written. I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Also, I have blogged about how you can add block storage to a Vultr server, backup and restore snapshots , syncing files with rsync along with using GitHub and Bitbucket but what do you do if you need to backup large amounts of data?

Backblaze has a Cloud storage solution that costs as low as $0.005c a GB (a month), The first 10G is free. Backblaze say “From bytes to petabytes Backblaze B2 is the lowest cost high-performance cloud storage in the world. ”

Back Blaze have open sourced internal drive enclosure designs and drive failure stats and it’s time I gave them a try.

Goto https://www.backblaze.com

Backblaze

Create or sign in.

Backblaze Login

After you login got the dashboard.

B2 Cloud

Click Backblaze B2 Cloud Storage

Activate B2 Cloud

Signup

Create a Bucket

Create Bucket

Name the bucket (long names with a GUID are good).

Name the Bucket

You can rename the bucket here and change public/private and or upload/download files manually.

Manage Bucket

The first thing I did was limit the versions of files under the lifecycle settings for the bucket.

Version Settings

Now I created a series of subfolders to store files from different servers (I could have used many buckets but one bucket will do).

Folders

I can upload files via the Backblaze bucket GUI if I needed to.

Upload and Download

Back Blaze has a command line tool for uploading: https://www.backblaze.com/b2/docs/quick_command_line.html

Install Steps

Backblaze state “The B2 command-line tool is available from the Python Package Index (PyPI) using the standard pip installation tool. Your first step is to make sure that you have either Python 2 (2.6 or later) or Python 3 (3.2 or later) installed.”

I have Python 2.7 installed

python --version
Python 2.7.10

Install PIP

sudo easy_install pip
Password:
Searching for pip
Best match: pip 1.5.6
Processing pip-1.5.6-py2.7.egg
pip 1.5.6 is already the active version in easy-install.pth
Installing pip script to /usr/local/bin
Installing pip2.7 script to /usr/local/bin
Installing pip2 script to /usr/local/bin

Using /Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg
Processing dependencies for pip
Finished processing dependencies for pip

I ran into issues updating b2 CLI

sudo pip install --upgrade b2
Requirement already up-to-date: b2 in /Library/Python/2.7/site-packages
Requirement already up-to-date: arrow>=0.8.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: logfury>=0.1.2 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: requests>=2.9.1 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: six>=1.10 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: tqdm>=4.5.0 in /Library/Python/2.7/site-packages (from b2)
Requirement already up-to-date: futures>=3.0.5 in /Library/Python/2.7/site-packages (from b2)
Downloading/unpacking python-dateutil from https://pypi.python.org/packages/bc/c5/3449988d33baca4e9619f49a14e28026399b0a8c32817e28b503923a04ab/python_dateutil-2.7.0-py2.py3-none-any.whl#md5=5a86a548fe776cc079bf4a835473e3f8 (from arrow>=0.8.0->b2)
  Downloading python_dateutil-2.7.0-py2.py3-none-any.whl (207kB): 207kB downloaded
Installing collected packages: python-dateutil
  Found existing installation: python-dateutil 1.5
    Uninstalling python-dateutil:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/zoneinfo-2010g.tar.gz'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/zoneinfo'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/relativedelta.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/parser.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/rrule.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/easter.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tz.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil/tzwin.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil', "[Errno 1] Operation not permitted: '/tmp/pip-jWEHna-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/dateutil'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

I tried installing via the alternative method (with no luck)

git clone https://github.com/Backblaze/B2_Command_Line_Tool.git
Cloning into 'B2_Command_Line_Tool'...
remote: Counting objects: 5084, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 5084 (delta 1), reused 1 (delta 0), pack-reused 5076
Receiving objects: 100% (5084/5084), 1.25 MiB | 689.00 KiB/s, done.
Resolving deltas: 100% (3622/3622), done.
cd B2_Command_Line_Tool/

I tried running the setup script (with no luck)

sudo python setup.py install
setuptools 20.2 or later is required. To fix, try running: pip install "setuptools>=20.2"

Upgrading setup tools also failed

sudo pip install "setuptools>=20.2"
Downloading/unpacking setuptools>=20.2
  Downloading setuptools-38.5.2-py2.py3-none-any.whl (490kB): 490kB downloaded
Installing collected packages: setuptools
  Found existing installation: setuptools 18.5
    Uninstalling setuptools:
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1431, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 598, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/req.py", line 1836, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-1.5.6-py2.7.egg/pip/util.py", line 295, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
    copytree(src, real_dst, symlinks=True)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
    raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-8Vu7xp-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]

Storing debug log for failure in /Users/simon/Library/Logs/pip.log

Backing up a Mac via command line with B2

More to come when I can get B2 CLI Installed.

Backing up an Ubuntu machine via command line with B2

More to come when I can get B2 CLI Installed.

Update

My ticket with Backblaze was automatically closed with this note “If the issue is persisting, it may be easiest to map the installation to the user folder, rather than the system level.”

No ideas how but something to research.

Ask a question or the recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V1.1 ticket closed

v1.0 Initial post

Filed Under: Backup Tagged With: an, B2, backblaze, Backing, cloud, command, from, line, or, OSX, server, storage, the, ubuntu, up, with

Setting up the Debian Kali Linux distro to perform penetration testing of your systems

March 7, 2018 by Simon

This post will show you how to setup the Kali Linux distro to perform penetration testing of your systems

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Securing your systems is very important (don’t stop) and keep learning (securing ubuntu in the cloud, securing checklist, run a Lynis system audit etc)

snip from: https://www.kali.org/about-us/

“Kali Linux is an open source project that is maintained and funded by Offensive Security, a provider of world-class information security training and penetration testing services. In addition to Kali Linux, Offensive Security also maintains the Exploit Database and the free online course, Metasploit Unleashed.”

Download Kali

I downloaded the torrent version (as the HTTP version kept stopping (even on 50/20 NBN).

Download Kali

After the download finished I checked the SHA sum to verify it’a integrity

cd /Users/username/Downloads/kali-linux-2018.1-amd64
shasum -a 256 ./kali-linux-2018.1-amd64.iso 
ed88466834ceeba65f426235ec191fb3580f71d50364ac5131daec1bf976b317  ./kali-linux-2018.1-amd64.iso

A least it matched the known (or hacked) hash here.

Installing Parallels in a VM on OSX

I use Parallels 11 on OSX to set up a VM os Demina Kali, you can use VirtualBox, VMWare etc.

VM Setup in Parallels

Hardware: 2x CPU, 2048MB Ram, 32MB Graphics, 64GB Disk.

I selected Graphical Install (English, Australia, American English, host: kali, network: hyrule, New South Wales, Partition: Guided – entire disk, Default, Default, Default, Continue, Yes, Network Mirror: Yes, No Proxy, Installed GRUB bootloader on VM HD.

Post Install

Install Parallel Tools

Official Guide: https://kb.parallels.com/en/123968

I opened the VM then selected the Actions then Install Parallels Tools, this mounted /media/cdrom/, I copied all contents to /temp/

As recommended by the Parallels instal bash script I updated headers.

apt install linux-headers-4.14.0-kali1-amd64

Then the following from https://kb.parallels.com/en/123968

apt-get clean
apt-get update
apt-get upgrade -y
apt-get dist-upgrade -y
apt-get install dkms kpartx printer-driver-postscript-hp

Parallels will not install, I think I need to upgrade to parallel 12 or 12 as the printer driver detection is not detecting (even though it is installed).

Installing Google Chrome

I used the video below

I have to run chrome with

/usr/bin/gogole-chrome-stable %U --no-sandbox --user-data=dir &

It works.

Chrome

Running your first remote vulnerability scan in Kali

I found this video useful in helping me scan and check my systems for exploits

Simple exploit search in Armitage (metasploit)

Armitage Scan

A quick scan of my server revealed three ports open and (22, 80 and 443). Port 80 redirects to 443 and port 22 is firewalled.  I have WordPress and exploits I rued failed to work thanks to patching (always stay ahead of patching and updating of software and the OS.

k006-ports

Without knowing what I was doing I was able to check my WordPress against known exploits. 

If you open the Check Exploits menu at the end of the Attacks menu you can do a bulk exploit check.

kali_bulk

WP Scan

Kali also comes with a WordPress scanner

wpscan --url https://fearby.com

This will try and output everything from your web server and WordPress plugins.

/xmlrpc.php was found and I was advised to deny access to that file in NGINX. xmlrpc.php is ok but can be used in denial of service attacks.

location = /xmlrpc.php {
	deny all;
	access_log off;
	log_not_found off;
}

I had a hit for a vulnerability in a Youtube Embed plugin but I had a patched version.

k007-wpscan

TIP: Check your WordPress often.

More to come (Draft Post).

  • OWASP scanner
  • WPSCAN
  • Ethical Hacker modules
  • Cybrary training
  • Sent tips to @FearbySoftware

Tips

Don’t have unwanted ports open, securely installed software, Use unattended security updates in Ubuntu, update WordPress frequently and limit plugins and also consider running more verbose audit tools like Lynis.

More Reading

Read my OWASP Zap guide on application testing and Cloudflare guide.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 added More Reading links.

v1.1 Added bulk exploit check.

v1.0 Initial post

Filed Under: Exploit, Linux, Malware, Security, Server, SSH, Vulnerability Tagged With: debian, distro, Kali, Linux, of, penetration, perform, Setting, systems, testing, the, to, up, your

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud

March 7, 2018 by Simon

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud is quite simple.

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.  I prefer Vultr as they are located in the country (Australia) and are easy to use.

First, you need to shut down the server from within the VM (SSH), I used the command.

sudo shutdown now

Once the VM is shut down (wait a few minutes) you can turn off the VM in the Vultr GUI.

Shutdown

You can then go to Settings, Change Plan and review upgrade options.

Upgrade Options

Snapshot

Don’t forget to take a final snapshot.

Snapshot reminder

Goto the Snapshots page (read this guide to restore a snapshot) and click Take Snapshot.

Take Snapshop

You can see snapshot progress on the main screen.

Snapshot Progress

It may take a while for your snapshot to change from Pending to Processing.

Processing

Upgrade

When the snapshot is done it will auto boot and allow you to upgrade.

Manage

Choose the Upgrade specifications (Settings, Change Plan)

Upgrade Specs

Click Upgrade

Upgrade

Confirm

Confirm

The upgrade process will take a few minutes (I could see the CU and Ram was updated but the Storage was pending)

Upgrade Pending

Testing

After the upgrade happened the VM will autoboot, login and check tour specifications (Useful Linux Commands).

I use the htop command to view specification information.

I did a quick benchmark pre-optimizing and I can see a speed bump of 0.2s. Time to optimize.

Benchmark

I threw 50 concurrent clients at my website (with loader.io) and the server handled it fine with no increase above memory capacity like before.

Concurent Users

Optimize

Now I need to Optimize.  Truth be told  I did optimize and harden PHP and crashed PHP-FPM so I had o restore a VM snapshot.

Troubleshooting

If all else fails (post-upgrade configuration) you can restore the Vultr VM from a snapshot.

I hope this guide helps someone.

P.S If you don’t have a VM on Vultr click this link to set one up in minutes (setup guide here).

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial post

Filed Under: Cloud, Server, Ubuntu, VM Tagged With: a, and, cloud, cpu, in, memory, on, ram, the, ubuntu, Upgrading, vm, vultr

Using the free Adminer GUI for MySQL on your website

February 8, 2018 by Simon

Adminer is a free GUI tool that can you can easily install on a PHP web server. Adminer allows you to easily connect to your MySQL instance, create databases/tables/indexes/rows and backup/import databases and much more.

You can read my other posts on Useful Linux Terminal Commands and Useful OSX Terminal Commands.

I used to use phpMyAdmin to manage MySQL databases on AWS, Digital Ocean and Vultr but switched to Adminer due to forgotten issues.  You can always manage MySQL via command line but that is quite boring.

The below screenshots were taken on my local Development Mac Laptop (with optional OSX Apache SSL Setup (that reports “Not Secure” (but it is good enough to use locally)). I prefer to code in SSL and warn when SSL is not detected.

Downloading and Installing Adminer

Navigate to https://www.adminer.org/ and click Download.

Adminer GUIClick English only (.php file)

Adminer

Save the Adminder for MySQL (.php) file to your web server and give it a random name and put in a folder also with a random name (I use https://www.grc.com/passwords.htm to generate strong password).

Tip: Uploading this file to a live serve offers hackers and unauthorized people potential access to your MySQL server.  I would remove this file from live serves when you are not using it not to be sure.

Tip: Read my guide here on setting up NGINX, MySQL and PHP here.  Basically, I did this to setup MySQL on Ubuntu 16.04.

sudo apt-get install mysql-common
sudo apt-get install mysql-server
mysql --version
>mysql Ver 14.14 Distrib 5.7.19, for Linux (x86_64) using EditLine wrapper
sudo mysql_secure_installation
>Y (Valitate plugin)
>2 (Strong passwords)
>N (Don't chnage root password)
>Y (Remove anon accounts)
>Y (No remote root login)
>Y (Remove test DB)
>Y (Reload)
service mysql status
> mysql.service - MySQL Community Server

TIP: Ensure MySQL is secure and has a good root password, also consider setting up Ubuntu Firewalls and Securing Ubuntu. Also, ensure the Server is patched and does not have exploits like Spectre and meltdown.

Now you can access your Admirer php file on your Web Server (hopefully with an obfuscated name).

Randomize

Login to Adminer with your MySQL root password.

Login

Click Create databaase

Create Database

Give the database a name and choose the character coding standard (e.g UTF8 general ci). Different standards have different performance impacts too.

Save

Now that you have a database you can create a table.

Adminer

Consider adding an auto-incrementing ID and say a Key and Value varchar column.

Adminer

When the table is created you can add a row to the table.

Adminer

I created one with a “TestKey” and “TestValue” row.

Adminer

The row was inserted.

Adminer

The final thing to do is add a database user that code can connect to the database with. Click Privileges.

Adminer

Click Create user

Adminer

Tick All privileges and click Save

Adminer

Now the user is added to the database

Adminer

Let’s create a PHP file and talk to the database. Let’s use parameterized queries

<?php

date_default_timezone_set('Australia/Sydney');
echo "Last modified: " . date ("F d Y H:i:s.", getlastmod()) . "<br /><br />";

// Turn on if you need to see errors
// error_reporting(E_ALL);
// ini_set('display_errors', 0);

$dbhost = '127.0.0.1';
$dbname = 'dbtest';
$dbusername = 'dbtestuser';
$dbpassword = '*****************************************'';

$con = mysqli_connect($dbhost, $dbusername, $dbpassword, $dbname);
 
// Turn on debug stuff if you need it
// echo var_dump($con);
// printf(" - Error: %s.n", $stmt->error);
 
if($con->connect_errno > 0){

    printf(" - Error: %s.n", $stmt->error);
    die("Error: Unable to connect to MySQL");

} else {

    echo "Charset set to utf8<br />";
    mysqli_set_charset($con,"utf8");
}
 
if (!$con) {

    echo "Error: Unable to connect to MySQL (E002)" . PHP_EOL;
    echo "Debugging errno: " . mysqli_connect_errno() . PHP_EOL;
    echo "Debugging error: " . mysqli_connect_error() . PHP_EOL;
    exit;

} else {

    echo "Database Connection OK<br />";
 
    echo "&nbsp; Success: A proper connection to MySQL was made! The $dbname database is great." . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Host information: " . mysqli_get_host_info($con) . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Server Info: '" . mysqli_get_server_info($con) . "'<br />";
    echo "&nbsp; &nbsp;- Server Protocol Info : ". mysqli_get_proto_info($con) . "<br />";
    echo "&nbsp; &nbsp;- Server Version: " . mysqli_get_server_version($con) . "<br />";
    //echo " - Server Connection Stats: " . print_r(vmysqli_get_connection_stats($con)) . "<br />";
    echo "&nbsp; &nbsp;- Client Version: " . mysqli_get_client_version($con) . "<br />";
    echo "&nbsp; &nbsp;- Client Info: '" . mysqli_get_client_info() . "'<br />";
 
    echo "Ready to Query the database '$dbname'.<br />";
 
    // Input Var's that are parameterized/bound into the query statement
    $in_key = mysqli_real_escape_string($con, 'TestKey');
 
    // Output Var's that the query fills after querying the database
    // These variables will be filled with data from the current returned row
    $out_id = "";
    $out_key = "";
    $out_value = "";
 
    echo "1. About to query the database: '$dbname'<br />";
    $stmt = mysqli_stmt_init($con);

    $sql = "SELECT testid, testkey, testvalue FROM tbtest WHERE testkey = ?";
    echo "SQL: $sql (In = $in_key)<br /";

    if (mysqli_stmt_prepare($stmt, $sql)) {

            echo "2. Query Returned<br />";
            /*
                Type specification chars
                Character   Description
                i   corresponding variable has type integer
                d   corresponding variable has type double
                s   corresponding variable has type string
                b   corresponding variable is a blob and will be sent in packets
            */
            mysqli_stmt_bind_param($stmt, 's', $in_key);
            mysqli_stmt_execute($stmt);
            mysqli_stmt_bind_result($stmt, $out_id, $out_key, $out_value);
            mysqli_stmt_fetch($stmt);
     
            // Do something with the 1st returned row        
            echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//

            // Do we have more rows to process
            while($stmt->fetch()) { 
                
                    // Output returned values
                    echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//
            
            }
            mysqli_stmt_close($stmt);
            
            echo "Done<br />";
        
        } else {
        
            echo "3. Error Querying<br/>";
            printf(" - Error: %s.n", $stmt->error);
        
        }
}    
?>

Result

Adminer Results

If you don’t have a server check out my guides on AWS, Digital Ocean and Vultr.

Happy coding and I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial Version

Filed Under: MySQLGUI Tagged With: Adminer, for, free, gui, MySQL, on, the, Using, website, your

The case of the overheating Mac Book Pro and Occam’s Razor

December 9, 2017 by Simon

Thank You William of Ockham (1287 – 1347) who coined the term “Ockham Razor” (“among competing hypotheses, the one with the fewest assumptions should be selected“) has helped my overheating Late 2012 Mac Book Pro and solve heat problems by removing dust, installing fan control software and re-applying thermal paste.

I knew summer was coming (Thanks to Troy Hunt), I was already accepting high temperatures on my Development laptop.

My Mac Book Pro was reaching 95~104c right after bootup and stayed there. The Intel processor (2.6Ghz i7) was doing a fantastic job at throttling the core down to protect the processor. At this point, I thought my laptop was powered by a Fitbit processor (everything was super slow).

Heat

I was using the TunaBelly temperature monitor software to see temps.

Cooler

I checked the Tunabelly Software website and found I was running an older version of the software (App Store version). I can now download a new version (direct from the author called TG Pro) and upgrade to a TB Pro ( see here ).

Old version (left screenshot below) and the new version (right screenshot below).

Old New

The new version has a nice drop-down menu (with stats).

Bar

TG Pro even has notifications when temps get too high 🙂

Notification

Why the High Temps

I had assumed that this heat was caused by extra CPU activity in the recent Apple High Sierra update (or Summer). I had noticed OSX locking up after I installed High Sierra. I had assumed that OSX 10.3 (High Sierra) was now pushing the CPU harder like Metal was pushing Apple GPU’s harder.

This morning the higher fan speed did not seem to help so I sat my Mac under a split system air conditioner (with no luck).

Air Con

Even with the split system air conditioner running (and pointing at the Mac), the Mac was still overheating.

I was a boiling frog in my own pot of assumptions. This bad assumption sent me down the wrong (potentially expensive path (new machine)).

My brain dived into solutions mode and concocted that High Sierra was crap and I needed a new iMac Pro with advanced cooling (the problem is Apple have not released them yet). The image of the new Apple iMac Pro (below) is really here just so I can drool at the new iMac Pro insides: https://www.apple.com/au/imac-pro/

iMac Pro

Simulated cooling in the new iMac Pro. **Mmmm Reality distortion field.**

After realising the high fan speeds and the air conditioner was no longer working I decided to open the laptop and check for dust. I had always assumed that because I don’t smoke, don’t let cats near the laptop and have a clean laptop area that dust was not the issue (hint: I was wrong).

Warning: Opening your laptop will void the warranty and you will probably kill it. You will need special screwdrivers, you will need to carefully follow anti-static procedures etc. I take no responsibility for your actions opening your devices.

P.S Watch Louis Rossman from https://www.rossmanngroup.com/ if you want to learn how to do things the right way. Here is a sample Louis Rossmann video (possibly NSFW) where he fixed a board (debugging).

Experience

I have pulled apart and modified heaps of PC’s before so I am comfortable opening up electronics and computers.

I have cut PCI-E 16x circuits down to PC’IE-8x to make them fit a server mainboard. Do not play with the insides of computers or electricity unless you know what you are doing period.

PCIE-8

I have setup watercooled PC’s and dealt with water leakages in the search for silence.

Water Cooling

Ok enough warnings.

Is Dust an Issue?

It seems I have ignored Ockham Razor (“among competing hypotheses, the one with the fewest assumptions should be selected”). Time to crack my laptop open and clean away any dust, Although I don’t have before photos of the dust (it was rather hidden near the exit port), dust was clearly the cause of the high temps.

It is not as if I am editing 4K videos here.

Troubleshooting: Start with the basics.

  • I should have considered dust inside the laptop,
  • I should have considered the old thermal paste needed replacing.

After Cleaning

After a quick clean with the kids paint brushes, tweezers, pin, the wife’s makeup brush and little rocket camera cleaning thingy I think I am finished.

After Cleaning Photos (Click for higher resolution)

fyi: These photos are just to show off my Late 2012 Mac Book Pro insides after cleaning.

fyi: SSD on the bottom left, memory in the middle and the heat pipe at the top of the photo.

Mainboard 01

fyi: SSD at the bottom, Fan and Heatsink near the top. The CPU fan was mostly clear and not an obvious overheating issue.

Mainboard 02

Most of the dust was on the heatsink at the end of the heat pipe (at the top of this photo, about 1/3 was blocked). The CPU fans had a slight coating of fine dust that I removed.

fyi: The heatsink below on the heat pipe was overheating due to 1/3 of the output duct being blocked.

Mainboard 03

fyi: This area (above) was rather dusty and possibly overheating.

Mainboard 04

Memory to CPU interconnect area (now clean).

Mainboard 05

fyi: Battery connector, battery power converter and fan.

Mainboard 06

Each primary air intake duct is now clean and free.

Removed Dust (and tools)

I removed this much dust from each of the fan’s outputs. The dust was mostly around the heat pipe area. The dust appears to be the same fibre as the protective case (Green Smart Neogreene case, that I slid the laptop into to protect it). This was not a lot of dust but was enough to block airflow, I will now clean my Mac Book every spring.

Dust and Tools

When I started up the laptop with the base on temps where better.

Conclusion

CPU Temperatures are about 40c cooler, GPU is about 35c cooler and CPU area is about 15c cooler. My Mac Book is no longer overheating and handles high CPU demands again without thermal throttling

Cooler

Here are idle and load temps with the older Tuna Belly Temperature Monitor (fyi: I now use the newer TunaBelly Software TG Pro temperature Software) and output from the Intel Power Gadget Utility.

Benchmark

As a precaution I have I configured TG Pro to Auto Boost fan speeds. This has helped keep the high temperatures under control (when they go above 80c).

fyi: TG Pro also allows you to set your own ramps in fan speeds, email alerts, sounds etc (review coming soon).

Auto

Summary

In future, I will ask myself what the simplest possible problem is (Occam’s Razor or use Rubber Duck Debugging) instead of dreaming up problems/solutions and yes Apple computers need maintenance too.

Hope this helps someone.

UPDATE After 1 week

Temperatures have crept back up with the heatwave in Australia. The CPU is now in the high 90’s again. I am now solely using the TG Pro software to mange fan speeds until the new machine. Sadly Apple deliver quiet computers over cool ones so I’ll install TG Pro on all future machines.

Auto

Another Thought

Should the thermal paste be replaced? If this was a PC I would have done this already.

I have ordered some thermal paste and will apply that to the GPU and CPU soon and update this post.

https://www.pccasegear.com/products/33322/thermal-grizzly-kryonaut-thermal-grease-1g/

I will try this when it arrives.

I could not wait and I did remove the stock thermal paste and applied some old (10-year-old) Vantec paste I had that was still good. I did have some Arctic Silver 5 (5x tubes) but it was 10 years old and gummy. The best paste I had was 1 tube of white silicone Vantec paste I had that came with a past PC heatsink. It was still the right consistency, not separated and spreadable.

Always test thermal paste and write the purchase date on the paste when your order i (it does go bad).

Thermal Compound

Removing my heatsink was easy (with the right screwdrivers) on my Mid 2012 MPB 15″ Retina. Consult ifixit.com or online guide to open your mac.

I removed these screws.

Screws

I removed the stock Mac Book heat spreader to reveal a terrible application of Apple stock thermal paste. The paste was very dry and the CPU die was not fully covered anymore.

Stock Paste

This was the paste under the heat sink. Dry and flakey.

This is the thickness of the MBP heatsink.

heatsink

I cleaned the old paste off the Geforce 650M GPU die.

Cleaned the GPU

I cleaned the Core i7 2.6Ghz CPU die too.

CPU Die

This is the size of the MBP heatsink.

MBP Heatsink

The Heatsink is not very thick.

Thickness

This was the cooler on the last Windows PC I built.

PC Cooler

I cleaned the old paste off the heatsink and applied the new (old) Vantec silicone paste using a new application method mentioned here. I will lift the heatsink in 1 week (when the new Grizzly compound arrives) and see if this new method is ok over the tried and tested grain of rice size dollop in the middle of the die.

I clean the base of the heatsink too

I reassembled the laptop and the results seemed to drop the CPU temps to 20c above ambient air temps. My CPU is no longer sitting in the high 90’s in the non-air-conditioned Aussie summer 🙂

I get the usual high CPU temp spikes but the temps quickly fall back down again.

Ambient temps are higher today (compared to past screenshots) so direct comparisons cannot be made.

Temps Now

I will wait for my order of Thermal Grizzly Kryonaut Thermal Grease 1g arrives and update this post.

Update: Applied Kyronaut – Ultra High-Performance Thermal Grease

Kyronaut - Ultra High Performance Thermal Grease

7mm of thermal grease

6mm of thermal grease

Old Silicone Grease Removal.

Old Silicone Grease Removal

I used this application method

Results

16c above CPU Proximity Temp at idle 🙂 The Silicone grease was 19c above ambient temps.

Results

Conclusion

I have no doubt the more aggressive TG Pro fan profiles and the Grizzly thermal grease application has brought life back to my old laptop. Idle temps are lower and when the system is pushed to high activity temperatures fall much sooner.

2018 Update

OMG, Apple still has NOT fixed the cooling on the Mac Book Pro (unless you plan to run it in a freezer)

I will never buy another Apple Mac Book Pro unless they solve the cooling issues.

Pro = Professional (not ProThetic)

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v2.0 Goodbye Apple, 2018 update

V1.9 Not editing 4K videos

v1.8 Applied Kyronaut – Ultra High-Performance Thermal Grease

v1.7 Applied old thermal paste information (before good thermal paste arrives).

V1.6 Added YouTube video

V1.5 Ordered some thermal paste.

V1.4 TG Pro, update, simulated cooling photo.

Filed Under: Occam's Razor Tagged With: and, Book, case, mac, Occam's, of, overheating, pro, Razor, the

Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin

December 2, 2017 by Simon

Below is my quick blog posts on using the EWWW IO ExactDN CDN plugin in WordPress to set up an ExactDN (Global Dis.cotribution Network (CDN)) to distribute images to my site’s visitors and shrink (optimize) images in posts.

I have blogged before on speeding up WordPress that has involved moving servers away from CPanel domains to self-managed servers (e.g on Digital Ocean or Vultr), using Lazy Load image plugins like BJ Lazy Load, Optimize images automatically in WordPress with EWWW.io and scaling and moving servers closer to your customers.

For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Today I am going to set up the https://www.ewww.io/resize/ ExactDN (CDN) delivery network. I am paying for this myself (and this is an unbiased review).

FYI: I use Ubuntu servers and not Windows.

Know your starting point. 

“If you fail to plan, you are planning to fail!” – Benjamin Franklin

In my original post about Speeding up WordPress, I used the webpagetest.org site to test my sites response times, I was getting an embarrassing 21 seconds load time and 6 second first-byte time. I have worked to speed up WordPress by moving WordPress to a self-managed server (away from CPanel), used the BJ Lazy Load plugin and the awesome ewww.io image optimize plugin, now I get about 4-5 seconds.

I am hoping adding a CDN will make things faster. My blog is delivering over two-thirds images, perfect for a CDN, this is why I am trying this out.

Image type cdn

TIP: Check where your customers/readers are located, and how many are New versus Returning customers? Do you need a CDN to deliver content (Images) that are closer to your customers/readers or do you need to move your web server somewhere else?  The more you know the more you can help them. Worst case you will be supporting a positive experience (and potentially turning a one time visitor into a returning visitor).

I looked at my Google Analytics data to see where my visitors are. Whether good or bad, they are all over the world (Hello)?

World

Other data is available in Google Analytics.  I can see the last few years growth is growing and I am getting more returning visitors, now is the time to ensure my site is ready for more traffic and returning visitors.

Data

Note: The fall in traffic in the Audience overview (right-hand side of the left image) is the unfinished month (not a reader fall off).

Personally, I set a goal to have a high page bounce rate of 90% be way lower (at present I am at about 80% and falling (good) and my page read time has gone from 40 seconds to 1 minute 40.  Every bit you can do will help create a positive experience and help your visitors. I can see from the data above the content is being read, I am building returning visitors and they are geographically spread out. A CDN will be great. After you know where your visitors are it is good to know the times of day that your visitors are hitting your site. Lucky for me it is spread evenly over a 24 hour period.

Data Quantity

FYI: My servers outgoing data (last 30 days), not huge but BJ LazyLoad and image optimization may be helping.

Outbound

Traffic Forecast

Do you know the forecasted growth of your website?

Site Growth

Measure Before Optimizing

Before I started to optimize my WordPress site (hosted on a shared CPanel server) I had the following Web Page Test score. I tested from Singapore as that’s was where my server was originally (and the closest to me).

TIP: Read more about Performance, First Byte, Start Render and Complete scores at my blog post here.

Web Page Performace Test Results (Before Optimizations):

  • Load Time: 23.672s
  • First Byte: 6.743s
  • Start Render: 11.8300s
  • Speed Index: 15024
  • Requests: 132/164 (Document complete v Fully Loaded)
  • Bytes: 3,346KB/3,454KB  (Document complete v Fully Loaded)

Quick Web Page Performace Score Card (Before Optimizations):

  • First Byte: F (I should have captured the subscores)
  • Keep Alive Enabled: F (I should have captured the subscores)
  • Compress Transfer: F (I should have captured the subscores)
  • Compress Images: A (I should have captured the subscores)
  • Cache Static Content: X (I should have captured the subscores)
  • Effective use of CDN: X (I should have captured the subscores)

My initial scores were bad across the range of tests (before optimisations). On the upside, I was manually compressing images with a tool on my desktop before uploading images and this showed an “A” but this scorecard overall was really bad.

Here are the results after Quick Optimizations (EWWW.io image compress, moved the server, reorganizing the site and Lazy Load Images)

Now I get these results after speeding up my site (after using the EWWW.io image resizing, reorganizing the site, minifying, lazy load images, moving servers etc.).

Web Page Performace Test Results (After Simple Optimizations):

  • Load Time: 8.823s (down 14.849s)
  • First Byte: 3.5533s (down 3.1897s)
  • Start Render: 5,4800s (down 6.35s)
  • Speed Index: 5594 (down 9430)
  • Requests: 73/76 (Document complete v Fully Loaded) (down 59/88)
  • Bytes: 848KB/855KB  (Document complete v Fully Loaded) (down 2,498KB/2,599KB)

Quick Web Page Performace Score Card (After Simple Optimizations):

  • First Byte: F (I should have captured the subscores)
  • Keep Alive Enabled: A (I should have captured the subscores)
  • Compress Transfer: A (I should have captured the subscores)
  • Compress Images: A (I should have captured the subscores)
  • Cache Static Content: B (I should have captured the subscores)
  • Effective Use of CDN: X  (I should have captured the subscores)

Tested my pagespeed tests

Even at 4 seconds web page “First Byte“, this is considered not good. My brain says I want sub 1 second, I doubt this is achievable with WordPress over thousands of miles away with SSL (read here about scalability).

I know https and non-geographically favourable servers add half a second to data. SSL will add processing overheads and latency period. If you only want speed don’t setup SSL but if you want SEO and security then setup SSL.

locationlocationlocation

WebPageTest.org test reveals there is no effective use of Content Delivery Networks (CDN) on fearby.com (that’s why I am about to install EWWW.io ExactDN).

ExactDN (Content Delivery Network)

I did try and set up a number of caching and CDN plugins in the past (e.g Max CDN, W3 Total Cache, WP Fastest Cache, Cache Enable, WP Rocket, WP Super Cache, etc.) but they either made results worse or were impossible to set up.

Now that EWWW.io has a CDN let’s give that a go.

What is EWWW.io’s ExactDN?

You can read more about EWWW.io’s two-pronged approach to delivering one plugin to A) “compress images” and B) “add a Content Delivery Network (CDN)” here: https://ewww.io/resize/

What is ExactDN

Ensure you read up about EWWW.io’s ExactDN here: https://ewww.io/resize/ . If you have not used EWWW.io check out my review of the EWWW.io image compression plugin here first.

Pre Signup

Purchasing and Installing ExactDN Exact DN

Login to your EWWW.io account (or signup then log in).

Signin

FYI: If you have used the Optimise images automatically in WordPress plugin from EWWW.io plugin before, then you will see past purchases here.

Signed in

Now you can go back to the EWWW.io ExactDN product page (https://ewww.io/resize/) and purchase a subscription (Make sure you are logged in with an EWWW.io account before you purchase ExactDN).

Pre purchase

I purchased an ExactDN monthly subscription for $9.00 monthly (with a $1 signup fee for the cloud compression service).

Order

Purchase confirmation screen.

Order

Post-purchase, I was advised to find my “Site Address (URL)” in WordPress and add it to EWWW.io Manage Sites screen.

Add site to the plugin

I now noticed I had a CDN for my domain ( fearby-com.exactdn.com ). Nice.

CDN Site Created

EWWW.io said to tick the CDN option in the Image Resize area of the EWWW.io plugin. But before I do that I  will update WordPress Core and WordPress Plugins before enabling the ExactDN as they are out of date.

Update

TIP: I update WordPress Core and WordPress plugins via command line (my guide here).

Backup WordPress (just in case)

It is always a good idea to backup your website, I logged into my Ubuntu server and checked the size of my site before backing it up.

du -hs ./www
513M    ./www

I copied the website folder (from “/www/” to “/www-backup”)

cp -rTv /www/ /www-backup

I confirmed the copied folder size (same, good)

du -hs ./www-backup/
513M    ./www-backup/

I dumped all databases with the MySQL dump command

mysqldump --all-databases > /my-sql-backup.sql -u root -p
> Enter password: ##############################

ls my-sql*.sql -al
-rw-r--r-- 1 removed removed 75605938 Dec  1 21:11 my-sql-backup.sql

While I was there I compress the SQL backup files (text files)

tar -czvf my-sql-backup.sql.tar.gz my-sql-backup.sql

Checking the CDN Status

I checked the DNS replication for fearby-com.exactdn.com with this DNS checker. It’s setup and ready to go across the globe 🙂

A quick CNAME check reveals its upstream provider is fearbycom-8ba8.kxcdn.com. Keycdn.com have been around since 2012. It is great that EWWW.io has paired with keycdn and included their image compression magic in a single WordPress plugin.

Configuring the EWWW.io plugin in WordPress.

Before you enable the CDN below do check the https://www.whatsmydns.net/ and see if your CDN has replicated around the world (Australia can be a bit slow at DNS replication from time to time). Do wait at least 10 minutes before proceeding.

Click the settings in your EWWW Image Optimizer Plugin.

Configure

Now click the Resize Settings tab and click Enable CDN in the EWWW Image Optimizer plugin screen.

Enable Exact DN

Don’t forget to click Save Changes

Save

Testing the CDN

https://www.webpagetest.org NOW reports that my site is using a CDN 🙂

FYI: I have always used webpage test from Singapore and I have done the same here to compare apples to apples.  Singapore is not a good test location in Australia (my server is in Australia) and it is 200ms away and the added layer of SSL adds latency to the connection (read here). This is a real-world test though (worst case).

CDN's Used:
fearby.com : 
fearby-com.exactdn.com : KeyCDN
fonts.googleapis.com : Google
fonts.gstatic.com : Google
static.addtoany.com : Cloudflare
www.googletagmanager.com : Google
pagead2.googlesyndication.com : Google
www.youtube.com : Google
adservice.google.com.sg : Google
adservice.google.com : Google
googleads.g.doubleclick.net : Google
www.google-analytics.com : Google
s.ytimg.com : Google
stats.g.doubleclick.net : Google
fearby-com.disqus.com : Fastly
ssl.google-analytics.com : Google

FYI: https://www.webpagetest.org does want other (all) static content to be on a CDN to give a higher score than 42 out 100, sorry I did not get a subscore before enabling the CDN.

I was expecting a green A here for CDN but Webpage Test has given me more items to investigate to ensure WordPress plugins are also fast (minify or move to CDN). It appears WordPress includes are my next target for optimizing.

Web Page Test indicates the following WordPress assets should also be in CDN.

FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/styles/ytprefs.min.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/svgxuse.js?ver=1.1.21
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/scripts/ytprefs.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/add-to-any/addtoany.min.js?ver=1.0
FAILED - https://fearby.com/wp-content/plugins/crayon-syntax-highlighter/js/min/crayon.min.js?ver=_2.7.2_beta
FAILED - https://fearby.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=1.4.1
FAILED - https://fearby.com/wp-includes/js/jquery/jquery.js?ver=1.12.4
FAILED - https://fearby.com/wp-content/plugins/contact-form-7/includes/js/scripts.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/menu/superfish.args.js?ver=2.5.3
FAILED - https://fearby.com/wp-content/plugins/crayon-syntax-highlighter/css/min/crayon.min.css?ver=_2.7.2_beta
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/skip-links.js?ver=2.5.3
FAILED - https://fearby.com/wp-includes/js/comment-reply.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/hoverIntent.min.js?ver=1.8.1
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/menu/superfish.js?ver=1.7.5
FAILED - https://fearby.com/wp-includes/js/jquery/ui/tabs.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/themes/news-pro/js/global.js?ver=3.2.2
FAILED - https://fearby.com/wp-content/themes/news-pro/js/responsive-menus.min.js?ver=3.2.2
FAILED - https://fearby.com/wp-includes/js/jquery/ui/core.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/themes/news-pro/style.css?ver=3.2.2
FAILED - https://fearby.com/wp-includes/js/jquery/ui/widget.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/themes/news-pro/js/jquery.matchHeight.min.js?ver=3.2.2
FAILED - https://fearby.com/wp-content/plugins/contact-form-7/includes/css/styles.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/scripts/fitvids.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/wp-embed.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/bj-lazy-load/js/bj-lazy-load.min.js?ver=2
FAILED - https://fearby.com/wp-content/plugins/disqus-comment-system/media/js/count.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/wp-emoji-release.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/wp-seo-html-sitemap/style.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/symbol-defs.svg
FAILED - https://fearby.com/wp-includes/css/dashicons.min.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/themes/news-pro/images/favicon.ico
FAILED - https://fearby.com/wp-content/plugins/genesis-tabs/style.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/css/style.css?ver=2.0.1
FAILED - https://fearby.com/wp-content/plugins/add-to-any/addtoany.min.css?ver=1.14

It would be a huge effort trying to read and keep static plugin and theme related files in a CDN. I’ll ask the EWWW.io developer to see if this is possible in a future version (that would be nice). The developer did promptly point me here to opt into using the CDN to deliver CSS, JS etc.

Is that it? It can’t be that simple!

I can load my site at https://fearby.com and my images load from https://fearby-com.exactdn.com 🙂 I am impressed, blog posts are now loading images from the CDN network and I did not have to edit posts. I did have to subscribe to ExactDN and tick a checkbox in WordPress though.

Here is a sample image (my largest) from this blog post.

https://fearby-com.exactdn.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-3.jpg?strip=all&quality=60&ssl=1

fearby.com WebPAGE TEST Results – Singapore

Web Page Performace Test Results – Singapore (After Setting up the EWWW.io ExactDN):

  • Load Time: 4.177s (down 4.646s from previous optimizations ((My Original load times were 23.672s))
  • First Byte: 2.240s (down 1.3133s from previous optimizations (My Original First Byte: 6.743s))
  • Start Render: 2.800s (down 2.68s from previous optimizations (My Original Start Render: 11.8300s))
  • Speed Index: 3009 (down 2585 from previous optimizations (Speed Index: 15024))
  • Requests: 67/68 (Document complete v Fully Loaded) 
  • Bytes: 535KB/538KB  (Document complete v Fully Loaded)

Quick Web Page Performace Score Card – Singapore (After Setting up the EWWW.io ExactDN):

  • First Byte: F (I should have captured the subscores). 
  • Keep Alive Enabled: A (I should have captured the subscores).
  • Compress Transfer: A (I should have captured the subscores).
  • Compress Images: A (I should have captured the subscores).
  • Cache Static Content: B (I should have captured the subscores).
  • Effective Use of CDN: X (I should have captured the subscores). plugins now need to be on a CDN.

The Web Page Test site does give detailed scores and recommendations if you scroll down or click the score car (A-F). Do read the recommendations and see what you may need to do next.  I am happy that I now have a CDN via EWWW.io. Clicking the first byte and CDN buttons at Web Page Test reveal sub-scores to allow you to see if you have made improvements, regrettably, I did not know of and capture sub-scores until after installing the CDN (I suggest you do).

Full Dump.

Details
First Byte Time (back-end processing): 0/100
2240 ms First Byte Time
ms Target First Byte Time

Use persistent connections (keep alive): 100/100

Use gzip compression for transferring compressable responses: 100/100
404.6 KB total in compressible text, target size = 404.6 KB - potential savings = 0.0 KB

Compress Images: 100/100
98.7 KB total in images, target size = 98.7 KB - potential savings = 0.0 KB

Use Progressive JPEGs: 100/100
97.4 KB of a possible 97.4 KB (100%) were from progressive JPEG images

Leverage browser caching of static assets: 88/100
FAILED - (No max-age or expires) - https://fearby-com.disqus.com/count.js
FAILED - (8.2 minutes) - https://www.google-analytics.com/analytics.js
FAILED - (15.0 minutes) - https://www.googletagmanager.com/gtag/js?id=UA-93963-1
FAILED - (60.0 minutes) - https://pagead2.googlesyndication.com/pagead/js/adsbygoogle.js
WARNING - (1.1 hours) - https://www.google-analytics.com/ga.js
WARNING - (12.0 hours) - https://pagead2.googlesyndication.com/pub-config/r20160913/ca-pub-9241521190070921.js
WARNING - (24.0 hours) - https://fonts.googleapis.com/css?family=Raleway%3A400%2C700&ver=3.2.2
WARNING - (2.0 days) - https://static.addtoany.com/menu/page.js
WARNING - (4.5 days) - https://s.ytimg.com/yts/jsbin/www-widgetapi-vflUJbESo/www-widgetapi.js

Use a CDN for all static assets: 46/100
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/css/style.css?ver=2.0.1
FAILED - https://fearby.com/wp-content/plugins/add-to-any/addtoany.min.css?ver=1.14
FAILED - https://fearby.com/wp-content/plugins/crayon-syntax-highlighter/css/min/crayon.min.css?ver=_2.7.2_beta
FAILED - https://fearby.com/wp-content/themes/news-pro/style.css?ver=3.2.2
FAILED - https://fearby.com/wp-content/plugins/contact-form-7/includes/css/styles.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/wp-seo-html-sitemap/style.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/css/dashicons.min.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/contact-form-7/includes/js/scripts.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/comment-reply.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/crayon-syntax-highlighter/js/min/crayon.min.js?ver=_2.7.2_beta
FAILED - https://fearby.com/wp-includes/js/hoverIntent.min.js?ver=1.8.1
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/menu/superfish.js?ver=1.7.5
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/menu/superfish.args.js?ver=2.5.3
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/skip-links.js?ver=2.5.3
FAILED - https://fearby.com/wp-content/themes/news-pro/js/jquery.matchHeight.min.js?ver=3.2.2
FAILED - https://fearby.com/wp-content/themes/news-pro/js/responsive-menus.min.js?ver=3.2.2
FAILED - https://fearby.com/wp-includes/js/jquery/ui/tabs.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/plugins/add-to-any/addtoany.min.js?ver=1.0
FAILED - https://fearby.com/wp-content/themes/news-pro/js/global.js?ver=3.2.2
FAILED - https://fearby.com/wp-includes/js/jquery/ui/core.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-includes/js/jquery/ui/widget.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/plugins/bj-lazy-load/js/bj-lazy-load.min.js?ver=2
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/scripts/fitvids.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/wp-embed.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/disqus-comment-system/media/js/count.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/styles/ytprefs.min.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/wp-emoji-release.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/symbol-defs.svg
FAILED - https://fearby.com/wp-content/plugins/genesis-tabs/style.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/jquery/jquery.js?ver=1.12.4
FAILED - https://fearby.com/wp-content/themes/news-pro/images/favicon.ico
FAILED - https://fearby.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=1.4.1
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/svgxuse.js?ver=1.1.21
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/scripts/ytprefs.min.js?ver=230d722825ddde2688088d563a906075

CDN's Used:
fearby.com : 
fonts.googleapis.com : Google
fonts.gstatic.com : Google
www.googletagmanager.com : Google
static.addtoany.com : Cloudflare
fearby-com.exactdn.com : KeyCDN
pagead2.googlesyndication.com : Google
www.youtube.com : Google
www.google-analytics.com : Google
adservice.google.com : Google
adservice.google.com.sg : Google
googleads.g.doubleclick.net : Google
s.ytimg.com : Google
fearby-com.disqus.com : Fastly
stats.g.doubleclick.net : Google

GT Metrix Page Speed Score

https://gtmetrix.com is giving a good score across the board (86%). It hints I should optimize Javascript files and “Remove query strings in static files” as some proxy servers do not cache URLs with “?” in them. That is (fortunately) not a problem with ExactDN, as the servers are configured to properly handle query strings.

GTMetrics

Gtmetrix.com does give some optimisation tips too (However, it does report low CDN optimizations if a single file is not delivered over a CDN).

I do like Gtmetrix.com email reports, you can see if your site performance is degrading.

GTMetrix Summary

Page Speed Insights

I am now looking at the Google PageSpeed Insights test for things to fix next. I think I can tweak my NGINX a little (adding caching). Read more about Google Page Speed Insights here.

fyi: My Google Page Speed Insights score (desktop) along with another local big corporate site I tested.

Page Insight

I am happy with 82 🙂

It appears if you want to get 100% you need to get..

  • Initial Response under 100ms
  • Animate, produce frame in under 10ms
  • Idle maximize idle time
  • Deliver all content in under 1000ms

Getting More from ExactDN by caching CSS and JS files.

I added the following to my /www/wp-config.php file as mentioned here: http://docs.ewww.io/article/47-getting-more-from-exactdn This will serve more resources from the CDN

define( 'EXACTDN_ALL_THE_THINGS', true );

I refreshed my site in a browser and now CSS, JS and fonts are loaded from the CDN too.

All

To reduce files served from my website I re-enabled the Fast Velocity Minify plugin in WordPress and pointed it at my CDN (https://fearby-com.exactdn.com/)

cdn-minify

Now I am getting a GTMatrix score of Page Speed Score = 93% (A) and YSlow Score  = 74% (C) and a 2.6-second load and a much lower 35 requests (post minification and pointing the minified files to the CDN).

WOW.

Minifiy

And WebPageTest.org is reporting Effective use of CDN 🙂 Awesome.

Effective use of CDN

I think I’m done (ExactDN CDN and the image optimiser and other tweaks have worked its magic).

How do I compare to other sites?

Given Apple, Microsoft and NBC speed scores are worse than mine I’m happy for now.  🙂

GTMetricx Graphs

GT Metrics graphs show the improvement, YSlow report does indicate I can reduce the website DOM elements to speed things up and tweak plugins but I’d rather keep my design for SEO and not play with plugins or they might break.

I could tweak the server side (NGINX/MySQL/Cache or DNS) but I don’t need too.

Conclusion

I still can’t believe setting up a CDN is a one-click solution and adding a wp-config.php lin (after you subscribe). Best of all the BJ Lazy Load plugin still works. I am very happy with the EWWW.io ExactDN. I now have a CDN and it has lowered my First Byte time, Start Render Times, Speed Index time (on more than just the front page) and all I did was subscribe and tick a checkbox.

It is nice that EWWW.io have not charged for Data delivered from the CDN network on top of a monthly subscription at this stage but they have left it open in the terms and conditions. It is important to ensure a service can sustain itself, so this is a good sign. FYI: my favorite Agile toolkit (Atlaz.io) that I reviewed here closed shop today so it is wise for aaS shops to plan ahead. I know how hard it can be to spin up servers and stuff and allocate times to keep them running. Good luck EWWW.io!

Also, I now have a clear set of steps (below) to resolve other non-optimized assets that are outside of the CDN.

Full Report

Thanks to EWWW.io and Web Page Test for helping make my site faster.

I doubt WordPress and my “down under”  server location can get me to under 1000ms but I will try.

Read on here to see how Cloudflare can increase your site’s performance.

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

DNS Improved

Don’t forget your sites performance/SEO and security.

Website

Update November 2018

I have a much faster loading website after moving it to a new host, read my guide here.

GTMetrix After YSlow

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v2.0 November 2018 Update

v1.9 added Cloudflare, SEO, Google Page Speed Insights test and future optimization, page insight speed image, added GTMetrics comments, growth image, gtmetrix.com reports (typos are free), updated change is domain to .com, added EXACTDN_ALL_THE_THINGS, 93 page speed, minify, compare, graph, fixed typos, tidied up the conclusion.

Filed Under: CDN, ExactDN Tagged With: and, cdn, compression, ewww.io, ExactDN, image, plugin, Speeding, the, up, with, wordpress

Security checklist for securing a self-managed Ubuntu server in the cloud

November 2, 2017 by Simon

Below is a (perpetually updated) security checklist for securing a self-managed Ubuntu server. Recently WordPress released patch v4.8.3  that fixed some SQL injection issues.  Is your OS, Database, Web Server, OS and software up to date?

Although I have recently blogged about securing Ubuntu in the cloud, and running a server Audit with Lynus,  this new post is really about obtaining a mindset change and allocating time (each week) to ensure your self-managed servers and software is kept up to date. You can easily list down the actions you need to follow but keeping a system up to date is hard work. Sites like www.shodan.io will reveal what servers or services are vulnerable, let software updates lapse long enough and an open exploit may open a hole to your system.  It only takes minutes to set up a $2.5  a month Ubuntu server with Vultr, $5 a month Digital Ocean Server or AWS server but you need to maintain it.

I highly recommend that you watch the following video that highlights the need for even minor vulnerabilities to be patched asap. If you leave one minor vulnerability open you will give hackers a foothold into your system.

Follow @jawache on twitter.

Troy Hunt has a great post about the simplicity of hacking. Hacking is child’s play.

General Security Checklist

  • Do Setup a Firewall and only allow needed ports to accept data (use tools like Portscan and Shodan.io to find open ports).
  • Use least access permissions (on NGINX, PHP and MySQL processes).
  • Use strong unique passwords for every service (1Password and sites like Gibson Research Corp have password generators, use www.howsecureismypassword.net to check tour passwords strength)
  • Enable logging.

Find log files on your system:

cd /
find -iname "*.log"

Output (handy logs to review):

./var/log/mongodb/mongod.log
./var/log/fail2ban.log
./var/log/mysql/error.log
./var/log/ufw.log
./var/log/lynis.log
./var/log/dpkg.log
./var/log/nginx/error.log
./var/log/nginx/nginxcriterror.log
./var/log/nginx/access.log
./var/log/audit/audit.log
./var/log/php7.0-fpm.log
./var/log/mail.log
./backup/backup.log
./scripts/boot.log
etc
  • Enable brute force detection and banning (fail2ban etc) Read more here.
  • Secure folders with service accounts.
  • Do secure software (e.g WordPress Wordfence)
  • Do use SSL Certificates (and use modern cyphers and test with https://www.ssllabs.com/ssltest/)
  • Monitor SSL vulnerabilities.
  • Do a Lynis security report.
  • Install a Virus scanner (read here).
  • Secure MySQL/Databases.

First, find the version of MySQL

mysql --version
mysql  Ver 14.14 Distrib 5.7.19, for Linux (x86_64)

Read the official MySQL manual here and security guidelines here.

Read this Digital Ocean guide on securing MySQL.

  • Other: _______

Application (coding) checklist

Retain and protect information.

  • Disable errors (PHP: turn off or here)
  • Enable logging (web server, PHP and or node)
  • Sanitize data (never trust uses data) in code (see how to do this in PHP 7)
  • Do no develop on production boxes (use parameterised queries and follow OWASP application security procedures.
  • Read the OWASP Secure Coding Practices – Quick Reference Guide

Infrastructure

Plan for the worst, hope for the best.

  • Use the latest Long Term Support (LTS) version or Ubuntu
  • Update packages

View app packages (Ubuntu 16.04) with updates

sudo /usr/lib/update-notifier/apt-check -p

View app packages (Ubuntu 16.04) with updates

apt list --upgradable

To update packages type (remember to backup data and config files first)

sudo apt-get update && sudo apt-get upgrade

Among other things, you will see the following information

The following packages will be upgraded:
  binutils certbot cracklib-runtime curl distro-info-data grub-common grub-pc grub-pc-bin grub2-common initramfs-tools initramfs-tools-bin initramfs-tools-core libapache2-mod-php7.0
  libcrack2 libcurl3 libcurl3-gnutls libgnutls-openssl27 libgnutls30 libicu55 libpam-systemd libsystemd0 libudev1 linux-firmware linux-libc-dev lshw mdadm mysql-client-5.7
  mysql-client-core-5.7 mysql-common mysql-server mysql-server-5.7 mysql-server-core-5.7 nodejs php7.0 php7.0-cli php7.0-common php7.0-curl php7.0-dev php7.0-fpm php7.0-gd php7.0-imap
  php7.0-intl php7.0-json php7.0-ldap php7.0-mbstring php7.0-mysql php7.0-opcache php7.0-pgsql php7.0-phpdbg php7.0-pspell php7.0-readline php7.0-recode php7.0-snmp php7.0-tidy
  php7.0-xml php7.0-zip python-acme python-certbot python-certbot-nginx python-cffi-backend python-chardet python-idna python-six python3-chardet python3-distupgrade python3-six
  python3-update-manager systemd systemd-sysv ubuntu-release-upgrader-core udev update-manager-core wget

Show available updates

/usr/lib/update-notifier/apt-check --human-readable
0 packages can be updated.
0 updates are security updates.
  • Only work on code checked into GitHub or BitBucket (You will thank me when data or servers disappear).
  • Backup configuration files or backup to remote servers (my rsync guide here)
  • Use snapshots of VM’s.
  • Use Green/Blue server deployments (toggle one server a Prod and the other and Dev/Test and have one ready for a hot spare). Digital Ocean has a good guide here.
  • Consider forcing Content Security Polic and Public Key Pinning or at least using LetsEncrypt SSL certificates.
  • Take Snapshots of VM’s (automate)
  • Backup MySQL databases:
sudo mysqldump --all-databases > /backup/dump-$( date '+%Y-%m-%d_%H-%M-%S' ).sql -u root -p

Other Useful Linus Terminal Commands.

Mindset/Culture

Dedicate time to securing your site.

  1. Spend one day a week (or automate) the updating of the OS/Software (no excuses).
  2. Follow people on twitter and subscribe to newsletters of those that are security conscious

Don’t forget to read securing Ubuntu in the cloud blog post here.

And check out the extensive Hardening a Linux Server guide at thecloud.org.uk: https://thecloud.org.uk/wiki/index.php?title=Hardening_a_Linux_Server

More to come..

Donate and make this blog better


Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.2 added link to Hardening Linux Server link

v1.1 added @jawache link

Short (Article):

Filed Under: OS, Secure, Security, VM Tagged With: a, checklist, cloud, for, in, securing, security, self managed, server, the, ubuntu

  • « Go to Previous Page
  • Go to page 1
  • Go to page 2

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) IoT (9) LetsEncrypt (7) Linux (21) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) Performance (6) PHP (13) Scalability (12) Scalable (14) Security (45) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (45) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT