Code, Security and Server Stuff

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View All Posts.

with

Scanning WordPress with Gravity scan for free to detect Supply Chain Attacks and WordPress malware

by

A recent trend with some WordPress Plugins (and Google Chrome Extensions) is malicious parties will purchase existing plugins (extensions) and inject malicious code into new versions to infect sites and software, this is called “Supply Chain Attacks”. This is a personal unpaid review of Gravity Scan.

Advertisement:


Update Feb 2018: Gravity Scan is shutting down 🙁

Recently WordFence wrote a blog post about Supply Chain Attacks found cases where older plugins are being purchased by malicious people in order to infect WordPress sites. WordPress CMS apparently runs 29% of the websites on the internet. Wordfence is a firewall and Gravity Scan is vulnerability scanner, they complement each other.

I have blogged here about setting up WordPress via Command line and setting up an Ubuntu server for as low as 42.5 a month on Vultr.

What can you do to protect your WordPress sites from “Supply Chain Attacks”? First, install the WordFence plugin (I blogged about it here). Wordfence gives you a great set of security settings and reports to keep your site safe. The Wordfence dashboard page on your site is a good place to stay up to date.

WordFence is a Firewall, Gravity scan is a vulnerability and malware scanner. Read more here.

wordfence dashboard

Gravity Scan

Gravity scan is also made by the WordFence people to enable external audits and reports.

Gravity Scan Website

Sign up at https://www.gravityscan.com/ Verify your email and log in.

At login, you will be prompted to add a domain.

Add A Site

Scan

tip: You may want to whitelist Gravity scan servers. Read my guide about securing Ubuntu in the cloud.

A site scan will automatically be started.

Scan Started

Scan Results

Scan Results

Post Scan Actions

Speed Up future scans by downloading the Install Gravity Scan Accelerator (by clicking “Not instaled” under “Accelerator” in scan results) and follow the instructions to download, upload and verify the accelerator.

Install Gravityscan Accelerator

Read the Gravity Scan Accelerator Install Instructions here.

tip: I had to run the following command to make the

Accelerator Installed

I also clicked “Trust Badge” link and added the script code to my site and verified it.

Trust Badle

I now have a scan badge in my site footer.

Future Scans

Future scans are all good to go.

Ready

New Scan Options

New Scan Options

It looks as if the accelerator gives more server-side verifications of checks of WordPress and PHP versions etc.

Go Pro

Gravity Scan also offers a non-free (paid) version where you can enable more options, enable scan schedules and set up SMS alerts and more for $4.95 a month per site.

Go Pro

To be honest I am happy with performing manual scans and I’d rather pay for a premium WordFence subscription first.

The Catch

Hang on Gravity scan requires a Pro membership to see High and Critical issues 🙁

Critical Issues

I decided not to go pro to reveal issues.

A few months later

I started receiving scan results with severity Critical (but I can’t see results until I start a trail (and enter payment details)).

Issues

Time to start a trial

I started a trial and full details were shown, the critical error was my fault

Gone Pro

Critical Issue

This was my fault, I left a previous version of WordPress in a subfolder from when I moved the site to a self-managed server. A quick few Linux commands later (removed) and this was fixed.

old files visible

High Issue

Publically accessible file (fixed with a chmod command)

File Visible

Current Scan Results

Current

Remote Scan Options

Daily Scans, Alter levels, Malware, Vulnerability and status checks. Definitely, install the Accelerator as it found my local backup of WordPress.

Manage Remote

Pros

  • Found a publically readable file
  • Found a past copy of my WordPress site (and all known issues with the old WordPress backup).
  • Can setup daily remote scans.

Cons

  • You have to go pro.
  • Cant read my NGINX version (“Nginx version not detected, Gravityscan is unable to detect any associated vulnerabilities.“). I logged a ticket. Surely they can add a shell to”nginx -v” to the scan accelerator.
  • No word fence discount bundle?
  • Gravity scan and Word fence on twitter are slow to respond.

More to come.

More Reading

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

V.1.5 Gravity Scan shutting down

v1.4 Added remote scan options

v1.3 Pros and Cons and current results.

v1.2 Added more

v1.1 Fixed a few issues

v1.0 Initial Version

How to upload and download files to a server with sftp (over the SSH protocol)

by

I have blogged about http://c9.io before and how it makes managing a remote Ubuntu server easier. Recently AWS acquired C9 and integrated it into AWS. This has triggered me to find a more open/free way to connect to my servers. I like AWS but I can tell that C9 will someday block you from talking to non-AWS servers.

Advertisement:



C9 Aquisition

C9 on AWS: https://aws.amazon.com/cloud9/?origin=c9io

C9 Aquisition: https://c9.io/announcement

I have blogged about connecting to an AWS EC2 instance with C9 (before AWS acquired C9), The quickest way to set up a scalable development IDE and web server when I was using Digital Ocean Servers, How to buy a new domain (dedicated server from digital ocean) and add an SSL certificate from NameCheap but when I moved to Vultr serves I decided to ditch C9 and save $9 a month.  I got used to setting up an SSH connection, using rmate to edit remote files locally with Sublime and SSH’ing into the box with vSSH on OSX but nothing replaced the C9 file management experience.

C9 was a good IDE (but recent price hikes and AWS purchase have made me cautious).

C9

Uploading files without C9 to Ubuntu

Uploading files from an OSX laptop to a remote Ubuntu server is possible with the SCP command

Server Actions (to upload the old way, with scp)

There has to be a better way

I have used the Forklift 3 program before (my review here) and in recent google searches, it was suggested that Forklift best SFTP program on OSX. Yay. This unix.stackexchange.com thread mentions the difference between SFTP and SSH.

Below are the steps to use SFTP in Foftlift 3 on OSX to connect to a Vultr server (you could use Digital Ocean). Ensure you have a working SSH connection setup from your server to your local OSX.

Setting up an SFTP Connection in Forklift 3

Read my Forklift 3 guide here first.

Update your Forklift to the latest version.

Add a Favorite in Forklift 3

Add Favorite

Name the connection

Name the connection

In Forklift edit the connection

Edit

Specify SFTP and your servers working SSH username and password, and “/” folders

Settings

Save.

Double Click the entry to connect to your server.

Connected

To upload or download files with Forklift over SFTP, simply drag and drop.

Upload

Simple

Upload

You can now use Forklift to navigate folders, delete files, set remote file permissions all from a GUI. You can still use SSH CLI and or Sublime.

Quik Edit works too.

Quick Edit

Nice, this gives me a convenient GUI way to upload, download and edit files instead of via CSP/SSH/CLI.

Now is there an SFTP plugin for Sublime?

More to come soon.

 

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial Draft

 

Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl

by

Below is my quick guide to showing how I recover lost files on a removable (or internal drive) with software on Windows. This is not a paid advert but this is what works for me (and has worked with everything I have thrown at it). Tell me what works for you (especially on OSX).

Advertisement:


“Helping one person might not change the world but it could change the world for one person” – Buddha.

I hope this guide helps someone, I have never seen someone happier than when I have been able to restore lost files or photos for someone. Not everyone backs up data (read my guide here).

Before you start

Don’t touch (format, access or install software) onto drives and USB drive that you want to restore files from or you may overwrite files on that drive. Doing this will potentially block the ability to restore files. Electronics die, software gets corrupted, this can be complex so If in doubt seek professional help and advice before proceeding. This is advice only and it’s your call with your data, Restoring files on damaged magnetic or USB drives can damage the data storage platter or storage chips, so proceed at your own risk. (legal disclaimer over, good luck).

More on how file systems and storage work

How File Systems Work

How the File Allocation Table Works

Installing DIY Data Recovery

The DIY Data Recovery software can restore images (photos) from a UBS, SD card or files (images or non-images) from data or operating system drives. If you have a system or data drive from a PC you can plug it into another PC running DIY Dat Recovery and scan and restore from the possibly bad drive to a good host drive, never restore files to a bad drive from a bad drive. I am an Apple Mac user now but only know of this Windows method for restoring files, please let me know of an Apple Mac way to restore files on FAT, NTFS and other file systems please.

I usually connect USB drives directly or connect SATA drives via a small caddy

On a working Windows computer install DIY Data Recovery software from here.

Install

Plug in your dead USB or another drive to restore files from (do not interact with the drive once Windows can see it).

Insert Drive

Do not format or modify the drive if prompted (cancel this screen).

Dont Format

Sometimes Windows will offer to fix it (do not interact with this dialog).

From memory, you can follow the steps below on a free trial version but you will need a paid version (serial number) to actually restore files. The free trial will show files that it can restore though. I paid about $60 for the software a few years ago, it is currently 59 EUROS. Enter your serial number if you have one.

Register iRecovery

After you register it (or continue with the free trial) select the drive to scan for files and click Next.

Select Drive

Here is another unreadable SATA drive

DIY Data Recovery will scan each sector and try and find files. On a smaller healthy drive, the scan won’t take long, on a larger drive (e.g a 2TB NTFS drive it may take 18+ hours).

In this case, we can see the USB drive has two bad sectors (red squares) at the front of the USB stick in the file system sectors (green squares). This possibly happened when the USB was pulled out when the operating system was writing to the USB stick. If the USB was dying the bad sectors would be at random positions.

Data fragments are listed as blue squares.

Let it scan

Here is another sample scan results

When the scan is done click ‘Save” to save the found sectors state (saving the state can save you time later).

Disclaimer: Clicking save does not save lost files it just saves the current scan state.

choose Files to recover

You can click the legend button to see the state of each file (Not processed, Presumably valid, Invalid, bad sectors etc)

Tick the files you want to be restored.

Legend

FYI: You will see some files you know, some files you forgot and some weird files. Restoring files is hit and miss and you may never see files again (do backup). Benjamin Franklin once said “Failing to plan is planning to fail”, You can also say “Failing to backup is guaranteeing lost data”.

Restoring Files

Do

  • Create a folder on your working Desktop to restore files to.
  • Do increase Read and Write cache
  • Do set longer Timeouts (e.g 1000, 5000, 10000, 60000)

Restore

Click “Start copying the selected files” to restore.

restore

Hopefully, you will have files restored, if not seek professional help. I’d recommend keeping the corrupt or source drive in a drawer, you may be able to use other tools to restore more files at a later date.

Restored Files

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

V1.1 Added more screenshot from a second DATA drive scan

v1.0 Initial Post

Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin

by

Below is my quick blog posts on using the EWWW IO ExactDN CDN plugin in WordPress to set up an ExactDN (Global Dis.cotribution Network (CDN)) to distribute images to my site’s visitors and shrink (optimize) images in posts.

Advertisement:



I have blogged before on speeding up WordPress that has involved moving servers away from CPanel domains to self-managed servers (e.g on Digital Ocean or Vultr), using Lazy Load image plugins like BJ Lazy Load, Optimize images automatically in WordPress with EWWW.io and scaling and moving servers closer to your customers.

For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Today I am going to set up the https://www.ewww.io/resize/ ExactDN (CDN) delivery network. I am paying for this myself (and this is an unbiased review).

FYI: I use Ubuntu servers and not Windows.

Know your starting point. 

“If you fail to plan, you are planning to fail!” – Benjamin Franklin

In my original post about Speeding up WordPress, I used the webpagetest.org site to test my sites response times, I was getting an embarrassing 21 seconds load time and 6 second first-byte time. I have worked to speed up WordPress by moving WordPress to a self-managed server (away from CPanel), used the BJ Lazy Load plugin and the awesome ewww.io image optimize plugin, now I get about 4-5 seconds.

I am hoping adding a CDN will make things faster. My blog is delivering over two-thirds images, perfect for a CDN, this is why I am trying this out.

Image type cdn

TIP: Check where your customers/readers are located, and how many are New versus Returning customers? Do you need a CDN to deliver content (Images) that are closer to your customers/readers or do you need to move your web server somewhere else?  The more you know the more you can help them. Worst case you will be supporting a positive experience (and potentially turning a one time visitor into a returning visitor).

I looked at my Google Analytics data to see where my visitors are. Whether good or bad, they are all over the world (Hello)?

World

Other data is available in Google Analytics.  I can see the last few years growth is growing and I am getting more returning visitors, now is the time to ensure my site is ready for more traffic and returning visitors.

Data

Note: The fall in traffic in the Audience overview (right-hand side of the left image) is the unfinished month (not a reader fall off).

Personally, I set a goal to have a high page bounce rate of 90% be way lower (at present I am at about 80% and falling (good) and my page read time has gone from 40 seconds to 1 minute 40.  Every bit you can do will help create a positive experience and help your visitors. I can see from the data above the content is being read, I am building returning visitors and they are geographically spread out. A CDN will be great. After you know where your visitors are it is good to know the times of day that your visitors are hitting your site. Lucky for me it is spread evenly over a 24 hour period.

Data Quantity

FYI: My servers outgoing data (last 30 days), not huge but BJ LazyLoad and image optimization may be helping.

Outbound

Traffic Forecast

Do you know the forecasted growth of your website?

Site Growth

Measure Before Optimizing

Before I started to optimize my WordPress site (hosted on a shared CPanel server) I had the following Web Page Test score. I tested from Singapore as that’s was where my server was originally (and the closest to me).

TIP: Read more about Performance, First Byte, Start Render and Complete scores at my blog post here.

Web Page Performace Test Results (Before Optimizations):

  • Load Time: 23.672s
  • First Byte: 6.743s
  • Start Render: 11.8300s
  • Speed Index: 15024
  • Requests: 132/164 (Document complete v Fully Loaded)
  • Bytes: 3,346KB/3,454KB  (Document complete v Fully Loaded)

Quick Web Page Performace Score Card (Before Optimizations):

  • First Byte: F (I should have captured the subscores)
  • Keep Alive Enabled: F (I should have captured the subscores)
  • Compress Transfer: F (I should have captured the subscores)
  • Compress Images: A (I should have captured the subscores)
  • Cache Static Content: X (I should have captured the subscores)
  • Effective use of CDN: X (I should have captured the subscores)

My initial scores were bad across the range of tests (before optimisations). On the upside, I was manually compressing images with a tool on my desktop before uploading images and this showed an “A” but this scorecard overall was really bad.

Here are the results after Quick Optimizations (EWWW.io image compress, moved the server, reorganizing the site and Lazy Load Images)

Now I get these results after speeding up my site (after using the EWWW.io image resizing, reorganizing the site, minifying, lazy load images, moving servers etc.).

Web Page Performace Test Results (After Simple Optimizations):

  • Load Time: 8.823s (down 14.849s)
  • First Byte: 3.5533s (down 3.1897s)
  • Start Render: 5,4800s (down 6.35s)
  • Speed Index: 5594 (down 9430)
  • Requests: 73/76 (Document complete v Fully Loaded) (down 59/88)
  • Bytes: 848KB/855KB  (Document complete v Fully Loaded) (down 2,498KB/2,599KB)

Quick Web Page Performace Score Card (After Simple Optimizations):

  • First Byte: F (I should have captured the subscores)
  • Keep Alive Enabled: A (I should have captured the subscores)
  • Compress Transfer: A (I should have captured the subscores)
  • Compress Images: A (I should have captured the subscores)
  • Cache Static Content: B (I should have captured the subscores)
  • Effective Use of CDN: X  (I should have captured the subscores)

Tested my pagespeed tests

Even at 4 seconds web page “First Byte“, this is considered not good. My brain says I want sub 1 second, I doubt this is achievable with WordPress over thousands of miles away with SSL (read here about scalability).

I know https and non-geographically favourable servers add half a second to data. SSL will add processing overheads and latency period. If you only want speed don’t setup SSL but if you want SEO and security then setup SSL.

locationlocationlocation

WebPageTest.org test reveals there is no effective use of Content Delivery Networks (CDN) on fearby.com (that’s why I am about to install EWWW.io ExactDN).

ExactDN (Content Delivery Network)

I did try and set up a number of caching and CDN plugins in the past (e.g Max CDN, W3 Total Cache, WP Fastest Cache, Cache Enable, WP Rocket, WP Super Cache, etc.) but they either made results worse or were impossible to set up.

Now that EWWW.io has a CDN let’s give that a go.

What is EWWW.io’s ExactDN?

You can read more about EWWW.io’s two-pronged approach to delivering one plugin to A) “compress images” and B) “add a Content Delivery Network (CDN)” here: https://ewww.io/resize/

What is ExactDN

Ensure you read up about EWWW.io’s ExactDN here: https://ewww.io/resize/ . If you have not used EWWW.io check out my review of the EWWW.io image compression plugin here first.

Pre Signup

Purchasing and Installing ExactDN Exact DN

Login to your EWWW.io account (or signup then log in).

Signin

FYI: If you have used the Optimise images automatically in WordPress plugin from EWWW.io plugin before, then you will see past purchases here.

Signed in

Now you can go back to the EWWW.io ExactDN product page (https://ewww.io/resize/) and purchase a subscription (Make sure you are logged in with an EWWW.io account before you purchase ExactDN).

Pre purchase

I purchased an ExactDN monthly subscription for $9.00 monthly (with a $1 signup fee for the cloud compression service).

Order

Purchase confirmation screen.

Order

Post-purchase, I was advised to find my “Site Address (URL)” in WordPress and add it to EWWW.io Manage Sites screen.

Add site to the plugin

I now noticed I had a CDN for my domain ( fearby-com.exactdn.com ). Nice.

CDN Site Created

EWWW.io said to tick the CDN option in the Image Resize area of the EWWW.io plugin. But before I do that I  will update WordPress Core and WordPress Plugins before enabling the ExactDN as they are out of date.

Update

TIP: I update WordPress Core and WordPress plugins via command line (my guide here).

Backup WordPress (just in case)

It is always a good idea to backup your website, I logged into my Ubuntu server and checked the size of my site before backing it up.

I copied the website folder (from “/www/” to “/www-backup”)

I confirmed the copied folder size (same, good)

I dumped all databases with the MySQL dump command

While I was there I compress the SQL backup files (text files)

Checking the CDN Status

I checked the DNS replication for fearby-com.exactdn.com with this DNS checker. It’s setup and ready to go across the globe 🙂

A quick CNAME check reveals its upstream provider is fearbycom-8ba8.kxcdn.com. Keycdn.com have been around since 2012. It is great that EWWW.io has paired with keycdn and included their image compression magic in a single WordPress plugin.

Configuring the EWWW.io plugin in WordPress.

Before you enable the CDN below do check the https://www.whatsmydns.net/ and see if your CDN has replicated around the world (Australia can be a bit slow at DNS replication from time to time). Do wait at least 10 minutes before proceeding.

Click the settings in your EWWW Image Optimizer Plugin.

Configure

Now click the Resize Settings tab and click Enable CDN in the EWWW Image Optimizer plugin screen.

Enable Exact DN

Don’t forget to click Save Changes

Save

 

Testing the CDN

https://www.webpagetest.org NOW reports that my site is using a CDN 🙂

FYI: I have always used webpage test from Singapore and I have done the same here to compare apples to apples.  Singapore is not a good test location in Australia (my server is in Australia) and it is 200ms away and the added layer of SSL adds latency to the connection (read here). This is a real-world test though (worst case).

CDN's Used:
fearby.com : 
fearby-com.exactdn.com : KeyCDN
fonts.googleapis.com : Google
fonts.gstatic.com : Google
static.addtoany.com : Cloudflare
www.googletagmanager.com : Google
pagead2.googlesyndication.com : Google
www.youtube.com : Google
adservice.google.com.sg : Google
adservice.google.com : Google
googleads.g.doubleclick.net : Google
www.google-analytics.com : Google
s.ytimg.com : Google
stats.g.doubleclick.net : Google
fearby-com.disqus.com : Fastly
ssl.google-analytics.com : Google

FYI: https://www.webpagetest.org does want other (all) static content to be on a CDN to give a higher score than 42 out 100, sorry I did not get a subscore before enabling the CDN.

I was expecting a green A here for CDN but Webpage Test has given me more items to investigate to ensure WordPress plugins are also fast (minify or move to CDN). It appears WordPress includes are my next target for optimizing.

Web Page Test indicates the following WordPress assets should also be in CDN.

FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/styles/ytprefs.min.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/svgxuse.js?ver=1.1.21
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/scripts/ytprefs.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/add-to-any/addtoany.min.js?ver=1.0
FAILED - https://fearby.com/wp-content/plugins/crayon-syntax-highlighter/js/min/crayon.min.js?ver=_2.7.2_beta
FAILED - https://fearby.com/wp-includes/js/jquery/jquery-migrate.min.js?ver=1.4.1
FAILED - https://fearby.com/wp-includes/js/jquery/jquery.js?ver=1.12.4
FAILED - https://fearby.com/wp-content/plugins/contact-form-7/includes/js/scripts.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/menu/superfish.args.js?ver=2.5.3
FAILED - https://fearby.com/wp-content/plugins/crayon-syntax-highlighter/css/min/crayon.min.css?ver=_2.7.2_beta
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/skip-links.js?ver=2.5.3
FAILED - https://fearby.com/wp-includes/js/comment-reply.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/hoverIntent.min.js?ver=1.8.1
FAILED - https://fearby.com/wp-content/themes/genesis/lib/js/menu/superfish.js?ver=1.7.5
FAILED - https://fearby.com/wp-includes/js/jquery/ui/tabs.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/themes/news-pro/js/global.js?ver=3.2.2
FAILED - https://fearby.com/wp-content/themes/news-pro/js/responsive-menus.min.js?ver=3.2.2
FAILED - https://fearby.com/wp-includes/js/jquery/ui/core.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/themes/news-pro/style.css?ver=3.2.2
FAILED - https://fearby.com/wp-includes/js/jquery/ui/widget.min.js?ver=1.11.4
FAILED - https://fearby.com/wp-content/themes/news-pro/js/jquery.matchHeight.min.js?ver=3.2.2
FAILED - https://fearby.com/wp-content/plugins/contact-form-7/includes/css/styles.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/youtube-embed-plus/scripts/fitvids.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/wp-embed.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/bj-lazy-load/js/bj-lazy-load.min.js?ver=2
FAILED - https://fearby.com/wp-content/plugins/disqus-comment-system/media/js/count.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-includes/js/wp-emoji-release.min.js?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/wp-seo-html-sitemap/style.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/symbol-defs.svg
FAILED - https://fearby.com/wp-includes/css/dashicons.min.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/themes/news-pro/images/favicon.ico
FAILED - https://fearby.com/wp-content/plugins/genesis-tabs/style.css?ver=230d722825ddde2688088d563a906075
FAILED - https://fearby.com/wp-content/plugins/simple-social-icons/css/style.css?ver=2.0.1
FAILED - https://fearby.com/wp-content/plugins/add-to-any/addtoany.min.css?ver=1.14

It would be a huge effort trying to read and keep static plugin and theme related files in a CDN. I’ll ask the EWWW.io developer to see if this is possible in a future version (that would be nice). The developer did promptly point me here to opt into using the CDN to deliver CSS, JS etc.

Is that it? It can’t be that simple!

I can load my site at https://fearby.com and my images load from https://fearby-com.exactdn.com 🙂 I am impressed, blog posts are now loading images from the CDN network and I did not have to edit posts. I did have to subscribe to ExactDN and tick a checkbox in WordPress though.

Here is a sample image (my largest) from this blog post.

https://fearby-com.exactdn.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-3.jpg?strip=all&quality=60&ssl=1

fearby.com WebPAGE TEST Results – Singapore

Web Page Performace Test Results – Singapore (After Setting up the EWWW.io ExactDN):

  • Load Time: 4.177s (down 4.646s from previous optimizations ((My Original load times were 23.672s))
  • First Byte: 2.240s (down 1.3133s from previous optimizations (My Original First Byte: 6.743s))
  • Start Render: 2.800s (down 2.68s from previous optimizations (My Original Start Render: 11.8300s))
  • Speed Index: 3009 (down 2585 from previous optimizations (Speed Index: 15024))
  • Requests: 67/68 (Document complete v Fully Loaded) 
  • Bytes: 535KB/538KB  (Document complete v Fully Loaded)

Quick Web Page Performace Score Card – Singapore (After Setting up the EWWW.io ExactDN):

  • First Byte: F (I should have captured the subscores). 
  • Keep Alive Enabled: A (I should have captured the subscores).
  • Compress Transfer: A (I should have captured the subscores).
  • Compress Images: A (I should have captured the subscores).
  • Cache Static Content: B (I should have captured the subscores).
  • Effective Use of CDN: X (I should have captured the subscores). plugins now need to be on a CDN.

The Web Page Test site does give detailed scores and recommendations if you scroll down or click the score car (A-F). Do read the recommendations and see what you may need to do next.  I am happy that I now have a CDN via EWWW.io. Clicking the first byte and CDN buttons at Web Page Test reveal sub-scores to allow you to see if you have made improvements, regrettably, I did not know of and capture sub-scores until after installing the CDN (I suggest you do).

Full Dump.

GT Metrix Page Speed Score

https://gtmetrix.com is giving a good score across the board (86%). It hints I should optimize Javascript files and “Remove query strings in static files” as some proxy servers do not cache URLs with “?” in them. That is (fortunately) not a problem with ExactDN, as the servers are configured to properly handle query strings.

GTMetrics

Gtmetrix.com does give some optimisation tips too (However, it does report low CDN optimizations if a single file is not delivered over a CDN).

I do like Gtmetrix.com email reports, you can see if your site performance is degrading.

GTMetrix Summary

Page Speed Insights

I am now looking at the Google PageSpeed Insights test for things to fix next. I think I can tweak my NGINX a little (adding caching). Read more about Google Page Speed Insights here.

fyi: My Google Page Speed Insights score (desktop) along with another local big corporate site I tested.

Page Insight

I am happy with 82 🙂

It appears if you want to get 100% you need to get..

  • Initial Response under 100ms
  • Animate, produce frame in under 10ms
  • Idle maximize idle time
  • Deliver all content in under 1000ms

Getting More from ExactDN by caching CSS and JS files.

I added the following to my /www/wp-config.php file as mentioned here: http://docs.ewww.io/article/47-getting-more-from-exactdn This will serve more resources from the CDN

define( 'EXACTDN_ALL_THE_THINGS', true );

I refreshed my site in a browser and now CSS, JS and fonts are loaded from the CDN too.

All

To reduce files served from my website I re-enabled the Fast Velocity Minify plugin in WordPress and pointed it at my CDN (https://fearby-com.exactdn.com/)

cdn-minify

Now I am getting a GTMatrix score of Page Speed Score = 93% (A) and YSlow Score  = 74% (C) and a 2.6-second load and a much lower 35 requests (post minification and pointing the minified files to the CDN).

WOW.

Minifiy

And WebPageTest.org is reporting Effective use of CDN 🙂 Awesome.

Effective use of CDN

I think I’m done (ExactDN CDN and the image optimiser and other tweaks have worked its magic).

How do I compare to other sites?

Given Apple, Microsoft and NBC speed scores are worse than mine I’m happy for now.  🙂

GTMetricx Graphs

GT Metrics graphs show the improvement, YSlow report does indicate I can reduce the website DOM elements to speed things up and tweak plugins but I’d rather keep my design for SEO and not play with plugins or they might break.

I could tweak the server side (NGINX/MySQL/Cache or DNS) but I don’t need too.

Conclusion

I still can’t believe setting up a CDN is a one-click solution and adding a wp-config.php lin (after you subscribe). Best of all the BJ Lazy Load plugin still works. I am very happy with the EWWW.io ExactDN. I now have a CDN and it has lowered my First Byte time, Start Render Times, Speed Index time (on more than just the front page) and all I did was subscribe and tick a checkbox.

It is nice that EWWW.io have not charged for Data delivered from the CDN network on top of a monthly subscription at this stage but they have left it open in the terms and conditions. It is important to ensure a service can sustain itself, so this is a good sign. FYI: my favorite Agile toolkit (Atlaz.io) that I reviewed here closed shop today so it is wise for aaS shops to plan ahead. I know how hard it can be to spin up servers and stuff and allocate times to keep them running. Good luck EWWW.io!

Also, I now have a clear set of steps (below) to resolve other non-optimized assets that are outside of the CDN.

Full Report

Thanks to EWWW.io and Web Page Test for helping make my site faster.

I doubt WordPress and my “down under”  server location can get me to under 1000ms but I will try.

Read on here to see how Cloudflare can increase your site’s performance.

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

DNS Improved

Don’t forget your sites performance/SEO and security.

Website

Update November 2018

I have a much faster loading website after moving it to a new host, read my guide here.

GTMetrix After YSlow

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v2.0 November 2018 Update

v1.9 added Cloudflare, SEO, Google Page Speed Insights test and future optimization, page insight speed image, added GTMetrics comments, growth image, gtmetrix.com reports (typos are free), updated change is domain to .com, added EXACTDN_ALL_THE_THINGS, 93 page speed, minify, compare, graph, fixed typos, tidied up the conclusion.

 

Connect to a remote server with ssh keys generated on OSX

by

Below is the way I connect to a remote server via SSH keys generated on OSX.

Advertisement:


Setting up a server

When you set up an Ubuntu server on Vultr (read my guide on setting up a Vultr server for as low as $2.5 a month) or Digital Ocean (use this link to get two months free when you setup an Ubuntu server on digital ocean) you can specify an SSH key to use for remote connections during the server create stage (old guide here).

How to create an SSH key on OSX to use to connect to a remote server

Run the following command (“sudo ssh-keygen -t rsa“) to generate an ssh key paid in “~/.ssh/” on OSX.

TIP: It is a good idea to also generate a passphrase to use with the key (double protection). You will be prompted to enter this password to use the RSA key.

You can now see the generated keys in ~/.ssh/

You can view the contents of the public file (you can use this when generating Digital Ocean, Vultr, AWS or Azure or other cloud servers).

fyi: Replace 123.123.123.123 with your remote serves ip.

How to connect to a server (the old way).

As long as your host added the desired public ssh key file contents to the server (adding the public ssh key contents to “~/.ssh/authorized_keys” you will be able to connect to the server.

Run the following command on OSX command line to connect to the server via SSH.

You should see..

Congratulations, you should now be able to connect to your server via SSH.

Securing your ubuntu Server

Read my guides here, here and here.

Don’t forget to add a firewall and set up an SSL certificate.

How to connect to a server (faster way).

todo: ~/.ssh/config method

Now we can connect to your remote server with the shorter method.

todo: ~/.ssh/config method

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.0 Initial Post

etc

Building faster web apps with google tools and exceed user expectations

by

@jawache pointed me in the direction of twitter user @addyosmani and a talk at Chrome Dev Summit 2017 called “Fast By Default: Modern Loading Best Practices

Advertisement:



Recently I blogged about speeding up my serves (by moving them closer to me and visitors) and Sped up WordPress by moving away from CPanel tp a  self-managed server.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

RAIL: Measure Performance with the RAIL Model (Fast Integrated, Reliable and Engaging)

https://developers.google.com/web/fundamentals/

  • Focus on the user
  • Initial Response under 100ms
  • Animate, produce frame in under 10ms
  • Idle maximize idle time
  • Deliver all content in under 1000ms

Read more here: https://developers.google.com/web/fundamentals/performance/rail

Replaced with: User happiness metrics

  • First Paint
  • First content Paint
  • First Meaningful Paint
  • Time to interactive

Potential Impacts

Do’s

  • Less JavaScript
  • More Caching
  • Lazy Loading ( I use the BJ Lazy Load plugin in WordPress )
  • Develop on Slow Devices/Links
  • CDN
  • DNS
  • Minify Code  ( fyi )
  • Optimize Images ( fyi )
  • Respect Visitor Data Plans/Devices (offer low data version)

Use online tools (Browser dev audit tools) to measure performance metrics.

Consider in-memory caching solutions

Use the following third-party tools while developing.

These are highly recommended tools.

Siege Command Line Benchmarking tool (Read guide here)

https://calibreapp.com/

https://speedcurve.com/

https://github.com/siddharthkp/bundlesize

http://loader.io/ – External Benchmarker

State of the Web:

Use http://beta.httparchive.org/ to see the average website stats and trends, is your site average or abusing the code type/size you push?

Sample httparchive.org Reports:

Average TCP Connection Per Page

Average percent of HTTPS requests (now over 50%)

Time in seconds to first meaningful paint

1st paint

Average time to first interactive paint

1st Interactic paint

Average time to first on load (finished)

On Load

Netcraft: October 2017 Web Server Survey:

https://news.netcraft.com/archives/2017/10/26/october-2017-web-server-survey-13.html

Web Server Technology Share

Web Server Share

(Image credit: netcraft.com)

Active Web Sites v Hostnames

Active Sites

(Image credit: netcraft.com)

State of Desktops (Gamers Survey)

Steam conducts a monthly survey to collect data about what kinds of computer hardware and software our customers are using. Participation in the survey is optional, and anonymous via the steam app.

Steam HW Scale

This gives a rough (biased) breakdown of high-end machines (gamers) but does reveal valid stats on installed memory and product share.

http://store.steampowered.com/hwsurvey

Don’t forget to track the visitors (location, time, devices, speed, gender etc.) to your site with Google Analytics https://support.google.com/sites/answer/97459?hl=en

Google Progressive Web APP LightHouse Site Audit (Chrome Dev Tools)

You can now run Audits in the Chrome DevTools. To run a report (Read More: https://developers.google.com/web/tools/lighthouse/#devtools )

Chrome Dev Tools

Click Perform and Audit (in the Chrome Developer Tools) on the site you have loaded.

Perform and Audit

Audit in Progress

Audit in Progress

My Report

FYI: Report scores will depend on the time of day and the speed of your computer, A  55 score may be 65 or 45 the following day.

PWA Score

Read the Progressive Web App Check List to help understand how to boost your score.

https://developers.google.com/web/progressive-web-apps/checklist

Google Mobile-Friendly Check List

Also run the mobile-friendly tesh over your site here.

https://search.google.com/test/mobile-friendly

Mobile Friendly

Google Page Speed Insights

Goto PageSpeed Insights and scan your site.

Page Speed Insights

Scanning

Page Speed Insights Report Results

page Insights Results

 

FYI: My site is a WordPress site and I speed up the sight by.

The next thing I want to try is (Sponsors needed).

  • Setting up self-managed Sub Domain CDN’s (https://img01cdn.fearby.com, https://img02cdn.feearby.com etc). – See Vultr and Digital Ocean Ubuntu Server setup guides.
  • Third Party CDN’s.
  • DNS optimizations.
  • Setup NGINX Caching.

Google User Experience Report (Advanced):

The Google User Experience Report (SQL like query and reporting system) is a public dataset of key user experience metrics for top origins on the web. All performance data included in the report is from real-world conditions, aggregated from Chrome users who have opted-in to syncing their browsing history and have usage statistic reporting enabled ( https://www.google.com/chrome/browser/privacy/whitepaper.html#usagestats ) .(end snip from  https://blog.chromium.org/2017/10/introducing-chrome-user-experience-report.html )

Getting started with Chrome User Experience Report Tool: https://developers.google.com/web/tools/chrome-user-experience-report/getting-started
Chrome User Experience Report Tool will allow you to query datasets on page load times.

Why

Better able to compete and deliver measurable advantages to your customers.

Conclusion

  • Know data – MOM (Measure, Optimise and Monitor).
  • Improve over time or before launch.
  • Set budgets on file sizes and page times.
  • Develop on slow devices/links.
  • Consider using PageSpeed Module on your server https://developers.google.com/speed/pagespeed/module/
  • Consider using the RPRL Pattern (https://developers.google.com/web/fundamentals/performance/prpl-pattern/ ) to split and cache code to speed up the time to interactivity.
  • Scale up or out?
  • Enabling web server caching (NGINX: https://www.nginx.com/blog/nginx-caching-guide/

Still Reading?

View more Google Dev Summit 2017 talks: https://www.youtube.com/playlist?list=PLNYkxOF6rcICUD5nBfRdAR6Fveosnqa5m

Also read: Can You Afford It?: Real-world Web Performance Budgets: https://infrequently.org/2017/10/can-you-afford-it-real-world-web-performance-budgets/

Still Reading:

Check out Googles HTTPS Transparency Report https://transparencyreport.google.com/https/overview

Read the introduction blog at Chromium Blog https://blog.chromium.org/2017/10/introducing-chrome-user-experience-report.html

Bookmark https://blog.chromium.org/

Update Jan 2018

Google begins mobile-first indexing of sites to improve mobile search results

What Crawl Budget Means for Googlebot

 

More to come.

Donate and make this blog better


Ask a question or recommend an article
[contact-form-7 404 "Not Found"]

v1.2 Jan 2018 additions on Google’s mobile first approach and increasing crawl rates.

v1.1 fixed typos