Code, InfoSec and Server Stuff

Views are personal and not my employers.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

"If you're not still learning, you're already dying."
- Ryan Holiday - Ego is the Enemy

Follow me on Twitter: @FearbySoftware

View All Posts.

Cache

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

by

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Buy a Domain 

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Cloudflare Benefits (Free Plan)

  • DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
  • Global CDN
  • Shared SSL certificate (I disabled this and opted to use my own)
  • Access to audit logs
  • 3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Cloudflare Global Network

Setup

You will need to sign up at cloudflare.com

Cloudflare

After you create an account you will be prompted to add a siteAdd SiteCloudflare will pull your public DNS records to import.

Query DNS

You will be prompted to select a plan (I selected free)

Plan Select

Verify DNS settings to import.

DNS Import

You will now be asked to change your DNS nameservers with your domain reseller

DNS Nameservers

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Strict SSL

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

The Cloudflare CTO responded.  🙂

Confirm Cloudflare link to a domain from the OSX Comand line

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

Page Rules

“fearby.com.com/wp-*” Cache level: Bypass

“fearby.com.com/wp-admin/post.php*” Cache level: Bypass

“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours.  having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before).  I just need to wait for caching and minification to kick in.

DNS Improved

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

  • Load Time: 1.80s
  • First Byte 0.176s
  • Start Render 1.200s

webpagetest

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Under Attack Flow

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here.  A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

cloudflare-attack-001

Thanks, Cloudflare.

Cloudflare Pros

  • Enabling Attack mode was simple.
  • Soaked up an attack.
  • Free Tier
  • Many Reports
  • Option to force HTTPS over HTTP
  • Option to ban/challenge suspicious IP’s and set challenge timeframes.
  • Ability to setup IP firewall rules and Application Firewalls.
  • User-agent blocking
  • Lockdown URL’s to IP’s (pro feature)
  • Option to minify Javascript, CSS and HTML
  • Option to accelerate mobile links
  • Brotli compression on assets served.
  • Optio to enable BETA Rocket loader for Javascript performance tweaks.
  • Run Javascript service workers from the 120+ CDN’s
  • Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
  • HTTP/2 on, IPV6 ON
  • Option to setup load balancing/failover
  • CTO of Cloudflare responded in Twitter 🙂
  • Option to enable rate limiting (charged at 10,000 hits for $0.05c)
  • Option to block countries (pro feature)
  • Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

  • No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
  • Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
  • Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
  • WordPress draft posts are being cached even though page riles block wp-admin page caching.
  • Would be nice to have ad automatic Under Attack mode
  • Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post

Improving the speed of WordPress

by

This post shows my never ending quest to speed up WordPress for free.

Advertisement:



I have used to use WP Total Cache in the past but decided to check out what other recommended, I found this post 6 Best WordPress Caching Plugins Compared. Some WordPress Caching Plugins.

What plugin do I use?

Benchmark (No Caching Plugin)

I tested my site before installing a caching plugin with https://www.webpagetest.org/ and my site was loading in 21s (loading over 141 files).

My site loaded in a terrible 21.3 seconds. My blog is hosted on Jumba (Net Registry) on and Ultimate plan for $25 a month.

My site seems to deliver 70% images so I wonder if a page caching plugin can help?

I do run the EWWW Image Optimizer plugin to automatically compress images when I upload them to my site. Read my blog post on the EWWW Image Optimizer here.  I do keep images at a high quality to capture all details.

WP Fastest Cache Plugin

I have decided to try the WP Fastest Cache because it’s source was updated 4 hours ago compared to WP Super Cahches update 5 months ago. Both these plugins offer similar GT Metrix performance improvements and WP Fastest Cache has been tested on WordPress v4.8.

Installing WP Fastest Cache

I looked for the WP Fastest Cache Plugin on the WP plugin directory but it was not there.

I downloaded the latest version WP Fastest Cache from https://wordpress.org/plugins/wp-fastest-cache/

I upload the WP Faster Cache plugin to my site.

I Activated the plugin.

WP Fastest Cache plugin is now installed 🙂

It appears to have auto cached/indexed my site?

Now it’s time to run the same benchmark and see if the site is faster (with the same settings (Singapore chrome))?

1 of 3 test are under way.

WP Fastest Cache Results

Wow WP Fastest Cache loaded my site 2 seconds slower (Try 1 = 23 seconds, Try 2 =  21 seconds and Try 3 =  28 Seconds).

This could have been because of weekend traffic or hosting issues but this was not what I expected.

I disabled the WP Fastest Cache plugin and ran the benchmarks again and it was still 23 seconds (weekend traffic?). I re enabled WP Fastest Cache and re ran the test but no improvement.

My bad I think I needed to manually configure the WP Faster Cache plugin by opening the new WP Faster Cache menu on the left-hand side of the WP admin dashboard.

There I enabled caching options in the WP Faster Cache options.

I ran https://www.webpagetest.org tests again and got 16s, 18s and 16s seconds results in three tests and an A on compressed images. It appears you need to manually configure the WP Total Cache plugin after installing it (I missed this step).

I disable WP Fastest Cache and tried the WP Super Cache plugin and the test results were 29s, 24s, 24s (slower than WP Faster Cache). then tried W3 Total cache and the results were ()

I tried the W3 Total Cache plugin and the results were (30s, 16s 26s).

I Tried Autoptimize and it was tested at 45s.

It looks like WP Faster Cache is the fastest, ill turn it back on until try setup a CDN.

Fast Forward to Sept 2017

Since writing this post I have moved away from a shared C-Panel host and have moved my domain to a self-managed Vultr server closer to me, I have moved my email to Google G-Suite. I have learned how to deploy and manage WordPress by command line tools. I have set up servers on Digital Ocean before but the servers are located in Singapore and not Sydney and latency and scalability was poor. SSL will make sites slower and servers far away will just compound the issues.

Re enabling the WP Fastest Cache Plugin

I tried reinstalling the WP Fastest Cache plugin and for me, the plugin just slows down my site by 6 seconds.

I opened my NGINX config and got my NGINX user

My user is: www-data

I enabled the WP Fastest Cache plugin and ensured the WP Fastest Cache has ownership and access to the cache folder.

Below are the settings I use.

WP Fastest Cache

installing the WP-Optimize Plugin

I recommend setting up WP-Optimize plugin as it will optimize your database and keep things fast, it only saves me a second on my load times but this helps.

WP Optimize

WP-Optimize will allow for to review database optimizations

WP-Optimize database savings

Setting up Nginx GZip Compression

I set up my Nginx config to include

I set the minimum size to gzip too

Benchmark with G-Zip, Caching and WP Fastest Cache 

With WP Fastest Cache I now load my site in 13.9 seconds from Singapore. Time to disable WP Fastest cache plugin as it does not seem to be helping without linking to a CDN.

With Cache plugi

Setting up Browser Caching

I also setup browser caching by editing in NginX.

Added

Not sure if caching CSS and JS will cause problems in future?

Benchmark with G-Zip, Caching and without WP Fastest Cache (Singapore)

I re-ran the tests and got 10.9 seconds and got a B for cached content. When in Started on C-Panel I was getting near 30s

Benchmark

Benchmark with G-Zip, Caching and without WP Fastest Cache (Sydney)

I have always benchmarked from Singapore (as Sydney was not an option when I started) but now it is.  Out f curiosity is my website load time in Sydney?

8.2 seconds. Distance does affect performance.

Google Speed Insights

Google has an awesome tools to help you increase your benchmark mobile and desktop website speeds and reccomend focus areas to resolve problems: https://developers.google.com/speed/pagespeed/insights/

Mobile Speed Score

Desktop Speed Score

Tips

I was getting SVG files failing compassion tests so I added the following under allowed mime types under “http gzip_types” in /etc/nginx/nginx.conf

Minifying JS and CSS

This needs to be done and 50% of my site files appears to be CSS and JS related.

It looks like 30%~40 of your sites google speed index is related to minified/combined JS/CSS.

Google Speed Test

I installed the Fast Velocity Minify WordPress plugin.

I ran this to install it from the command line

Unzip

I activated the plugin and set some settings

Minify Settings

Verified minify logs

Logs

Google Page Insights can now see the minifies, css, js and html

Minified

Google Page Insights – Possible Optimizations

issues

And Google Ad Words and Google Analytics appear to be holding back Google Page Insight scores

Google adwords and Analytics

I am getting a few false positives with plugins javascript but that can be resolved another day.

Pingdom (Melbourne results)

3.2 seconds, a few false positives though.

Kingdom

I was going to test with https://www.webpagetest.org/ (from Singapore) but the service kept stalling and had too many tests before me (even from Sydney).

Wait

Address First Byte Time (todo)

If I look at the first-byte load results in the waterfall view my site is taking many seconds to deliver the first byte, this lowers the performance scores about 20%. I need to setup a CDN and or configure NGINX following this guide based on this manual configuration entry (I tried some of the Nginx settings but it appears I need to compile some performance settings into Nginx).

CDN (todo)

I am sure a Content Delivery Network (CDN) will help with the whole page deliver and first-byte times but I am trying to milk as much free as possible and limit future costs. A CDN will trigger higher monthly costs (any CDN providers want to donate a temporary pro plan for review purposes).

Misc Speed Articles

Configuring Ubuntu for Performance

Preventing applications swapping for disk (read more here)

I added this memory related setting.

This will all but prevent applications writing to disk (swap) when they are not active. I had free memory on my VM so I may as well use it.

I will monitor the free ram after reboot and play with php memory settings.

ram

Setup Lazyload for images in posts

Lazyload Plugin Settings

Lazyload

Placeholder Image ( Image: https://fearby-com.exactdn.com/wp-content/uploads/2017/09/placeholder.jpg )

Web Performance Test from Sydney

8.4 seconds ( Score Card F A A A C, was F F F A F ).  I was getting up to 28 second load times with Net Registry C Panel servers.

Static Content is cached but Googe Ad Sence, Google Analytics, and some plugins do block the score. The front page does have some features content that has to be loaded and can’t be minified or cached much.

Sydney Results

It is obvious I need to work in the initial websitee load (DNS, CDN or SSL), there sis  3 seconds I can save here.

3 sec

Configuring PHP for Performance

todo: PHP base config.

todo: PHP caching.

Conclusion

I was expecting WP Fastest Cache to deliver faster speeds but in reality but I am getting 4 seconds faster in WordPress. I was going to configure MaxCDN but they are to expensive. Fast Velocity Minify Plugin is working a treat 🙂

I ended up ditching the shared CPanel hosted domain and setup my own server for WordPress. My site seems a lot faster now. A friend set up CloudFlare with great success, more soon. I blogged about my server setup here.

Adding browser cache and compressing and moving away from CPanel to a self-managed server helped.

The only things to try now is to use a CDN and speed up the delivery of my site and improve the First Byte Time.

Donate and make this blog better




Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.932 added lazy load information (24th Sep 2017)

v1.952 added small changes (23rd Sep 2017)

etc

Securing Ubuntu in the cloud

by

It is easy to deploy servers to the cloud within a few minutes, you can have a cloud-based server that you (or others can use). ubuntu has a great guide on setting up basic security issues but what do you need to do.

Advertisement:



If you do not secure your server expect it to be hacked into. Below are tips on securing your cloud server.

First, read more on scanning your server with Lynis security scan.

Always use up to date software

Always use update software, malicious users can detect what software you use with sites like shodan.io (or use port scan tools) and then look for weaknesses from well-published lists (e.g WordPress, Windows, MySQL, node, LifeRay, Oracle etc). People can even use Google to search for login pages or sites with passwords in HTML (yes that simple).  Once a system is identified by a malicious user they can send automated bots to break into your site (trying millions of passwords a day) or use tools to bypass existing defences (Security researcher Troy Hunt found out it’s child’s play).

Portscan sites like https://mxtoolbox.com/SuperTool.aspx?action=scan are good for knowing what you have exposed.

You can also use local programs like nmap to view open ports

Instal nmap

Find open ports

Limit ssh connections

Read more here.

Use ufw to set limits on login attempts

Only allow known IP’s access to your valuable ports

Delete unwanted firewall rules

Only allow known IP’s to certain ports

Also, set outgoing traffic to known active servers and ports

Don’t use weak/common Diffie-Hellman key for SSL certificates, more information here.

More info on generating SSL certs here and setting here and setting up Public Key Pinning here.

Intrusion Prevention Software

Do run fail2ban: Guide here https://www.linode.com/docs/security/using-fail2ban-for-security

I use iThemes Security to secure my WordPress and block repeat failed logins from certain IP addresses.

iThemes Security can even lock down your WordPress.

You can set iThemes to auto lock out users on x failed logins

Remember to use allowed whitelists though (it is so easy to lock yourself out of servers).

Passwords

Do have strong passwords and change the root password provided by the hosts. https://howsecureismypassword.net/ is a good site to see how strong your password is from brute force password attempts. https://www.grc.com/passwords.htm is a good site to obtain a strong password.  Do follow Troy Hunt’s blog and twitter account to keep up to date with security issues.

Configure a Firewall Basics

You should install a firewall on your Ubuntu and configure it and also configure a firewall with your hosts (e.g AWS, Vultr, Digital Ocean).

Configure a Firewall on AWS

My AWS server setup guide here. AWS allow you to configure the firewall here in the Amazon Console.

TypeProtocolPort RangeSourceComment
HTTPTCP800.0.0.0/0Opens a web server port for later
All ICMPALLN/A0.0.0.0/0Allows you to ping
All trafficALLAll0.0.0.0/0Not advisable long term but OK for testing today.
SSHTCP220.0.0.0/0Not advisable, try and limit this to known IP’s only.
HTTPSTCP4430.0.0.0/0Opens a secure web server port for later

Configure a Firewall on Digital Ocean

Configuring a firewall on Digital Ocean (create a $5/m server here).  You can configure your Digital Ocean droplet firewall by clicking Droplet, Networking then Manage Firewall after logging into Digital Ocean.

Configure a Firewall on Vultr

Configuring a firewall on Vultr (create a $2.5/m server here).

Don’t forget to set IP rules for IPV4 and IPV6, Only set the post you need to allow and ensure applications have strong passwords.

Ubuntu has a firewall built in (documentation).

Enable the firewall

Adding common ports

Add a whitelist for your IP (use http://icanhazip.com/ to get your IP) to ensure you won’t get kicked out of your server.

More help here.  Here is a  good guide on ufw commands. Info on port numbers here.

If you don’t have a  Digital Ocean server for $5 a month click here and if a $2.5 a month Vultr server here.

Backups

rsync is a good way to copy files to another server or use Bacula

Basics

Initial server setup guide (Digital Ocean).

Sudo (admin user)

Read this guide on the Linux sudo command (the equivalent if run as administrator on Windows).

Users

List users on an Ubuntu OS (or compgen -u)

Common output

Add User

e.g

Add user to a group

Show users in a group

This will show users in a group

Remove a user

Rename user

Change user password

Groups

Show all groups

Common output

You can create your own groups but first, you must be aware of group ids

Then you can see your systems groups and ids.

Create a group

Permissions

Read this https://help.ubuntu.com/community/FilePermissions

How to list users on Ubuntu.

Read more on setting permissions here.

Chmod help can be found here.

Install Fail2Ban

I used this guide on installing Fail2Ban.

Check Fail2Ban often and add blocks to the firewall of known bad IPs

Best practices

Ubuntu has a guide on basic security setup here.

Startup Processes

It is a good idea to review startup processes from time to time.

Accounts

  • Read up on the concept of least privilege access for apps and services here.
  • Read up on chmod permissions.

Updates

Do update your operating system often.

Minimal software

Only install what software you need

Exploits and Keeping up to date

Do keep up to date with exploits and vulnerabilities

Secure your applications

  • NodeJS: Enable logging in applications you install or develop.

Ban repeat Login attempts with FailBan

Fail2Ban config

Hosts File Hardening

Add

Add a whitelist with your ip on /etc/fail2ban/jail.conf (see this)

Restart the service

Intrusion detection (logging) systems

Tripwire will not block or prevent intrusions but it will log and give you a heads up with risks and things of concern

Install Tripwire.

Running Tripwire

This will scan your system for issues of note

My Output.

More on Tripwire here.

Hardening PHP

Hardening PHP config (and backing the PHP config it up), first create an info.php file in your website root folder with this info

Now look for what PHP file is loadingPHP Config

Back that your PHP config file

TIP: Delete the file with phpinfo() in it as it is a security risk to leave it there.

TIP: Read the OWASP cheat sheet on using PHP securely here and securing php.ini here.

Some common security changes

Don’t forget to review logs, more config changes here.

Antivirus

Yes, it is a good idea to run antivirus in Ubuntu, here is a good list of antivirus software

I am installing ClamAV as it can be installed on the command line and is open source.

ClamAV help here.

Scan a folder

Setup auto update antivirus definitions

I set auto updates 24 times a day (every hour) via daemon updates.

tip: Download manual antivirus update definitions. If you only have a 512MB server your update may fail and you may want to stop fresh claim/php/nginx and mysql before you update to ensure the antivirus definitions update. You can move this to a con job and set this to update at set times over daemon to ensure updates happen.

Manual scan with a bash script

Create a bash script

Bash script contents to update antivirus definitions.

Edit the crontab to run the script every hour

Uninstalling Clam AV

You may need to uninstall Clamav if you don’t have a lot of memory or find updates are too big.

Setup Unattended Ubuntu Security updates

At login, you should receive

Other

  • Read this awesome guide.
  • install Fail2Ban
  • Do check your log files if you suspect suspicious activity.

Check out the extensive Hardening a Linux Server guide at thecloud.org.uk: https://thecloud.org.uk/wiki/index.php?title=Hardening_a_Linux_Server

 

Donate and make this blog better




Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

v1.92 added hardening a linux server link