Programming, IoT and Server Stuff

Views are personal and not my employers.

Header Right

Development Blog

Coding for fun since 1996, Learn by doing and sharing.

Main navigation

and

Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare

by Leave a Comment

This post will show you how you can setup Sender Policy Framework (SPF), DomainKeys Identified Mail (DKIM) Signatures and Domain-based Message Authentication, Reporting, and Conformance (DMARC) on your GMail (G Suite) email to limit spam and increase security.

Advertisement:



I have a number of guides on moving away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. I use Google G Suite to send and receive emails that are linked to my domain (even via the command line) using multiple domains (with aliases).

Buy how can you extend your email security and limit spam?

Enter..

Sender Policy Framework (SPF) for Authorizing Use of Domains in Email

Background: SPF summary from the RFC document from the Internet Engineering Task Force (IETF).

“Email on the Internet can be forged in a number of ways. In particular, existing protocols place no restriction on what a sending host can use as the “MAIL FROM” of a message or the domain given on the SMTP HELO/EHLO commands. This document describes version 1 of the Sender Policy Framework (SPF) protocol, whereby ADministrative Management Domains (ADMDs) can explicitly authorize the hosts that are allowed to use their domain names, and a receiving host can check such authorization.”

Google has a guide in setting up SPF records for your G Suite account.

Scan your site for SPF, DKIM and DMARC configuration(s).

Gmail has a test site where you can check your site SPF, DKIM and DMARC etc: https://toolbox.googleapps.com/apps/checkmx/

Secure GSuite

How to set up an SPF Record

I followed this guide to set up an SPF record on my G Suite account. I use Cloudflare for my DNS provider so I’ll make my DNS changes there.

Add SPF Record

Update: Google instructions were wrong, use a TXT record and not a SPF record.

Read more on SPF at Wikipedia here.

DNS will take a while to replicate so do wait a few hours before checking again with the checkmx tool.

Now let’s set up DomainKeys Identified Mail (DKIM) Signatures

Read more on DKIM at Wikipedia here.

Background: The DKIM RFC form the Internet Engineering Task Force (IETF) states…

“DomainKeys Identified Mail (DKIM) permits a person, role, or organization that owns the signing domain to claim some responsibility for a message by associating the domain with the message. This can be an author’s organization, an operational relay, or one of their agents. DKIM separates the question of the identity of the Signer of the message from the purported author of the message. Assertion of responsibility is validated through a cryptographic signature and by querying the Signer’s domain directly to retrieve the appropriate public key. Message transit from author to recipient is through relays that typically make no substantive change to the message content and thus preserve the DKIM signature.”

Google has a DKIM FAQ: https://support.google.com/a/answer/174124

Login to your G Suite account and load this FAQ.

The FAQ page states..

“You can help prevent spoofing by adding a digital signature to outgoing message headers using the DKIM standard. This involves using a private domain key to encrypt your domain’s outgoing mail headers, and adding a public version of the key to the domain’s DNS records. Recipient servers can then retrieve the public key to decrypt incoming headers and verify that the message really comes from your domain and hasn’t been changed along the way.”

Click Generate the Domain Key

Generate Domain Key

Follow the steps and generate a key

Generate Key

Generate a new record

Generate key

Add the DKIM key to your DNS record

Add DNS record

DNS will take a while to replicate so do wait a few hours before checking again with the checkmx tool.

Domain-based Message Authentication, Reporting, and Conformance (DMARC)

Read more on DMARC at Wikipedia here. Read the official page here https://dmarc.org/.

Background: The DMARC RFC form the Internet Engineering Task Force (IETF) states…

DMARC Flow

DMARC Flow

“Domain-based Message Authentication, Reporting, and Conformance (DMARC) is a scalable mechanism by which a mail-originating organization can express domain-level policies and preferences for message validation, disposition, and reporting, that a mail-receiving organization can use to improve mail handling.

Originators of Internet Mail need to be able to associate reliable and authenticated domain identifiers with messages, communicate policies about messages that use those identifiers, and report about mail using those identifiers. These abilities have several benefits: Receivers can provide feedback to Domain Owners about the use of their domains; this feedback can provide valuable insight about the management of internal operations and the presence of external domain name abuse.

DMARC does not produce or encourage elevated delivery privilege of authenticated email. DMARC is a mechanism for policy distribution that enables increasingly strict handling of messages that fail authentication checks, ranging from no action, through altered delivery, up to message rejection.”

Google G Suite has a guide to setting up a DMARC record here

Snip from the Google guide here..

“Spammers can sometimes forge the “From” address on email messages so the spam appears to come from a user in your domain. To help prevent this sort of abuse, Google is participating in DMARC.org, which gives domain owners more control over what Gmail does with spam email messages from their domain.

G Suite follows the DMARC.org standard and allows you to decide how Gmail treats unauthenticated emails coming from your domain. Domain owners can publish a policy telling Gmail and other participating email providers how to handle unauthenticated messages sent from their domain. By defining a policy, you can help combat phishing to protect users and your reputation.“

Login to your G Suite account and load this FAQ

Click Add A DMARC Record

Add DMARC

You will then need to set up a DKIM Domain Key (if you have not done so yet)

When you are done you need to choose your DMARC rules, I would suggest you go to https://mxtoolbox.com/DMARCRecordGenerator.aspx to generate a record

I generated these rules

Dmarc Rules

Warning: Setting a DMARC policy that is too strict may block mail from being delivered. Tighten rules over time.

Login to your DNS provider and add your TXT record.

DMARC Record

You should now have an SPF, DKIM and DMARC record in DNS.

DNS

Update: The SPD record above should be a TXT (Google led me astray)

DNS will take a while to replicate so do wait a few hours before checking again with the checkmx tool.

Now go to bed and wait for DNS to replicate.

Troubleshooting SPF

My TXT record would not validate with https://toolbox.googleapps.com/apps/checkmx/check

Google Toolbox

The MX Toolbox SPF checker reports that SPF records are deprecated and to use TXT records instead.

SPF TXT Record

Fix (remove the SPF record and add a TXT record with the same contents). Don’t forget to delete the old SPF record.

TXT Record

Results

SPF Setup

Reports

SPF/DKIM reports will let me know when unauthorized people send email from my domain.

Also, DMARC will alert me to unauthorized activity

 

I hope this guide helps someone.

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.4 Reports

v1.3 DMARC Flow image

V1.2 Updated wording

V1.1 Fixed typos (they were free)

v1.0 Initial post

Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

by Leave a Comment

This guide will help you install and setup the pm2 NodejJS process monitor PM2 from Keymetrics.io for free and manage your node apps performance and exceptions.

Advertisement:



What is PM2?

PM2 is a production process manager for Node.js applications with a built-in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks. This is the steps I used on Ubuntu 16.04. This is NOT a paid endorsement (just self-documenting).

Key Features of PM2

PM2 offers web-based monitoring dashboard, exception reporting, load balancer, CPU and memory monitoring, transaction tracer and much more for NodeJS apps.

pm2-features

What is PM2?

Official page: http://pm2.keymetrics.io/

More info https://www.npmjs.com/package/pm2

Install PM2

Install Output

PM2 Pricing

PM2 appears to be for high-end apps but I am only using the free version or PM2 (thanks KeyMetrics)

pm2-pricing

Create a bucket for your node app

Login to keymetrics.io,

Click Generate New Bucket

Create New Bucket

Give the bucket a name etc.

Node Bucket Name

You can now link your bucket with your local pm2 installation (keep the keys private (this one no longer exists))

pm2-link

Linking your local pm2 installation with your keymetrics bucket

To add an existing node app to PM2 type the following.

You can view node apps that pm2 is managing by typing

I had a two CPU VM and I found that the app I added was added to each of the two CPU (I only needed one) so I needed to delete the second app on my second core

Restart the API

You can add a single node apps one 1, 3 or max available CPU’s

Again, to add an existing node app to PM2 type the following.

Now you can view node app data online. If you don’t have a node app ready you can use the test app.

monitor output

You can monitor your node app locally too from the CLI.

local monitoring

You can also view a demo bucket at keymetrix.io

pm2-demo-bucket

PM2’s one age documentation can be found here.

I hope this guide helps someone.

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.0 Initial post

Using Cloudflare DNS servers to speed up the internet and add privacy on OSX

by Leave a Comment

Below is how I setup my OSX to use Cloudflare’s new DNS servers to speed up internet browsing and add privacy on OSX

Advertisement:



Cloudflare has launched a DNS service: https://blog.cloudflare.com/announcing-1111/

DNS Performance

You can view worldwide DNS performance by viewing https://www.dnsperf.com/#!dns-providers

DNS Performance

I check the DNS at my router, I am using ISP provided DNS servers.

Review DNS

Cloudflare DNS

On April Fools 2018 Cloudflare Released a DNS server service.

Snip from here: “DNS: Internet’s Directory Nearly everything on the Internet starts with a DNS request. DNS is the Internet’s directory. Click on a link, open an app, send an email and the first thing your device does is ask the directory: Where can I find this? Unfortunately, by default, DNS is usually slow and insecure. Your ISP, and anyone else listening in on the Internet, can see every site you visit and every app you use — even if their content is encrypted. Creepily, some DNS providers sell data about your Internet activity or use it target you with ads.”

https://1.1.1.1/

Set Cloudflare Nameservers using OSX

Open the Apple System Preferences, click Network, click on your Network (Wifi or ethernet), Click Advanced then DNS and add 1.1.1.1 and 1.0.0.1

Alternatively, you can manually set your DNS servers in OSX by editing the /etc/resolv.conf, by default SX will inherit DNS settings from our router.

Troubleshooting: Clear DNS Cache

Debug DNS Data

Confirm Cloudflare DNS from the OSX Comand line

Privacy

I am not sure if Cloudflare is any more private than using ISP DNS but I’ll happily use it.

Speed

I can’t tell if DNS is faster, I did ping my ISP DNS before switching and it was about the same (sub 25ms), time will tell.

Conclusion

I have used https://www.opendns.com/ before and loved the dashboards, I hope Cloudflare add dashboard options too.

I hope this guide helps someone.

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.0 Initial post

How to purchase your own domain name and set up a web server from $5 a month

by Leave a Comment

This guide will show technically minded people how you can purchase your own domain name, set up a web server on Vultr with an online store using WordPress/WooCommerce from $5 a month. Warning this post is technical (if you have never used SSH, Ubuntu, Linux Command Line, hate risk or are not patient then this is NOT the guide you are after).

Advertisement:


Draft Post (released early to share)

I personally recommend (not a paid endorsement) the free WooCommerce plugin for the free WordPress.org CMS on the free Ubuntu Operating system with the free NGINX web server and the free MYSQL database engine and free SSL certificates from Lets Encrypt.

Sorry for using the word free a lot but I like free things.  One of the benefits of a using a self-managed server is you get the option to install free software and configure the server how you want and secure it how you want. Truth be told managed ho (e.g CPanel, etc) are in the business of making money via monthly feed, expensive SSL certificates, taxing your transactions or pushing you to higher priced tiers.

Legend:

  • Self Managed Server = A server that you create, you configure patch and support (all the reward and risk is owned by you and costs are low).
  • Hosted Server = A server you have partial control of and the hosts manage the server and support (You hand away all risk and most of the control and pay for support/features).

I moved to a self-managed server after I was paying $25/m for a poorly performing website and $150/y for a poor quality  SSL certificate and a slice of a server that seemed to always say “Usage Limit Exceeded”. Why pay for an insecure website that my visitors could not view because the usage limit was exceeded.

Bad CPanel SSL Certificate

fyi: Fearby.com costs me $10 a month for a server and $5/m for CDN abilities.

CPanel hosts are an option when you don’t want to self-manage a service and take on the hassle but be prepared for server limitations (The image below was taken on an older CPanel based hosts before I moved to a self-managed Vultr server)

cpenal_usage_exceeded

I recently discovered a well known and established website hosting service (that I used to use) and a friend is still using is insecure. My friend’s site has a static website on it but the server underneath was very old and insecure. Having a secure web server should be at the top of your list with any self-managed or hosted website (this will help search engine optimization and prevent risks to your website visitors).

Static Website

Sites like Virus Total, SSL Labs and Alexa Site Info , Qualys are good ways to review a site’s credibility.

fyi: The awesome https://seositecheckup.com/ is awesome for evaluating our sites SEO score.

Before we set up a server with WordPress on your own server let’s quickly look at the alternative commercial ready to go website builders.

 

Alternative (paid) DIY Website Builders

The following leading commercial sites will allow you to build a site online.

In my opinion, five things matter with setting up site online website.

  • Setup Costs, Monthly Cost and Commissions (what are the hidden charges)
  • Security (having a food SSL Certificate is key to having a good organic traffic from search engines)
  • Site Speed (Having a slow site will impact search engine optimization and drive visitors away)
  • Accessibility (if your site is not WCAG accessible it will not rank high on search engines).
  • Control (will you be able to do everything you want too, nothing worse than going so far and being limited)

Ok, let’s see how much it will cost to set up a simple business site on the sites above.

 

Wix 

Setup: Goto https://www.wix.com/, Login to Wix, click Create Site, click Business, Click Choose a Template, Edit the page, Click Save, Click “Connect your own customized domain“, Click “Connect a domain you already own“.

I was redirected to a Wix plan pricing page where I need to choose a plan to continue. From what I researched you cant control HTML on Wix so can’t add a MailChimp newsletter signup form so you would have to go with the $24.5/m option to enable Email Campaigns.

Wix Plans Chooser

I could not see information about included SSL certificates, SEO or other chargers.  SSL is free after you pay right?

The Wix editor appears OK (it may take a bit of learning though).

Wix Editoer

I clicked publish and the site was live

Wix site published

A quick check of the SSL, Accessibility and SEO and no obvious deal breakers here apart from the price and platform lock-in.

Wix Checkup

I performed a security check on the site with https://freescan.qualys.com (passed)

Conclusion: I hear Wix templates are hard to change so choose your template wisely, A large collection of apps are available that you can add to the site.

Although Wix was nice and it does include a full-featured look at the engine it is not for me ($24/m USD is too expensive).

 

Squarespace 

Squarespace basic websites cost $16/$25 a month or $34/52 for online stores: https://www.squarespace.com/pricing/

SquareSpace Pricing

Setup a Squarespace website: Goto https://www.squarespace.com/, Click Start a Free Trial, Choose a Template, Create an Account (a quick read of the terms of service and privacy policy, #scary), SpareSpace sites are pre-published?

Square Space Build

Loading the webpage on a non-logged-in (with SquareSpace login) browser displays a trial warning.  Trial pages are essentially restricted (unlike Wix).

Login Challenge

The mobile view does not match the template?  I guess the chosen template is more of a vibe and not a template.

Mobile view

Setting up a Squarespace website may take some time. Squarespace does have some nifty advance options in a slide-out menu though.

Squarespace Settings

Because the public view of the page is restricted I cannot scan it with WCAG accessibility tools. Scanning the site performance speeds with gtmetrix also fails.

Performance Rejected

Squarespace is well known to be difficult to set up a website when compared to other drag and drop editors (but Squarespace sites do look nice).

I am not paying $54/m for a website so let’s move on.

 

Shopify

Shopify Setup: Goto https://shopify.com and click Create, Sign up and enter your store name. Complete the wizard. 

Shopify

Choose a Shopify Plan

Shopify Plan

Scalping transactions, no thanks. let’s move on.

 

Weebly

Weebly Setup: Goto https://www.weebly.com/au and click Get Started under Create Store. Enter your account details and click Create Your Site, enter the name of the store, Click I’m just trying Weebly, click the type of product you will be selling.

Weebly Site Setup

Weebly Setup

Theme Selection

Theme Select

Choose a Domain

Domain Select

Publish the site

Publish

Clicking publish appears to be a dead end.

Verify Weebly

“Please contact Weebly Support to verify your account”, No Thanks, let’s move on.

One candidate remains and that is WordPress hosted (wordpress.com not wordpress.org).

 

WordPress.com

WordPress.com offer hosted plans for WordPress in the cloud.

Setup a WordPress site, the only one that removes WordPress branding and allows third-party plugins to be installed it the Business plans for $33 a month.

WordPress Plans

Setup Basics

Wordpress

Choose a WordPress theme.

Choose Theme

Assign a Domain

WordPress Domain

In order to buy a domain, you need to log in (top right) with an account

My working WordPress account (is no longer working), it was in my password manager.

wordpresscomerror

I seem to be stuck in a signup loop

Wordpress

Time to move on. Time to set up my own server on Vultr and setup WordPress and  WooCommerce,

But, before we do, let’s ensure our name is secure online.

Search for your Name/Brand

Do search for your website (or thing) in search engines to see if your name is already taken, don’t buy a domain that is owned or has IP or trademark presence. It is a  good idea to use sites like https://namechk.com/ to see if your site or social media is already taken.

https://namechk.com/

namechk.com will allow you to search for name availability online.  The name “mything” is not fully available online.

https://namechk.com/ 2

You will want to see all green squares (name available) below before buying a domain name. This looks better.

Namechk ok

I would recommend you create your social media accounts before or right after buying your domain. Sites like Twitter will insist on short usernames names so get your social media sites first.

 

Trademark and Brand Search

Also, perform a trademark and IP search.

Australian Trademark Search: https://search.ipaustralia.gov.au/trademarks/search/quick

United States Trademark Database: https://www.uspto.gov/trademarks-application-process/search-trademark-database

Global brand Search: http://www.wipo.int/branddb/en/

etc

 

 

Self Managed Warning

I tend to go the “self-managed server route” and install the free WordPress CMS because:

There are many reasons why you would not want to “self-manage” your own server

  • Technical Requirements (and time to support).
  • Higher Risk.
  • Applying Updates and Patches.
  • etc.

Being technically minded and choosing a “self-managed web servers” can take away time from the fun stuff like SEO, Site Design, customer needs, branding etc. If you do not have experience with SEO, designing, branding, social media, websites etc (and need help) please talk to my friends at Niche Creative (this is not a not a paid endorsement, they are passionate and know thier stuff). I blogged my journey away from a Panel host self-managed server here..

Self Managed Costs

For $5 a month you can buy a server with enough memory to install WordPress (cheaper if you don’t need WordPress)

Vultr is great. Vultr does have ready to go servers that you can deploy that have WordPress all set up.

wordpress-template

The Vultr template above does use the Centos OS (read my guide setting up Centos on a different service provider here) but I prefer to manually setup a server with Ubuntu 16.04 OS on Vultr.

Wih a $5 server you can do what you want with it.  I have blogged before about setting up your own Server. e.g Installing Centos and Ubuntu server on Digital Ocean.  Digital Ocean does not have data centres in Australia and this kills scalability. AWS is good but 4x the price of Vultr. I have blogged about setting up and AWS server here (and upgrading an AWS instance). I tried to check out Alibaba Cloud but the verification process was broken so I decided to check our Vultr.

Manual Setup of Vultr on an Ubuntu 16.04 server

  • Deploy a Vultr Server – Guide here  ($2.5/m to New Jersey or Florida or $5/m to Sydney,  I would recommend you opt in for the auto backup for $0.50c/m and $1/m respectively).
  • Setup NGINX.
  • Setup PHP and PHP-FPM (see guide above), consider adding PHP child workers.
  • Setup and secure MySQL (see guide above), create a database for WordPress to use.
  • Instal Adminder MySQL GUI (guide here).
  • Setup a free Lets Encrypt SSL certificate (guide here).
  • Install WordPress (and Jetpack plugin).
  • Install WordPress CLI.
  • Instal the WooCommerce Storefront WordPress Theme.
  • Install WooCommerce Plugin.
  • Secure Ubuntu.
  • Also consider linking your domain to Cloudflare to boost performance, scanning your site with Qualys Freescan and OWASP ZAP).
  • Consider setting up a WordPress image compressor and CDN plugin. like EWWW.io

Manual WooCommerce Plugin Setup

Once you setup Woocommerce you can setup the store defaults. Goto the WordPress dashboard and click WooCommerce Settings

Woo Commerce Settings

Settings – General

General

  • Set Address, City and State and Postcode
  • Set allowed countries to sell in (e.g Australia)
  • Set allowed countries to ship items to (e.g Australia)
  • Set Enable Taxes
  • Set Currency
  • etc

Settings – Products

Products

  • Set Weight
  • Set Dimensions
  • Enable Product Reviews
  • Enable Star Ratings on Reviews
  • etc

Settings – Shipping

Shipping

  • Enable Shipping Calculator
  • Add Shipping Classes
  • Shipping Zones
  • etc.

Settings – Checkout

Settings checkout

  • Force Secure Checkup
  • Create a Terms and Conditions page (and set).
  • etc

Settings – Account

Accounts

  • Set Account Options
  • etc

Settings – Emails

Emails

  • Set Email Preferences
  • Set Email Header Image
  • Set Email Colour
  • Set Footer Text
  • etc

Settings – API

API

  • API can be disabled if you don’t need it.

Optional Actions

  • Setup Yoast Plugin
  • Setup other plugins

Post Site Setup

Just because your site is live does not mean you can rest.

SEO Optimization

Do use sites like https://seositecheckup.com/ and follow recommended actions to improve your SEO like updating meta tags.

More Reading

Attaching an email to your domain

You can pay $5 a month and link a G Suite email to your domain.

  • Dedicated professional Google G Suite email account for $5 a month with 30GB storage (If you don’t want ot to buy a G Suite email and link it to your domain then you don’t need this).

Once you have a G Suite account you can link other domains (and domain emails) to it. You can login to your G Suite emails via G Mail and send emails from apps or the command line.

Why Vultr

I use the server host Vultr as they have data centres all around the world and the support of great, Digital Ocean is good too but they don’t have data centres in my country (Australia). Vultr allows you to deploy all over the world upgrade servers, move servers, add storage and restore servers.

Alternatively, you can buy a $2.5/m server and generate  a static website

I use the Platforma Web HTML generator to build mobile and WCAG compliant websites.

Buying a domain,  I buy my domains from https://www.namecheap.com/  it is a good idea to look for coupons first at https://www.namecheap.com/promos/coupons.aspx before buying a domain.

Once you buy a domain you can point it to a Vultr server and upload your website.

I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v.1.2 WordPress WooCommerce

v1.1 SEO

v1.0 Initial Draft

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

by 1 Comment

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Cloudflare Benefits (Free Plan)

  • DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
  • Global CDN
  • Shared SSL certificate (I disabled this and opted to use my own)
  • Access to audit logs
  • 3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Cloudflare Global Network

Setup

You will need to sign up at cloudflare.com

Login to Cloudflare

After you create an account you will be prompted to add a siteAdd SiteCloudflare will pull your public DNS records to import.

Query DNS

You will be prompted to select a plan (I selected free)

Plan Select

Verify DNS settings to import.

DNS Import

You will now be asked to change your DNS nameservers with your domain reseller

DNS Nameservers

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Strict SSL

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

The Cloudflare CTO responded.  🙂

Confirm Cloudflare link to a domain from the OSX Comand line

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

Page Rules

“fearby.com.com/wp-*” Cache level: Bypass

“fearby.com.com/wp-admin/post.php*” Cache level: Bypass

“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours.  having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before).  I just need to wait for caching and minification to kick in.

DNS Improved

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

  • Load Time: 1.80s
  • First Byte 0.176s
  • Start Render 1.200s

webpagetest

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Under Attack Flow

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here.  A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

cloudflare-attack-001

Thanks, Cloudflare.

Cloudflare Pros

  • Enabling Attack mode was simple.
  • Soaked up an attack.
  • Free Tier
  • Many Reports
  • Option to force HTTPS over HTTP
  • Option to ban/challenge suspicious IP’s and set challenge timeframes.
  • Ability to setup IP firewall rules and Application Firewalls.
  • User-agent blocking
  • Lockdown URL’s to IP’s (pro feature)
  • Option to minify Javascript, CSS and HTML
  • Option to accelerate mobile links
  • Brotli compression on assets served.
  • Optio to enable BETA Rocket loader for Javascript performance tweaks.
  • Run Javascript service workers from the 120+ CDN’s
  • Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
  • HTTP/2 on, IPV6 ON
  • Option to setup load balancing/failover
  • CTO of Cloudflare responded in Twitter 🙂
  • Option to enable rate limiting (charged at 10,000 hits for $0.05c)
  • Option to block countries (pro feature)
  • Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

  • No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
  • Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
  • Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
  • WordPress draft posts are being cached even though page riles block wp-admin page caching.
  • Would be nice to have ad automatic Under Attack mode
  • Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud

by Leave a Comment

Upgrading the RAM, CPU and Memory on a Vultr Ubuntu VM in the cloud is quite simple.

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.  I prefer Vultr as they are located in the country (Australia) and are easy to use.

First, you need to shut down the server from within the VM (SSH), I used the command.

Once the VM is shut down (wait a few minutes) you can turn off the VM in the Vultr GUI.

Shutdown

You can then go to Settings, Change Plan and review upgrade options.

Upgrade Options

Snapshot

Don’t forget to take a final snapshot.

Snapshot reminder

Goto the Snapshots page (read this guide to restore a snapshot) and click Take Snapshot.

Take Snapshop

You can see snapshot progress on the main screen.

Snapshot Progress

It may take a while for your snapshot to change from Pending to Processing.

Processing

Upgrade

When the snapshot is done it will auto boot and allow you to upgrade.

Manage

Choose the Upgrade specifications (Settings, Change Plan)

Upgrade Specs

Click Upgrade

Upgrade

Confirm

Confirm

The upgrade process will take a few minutes (I could see the CU and Ram was updated but the Storage was pending)

Upgrade Pending

Testing

After the upgrade happened the VM will autoboot, login and check tour specifications (Useful Linux Commands).

I use the htop command to view specification information.

I did a quick benchmark pre-optimizing and I can see a speed bump of 0.2s. Time to optimize.

Benchmark

I threw 50 concurrent clients at my website (with loader.io) and the server handled it fine with no increase above memory capacity like before.

Concurent Users

Optimize

Now I need to Optimize.  Truth be told  I did optimize and harden PHP and crashed PHP-FPM so I had o restore a VM snapshot.

Troubleshooting

If all else fails (post-upgrade configuration) you can restore the Vultr VM from a snapshot.

I hope this guide helps someone.

 

P.S If you don’t have a VM on Vultr click this link to set one up in minutes (setup guide here).

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.0 Initial post

Using Chrome 65 to measure website Performance, Progressive Web Apps, Basic Practices, Accessibility and SEO

by 1 Comment

Chrome 65 beta has added SEO to audits in the Developer Tools Audit tab.

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line.

When you develop any site you need to perform regular audits to ensure it is up-to-date and compliant.  Google Chrome has audit tools built right in (Chrome 65 has SEO audit tools (yay)

Chrome 64Developer Tools – Audit tool.

Chroms 64 Audit

Chrome 64 Audit Scan options

Chrome 64 Audit Scan

Chrome 64 Audit Results

The performance results seem off given I have set up a CDN, optimized WordPress.

Chrome 64 Audit Results

Time to download Chrome 65 beta and get new audit tools.

Time to download Chrome 65 and get new audit tools

Chrome version confirmed

Chrome Version

Audit Options

Chrome 65 BETA Audit Options

Chrome 65 Audit Results

Hmmm, results are different (same website, network and time)

Chrome 64 Audit Results:

  • Performance: 45
  • Progressive Web App: 30
  • Accessibility: 86
  • Best Practice: 56
  • SEO: N/A

Chrome 65 BETA Audit Results:

  • Performance: 24
  • Progressive Web App: 45
  • Accessibility: 65
  • Best Practice: 63
  • SEO: 90

Chrome 65 BETA Audio Results
Google recommends you load pages in under 1s, I would suggest using multiple tools to indicate site speed.

https://gtmetrix.com is a great site for testing your sites speed. I used GT Metrix to move my site from 26s to 6 by setting up a WordPress CDN, moving away from CPanel to a self-managed server on Vultr then to 4s with optimising PHP by setting up child workers.

GT Metrix

https://www.webpagetest.org is also good for testing websites.

I will continue to use Chrome Audit Tools for other results like SEO

Chrome SEO Tools

That’s weird Chrome Audit Tools, I thought I did not have meta tags?

I downloaded the WP Meta SEO plugin, I use the wp command from the SLI (setup guide here)

I activated the plugin in WordPress the loaded the dashboard.  WP Meta SEO asked to import data from the Yoast plugin.

meta-dashboard

Now I can see my metadata is falling behind.

wp-meta-seo

With WP Meta SEO I was able to apply individual page Meta tags.

meta-apply

 

I added the following meta tags with WP Meta SEO plugin for each page (the metadata below was for this page)

 

fyi

I use the free version of the Yoast for SEO plugin in WordPress.  The premium version has a lot more to offer, I might have to check Premium out.

Yoast Compare

I am now on the hunt for meta plugins for WordPress

I hope this guide helps someone.

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.2 WP-Meta SEO

v1.1 Added webpagetest.org link and results

v1.0 Initial post

Using Fritzing to draw electronic schematics for Arduino, Raspberry Pi and ESP8266

by Leave a Comment

This guide will show how you can create an electronics schematic to represent elements of a circuit for Arduino, Raspberry Pi and ESP8266 micro-controllers.

Advertisement:


Todo: Content
Wikipedia states here: “A schematic, or schematic diagram, is a representation of the elements of a system using abstract, graphic symbols rather than realistic pictures. A schematic usually omits all details that are not relevant to the information the schematic is intended to convey, and may add unrealistic elements that aid comprehension.” This guide will help you create schematics using Fritzing on OSX, you may still want to document your system requirements etc in Github or Bitbucket.

I am creating some small personal weather station on Arduino, Raspberry Pi and ESP8266 to submit data to cloud servers on Vutr (or AWS or Digital Ocean), read my guide on using Adminer to create and manage MySQL databases.

What are sensors

A sensor could detect temperature, humidity, barometric pressure, light and just about anything else.

Sensor

Temperature, Humidity and Barometric sensor (bme280)

GY-BME280

Most sensors work on low voltage have analogue or digital outputs (I2C or SPI) and require minimal wires. The real problem is having multiple sensors and wiring gets out of hand.

What is Arduino

Arduino is a low power 8bit and 32bit hardware/software product that you can rapidly wire up circuits and sensors. Sites like Adafruit sell loads of sensors and develop a lot of software libraries to drive sensors.

Non-genuine Arduino boards age mega cheap on eBay.

Arduino has slide-on shields that can be pushed onto an Arduino board to expand them.

Pros

  • Cheap
  • Loads of support
  • Good analogue sensor support

Cons

  • Limited monitor support

Sensors or expansions can come in shields or be connected to pins with wires.

What is Raspberry Pi

Raspberry Pi is also a single board computer but with a more powerful ARM processor that runs a Linux operating system from an SD card. Raspberry Pi’s are essentially small desktop computers. Raspberry Pi’s have USB, can plug into a monitor, have a bootable desktop have Arduino like GPIO pins that can talk to sensors (but not as many analogue sensors).

Pros

  • Very Powerful.
  • Desktop Operating System included.
  • Can talk to sensors.
  • Good GPIO pins for controlling the real world.
  • Has HDMI monitor output.

Cons

  • Full kits are expensive.
  • Low analogue read pins.
  • Failure to shut down correctly can corrupt the OS.

Pi

What is ESP8266

Pros

  • Cheap
  • Good Analog and digital puns
  • Code in Arduino IDE
  • Has WiFi

Cons

  • Limited monitor support

ss

Installing Fritzing

Fritzing is a software package that can allow you to design and learn electronic schematics (filter by kid level, amateurs, master or higher).

Goto: http://fritzing.org/

Fritzing site

Click Download and Donate

Download Fritzing

Click Download and choose the right version for your Operating System

Download

Copy the app to your Application folder

Copy the app to your Application folder

Open the app a few times (open, wait, proceed, close, open wait proceed etc) to ensure parts updates are installed.

Open the app a few times to instal updates

Fritzing Introduction on YouTube

Watch the Introduction video for Fritzing here

Fritzing Killer Tips Series on YouTube

Watch Fritzing Killer Tips 001 The generic IC

Extending Fritzing

Fritzing allows you to import new parts and design your own parts or ask Fritzing to design a part for you.

Import

I was able to import a bme280 sensor within seconds.

bme280

If a part does not exist in the forums search google for “partname” and “.fzpz”

I was able to find an ESP8266 Node MCU from https://github.com/squix78/esp8266-fritzing-parts/tree/master/nodemcu-v1.0

NodeMCUV1.0

It looks like Fritzing has all the parts I need to work with (Pi, Arduino and Node MCU’s).

All Parts

Fritzing Views

Coming soon (Views: Breadboard, schematic, PCB, code).

Designing and Ordering a PCB

Coming soon.

Find Fritzing Projects

You can find all community created Raspberry Pi Fritzing Projects below.

http://fritzing.org/projects/by-tag/raspberry%20pi/

Arduino Fritzing Projets:

http://fritzing.org/projects/by-tag/arduino/

ESP8266 Fritzing Projets:

http://fritzing.org/projects/by-tag/esp8266/

Happy coding and I hope this helps someone.

More to come.

Making a PCB

Bonus: It is possible to order your own physical PCB from say
https://easyeda.com but you will need to export your schematic from Fritzing to Eagle (or manually create your PCB in Easy EDA online IDE).

Download Eagle:
https://www.autodesk.com/products/eagle/free-download

Easy EDA guides here:

How to make a custom PCB in Easy EDA (Part 1):

How to make a custom PCB in Easy EDA (Part 2):

I will try and find a PCB manufacturer that accepts orders from Fritzing exports.

More Reading

https://www.raspberrypi.org/magpi/

Free MagPi magazine Issues.

Donate and make this blog better

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

V1.1 Making a PCB info

v1.0 Initial Draft

How to upgrade a Digital Ocean Ubuntu VM and increase the vCPU or memory

by Leave a Comment

This blog post will show you how you can increase the memory and CPU allocation of an Ubuntu Server (Droplet) on Digital Ocean.

Advertisement:



If you don’t have an Ubuntu server on Digital Ocean use this link ( https://m.do.co/c/99a5082b6de5 ) and get $10 free credit (2 months free). Read my guide here on setting it up.

Before you begin, ensure you have backed up your server.  You can read here about setting up a new server on Digital Ocean from scratch, connecting to your server via SCP or automatically syncing files away from your server to another server with rsync .

In Jan 2018 Digital Ocean doubled the ram of $5/m servers from 512MB to 1GB so it’s time for me to get the free upgrade.

Connect to your server (via SSH or Web Console) and shut it down

After the server has shut down login to digital Ocean GUI and click (open) the server you want to upgrade.

resize-droplet-002

Click Resize and choose the new upgraded server capacity

resize-droplet-003

Click Resize (if the resize button is disabled you need to power off the server (via command line or via the power menu in Digital Ocean for the Droplet))

resize-droplet-004

Click the Power On button under the Access tab when the resize is completed to restart the VM.

resize-droplet-005

Congratulations, you will now have an upgraded server 🙂 Thank You Digital Ocean for the free RAM.

If you don’t have an Ubuntu server on Digital Ocean use this link ( https://m.do.co/c/99a5082b6de5 ) and get $10 free credit (2 months free). Read my guide here on setting it up.

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History

v1.0 Initial Post

OSX speed tested before applying Spectre and Meltdown patches in 10.13.2

by Leave a Comment

Below is a quick post of how OSX speed fairs before and after application of the OSX 10.13.2 update (that fixed Spectre and Meltdown CPU issues).

Advertisement:



I read of up to 30% speed drops with Redis after Spectre and Meltdown patches and was curious how OSX faired.

News

Apple Confirms ‘Meltdown’ and ‘Spectre’ Vulnerabilities Impact All Macs and iOS Devices, Some Fixes Already Released [Updated]

MacOS High Sierra 10.13.2 Update Released with Bug Fixes 

Patches

macOS High Sierra 10.13.2 Combo Update + Supplement Update

Benchmarks (before applying patches)

Here are Cinebench R15 benchmarks on OSX 10.13.1 before upgrading to 10.13.2 on a 2-year-old  core i5 based iMac.

High Sierra 10.13.1, Cinebench R15 and TG Pro running in the background.

osx-speculative-001-before-cinebench-cpu

CPU: 490 cb points.

Now time to test the GPU in Cinebench…
osx-speculative-002-before-cinebench-cpu-gpu

GPU: 79.33 fps.

Now time to run a Geekbench CPU benchmark.

Here is a breakdown of the scores (2 mins after a fresh reboot and the best of three runs).
osx-speculative-003-before-geekbench-simple

Geekbench Single-Core Score: 4433

Geekbench Multi-Core Score: 13,005

Here is a breakdown of advanced GeekBench tests…
osx-speculative-004-before-geekbench-detailed

I tried to update to 10.3.2 via the software updates but my network was having troubles (human error, my firewall was too aggressive)
osx-speculative-005-update-failed

I downloaded the manual 10.13.2 update from Apple (not realising my firewall was blocking the app updates (I blocked app updated as soon as spectre news hit so I could write this post)).

 

I downloaded and opened the 2.8GB 10.13.2 update.
osx-speculative-006-manual-update-002osx-speculative-006-manual-update-001

PKG was extracted from the DMG.

osx-speculative-006-manual-update-003

10.13.2 Installation Wizard.

osx-speculative-006-manual-update-004

I solved my networking issues (disable my firewall) and was able to verify the installation of the 10.13.2 patch and download further 10.13.2 supplemental updates.
osx-speculative-006-manual-update-005

I verified all 10.13.2 updates were downloaded and installed and rebooted three times.
osx-speculative-006-manual-update-006

2 Minutes after a reboot I ran the same Cinebench R15 CPU benchmark.
osx-speculative-007-after-cinebench-cpu

CPU: 484 cb 

osx-speculative-008-after-cinebench-cpu-gpu

Cinebench R15 CPU dropped  about 6 points (about a 1.22448979591837% drop, not massive)

osx-speculative-009-after-geekbench-simpleb

Geek bench Single CPU scores fell from 4433 to 4194 (about 5.436498984886081% slower, not as bad as I expected).

osx-speculative-009-after-geekbench-advanced

Geek Bench Multi CPU scores fell from 13,005 to 12,868 (about 1.053440984236832% slower, not as bad as I expected).

Versions of Software

Cinebench R15

osx-speculative-010-after-cinebench-about

Geek Bench

osx-speculative-010-after-geekbench-about

Results

Cinebench CPU

Graphs are unfortunately zoomed in (I could not find out how to tell Excel to zoom out).  I will update when I find out how to zoom out results.

Green/Left = OSX 10.3.1.

Blue/Right = OSX 10.3.2 (post Spectre/Meltdown Patch).

osx-speculative-011-cinebench-results-cpu

Cinebench GPU

Green/Left = OSX 10.3.1.

Blue/Right = OSX 10.3.2 (post Spectre/Meltdown Patch).

osx-speculative-012-cinebench-results-gpu

Geekbench Single CPU

Green/Left = OSX 10.3.1.

Blue/Right = OSX 10.3.2 (post Spectre/Meltdown Patch).

osx-speculative-013-geekbench-results-single-cpu

Geekbench Multi CPU

Green/Left = OSX 10.3.1.

Blue/Right = OSX 10.3.2 (pst Spectre/Meltdown Patch.

osx-speculative-014-geekbench-results-multi-cpu

Geekbench Other

Green= OSX 10.3.1.

Blue = OSX 10.3.2 (post Spectre/Meltdown Patch).

Note: Most results were slightly slower but some were faster post patch.

osx-speculative-015-geekbench-results-detailed

Conclusion

It looks like performance falls 1-5% on a 2 year old system, not as bad as I was expecting.

Donate and make this blog better

Ask a question or recommend an article

Your Name (required)

Your Email (required)

Your Question

Revision History
v1.0 Initial Draft

Hope this helps someone.