SEO

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

June 5, 2018 by Simon

How can you measure VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

Read Part 1, Part 2, Part 3 or Part 4

Measure Disk Performance with Bonnie++

Installing Bonnie++ on Ubuntu

apt-get install bonnie++

1	apt-get install bonnie++

Read this. post on using Bonnie++

Benchmark disk IO with DD and Bonnie++

Starting Bonnie++

bonnie++ -d /tmp -r 2048 -u username

1	bonnie++ -d /tmp -r 2048 -u username

Bonnie++ Readme.

Disk io with bonnie++ on Vultr/Sydney

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 656 99 308954 68 113706 33 1200 92 188671 30 10237 251
Latency 26067us 119ms 179ms 29139us 26069us 16118us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1463us 703us 880us 263us 119us 593us
1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us

Writing a byte at a time...done

Writing intelligently...done

Rewriting...done

Reading a byte at a time...done

Reading intelligently...done

start 'em...done...done...done...done...done...

Create files in sequential order...done.

Stat files in sequential order...done.

Delete files in sequential order...done.

Create files in random order...done.

Stat files in random order...done.

Delete files in random order...done.

Version 1.97 ------Sequential Output------ --Sequential Input- --Random-

Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP

servername 4G 656 99 308954 68 113706 33 1200 92 188671 30 10237 251

Latency 26067us 119ms 179ms 29139us 26069us 16118us

Version 1.97 ------Sequential Create------ --------Random Create--------

servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP

16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++

Latency 1463us 703us 880us 263us 119us 593us

1.97,1.97,servername,1,1528177870,4G,,656,99,308954,68,113706,33,1200,92,188671,30,10237,251,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,26067us,119ms,179ms,29139us,26069us,16118us,1463us,703us,880us,263us,119us,593us

Disk io with bonnie++ on Digital Ocean/London

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 699 99 778636 74 610414 60 1556 99 1405337 59 +++++ +++
Latency 17678us 10099us 17014us 7027us 3067us 2366us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 1243us 376us 611us 108us 59us 181us
1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us

Writing a byte at a time...done

Writing intelligently...done

Rewriting...done

Reading a byte at a time...done

Reading intelligently...done

start 'em...done...done...done...done...done...

Create files in sequential order...done.

Stat files in sequential order...done.

Delete files in sequential order...done.

Create files in random order...done.

Stat files in random order...done.

Delete files in random order...done.

Version 1.97 ------Sequential Output------ --Sequential Input- --Random-

Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP

servername 4G 699 99 778636 74 610414 60 1556 99 1405337 59 +++++ +++

Latency 17678us 10099us 17014us 7027us 3067us 2366us

Version 1.97 ------Sequential Create------ --------Random Create--------

servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP

16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++

Latency 1243us 376us 611us 108us 59us 181us

1.97,1.97,servername,1,1528186398,4G,,699,99,778636,74,610414,60,1556,99,1405337,59,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,17678us,10099us,17014us,7027us,3067us,2366us,1243us,376us,611us,108us,59us,181us

Disk io with bonnie++ on UpCloud/Singapore

Writing a byte at a time...done
Writing intelligently...done
Rewriting...done
Reading a byte at a time...done
Reading intelligently...done
start 'em...done...done...done...done...done...
Create files in sequential order...done.
Stat files in sequential order...done.
Delete files in sequential order...done.
Create files in random order...done.
Stat files in random order...done.
Delete files in random order...done.
Version 1.97 ------Sequential Output------ --Sequential Input- --Random-
Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
servername 4G 1014 99 407179 24 366622 32 2137 99 451886 17 +++++ +++
Latency 11297us 54232us 16443us 4949us 44883us 1595us
Version 1.97 ------Sequential Create------ --------Random Create--------
servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++
Latency 264us 340us 561us 138us 66us 327us
1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us

Writing a byte at a time...done

Writing intelligently...done

Rewriting...done

Reading a byte at a time...done

Reading intelligently...done

start 'em...done...done...done...done...done...

Create files in sequential order...done.

Stat files in sequential order...done.

Delete files in sequential order...done.

Create files in random order...done.

Stat files in random order...done.

Delete files in random order...done.

Version 1.97 ------Sequential Output------ --Sequential Input- --Random-

Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--

Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP

servername 4G 1014 99 407179 24 366622 32 2137 99 451886 17 +++++ +++

Latency 11297us 54232us 16443us 4949us 44883us 1595us

Version 1.97 ------Sequential Create------ --------Random Create--------

servername -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--

files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP

16 +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++ +++++ +++

Latency 264us 340us 561us 138us 66us 327us

1.97,1.97,servername,1,1528226703,4G,,1014,99,407179,24,366622,32,2137,99,451886,17,+++++,+++,16,,,,,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,+++++,+++,11297us,54232us,16443us,4949us,44883us,1595us,264us,340us,561us,138us,66us,327us

Now read this site on how to make sense of this data

< Previous – Next >

Read Part 1, Part 2, Part 3 or Part 4

Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX

May 15, 2018 by Simon

This guide will aim to inform you of strong cryptographic protocols and ciphers to use on a web server on Ubuntu 16.04 and NGINX.

Secure encryption protocols are used to secure communications between a server and client. Older SSL protocols like Netscape’s Secure Sockets Layer (SSL) are flagged as DO NOT USE use by the Internet Engineering Task Force (IETF). Newer protocols like Transport Layer Security (TLS) are the newer recommended SSL protocols to use.

Wikipedia Article on Cryptographic Protocol’s

A security protocol (cryptographic protocol or encryption protocol) is an abstract or concrete protocol that performs a security-related function and applies cryptographic methods, often as sequences of cryptographic primitives. A protocol describes how the algorithms should be used. A sufficiently detailed protocol includes details about data structures and representations, at which point it can be used to implement multiple, interoperable versions of a program.

Wikipedia on Ciphers

In cryptography, a cipher (or cypher) is an algorithm for performing encryption or decryption—a series of well-defined steps that can be followed as a procedure. An alternative, less common term is encipherment. To encipher or encode is to convert information into cipher or code. In common parlance, “cipher” is synonymous with “code,” as they are both a set of steps that encrypt a message; however, the concepts are distinct in cryptography, especially classical cryptography.

Wikipedia article on Elliptic-curve cryptography

Wikipedia article on Diffie–Hellman key exchange

Bad SSL Assumptions I have heard for not using HTTPS

I am not a bank so I don’t need HTTPS
SSL overhead is was too high on servers.
My site only has static content, I don’t need HTTPS
I don’t need SSL to secure my site I just need to be less of a target than others
I don’t hold confidential information (Wrong)

Don’t be Lazy and secure a site poorly

A local business that wanted me to buy their goods is not convincing me.

(tested with SSL labs and asafaweb)

Why SSL

If you are unsure of why you need SSL visit https://doesmysiteneedhttps.com/, Avoiding the Not Secure Warning in Chrome, Why HTTPS matters and securing your site with HTTPS.

Google has an HTTPS usage graph for all communications to its services (hint it’s growing): https://transparencyreport.google.com/https/overview?hl=en

SSL Future

SSL is here to stay, Non-SSL sites will soon be labelled insecure, Non-SSL sites will have Search Engine Optimization (SEO) adversely affected.

Also, secure pages will be treated as normal (not flagged as secure)

In October, Chrome will remove the “secure” indicator on all HTTPS pages and mark pages that do no use the secure version of the HTTP protocol with a red “not secure” warning. This change will make the web safer to use by default. https://t.co/ar3lwB9aRt
— J-François Lavigne (@jflavigne) May 25, 2018

History of Protocol’s – Launch Dates

SSL 1.0 (never launched)
SSL 2.0 1995
SSL 3.0 1996
TLS 1.0 1999
TLS 1.1 2006
TLS 1.2 2008
TLS 1.3 2018

Sites like https://caniuse.com can show you if our browser an use new protocols like TLS (e.g TLS 1.0, TLS 1.1, TLS 1.2, TLS 1.3)

TLS 1.0 is supported by All Browsers
TLS 1.1 is supported on IE11+, Edge, Firefox 24+, Chrome 22+, Safari 7+, Opera 12.1+, iOS Safari 5.1+, Chrome 62 on Android 5+ etc
TLS 1.2 is supported on IE11+, Edge, Firefox 27, Chrome 30+, Safari 7+, Opera 17+, iOS Safari 5.1, Chrome 62 on Android 5+ etc
TLS 1.3 is not Supported by IE, Edge, Safari, iOS Safari, Android but is supported by Firefox 52, Chrome 56, Opera 43.

I have a guide here on setting up TLS 1.3 on Ubuntu 16.05 and Chrome, I use the draft build of OpenSSL but Open SSL 1.1.1 will support TLS 1.3. I am still figuring our TLS 1.3 on Ubuntu 18.04.

At the time of writing, you need to opt into TLS 1.3 draft specification in Chrome.

Cipher or Cypher

Read this page to see the history of the word Cipher or Cypher?

Buying an SSL certificate

Opening your wallet may not buy you the best certificate either, this was an SSL Labs review of a $150 SSL certificate Ii purchased a few years ago from a CPanel web host.

I don’t buy commercial certificates anymore, I prefer free SSL certificates from Lets Encrypt

SSL Strength

I prefer to set up my own (free) SSL certificate with Lest Encrypt and tets those certificated with https://dev.ssllabs.com/ssltest/

You can configure your web server to only use certain protocols.

ssl_prefer_server_ciphers on;
ssl_protocols TLSv1.2 TLSv1.3;

1 2	ssl_prefer_server_ciphers on; ssl_protocols TLSv1.2 TLSv1.3;

And define preferred ciphers

ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256;

1	ssl_ciphers ECDHE-RSA-AES256-GCM-SHA512:DHE-RSA-AES256-GCM-SHA512:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256;

Don’t forget to renew your SSL certificates ahead of time.

Also run a modern browser like Google Chrome Canary as some old browsers thnk expired SSL certificates are Secure

Ciphers

OpenSSL has implemented support for five TLS v1.3 cipher suites:

TLS13-AES-256-GCM-SHA384
TLS13-CHACHA20-POLY1305-SHA256
TLS13-AES-128-GCM-SHA256
TLS13-AES-128-CCM-8-SHA256
TLS13-AES-128-CCM-SHA256

Test OpenSSL Cipher Suites

openssl ciphers -s -v
TLS_AES_256_GCM_SHA384  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(256) Mac=AEAD
TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any      Au=any  Enc=CHACHA20/POLY1305(256) Mac=AEAD
TLS_AES_128_GCM_SHA256  TLSv1.3 Kx=any      Au=any  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(256) Mac=AEAD
ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(256) Mac=AEAD
DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(256) Mac=AEAD
ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH     Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH       Au=RSA  Enc=CHACHA20/POLY1305(256) Mac=AEAD
ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AESGCM(128) Mac=AEAD
ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AESGCM(128) Mac=AEAD
DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH       Au=RSA  Enc=AESGCM(128) Mac=AEAD
ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA384
ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA384
DHE-RSA-AES256-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA256
ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA256
ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA256
DHE-RSA-AES128-SHA256   TLSv1.2 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA256
ECDHE-ECDSA-AES256-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(256)  Mac=SHA1
ECDHE-RSA-AES256-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
ECDHE-ECDSA-AES128-SHA  TLSv1 Kx=ECDH     Au=ECDSA Enc=AES(128)  Mac=SHA1
ECDHE-RSA-AES128-SHA    TLSv1 Kx=ECDH     Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
AES256-GCM-SHA384       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(256) Mac=AEAD
AES128-GCM-SHA256       TLSv1.2 Kx=RSA      Au=RSA  Enc=AESGCM(128) Mac=AEAD
AES256-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA256
AES128-SHA256           TLSv1.2 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA256
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1

openssl ciphers -s -v

TLS_AES_256_GCM_SHA384 TLSv1.3 Kx=any Au=any Enc=AESGCM(256) Mac=AEAD

TLS_CHACHA20_POLY1305_SHA256 TLSv1.3 Kx=any Au=any Enc=CHACHA20/POLY1305(256) Mac=AEAD

TLS_AES_128_GCM_SHA256 TLSv1.3 Kx=any Au=any Enc=AESGCM(128) Mac=AEAD

ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(256) Mac=AEAD

ECDHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(256) Mac=AEAD

DHE-RSA-AES256-GCM-SHA384 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(256) Mac=AEAD

ECDHE-ECDSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=ECDSA Enc=CHACHA20/POLY1305(256) Mac=AEAD

ECDHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=ECDH Au=RSA Enc=CHACHA20/POLY1305(256) Mac=AEAD

DHE-RSA-CHACHA20-POLY1305 TLSv1.2 Kx=DH Au=RSA Enc=CHACHA20/POLY1305(256) Mac=AEAD

ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AESGCM(128) Mac=AEAD

ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD

DHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AESGCM(128) Mac=AEAD

ECDHE-ECDSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA384

ECDHE-RSA-AES256-SHA384 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA384

DHE-RSA-AES256-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(256) Mac=SHA256

ECDHE-ECDSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA256

ECDHE-RSA-AES128-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA256

DHE-RSA-AES128-SHA256 TLSv1.2 Kx=DH Au=RSA Enc=AES(128) Mac=SHA256

ECDHE-ECDSA-AES256-SHA TLSv1 Kx=ECDH Au=ECDSA Enc=AES(256) Mac=SHA1

ECDHE-RSA-AES256-SHA TLSv1 Kx=ECDH Au=RSA Enc=AES(256) Mac=SHA1

DHE-RSA-AES256-SHA SSLv3 Kx=DH Au=RSA Enc=AES(256) Mac=SHA1

ECDHE-ECDSA-AES128-SHA TLSv1 Kx=ECDH Au=ECDSA Enc=AES(128) Mac=SHA1

ECDHE-RSA-AES128-SHA TLSv1 Kx=ECDH Au=RSA Enc=AES(128) Mac=SHA1

DHE-RSA-AES128-SHA SSLv3 Kx=DH Au=RSA Enc=AES(128) Mac=SHA1

AES256-GCM-SHA384 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(256) Mac=AEAD

AES128-GCM-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AESGCM(128) Mac=AEAD

AES256-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA256

AES128-SHA256 TLSv1.2 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA256

AES256-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(256) Mac=SHA1

AES128-SHA SSLv3 Kx=RSA Au=RSA Enc=AES(128) Mac=SHA1

A handy guide about using ciphers

SSL/TLS: How to choose your cipher suite

Testing a remote host’s ciphers and protocols with cipherscan

Clone this repository: https://github.com/mozilla/cipherscan

Scan a site

./cipherscan fearby.com

1	./cipherscan fearby.com

Result

Target: fearby.com:443

prio  ciphersuite                        protocols  pfs                 curves
1     ECDHE-ECDSA-CHACHA20-POLY1305-OLD  TLSv1.2    ECDH,P-256,256bits  prime256v1
2     ECDHE-ECDSA-AES128-GCM-SHA256      TLSv1.2    ECDH,P-256,256bits  prime256v1
3     ECDHE-ECDSA-AES128-SHA             TLSv1.2    ECDH,P-256,256bits  prime256v1
4     ECDHE-ECDSA-AES128-SHA256          TLSv1.2    ECDH,P-256,256bits  prime256v1
5     ECDHE-ECDSA-AES256-GCM-SHA384      TLSv1.2    ECDH,P-256,256bits  prime256v1
6     ECDHE-ECDSA-AES256-SHA             TLSv1.2    ECDH,P-256,256bits  prime256v1
7     ECDHE-ECDSA-AES256-SHA384          TLSv1.2    ECDH,P-256,256bits  prime256v1

Certificate: trusted, 256 bits, ecdsa-with-SHA256 signature
TLS ticket lifetime hint: 64800
NPN protocols: h2,http/1.1
OCSP stapling: supported
Cipher ordering: server
Curves ordering: server - fallback: no
Server supports secure renegotiation
Server supported compression methods: NONE
TLS Tolerance: yes

Intolerance to:
 SSL 3.254           : absent
 TLS 1.0             : PRESENT
 TLS 1.1             : PRESENT
 TLS 1.2             : absent
 TLS 1.3             : absent
 TLS 1.4             : absent

Target: fearby.com:443

prio ciphersuite protocols pfs curves

1 ECDHE-ECDSA-CHACHA20-POLY1305-OLD TLSv1.2 ECDH,P-256,256bits prime256v1

2 ECDHE-ECDSA-AES128-GCM-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1

3 ECDHE-ECDSA-AES128-SHA TLSv1.2 ECDH,P-256,256bits prime256v1

4 ECDHE-ECDSA-AES128-SHA256 TLSv1.2 ECDH,P-256,256bits prime256v1

5 ECDHE-ECDSA-AES256-GCM-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1

6 ECDHE-ECDSA-AES256-SHA TLSv1.2 ECDH,P-256,256bits prime256v1

7 ECDHE-ECDSA-AES256-SHA384 TLSv1.2 ECDH,P-256,256bits prime256v1

Certificate: trusted, 256 bits, ecdsa-with-SHA256 signature

TLS ticket lifetime hint: 64800

NPN protocols: h2,http/1.1

OCSP stapling: supported

Cipher ordering: server

Curves ordering: server - fallback: no

Server supports secure renegotiation

Server supported compression methods: NONE

TLS Tolerance: yes

Intolerance to:

SSL 3.254 : absent

TLS 1.0 : PRESENT

TLS 1.1 : PRESENT

TLS 1.2 : absent

TLS 1.3 : absent

TLS 1.4 : absent

Cipher scan can also recommend settings to change to help you harden a server (based on https://wiki.mozilla.org/Security/Server_Side_TLS)

Analyze Command

./analyze.py -t fearby.com

1	./analyze.py -t fearby.com

Results

fearby.com:443 has bad ssl/tls

Things that are bad:
* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD

Changes needed to match the old level:
* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD
* enable TLSv1.1
* enable TLSv1
* enable SSLv3
* add cipher DES-CBC3-SHA
* use a certificate with sha1WithRSAEncryption signature
* use DHE of 1024bits and ECC of 160bits

Changes needed to match the intermediate level:
* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD
* consider enabling TLSv1.1
* consider enabling TLSv1
* add cipher AES128-SHA
* use a certificate signed with sha256WithRSAEncryption

Changes needed to match the modern level:
* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD
* remove cipher ECDHE-ECDSA-AES128-SHA
* remove cipher ECDHE-ECDSA-AES256-SHA

fearby.com:443 has bad ssl/tls

Things that are bad:

* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD

Changes needed to match the old level:

* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD

* enable TLSv1.1

* enable TLSv1

* enable SSLv3

* add cipher DES-CBC3-SHA

* use a certificate with sha1WithRSAEncryption signature

* use DHE of 1024bits and ECC of 160bits

Changes needed to match the intermediate level:

* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD

* consider enabling TLSv1.1

* consider enabling TLSv1

* add cipher AES128-SHA

* use a certificate signed with sha256WithRSAEncryption

Changes needed to match the modern level:

* remove cipher ECDHE-ECDSA-CHACHA20-POLY1305-OLD

* remove cipher ECDHE-ECDSA-AES128-SHA

* remove cipher ECDHE-ECDSA-AES256-SHA

More info on hardening here.

TLS 1.3 Information

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

March 13, 2018 by Simon

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Buy a Domain

Buy a domain name from Namecheap here.

Cloudflare Benefits (Free Plan)

DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
Global CDN
Shared SSL certificate (I disabled this and opted to use my own)
Access to audit logs
3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Setup

You will need to sign up at cloudflare.com

After you create an account you will be prompted to add a siteCloudflare will pull your public DNS records to import.

You will be prompted to select a plan (I selected free)

Verify DNS settings to import.

You will now be asked to change your DNS nameservers with your domain reseller

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

Could I kindly ask if you are reading this that you visit https://t.co/9x5TFARLCt, I am writing a @Cloudflare blog post and need to screenshot stats. Thanks in advance
— Simon Fearby (Developer) (@FearbySoftware) March 13, 2018

The Cloudflare CTO responded. 🙂

Sure thing 🙂
— John Graham-Cumming (@jgrahamc) March 13, 2018

Confirm Cloudflare link to a domain from the OSX Comand line

host -t NS fearby.com
fearby.com name server dane.ns.cloudflare.com.
fearby.com name server nora.ns.cloudflare.com.

host -t NS fearby.com

fearby.com name server dane.ns.cloudflare.com.

fearby.com name server nora.ns.cloudflare.com.

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

“fearby.com.com/wp-*” Cache level: Bypass
“fearby.com.com/wp-admin/post.php*” Cache level: Bypass
“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours. having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before). I just need to wait for caching and minification to kick in.

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

Load Time: 1.80s
First Byte 0.176s
Start Render 1.200s

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here. A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

Thanks, Cloudflare.

Cloudflare Pros

Enabling Attack mode was simple.
Soaked up an attack.
Free Tier
Many Reports
Option to force HTTPS over HTTP
Option to ban/challenge suspicious IP’s and set challenge timeframes.
Ability to setup IP firewall rules and Application Firewalls.
User-agent blocking
Lockdown URL’s to IP’s (pro feature)
Option to minify Javascript, CSS and HTML
Option to accelerate mobile links
Brotli compression on assets served.
Optio to enable BETA Rocket loader for Javascript performance tweaks.
Run Javascript service workers from the 120+ CDN’s
Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
HTTP/2 on, IPV6 ON
Option to setup load balancing/failover
CTO of Cloudflare responded in Twitter 🙂
Option to enable rate limiting (charged at 10,000 hits for $0.05c)
Option to block countries (pro feature)
Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
WordPress draft posts are being cached even though page riles block wp-admin page caching.
Would be nice to have ad automatic Under Attack mode
Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post

Manage Social Media posts with Buffer

October 10, 2017 by Simon

Here is a quick setup guide for Buffer.com where you can connect to and post (manually or scheduled) to multiple social media platforms.

You can view pricing here. You can signup for a free Buffer Individual plan: https://buffer.com/signup. Signup to Buffer (Free, Limited)

Post Signup Setup

Connect Buffer to Social Media Platforms

Type post content

Change the default image

Choose the platforms and images

Schedule the Post

You can manually share to a platform at any time.

TIP: If you share now you will need to manually share on each platform separately.

Results

Buffer features I like

Good Free Plan
Post Scheduling
Image Creation Integration (Paid)
Reply integration (Paid)
Manage all your social accounts from one simple dashboard
Ability to set custom posting slots.

Buffer features I Don’t like

Manual Share to all feature missing.
Timezones earlier than US Timezones appear to be untouchable (my Timezone is set)

Buffer FAQ’s: https://faq.buffer.com/

Tip: Create custom posting slots

More soon (reply automation and image creation).

Donate and make this blog better

Ask a question or recommend an article

Revision History

v1.1 Added Posting Slots Info

etc

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads

September 9, 2017 by Simon

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads.

This guide is a shorter post around setting up SEO (Search Engine Optimization) and driving more traffic to your site without buying ADs. In a nutshell, to have better SEO you need to jump some technical hurdles in order to drive more traffic to your site from search engines along with understanding your customer’s needs and making things easier for them.

I have blogged about this topics before but these posts are too long in reflection.

Buying Ad’s?

Facebook, Google, Bing and advertising agencies will recommend you set goals around growth and site traffic and pay for those goals to succeed (usually by advertisements).

Don’t get me wrong Advertising works but it is a competitive market, Online sites can easily setup the display of Ad’s on their site (my guide here Add Google AdWords to your WordPress blog, https://fearby.com/article/add-google-adwords-wordpress-blog/ ). You can buy physical billboard ad’s on the side of roads (e.g http://www.buythisspace.com.au/). I tried to enquire about the costs of a physical billboard but the agencies robot verification rejected my enquiry submission so I gave up. Advertising is buying peoples times and people now how to avoid ad’s and not interact with them (7 Marketing Lessons from Eye-Tracking Studies https://blog.kissmetrics.com/eye-tracking-studies/)

Do more of what works

Spoiler: This guide will recommend you do more of what works over buying millions of ad’s and hoping for new and engaged customers and customer growth.

If you don’t already have Google Analytics setup on your site then do it, you cannot identify your customers or identify what is broken or in turn fix it (Setting up Google Analytics on your website, https://fearby.com/article/setting-up-google-analytics-on-your-website/ )
Monitor Data – Do review your logs and customer related data (review orders, customers and try and identify what works. Software like https://www.zoho.com/one/applications/web.html will help you connect the dots.
Adobe Audience Cloud: http://www.adobe.com/au/experience-cloud.html is a more expensive software suite for driving decisions based on data.
Benchmarks – Set goals and work toward them (e.g I want 10x more customers).

SEO Tip’s

This older article on How to boost your site’s SEO attempts to mention what you need to do it to get better SEO.

Do run a modern great site

I am a big fan of word of mouth over free/organic traffic over paid customers via advertising (Mostly because I am tight and realize advertising can be a bottomless pit). The single biggest thing you can do to have more organic traffic from search engines is run a modern and fast website, have valuable content and make it as easy for the customer as possible. This is why I moved my site and setup an SSL certificate (link to article).

Search engines like your site to be fast, updated frequently, have sitemaps to make their jobs easier and have an SSL certificate to keep the web safe etc.

Google, Bing and other search engines will not send traffic your way if you do not satisfy them that your site is liked or has valuable content. Google makes money from Google Analytics by helping people understand their site’s visitors then recommend you pay for ad’s to use on sites that have AdWords on their site ( WordPress to a new self-managed server away from CPanel ).

How to boost your site’s SEO https://fearby.com/article/how-to-boost-your-sites-seo/
Your website needs to be fast, use sites like https://www.webpagetest.org to measure how fast your site is (Aim for all A’s). Read this page for information on the impact of slow websites https://www.searchenginejournal.com/mobile-page-speed-benchmarks/194511/
Mobile friendly – Ensure your site is mobile friendly (or risk being dropped from search engine results)
SSL – Do have a secure SSL certificate on your website (view mine here https://www.ssllabs.com/ssltest/analyze.html?d=www.fearby.com&s=45.63.29.217&latest).
Incoming links – Having incoming links to your site tell search engines that your site is popular.

Traffic Source types

Organic – An organic visitor to your site is one who found your site by searching something that was relevant to their search term and not by clicking on an advertisement.
Paid – A paid user is someone who has clicked an ad to come to your site.
Social – A social visitor is one who is known to come from a social media site, using social media sites like Twitter, Facebook or Instagram is a must to driving organic traffic (go where the people are).

Engagement

How engaged are your customers? Have you asked your customers recently what they value or appreciate about your business or product? Have you asked for feedback recently?

User Engagement Levels

None – Do you have landing pages that quickly inform customers of your products or services?
Low – What do they need to know about your product or service?
Medium – Aware (engaged)
High – Can this person be an advocate for your business?
Gone – Did you get exit Feedback?

Ways to engage already engaged customers.

Setup a free MailChimp Newsletter to allow willing people to be alerted of new communication https://login.mailchimp.com/signup/?source=website&pid=GAW
Web Browser popup Alerts can be a great way to engage with users when new content is added to your site (Read the guide here https://documentation.onesignal.com/docs/web-push-setup )
Mobile apps or mobile friendly website are a no brainer given 2 billion people use mobile phones ( http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ ).

What can you do to help understand your customer’s needs and make their purchase processes easier?

Why are your customers leaving?

Understand more about your customers reasons for leaving and act upon preventing others from leaving.

Trying something new (Does your website need to be simpler?)
Are your products too expensive?.
Your site (or ordering) is not convenient (Do you need to setup online ordering/subscriptions and delivery?)
etc

Who are your customers

Personas – Do setup customer personas in order to focus on your customer segments (get a free customer persona template here https://blog.hubspot.com/blog/tabid/6307/bid/33491/everything-marketers-need-to-research-create-detailed-buyer-personas-template.aspx )
Does your website match these personas?

Are your customers.

Engaged
Informed
Advocates

Feedback

Do you have feedback loops (A simple feedback form can solve this)?

What do you know about your customers?

Product Satisfaction
Product Loyalty
Product Awareness

Paid Traffic (Ad’s)

Google Ad’s – Signup Here http://www.google.com.au/adwords/get-started/
Bing – Advertise on Bing here https://advertise.bingads.microsoft.com/
Facebook – Advertise on Facebook here https://www.facebook.com/business/products/ads

Free Traffic (SEO + Organic Ad’s)

Blog Posts (Sharing value/passion)
Social Media Posts (use hashtags)
Instagram (Post value/passion)

Most importantly Do what works (Measure and replicate).

Focus on Business Value

Generate a SWOT Analysis ( Free tool here https://xtensio.com/ )

What are your Strengths?
What are your Weaknesses?
What are your Opportunities?
What are your Threats?

Goals

Goals allow you to investigate, learn, act and measure I order to improve.

Investigate – Data.
Learn/Insight – Make Assumptions.
Act – Act and measure.

Read more about customer engagement here https://en.wikipedia.org/wiki/Customer_engagement

Bonus

Do ensure your website is compliant with accessibility and technical standards

Test our sites Accessibility – https://achecker.ca/checker/index.php
Test your sites HTML5 Compliance – https://validator.w3.org
Test your Google PageSpeed Test – https://developers.google.com/speed/pagespeed/insights/
Do A B testing to determine the statistical significance of changes to your site.

Conclusion

The more you know the better you can connect, Do set goals and as a minimum setup Google Analytics, SSL certificate and submit your site to search engines, then focus on a fast site that makes things simple for your customers.

Donate and make this blog better

Ask a question or recommend an article

v1.0 Initial version

Securing Ubuntu in the cloud

August 9, 2017 by Simon

It is easy to deploy servers to the cloud within a few minutes, you can have a cloud-based server that you (or others can use). ubuntu has a great guide on setting up basic security issues but what do you need to do.

If you do not secure your server expect it to be hacked into. Below are tips on securing your cloud server.

First, read more on scanning your server with Lynis security scan.

Always use up to date software

Always use update software, malicious users can detect what software you use with sites like shodan.io (or use port scan tools) and then look for weaknesses from well-published lists (e.g WordPress, Windows, MySQL, node, LifeRay, Oracle etc). People can even use Google to search for login pages or sites with passwords in HTML (yes that simple). Once a system is identified by a malicious user they can send automated bots to break into your site (trying millions of passwords a day) or use tools to bypass existing defences (Security researcher Troy Hunt found out it’s child’s play).

Portscan sites like https://mxtoolbox.com/SuperTool.aspx?action=scan are good for knowing what you have exposed.

You can also use local programs like nmap to view open ports

Instal nmap

sudo apt-get install nmap

1	sudo apt-get install nmap

Find open ports

nmap -v -sT localhost

Starting Nmap 7.01 ( https://nmap.org ) at 2017-08-08 23:57 AEST
Initiating Connect Scan at 23:57
Scanning localhost (127.0.0.1) [1000 ports]
Discovered open port 80/tcp on 127.0.0.1
Discovered open port 3306/tcp on 127.0.0.1
Discovered open port 22/tcp on 127.0.0.1
Discovered open port 9101/tcp on 127.0.0.1
Discovered open port 9102/tcp on 127.0.0.1
Discovered open port 9103/tcp on 127.0.0.1
Completed Connect Scan at 23:57, 0.05s elapsed (1000 total ports)
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00020s latency).
Not shown: 994 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
80/tcp   open  http
3306/tcp open  mysql
9101/tcp open  jetdirect
9102/tcp open  jetdirect
9103/tcp open  jetdirect

Read data files from: /usr/bin/../share/nmap
Nmap done: 1 IP address (1 host up) scanned in 0.17 seconds
           Raw packets sent: 0 (0B) | Rcvd: 0 (0B)

nmap -v -sT localhost

Starting Nmap 7.01 ( https://nmap.org ) at 2017-08-08 23:57 AEST

Initiating Connect Scan at 23:57

Scanning localhost (127.0.0.1) [1000 ports]

Discovered open port 80/tcp on 127.0.0.1

Discovered open port 3306/tcp on 127.0.0.1

Discovered open port 22/tcp on 127.0.0.1

Discovered open port 9101/tcp on 127.0.0.1

Discovered open port 9102/tcp on 127.0.0.1

Discovered open port 9103/tcp on 127.0.0.1

Completed Connect Scan at 23:57, 0.05s elapsed (1000 total ports)

Nmap scan report for localhost (127.0.0.1)

Host is up (0.00020s latency).

Not shown: 994 closed ports

PORT STATE SERVICE

22/tcp open ssh

80/tcp open http

3306/tcp open mysql

9101/tcp open jetdirect

9102/tcp open jetdirect

9103/tcp open jetdirect

Read data files from: /usr/bin/../share/nmap

Nmap done: 1 IP address (1 host up) scanned in 0.17 seconds

Raw packets sent: 0 (0B) | Rcvd: 0 (0B)

Limit ssh connections

Read more on setting permissions here.

Chmod help can be found here.

Install Fail2Ban

I used this guide on installing Fail2Ban.

apt-get install fail2ban

1	apt-get install fail2ban

Check Fail2Ban often and add blocks to the firewall of known bad IPs

fail2ban-client status

1	fail2ban-client status

Best practices

Ubuntu has a guide on basic security setup here.

Startup Processes

It is a good idea to review startup processes from time to time.

sudo apt-get install rcconf
sudo rcconf

1 2	sudo apt-get install rcconf sudo rcconf

Accounts

Read up on the concept of least privilege access for apps and services here.
Read up on chmod permissions.

Updates

Do update your operating system often.

sudo apt-get update
sudo apt-get upgrade

1 2	sudo apt-get update sudo apt-get upgrade

Minimal software

Only install what software you need

Exploits and Keeping up to date

Do keep up to date with exploits and vulnerabilities

Follow 0xDUDE on twitter.
Read the GDI.Foundation page.
Visit the Exploit Database
Vulnerability & Exploit Database
Subscribe to the Security Now podcast.

Secure your applications

NodeJS: Enable logging in applications you install or develop.

Ban repeat Login attempts with FailBan

Fail2Ban config

sudo nano /etc/fail2ban/jail.conf

1	sudo nano /etc/fail2ban/jail.conf

[sshd]

enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 3

[sshd]

enabled = true

port = ssh

filter = sshd

logpath = /var/log/auth.log

maxretry = 3

Hosts File Hardening

sudo nano /etc/host.conf

1	sudo nano /etc/host.conf

Add

order bind,hosts
nospoof on

1 2	order bind,hosts nospoof on

Add a whitelist with your ip on /etc/fail2ban/jail.conf (see this)

[DEFAULT]
# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not                          
# ban a host which matches an address in this list. Several addresses can be                             
# defined using space separator.
                                                                         
ignoreip = 127.0.0.1 192.168.1.0/24 8.8.8.8

[DEFAULT]

# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not

# ban a host which matches an address in this list. Several addresses can be

# defined using space separator.

ignoreip = 127.0.0.1 192.168.1.0/24 8.8.8.8

Restart the service

sudo service fail2ban restart
sudo service fail2ban status

1 2	sudo service fail2ban restart sudo service fail2ban status

Intrusion detection (logging) systems

Tripwire will not block or prevent intrusions but it will log and give you a heads up with risks and things of concern

Install Tripwire.

sudo apt-get install tiger tripwire

1	sudo apt-get install tiger tripwire

Running Tripwire

sudo tiger

1	sudo tiger

This will scan your system for issues of note

sudo tiger
Tiger UN*X security checking system
   Developed by Texas A&M University, 1994
   Updated by the Advanced Research Corporation, 1999-2002
   Further updated by Javier Fernandez-Sanguino, 2001-2015
   Contributions by Francisco Manuel Garcia Claramonte, 2009-2010
   Covered by the GNU General Public License (GPL)

Configuring...

Will try to check using config for 'x86_64' running Linux 4.4.0-89-generic...
--CONFIG-- [con005c] Using configuration files for Linux 4.4.0-89-generic. Using
           configuration files for generic Linux 4.
Tiger security scripts *** 3.2.3, 2008.09.10.09.30 ***
20:42> Beginning security report for simon.
20:42> Starting file systems scans in background...
20:42> Checking password files...
20:42> Checking group files...
20:42> Checking user accounts...
20:42> Checking .rhosts files...
20:42> Checking .netrc files...
20:42> Checking ttytab, securetty, and login configuration files...
20:42> Checking PATH settings...
20:42> Checking anonymous ftp setup...
20:42> Checking mail aliases...
20:42> Checking cron entries...
20:42> Checking 'services' configuration...
20:42> Checking NFS export entries...
20:42> Checking permissions and ownership of system files...
--CONFIG-- [con010c] Filesystem 'fuse.lxcfs' used by 'lxcfs' is not recognised as a valid filesystem
20:42> Checking for indications of break-in...
--CONFIG-- [con010c] Filesystem 'fuse.lxcfs' used by 'lxcfs' is not recognised as a valid filesystem
20:42> Performing rootkit checks...
20:42> Performing system specific checks...
20:46> Performing root directory checks...
20:46> Checking for secure backup devices...
20:46> Checking for the presence of log files...
20:46> Checking for the setting of user's umask...
20:46> Checking for listening processes...
20:46> Checking SSHD's configuration...
20:46> Checking the printers control file...
20:46> Checking ftpusers configuration...
20:46> Checking NTP configuration...
20:46> Waiting for filesystems scans to complete...
20:46> Filesystems scans completed...
20:46> Performing check of embedded pathnames...
20:47> Security report completed for simon.
Security report is in `/var/log/tiger/security.report.simon.170809-20:42'.

sudo tiger

Tiger UN*X security checking system

Developed by Texas A&M University, 1994

Updated by the Advanced Research Corporation, 1999-2002

Further updated by Javier Fernandez-Sanguino, 2001-2015

Contributions by Francisco Manuel Garcia Claramonte, 2009-2010

Covered by the GNU General Public License (GPL)

Configuring...

Will try to check using config for 'x86_64' running Linux 4.4.0-89-generic...

--CONFIG-- [con005c] Using configuration files for Linux 4.4.0-89-generic. Using

configuration files for generic Linux 4.

Tiger security scripts *** 3.2.3, 2008.09.10.09.30 ***

20:42> Beginning security report for simon.

20:42> Starting file systems scans in background...

20:42> Checking password files...

20:42> Checking group files...

20:42> Checking user accounts...

20:42> Checking .rhosts files...

20:42> Checking .netrc files...

20:42> Checking ttytab, securetty, and login configuration files...

20:42> Checking PATH settings...

20:42> Checking anonymous ftp setup...

20:42> Checking mail aliases...

20:42> Checking cron entries...

20:42> Checking 'services' configuration...

20:42> Checking NFS export entries...

20:42> Checking permissions and ownership of system files...

--CONFIG-- [con010c] Filesystem 'fuse.lxcfs' used by 'lxcfs' is not recognised as a valid filesystem

20:42> Checking for indications of break-in...

--CONFIG-- [con010c] Filesystem 'fuse.lxcfs' used by 'lxcfs' is not recognised as a valid filesystem

20:42> Performing rootkit checks...

20:42> Performing system specific checks...

20:46> Performing root directory checks...

20:46> Checking for secure backup devices...

20:46> Checking for the presence of log files...

20:46> Checking for the setting of user's umask...

20:46> Checking for listening processes...

20:46> Checking SSHD's configuration...

20:46> Checking the printers control file...

20:46> Checking ftpusers configuration...

20:46> Checking NTP configuration...

20:46> Waiting for filesystems scans to complete...

20:46> Filesystems scans completed...

20:46> Performing check of embedded pathnames...

20:47> Security report completed for simon.

Security report is in `/var/log/tiger/security.report.simon.170809-20:42'.

My Output.

sudo nano /var/log/tiger/security.report.username.170809-18:42

Security scripts *** 3.2.3, 2008.09.10.09.30 ***
Wed Aug  9 18:42:24 AEST 2017
20:42> Beginning security report for username (x86_64 Linux 4.4.0-89-generic).

# Performing check of passwd files...
# Checking entries from /etc/passwd.
--WARN-- [pass014w] Login (bob) is disabled, but has a valid shell.
--WARN-- [pass014w] Login (root) is disabled, but has a valid shell.
--WARN-- [pass015w] Login ID sync does not have a valid shell (/bin/sync).
--WARN-- [pass012w] Home directory /nonexistent exists multiple times (3) in
         /etc/passwd.
--WARN-- [pass012w] Home directory /run/systemd exists multiple times (2) in
         /etc/passwd.
--WARN-- [pass006w] Integrity of password files questionable (/usr/sbin/pwck
         -r).

# Performing check of group files...

# Performing check of user accounts...
# Checking accounts from /etc/passwd.
--WARN-- [acc021w] Login ID dnsmasq appears to be a dormant account.
--WARN-- [acc022w] Login ID nobody home directory (/nonexistent) is not
         accessible.

# Performing check of /etc/hosts.equiv and .rhosts files...

# Checking accounts from /etc/passwd...

# Performing check of .netrc files...

# Checking accounts from /etc/passwd...

# Performing common access checks for root (in /etc/default/login, /securetty, and /etc/ttytab...
--WARN-- [root001w] Remote root login allowed in /etc/ssh/sshd_config

# Performing check of PATH components...
--WARN-- [path009w] /etc/profile does not export an initial setting for PATH.
# Only checking user 'root'

# Performing check of anonymous FTP...

# Performing checks of mail aliases...
# Checking aliases from /etc/aliases.

# Performing check of `cron' entries...
--WARN-- [cron005w] Use of cron is not restricted

# Performing check of 'services' ...
# Checking services from /etc/services.
--WARN-- [inet003w] The port for service ssmtp is also assigned to service
         urd.
--WARN-- [inet003w] The port for service pipe-server is also assigned to
         service search.

# Performing NFS exports check...

# Performing check of system file permissions...
--ALERT-- [perm023a] /bin/su is setuid to `root'.
--ALERT-- [perm023a] /usr/bin/at is setuid to `daemon'.
--ALERT-- [perm024a] /usr/bin/at is setgid to `daemon'.
--WARN-- [perm001w] The owner of /usr/bin/at should be root (owned by daemon).
--WARN-- [perm002w] The group owner of /usr/bin/at should be root.
--ALERT-- [perm023a] /usr/bin/passwd is setuid to `root'.
--ALERT-- [perm024a] /usr/bin/wall is setgid to `tty'.

# Checking for known intrusion signs...
# Testing for promiscuous interfaces with /bin/ip
# Testing for backdoors in inetd.conf

# Performing check of files in system mail spool...

# Performing check for rookits...
# Running chkrootkit (/usr/sbin/chkrootkit) to perform further checks...
--WARN-- [rootkit004w] Chkrootkit has detected a possible rootkit installation
Possible Linux/Ebury - Operation Windigo installetd

# Performing system specific checks...
# Performing checks for Linux/4...

# Checking boot loader file permissions...
--WARN-- [boot02] The configuration file /boot/grub/menu.lst has group
         permissions. Should be 0600
--FAIL-- [boot02] The configuration file /boot/grub/menu.lst has world
         permissions. Should be 0600
--WARN-- [boot06] The Grub bootloader does not have a password configured.

# Checking for vulnerabilities in inittab configuration...

# Checking for correct umask settings for init scripts...
--WARN-- [misc021w] There are no umask entries in /etc/init.d/rcS

# Checking Logins not used on the system ...

# Checking network configuration
--FAIL-- [lin013f] The system is not protected against Syn flooding attacks
--WARN-- [lin017w] The system is not configured to log suspicious (martian)
         packets

# Verifying system specific password checks...

# Checking OS release...
--WARN-- [osv004w] Unreleased Debian GNU/Linux version `stretch/sid'

# Checking installed packages vs Debian Security Advisories...

# Checking md5sums of installed files

# Checking installed files against packages...
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.dep' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.alias.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.devname' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.softdep' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.alias' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.symbols.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.builtin.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.symbols' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.dep.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.dep' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.alias.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.devname' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.softdep' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.alias' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.symbols.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.builtin.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.symbols' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.dep.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/udev/hwdb.bin' does not belong to any package.

# Performing check of root directory...

# Checking device permissions...
--WARN-- [dev003w] The directory /dev/block resides in a device directory.
--WARN-- [dev003w] The directory /dev/char resides in a device directory.
--WARN-- [dev003w] The directory /dev/cpu resides in a device directory.
--FAIL-- [dev002f] /dev/fuse has world permissions
--WARN-- [dev003w] The directory /dev/hugepages resides in a device directory.
--FAIL-- [dev002f] /dev/kmsg has world permissions
--WARN-- [dev003w] The directory /dev/lightnvm resides in a device directory.
--WARN-- [dev003w] The directory /dev/mqueue resides in a device directory.
--FAIL-- [dev002f] /dev/rfkill has world permissions
--WARN-- [dev003w] The directory /dev/vfio resides in a device directory.

# Checking for existence of log files...
--FAIL-- [logf005f] Log file /var/log/btmp permission should be 660
--FAIL-- [logf007f] Log file /var/log/messages does not exist

# Checking for correct umask settings for user login shells...
--WARN-- [misc021w] There is no umask definition for the dash shell
--WARN-- [misc021w] There is no umask definition for the bash shell

# Checking symbolic links...

# Performing check of embedded pathnames...
20:47> Security report completed for username.

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

sudo nano /var/log/tiger/security.report.username.170809-18:42

Security scripts *** 3.2.3, 2008.09.10.09.30 ***

Wed Aug 9 18:42:24 AEST 2017

20:42> Beginning security report for username (x86_64 Linux 4.4.0-89-generic).

# Performing check of passwd files...

# Checking entries from /etc/passwd.

--WARN-- [pass014w] Login (bob) is disabled, but has a valid shell.

--WARN-- [pass014w] Login (root) is disabled, but has a valid shell.

--WARN-- [pass015w] Login ID sync does not have a valid shell (/bin/sync).

--WARN-- [pass012w] Home directory /nonexistent exists multiple times (3) in

/etc/passwd.

--WARN-- [pass012w] Home directory /run/systemd exists multiple times (2) in

/etc/passwd.

--WARN-- [pass006w] Integrity of password files questionable (/usr/sbin/pwck

-r).

# Performing check of group files...

# Performing check of user accounts...

# Checking accounts from /etc/passwd.

--WARN-- [acc021w] Login ID dnsmasq appears to be a dormant account.

--WARN-- [acc022w] Login ID nobody home directory (/nonexistent) is not

accessible.

# Performing check of /etc/hosts.equiv and .rhosts files...

# Checking accounts from /etc/passwd...

# Performing check of .netrc files...

# Checking accounts from /etc/passwd...

# Performing common access checks for root (in /etc/default/login, /securetty, and /etc/ttytab...

--WARN-- [root001w] Remote root login allowed in /etc/ssh/sshd_config

# Performing check of PATH components...

--WARN-- [path009w] /etc/profile does not export an initial setting for PATH.

# Only checking user 'root'

# Performing check of anonymous FTP...

# Performing checks of mail aliases...

# Checking aliases from /etc/aliases.

# Performing check of `cron' entries...

--WARN-- [cron005w] Use of cron is not restricted

# Performing check of 'services' ...

# Checking services from /etc/services.

--WARN-- [inet003w] The port for service ssmtp is also assigned to service

urd.

--WARN-- [inet003w] The port for service pipe-server is also assigned to

service search.

# Performing NFS exports check...

# Performing check of system file permissions...

--ALERT-- [perm023a] /bin/su is setuid to `root'.

--ALERT-- [perm023a] /usr/bin/at is setuid to `daemon'.

--ALERT-- [perm024a] /usr/bin/at is setgid to `daemon'.

--WARN-- [perm001w] The owner of /usr/bin/at should be root (owned by daemon).

--WARN-- [perm002w] The group owner of /usr/bin/at should be root.

--ALERT-- [perm023a] /usr/bin/passwd is setuid to `root'.

--ALERT-- [perm024a] /usr/bin/wall is setgid to `tty'.

# Checking for known intrusion signs...

# Testing for promiscuous interfaces with /bin/ip

# Testing for backdoors in inetd.conf

# Performing check of files in system mail spool...

# Performing check for rookits...

# Running chkrootkit (/usr/sbin/chkrootkit) to perform further checks...

--WARN-- [rootkit004w] Chkrootkit has detected a possible rootkit installation

Possible Linux/Ebury - Operation Windigo installetd

# Performing system specific checks...

# Performing checks for Linux/4...

# Checking boot loader file permissions...

--WARN-- [boot02] The configuration file /boot/grub/menu.lst has group

permissions. Should be 0600

--FAIL-- [boot02] The configuration file /boot/grub/menu.lst has world

permissions. Should be 0600

--WARN-- [boot06] The Grub bootloader does not have a password configured.

# Checking for vulnerabilities in inittab configuration...

# Checking for correct umask settings for init scripts...

--WARN-- [misc021w] There are no umask entries in /etc/init.d/rcS

# Checking Logins not used on the system ...

# Checking network configuration

--FAIL-- [lin013f] The system is not protected against Syn flooding attacks

--WARN-- [lin017w] The system is not configured to log suspicious (martian)

packets

# Verifying system specific password checks...

# Checking OS release...

--WARN-- [osv004w] Unreleased Debian GNU/Linux version `stretch/sid'

# Checking installed packages vs Debian Security Advisories...

# Checking md5sums of installed files

# Checking installed files against packages...

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.dep' does not

belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.alias.bin' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.devname' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.softdep' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.alias' does not

belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.symbols.bin'

does not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.builtin.bin'

does not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.symbols' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.dep.bin' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.dep' does not

belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.alias.bin' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.devname' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.softdep' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.alias' does not

belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.symbols.bin'

does not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.builtin.bin'

does not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.symbols' does

not belong to any package.

--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.dep.bin' does

not belong to any package.

--WARN-- [lin001w] File `/lib/udev/hwdb.bin' does not belong to any package.

# Performing check of root directory...

# Checking device permissions...

--WARN-- [dev003w] The directory /dev/block resides in a device directory.

--WARN-- [dev003w] The directory /dev/char resides in a device directory.

--WARN-- [dev003w] The directory /dev/cpu resides in a device directory.

--FAIL-- [dev002f] /dev/fuse has world permissions

--WARN-- [dev003w] The directory /dev/hugepages resides in a device directory.

--FAIL-- [dev002f] /dev/kmsg has world permissions

--WARN-- [dev003w] The directory /dev/lightnvm resides in a device directory.

--WARN-- [dev003w] The directory /dev/mqueue resides in a device directory.

--FAIL-- [dev002f] /dev/rfkill has world permissions

--WARN-- [dev003w] The directory /dev/vfio resides in a device directory.

# Checking for existence of log files...

--FAIL-- [logf005f] Log file /var/log/btmp permission should be 660

--FAIL-- [logf007f] Log file /var/log/messages does not exist

# Checking for correct umask settings for user login shells...

--WARN-- [misc021w] There is no umask definition for the dash shell

--WARN-- [misc021w] There is no umask definition for the bash shell

# Checking symbolic links...

# Performing check of embedded pathnames...

20:47> Security report completed for username.

How to boost your site’s SEO

July 21, 2017 by Simon

In these olden days of the Internet you just needed to submit your site to search engines and wait a few days and every thing was fine and dandy. Making your site appear high in search engines these days is next to impossible due to changing rules and formulas defined by search engines.

It is advisable to avoid so called experts that spam your inbox saying they can improve your site’s SEO for $99 (but strangely do not have a website themselves).

There is no golden rule or trade secret to getting pages ranked high on search engines. You will need to do a number of things to get higher in search results or even appear on search results at all.

Google is not the only search engine and you will need to do a number of things to appear on all of them. Other search engines include DuckDuckGo, Bing, Yahoo, Ask and 14 more search engines here.

Site maps

Google and other top search engines expect your website to declare pages on your site via a sitemap.xml file (specifications here). There is a number of ways you can generate a site map file. If you use the WordPress CMS there are a number of plugins that generate sitemap.xml files for you. Search Engines have automatic bots that will read known sitemap files in iorder to update thir search results.

Q1) Does your web site have a sitemap file?

Warnings

Most search engines will ignore your website if it is not trustworthy. You will be classified trustworthy if:

Your website is linked to from many other trustworthy sites.
Your website is not in an untrustworthy country or on an untrustworthy web site host.
Your website does not have banned words or linked banned media.
You don’t serve malicious files (or your Web servers IP address has never served malicious files).
Your website is updated frequently. Sites like http://web.archive.org/web/*/www.microsoft.com will let you know how frequently a site has been updated.
Your website needs to be secure (must have a secure SSL certificate installed and enabled). Google tells you HTTPS sites get better ranking here.
Your website needs to be fast, a search engine will not point people to slow sites period.
Your website also needs to have a lot of past and current traffic.

Content

Your site will also need to have good content in order to be seen as a valuable site. A blog is a good way to create new content and get new organic traffic to your site. You will need a desirable product or service to be high in the search engines result (e.g Chart generator or sell a good product or service ). Your site also needs to contain the matching keywords to be referred to when people search for a particular thing.

It is easy to appear high in the search results if you use feed google enough words that match your article but it is hard enough to get to the top of the rankings for the term “SEO” with “144,00,000” other matching pages in google coming up for “SEO”

If you are going to pay someone to improve your SEO ask for their track record on SEO ranks, ask for examples, adding the more words to search results to get higher is lazy. I think being high in google rankings with 2 words is near impossible unless you use a location in the search term. Google search is a trying to match the right results based on what you feed it and if you fail in any way your site is ranked down.

Analytics

Using search engines analytics like Google Analytics is a good way to ensure search engines know you exist (Google at least) and let you know what works and allows you to generate more content of what works.

Inbound Links

Most search engines prioritize your site over others if you have a high number of links to your site (inbound links), you can have the best site but with no inbound links to your site, you are in trouble. Creating fake links from non-important sites won’t cheat the system.

Site Speed

Your web site need needs to be fast in order to be deemed trustworthy from search engines. A slow site will fall down the search engine page listing. Do everything you can do to speed up your site (start with ensuring the server is in the right location on a non-shared server with a fast website utilizing cached content).

I have updated my post on Speeding up WordPress, my site used to load in 28 seconds, it now loads less than 9 seconds.

Secure sites

Google and other search engines will soon require your site to have a strong SSL certificate and high security (read my guide on SSL here and content security policy and public key pinning here).

You can check your site’s SSL certificate here. Check your site with shodan.io often for security flaws.

Blacklists

You will need to ensure your site is hosted on non-malicious serves and is using modern clean web technology. Search engines prefer to redirect people to sites that use modern web technologies like HTML5 over transitional HTML websites.

Metadata (Dublin Core)

Meta data is important to allow search engines know what is on a web page without remembering everything on each page. Read more on the latest meta data standards here.

Logs

Do check your logs (error and activity logs) for website crawlers activity (active and missed).

Search your website logs for known crawler activity. Are you missing search engines?

Your logs will tell you how many hits are coming from search engines. I have had 1300+ people visit my site from Google this month alone.

Set goals around search engine targets and work toward them (percentage or total traffic).

Campaigns

Although campaigns are not directly related to SEO they do allow you to combine stats around page loads and advertising campaigns. I prefer to track campaigns with a custom central redirect page (e.g /go/index.php) that logs the inbound clicks.

// Save this to a file called index.php (maybe under a folder called "/go/" at the top of your server (e.g "https://www.fearby.com/go/index.php")
header("Cache-Control: no-cache, must-revalidate");
header("Expires: Sat, 26 Jul 1997 05:00:00 GMT");
header('Content-Type: text/html; charset=utf-8');

// Insert campaign logging code here
// todo

/* Sample Usage
https://www.fearby.com/go/?seo
https://www.fearby.com/go/?aws
*/
switch ($_SERVER['QUERY_STRING']) { 

	case "seo":
		header( 'Location: https://fearby.com/article/how-to-boost-your-sites-seo/' ) ;
		break;

	case ("beyondssl"):
		header( 'Location: https://fearby.com/article/beyond-ssl-with-content-security-policy-public-key-pinning-etc/' ) ;
		break;

	case ("mysql"):
		header( 'Location: https://fearby.com/article/setting-up-a-fast-distributed-mysql-environment-with-ssl/' ) ;
		break;
	
	case ("aws"):
		header( 'Location: https://fearby.com/article/creating-an-aws-ec2-ubuntu-14-04-server-with-nginx-node-and-mysql-and-phpmyadmin/' ) ;
		break;

	case ("do"):
		header( 'Location: https://fearby.com/article/how-to-buy-a-new-domain-and-ssl-cert-from-namecheap-a-server-from-digital-ocean-and-configure-it/' ) ;
		break;

	default: 
		echo "I could not find a page for the URL you specified.";
		break;
}

// Save this to a file called index.php (maybe under a folder called "/go/" at the top of your server (e.g "https://www.fearby.com/go/index.php")

header("Cache-Control: no-cache, must-revalidate");

header("Expires: Sat, 26 Jul 1997 05:00:00 GMT");

header('Content-Type: text/html; charset=utf-8');

// Insert campaign logging code here

// todo

/* Sample Usage

https://www.fearby.com/go/?seo

https://www.fearby.com/go/?aws

switch ($_SERVER['QUERY_STRING']) {

case "seo":

header( 'Location: https://fearby.com/article/how-to-boost-your-sites-seo/' ) ;

break;

case ("beyondssl"):

header( 'Location: https://fearby.com/article/beyond-ssl-with-content-security-policy-public-key-pinning-etc/' ) ;

break;

case ("mysql"):

header( 'Location: https://fearby.com/article/setting-up-a-fast-distributed-mysql-environment-with-ssl/' ) ;

break;

case ("aws"):

header( 'Location: https://fearby.com/article/creating-an-aws-ec2-ubuntu-14-04-server-with-nginx-node-and-mysql-and-phpmyadmin/' ) ;

break;

case ("do"):

header( 'Location: https://fearby.com/article/how-to-buy-a-new-domain-and-ssl-cert-from-namecheap-a-server-from-digital-ocean-and-configure-it/' ) ;

break;

default:

echo "I could not find a page for the URL you specified.";

break;

}

Long Tail Titles.

I use long and descriptive titles to mention what my blog post is about. I find that this helps with matching what and Google search loves it.

Some of my descriptive post titles

Using long page titles can ensure people click on your links with confidence ( and they will know what your site is about ). Page views are not everything, engagement is.

My most recent popular posts.

I think I need to create some more technical guides as they seem popular.

Stats

Viewing statistics and reports can help you determine the popularity of your site.

Analytics

Reviewing Logs, viewing campaign data, viewing misc reports can help you identify incoming traffic to verify working acquisition sources.

I like looking at Google Analytics Behaviour reports for viewing inbound visits. I can see over 19,000 incoming visitors from google over the last 2.5 years with no sightmap.xml (this proves that content and long titles work).

Adding your site to bing searches

Go to https://www.bing.com/toolbox/submit-site-url and submit your site.

I tried to submit my XML files to Bing but the form stalled (my bad I did not see the organization size mandatory field).

I changed the domain above to just the domain (no sitemap).

I added my 6 sitemaps to the Bing Webmaster tools.

Submit your sitemp files in the Bing Webmaster tools.

Adding Your Site to the Google Search Console

I am trying the WordPress SEO plugin by Yoest but before I do I am adding my site to the Google Search Console (linked to my Gmail account). You will need to add your site to Google Search Console to have Google index your site.

Before Google trusts your domain you need to verify ownership of the domain ( for me I uploaded the verification files with the free FileZilla FTP program then loaded the file, then clicked verify at the Google Search Console page ).

Now my domain is verified and the Google Search console allows me to view the search status of my domain.

Installing a WordPress SEO Plugin – Yoast

Install the Yoast Plugin in WordPress (follow the install steps after you activate from step 1-8). You can add Yoast to your WordPress via the Add Plugins screen or download then upload it.

Don’t forget to activate then run the advanced Yoast configuration wizard.

After you get an authorization code it is important that you select your website from the profile list (if you don’t see this STOP and add your site to the Google Search Console (you cannot skip this step)).

Continue through steps 9-12 and finish installing the Yoast plugin.

Don’t forget to enable sitemaps (and review your settings). Now you can open the SEO plugin advanced options (/wp-admin/admin.php?page=wpseo_dashboard) and review the plugin options.

I installed the plugin but no sitemap.xml was created???

Twitter user https://www.twitter.com/atimmer10 said I may need to save the/wp-admin/options-permalink.php settings (no luck, this did not fix it and no sitemap.xml was created ).

I edited an existing published post and changed the short link and re-published a post ( no luck, this did not fix it and no sitemap.xml was created ).

I read this Yoast FAQ about the sitemap.xml not being created. I added the following to my .htaccess file ( no luck, this did not fix it and no sitemap.xml was created ).

# Yoast SEO - XML Sitemap Rewrite Fix
RewriteEngine On
RewriteBase /
RewriteRule ^sitemap_index.xml$ /index.php?sitemap=1 [L]
RewriteRule ^locations.kml$ /index.php?sitemap=wpseo_local_kml [L]
RewriteRule ^geo_sitemap.xml$ /index.php?sitemap=geo [L]
RewriteRule ^([^/]+?)-sitemap([0-9]+)?.xml$ /index.php?sitemap=$1&sitemap_n=$2 [L]
RewriteRule ^([a-z]+)?-?sitemap.xsl$ /index.php?xsl=$1 [L]
# END Yoast SEO - XML Sitemap Rewrite Fix

# Yoast SEO - XML Sitemap Rewrite Fix

RewriteEngine On

RewriteBase /

RewriteRule ^sitemap_index.xml$ /index.php?sitemap=1 [L]

RewriteRule ^locations.kml$ /index.php?sitemap=wpseo_local_kml [L]

RewriteRule ^geo_sitemap.xml$ /index.php?sitemap=geo [L]

RewriteRule ^([^/]+?)-sitemap([0-9]+)?.xml$ /index.php?sitemap=$1&sitemap_n=$2 [L]

RewriteRule ^([a-z]+)?-?sitemap.xsl$ /index.php?xsl=$1 [L]

# END Yoast SEO - XML Sitemap Rewrite Fix

I still cannot get Yoast to create a sitemap.xml???

It turns out Yoast was creating a hidden /sitemap_index.xml

I then added the 6 individual sitemaps to the Google Search Console and Google started indexing my site.

Now Google will index my site and Yoast will auto update my sitemaps when I add, Edit or delete a piece of content.

Do monitor your sitemap syncs from time to time.

Add your sitemaps(s) to Yahoo, Bing and Ask search engines

More on adding your site to search engines on the Yoast website here. Also, read this article.

robots.txt

robots.txt is also an older sitemap.xml file format, read more here.

I made a backward compatible robots.txt file

User-agent: *
Disallow: 
Disallow: /cgi-bin/
Disallow: /wp-admin/
Sitemap: https://fearby.com/post-sitemap.xml
Sitemap: https://fearby.com/page-sitemap.xml
Sitemap: https://fearby.com/news-sitemap.xml
Sitemap: https://fearby.com/category-sitemap.xml
Sitemap: https://fearby.com/post_tag-sitemap.xml
Sitemap: https://fearby.com/news-category-sitemap.xml

User-agent: *

Disallow:

Disallow: /cgi-bin/

Disallow: /wp-admin/

Sitemap: https://fearby.com/post-sitemap.xml

Sitemap: https://fearby.com/page-sitemap.xml

Sitemap: https://fearby.com/news-sitemap.xml

Sitemap: https://fearby.com/category-sitemap.xml

Sitemap: https://fearby.com/post_tag-sitemap.xml

Sitemap: https://fearby.com/news-category-sitemap.xml

Generate a robots.txt here and read more here.

Other Plugins

Generate sitemaps from Yoast sitemaps with WP SEO HTML Sitemaps.

Other Tools

Google has a Structured Data Testing Tool to allow you to diagnose various search issues.
Webpage accessibility checker.
Webpage speed checker

Top WordPress Plugins

Do More of What Works

Review pages and see what works, you may find old pages or posts are more popular than new ones.

Set Goals and Measure Them

Goals like page views are popular targets for SEO results but focusing on lowering bounce rates and increasing customer session time is just as important.

Through changes in the last 30 days, I have lowered my bounce rate from 94% to 80%. I have made some changes to my site to pick up chicks at the end of posts and hope to get a 50% or lower bounce rate.

Where are your visitors coming from?

Google Analytics will reveal where visitors come from. Twitter, Facebook, etc. Knowing where your readers are located allows you to target more users.

Time of Day

Knowing the time of day (from Google Analytics) will show you when it is best to post and promote.

Visitor Frequency

Google Analytics will tell you if your visitors are one-time visitors or regular visitors.

Conclusion

Ensure your sitemaps is up to date and keep adding new content.

Donate and make this blog better

Ask a question or recommend an article

V1.7 added info 1 month later

Type	Protocol	Port Range	Source	Comment
HTTP	TCP	80	0.0.0.0/0	Opens a web server port for later
All ICMP	ALL	N/A	0.0.0.0/0	Allows you to ping
All traffic	ALL	All	0.0.0.0/0	Not advisable long term but OK for testing today.
SSH	TCP	22	0.0.0.0/0	Not advisable, try and limit this to known IP’s only.
HTTPS	TCP	443	0.0.0.0/0	Opens a secure web server port for later

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

"If you're not still learning, you're already dying."
- Ryan Holiday - Ego is the Enemy

Follow me on Twitter: @FearbySoftware

View All Posts.

SEO

Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 2 of 4

Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads

How to boost your site’s SEO

Popular

Security

Code

Tech

Wordpress

General

SEO

Footer

Popular

Security

Code

Tech

Wordpress

General