Website

Setup Ruby, Rails, Gem and a command line twitter tool to query Twitter on Ubuntu 16.04 via a Twitter App

September 17, 2017 by Simon 2 Comments

Below is how I setup Ruby, Rails, Gem and a command line twitter tool to query Twitter on Ubuntu 16.04 via a Twitter App

Setup Twitter feed scraping on Ubuntu 16.04

At first, I had no network (I could not ping, run a system update or install packages (even though I had opened the firewall ports and disabled the firewall temporarily)? I fixed this by editing /etc/resolv.conf and added a google DNS entry.

sudo nano /etc/resolv.conf

1	sudo nano /etc/resolv.conf

Added the Google DNS server.

nameserver 8.8.8.8

1	nameserver 8.8.8.8

Bingo, I can now ping and update my system.

Setup Ruby and Pre-Requisites

sudo apt-get update
sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev
git clone https://github.com/rbenv/rbenv.git ~/.rbenv
echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc
echo 'eval "$(rbenv init -)"' >> ~/.bashrc
exec $SHELL
git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build
echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc
exec $SHELL
rbenv install 2.4.1
rbenv global 2.4.1

sudo apt-get update

sudo apt-get install git-core curl zlib1g-dev build-essential libssl-dev libreadline-dev libyaml-dev libsqlite3-dev sqlite3 libxml2-dev libxslt1-dev libcurl4-openssl-dev python-software-properties libffi-dev

git clone https://github.com/rbenv/rbenv.git ~/.rbenv

echo 'export PATH="$HOME/.rbenv/bin:$PATH"' >> ~/.bashrc

echo 'eval "$(rbenv init -)"' >> ~/.bashrc

exec $SHELL

git clone https://github.com/rbenv/ruby-build.git ~/.rbenv/plugins/ruby-build

echo 'export PATH="$HOME/.rbenv/plugins/ruby-build/bin:$PATH"' >> ~/.bashrc

exec $SHELL

rbenv install 2.4.1

rbenv global 2.4.1

Ruby is now installed

ruby -v
ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

1 2	ruby -v ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]

Install Bundler Gem

gem install bundler

1	gem install bundler

Install Rails

curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -

1	curl -sL https://deb.nodesource.com/setup_8.x \| sudo -E bash -

I skipped the Install (as I had node installed).

node -v
v6.11.3

1 2	node -v v6.11.3

Continue with rails install

gem install rails -v 5.1.3
rbenv rehash

1 2	gem install rails -v 5.1.3 rbenv rehash

Rails is installed

rails -v
Rails 5.1.3

1 2	rails -v Rails 5.1.3

Install the t gem (read more at the https://github.com/sferik/t GitHub repository)

gem install t

1	gem install t

You can now authorize the t gem to use your twitter.

t authorize
Welcome! Before you can use t, you'll first need to register an
application with Twitter. Just follow the steps below:
  1. Sign in to the Twitter Application Management site and click
     "Create New App".
  2. Complete the required fields and submit the form.
     Note: Your application must have a unique name.
  3. Go to the Permissions tab of your application, and change the
     Access setting to "Read, Write and Access direct messages".
  4. Go to the Keys and Access Tokens tab to view the consumer key
     and secret which you'll need to copy and paste below when
     prompted.

t authorize

Welcome! Before you can use t, you'll first need to register an

application with Twitter. Just follow the steps below:

1. Sign in to the Twitter Application Management site and click

"Create New App".

2. Complete the required fields and submit the form.

Note: Your application must have a unique name.

3. Go to the Permissions tab of your application, and change the

Access setting to "Read, Write and Access direct messages".

4. Go to the Keys and Access Tokens tab to view the consumer key

and secret which you'll need to copy and paste below when

prompted.

But first let’s create a twitter app.

Goto https://apps.twitter.com/, login and create an app.

Twitter will provide app details when you create the app.

Go to Permissions and set “Read, Write and Access direct messages” and save changes.

Linking the Twitter app to t Gem

Now that the Twitter app is created let’s activate and link it to the t gem,

Run the following command to start the authorization process.

t authorize

1	t authorize

The authorization process is dead simple, just follow the on-screen prompts.

Welcome! Before you can use t, you'll first need to register an
application with Twitter. Just follow the steps below:
  1. Sign in to the Twitter Application Management site and click
     "Create New App".
  2. Complete the required fields and submit the form.
     Note: Your application must have a unique name.
  3. Go to the Permissions tab of your application, and change the
     Access setting to "Read, Write and Access direct messages".
  4. Go to the Keys and Access Tokens tab to view the consumer key
     and secret which you'll need to copy and paste below when
     prompted.

Press [Enter] to open the Twitter Developer site.

Open: https://apps.twitter.com
Enter your API key: ########################
Enter your API secret: ################################################

In a moment, you will be directed to the Twitter app authorization page.
Perform the following steps to complete the authorization process:
  1. Sign in to Twitter.
  2. Press "Authorize app".
  3. Copy and paste the supplied PIN below when prompted.

Press [Enter] to open the Twitter app authorization page.

Open: https://api.twitter.com/oauth/authorize?oauth_callback=oob&oauth_consumer_key=################################################&oauth_signature=################################################&oauth_signature_method=HMAC-SHA1&oauth_timestamp=123456789&oauth_token=########################&oauth_version=1.0
Enter the supplied PIN: ######
Authorization successful.

Welcome! Before you can use t, you'll first need to register an

application with Twitter. Just follow the steps below:

1. Sign in to the Twitter Application Management site and click

"Create New App".

2. Complete the required fields and submit the form.

Note: Your application must have a unique name.

3. Go to the Permissions tab of your application, and change the

Access setting to "Read, Write and Access direct messages".

4. Go to the Keys and Access Tokens tab to view the consumer key

and secret which you'll need to copy and paste below when

prompted.

Press [Enter] to open the Twitter Developer site.

Open: https://apps.twitter.com

Enter your API key: ########################

Enter your API secret: ################################################

In a moment, you will be directed to the Twitter app authorization page.

Perform the following steps to complete the authorization process:

1. Sign in to Twitter.

2. Press "Authorize app".

3. Copy and paste the supplied PIN below when prompted.

Press [Enter] to open the Twitter app authorization page.

Open: https://api.twitter.com/oauth/authorize?oauth_callback=oob&oauth_consumer_key=################################################&oauth_signature=################################################&oauth_signature_method=HMAC-SHA1&oauth_timestamp=123456789&oauth_token=########################&oauth_version=1.0

Enter the supplied PIN: ######

Authorization successful.

This was easy.

fyi: Authorization (Twitter authorize app screenshot)

fyi: Authorization (Twitter authorize app pin screenshot)

Using T

T Help

t help
Commands:
  t accounts                          # List accounts
  t authorize                         # Allows an application to request user authorization
  t block USER [USER...]              # Block users.
  t delete SUBCOMMAND ...ARGS         # Delete Tweets, Direct Messages, etc.
  t direct_messages                   # Returns the 20 most recent Direct Messages sent to you.
  t direct_messages_sent              # Returns the 20 most recent Direct Messages you've sent.
  t dm USER MESSAGE                   # Sends that person a Direct Message.
  t does_contain [USER/]LIST USER     # Find out whether a list contains a user.
  t does_follow USER [USER]           # Find out whether one user follows another.
  t favorite TWEET_ID [TWEET_ID...]   # Marks Tweets as favorites.
  t favorites [USER]                  # Returns the 20 most recent Tweets you favorited.
  t follow USER [USER...]             # Allows you to start following users.
  t followers [USER]                  # Returns a list of the people who follow you on Twitter.
  t followings [USER]                 # Returns a list of the people you follow on Twitter.
  t followings_following USER [USER]  # Displays your friends who follow the specified user.
  t friends [USER]                    # Returns the list of people who you follow and follow you back.
  t groupies [USER]                   # Returns the list of people who follow you but you don't follow back.
  t help [COMMAND]                    # Describe available commands or one specific command
  t intersection USER [USER...]       # Displays the intersection of users followed by the specified users.
  t leaders [USER]                    # Returns the list of people who you follow but don't follow you back.
  t list SUBCOMMAND ...ARGS           # Do various things with lists.
  t lists [USER]                      # Returns the lists created by a user.
  t matrix                            # Unfortunately, no one can be told what the Matrix is. You have to see it for y...
  t mentions                          # Returns the 20 most recent Tweets mentioning you.
  t mute USER [USER...]               # Mute users.
  t muted [USER]                      # Returns a list of the people you have muted on Twitter.
  t open USER                         # Opens that user's profile in a web browser.
  t reach TWEET_ID                    # Shows the maximum number of people who may have seen the specified tweet in th...
  t reply TWEET_ID [MESSAGE]          # Post your Tweet as a reply directed at another person.
  t report_spam USER [USER...]        # Report users for spam.
  t retweet TWEET_ID [TWEET_ID...]    # Sends Tweets to your followers.
  t retweets [USER]                   # Returns the 20 most recent Retweets by a user.
  t retweets_of_me                    # Returns the 20 most recent Tweets of the authenticated user that have been ret...
  t ruler                             # Prints a 140-character ruler
  t search SUBCOMMAND ...ARGS         # Search through Tweets.
  t set SUBCOMMAND ...ARGS            # Change various account settings.
  t status TWEET_ID                   # Retrieves detailed information about a Tweet.
  t stream SUBCOMMAND ...ARGS         # Commands for streaming Tweets.
  t timeline [USER]                   # Returns the 20 most recent Tweets posted by a user.
  t trend_locations                   # Returns the locations for which Twitter has trending topic information.
  t trends [WOEID]                    # Returns the top 50 trending topics.
  t unfollow USER [USER...]           # Allows you to stop following users.
  t update [MESSAGE]                  # Post a Tweet.
  t users USER [USER...]              # Returns a list of users you specify.
  t version                           # Show version.
  t whoami                            # Retrieves profile information for the authenticated user.
  t whois USER                        # Retrieves profile information for the user.

Options:
  -C, [--color=COLOR]   # Control how color is used in output
                        # Default: auto
                        # Possible values: icon, auto, never
  -P, [--profile=FILE]  # Path to RC file
                        # Default: /root/.trc

t help

Commands:

t accounts # List accounts

t authorize # Allows an application to request user authorization

t block USER [USER...] # Block users.

t delete SUBCOMMAND ...ARGS # Delete Tweets, Direct Messages, etc.

t direct_messages # Returns the 20 most recent Direct Messages sent to you.

t direct_messages_sent # Returns the 20 most recent Direct Messages you've sent.

t dm USER MESSAGE # Sends that person a Direct Message.

t does_contain [USER/]LIST USER # Find out whether a list contains a user.

t does_follow USER [USER] # Find out whether one user follows another.

t favorite TWEET_ID [TWEET_ID...] # Marks Tweets as favorites.

t favorites [USER] # Returns the 20 most recent Tweets you favorited.

t follow USER [USER...] # Allows you to start following users.

t followers [USER] # Returns a list of the people who follow you on Twitter.

t followings [USER] # Returns a list of the people you follow on Twitter.

t followings_following USER [USER] # Displays your friends who follow the specified user.

t friends [USER] # Returns the list of people who you follow and follow you back.

t groupies [USER] # Returns the list of people who follow you but you don't follow back.

t help [COMMAND] # Describe available commands or one specific command

t intersection USER [USER...] # Displays the intersection of users followed by the specified users.

t leaders [USER] # Returns the list of people who you follow but don't follow you back.

t list SUBCOMMAND ...ARGS # Do various things with lists.

t lists [USER] # Returns the lists created by a user.

t matrix # Unfortunately, no one can be told what the Matrix is. You have to see it for y...

t mentions # Returns the 20 most recent Tweets mentioning you.

t mute USER [USER...] # Mute users.

t muted [USER] # Returns a list of the people you have muted on Twitter.

t open USER # Opens that user's profile in a web browser.

t reach TWEET_ID # Shows the maximum number of people who may have seen the specified tweet in th...

t reply TWEET_ID [MESSAGE] # Post your Tweet as a reply directed at another person.

t report_spam USER [USER...] # Report users for spam.

t retweet TWEET_ID [TWEET_ID...] # Sends Tweets to your followers.

t retweets [USER] # Returns the 20 most recent Retweets by a user.

t retweets_of_me # Returns the 20 most recent Tweets of the authenticated user that have been ret...

t ruler # Prints a 140-character ruler

t search SUBCOMMAND ...ARGS # Search through Tweets.

t set SUBCOMMAND ...ARGS # Change various account settings.

t status TWEET_ID # Retrieves detailed information about a Tweet.

t stream SUBCOMMAND ...ARGS # Commands for streaming Tweets.

t timeline [USER] # Returns the 20 most recent Tweets posted by a user.

t trend_locations # Returns the locations for which Twitter has trending topic information.

t trends [WOEID] # Returns the top 50 trending topics.

t unfollow USER [USER...] # Allows you to stop following users.

t update [MESSAGE] # Post a Tweet.

t users USER [USER...] # Returns a list of users you specify.

t version # Show version.

t whoami # Retrieves profile information for the authenticated user.

t whois USER # Retrieves profile information for the user.

Options:

-C, [--color=COLOR] # Control how color is used in output

# Default: auto

# Possible values: icon, auto, never

-P, [--profile=FILE] # Path to RC file

# Default: /root/.trc

Show linked twitter accounts with t

t accounts
yourappnamehere
  ###################### (active)

t accounts

yourappnamehere

###################### (active)

Show authorized twitter accounts

t set active yourtwitterappnamehere ########################
Active account has been updated to yourtwitterappnamehere.

1 2	t set active yourtwitterappnamehere ######################## Active account has been updated to yourtwitterappnamehere.

Using t to query a Twitter user

t whois @fearbysoftware
ID           1468627891
Since        May 30  2013 (4 years ago)
Last update  Editing remote files locally with sublime text editor over ssh https://t.co/k5qSnHmUrP #VoteYes (7 hours ago)
Screen name  @FearbySoftware
Name         Simon Fearby
Tweets       4,797
Favorites    940
Listed       88
Following    1,933
Followers    616
Bio          Developing augmented reality mobile apps, websites, ardrino and raspberry pi code/circuits etc. Tweets are my own not my employer. Blog at https://t.co/Azo81pi8Yt
Location     Tamworth NSW, Australia
URL          http://www.fearby.com

t whois @fearbysoftware

ID 1468627891

Since May 30 2013 (4 years ago)

Last update Editing remote files locally with sublime text editor over ssh https://t.co/k5qSnHmUrP #VoteYes (7 hours ago)

Screen name @FearbySoftware

Name Simon Fearby

Tweets 4,797

Favorites 940

Listed 88

Following 1,933

Followers 616

Bio Developing augmented reality mobile apps, websites, ardrino and raspberry pi code/circuits etc. Tweets are my own not my employer. Blog at https://t.co/Azo81pi8Yt

Location Tamworth NSW, Australia

URL http://www.fearby.com

Search Twitter for “fearby”

t search all "lang:en fearby"

1	t search all "lang:en fearby"

Output:

t search all "lang:en fearby"

@FearbySoftware
@troyhunt Google AdWords have worked for me https://t.co/KGZAd0sWkG

@FearbySoftware
Blogged setting up my own Ubuntu server to replace Cpanel for $2.5 a month https://t.co/GZCIMesaqJ

@FearbySoftware
Blogged Securing an Ubuntu VM with a free LetsEncrypt SSL certificate in 1 Minute https://t.co/QWiyR2I9ur

@MedinaSports
JV boys ⚽️ tied Roy-Hart 1-1. AJ Seefeldt scored the lone goal w/ 20' left in regulation. Zach Fike & Cooper Fearby
made great saves in goal

@FearbySoftware
Today's SEO experiment : Not pimping blog posts results in half the user hits and impressions. https://t.co/Q9eCoUZy9n

@FearbySoftware
@0xDUDE any advice on security my MongoDB more? Need whitlist IP, use the non standard port and have usr/pwd
https://t.co/5TEDz8LCJo

@FearbySoftware
Creating and configuring a CentOS server on Digital Ocean https://t.co/aI3FYKSFQC

@FearbySoftware
Self Service Status Pages https://t.co/F6ZjN2sdfM

@FearbySoftware
Alibaba Cloud how good is it? https://t.co/YbuWgvyDz8

@breakingnewsng_
IGBONLA SIX: Four of freed students resume studies - …Say there’s nothing to fearBy Monsuru Olowoopejolagos—Two...
https://t.co/sosKnSpT3h

@AFairymary
RT @AFairymary: Congratulations to William Fearby, Author of the Month. https://t.co/jtP0Sn0QCl

@FearbySoftware
I guess I need to get an #iPhoneX to develop apps on an post updates to my free dev blog https://t.co/9x5TFARLCt
#apple #iOS11 #AppleEvent

@fearby_nick
RT @ndonnelly88_: @NJDevils How many retweets for free season tickets?

@PrincessMutanu
RT @CBooksFree: $0.99—Imagine Your Life Without Fear—by Max Lucado https://t.co/pUVi2wWM1J https://t.co/7YakQTjzZf

@corund
RT @FearbySoftware: Securing an Ubuntu VM with a free LetsEncrypt SSL certificate in 1 Minute https://t.co/QWiyR2I9ur
#free #SSL #website #wordpress #nodejs
...

t search all "lang:en fearby"

@FearbySoftware

@troyhunt Google AdWords have worked for me https://t.co/KGZAd0sWkG

@FearbySoftware

Blogged setting up my own Ubuntu server to replace Cpanel for $2.5 a month https://t.co/GZCIMesaqJ

@FearbySoftware

Blogged Securing an Ubuntu VM with a free LetsEncrypt SSL certificate in 1 Minute https://t.co/QWiyR2I9ur

@MedinaSports

JV boys ⚽️ tied Roy-Hart 1-1. AJ Seefeldt scored the lone goal w/ 20' left in regulation. Zach Fike & Cooper Fearby

made great saves in goal

@FearbySoftware

Today's SEO experiment : Not pimping blog posts results in half the user hits and impressions. https://t.co/Q9eCoUZy9n

@FearbySoftware

@0xDUDE any advice on security my MongoDB more? Need whitlist IP, use the non standard port and have usr/pwd

https://t.co/5TEDz8LCJo

@FearbySoftware

Creating and configuring a CentOS server on Digital Ocean https://t.co/aI3FYKSFQC

@FearbySoftware

Self Service Status Pages https://t.co/F6ZjN2sdfM

@FearbySoftware

Alibaba Cloud how good is it? https://t.co/YbuWgvyDz8

@breakingnewsng_

IGBONLA SIX: Four of freed students resume studies - …Say there’s nothing to fearBy Monsuru Olowoopejolagos—Two...

https://t.co/sosKnSpT3h

@AFairymary

RT @AFairymary: Congratulations to William Fearby, Author of the Month. https://t.co/jtP0Sn0QCl

@FearbySoftware

I guess I need to get an #iPhoneX to develop apps on an post updates to my free dev blog https://t.co/9x5TFARLCt

#apple #iOS11 #AppleEvent

@fearby_nick

RT @ndonnelly88_: @NJDevils How many retweets for free season tickets?

@PrincessMutanu

RT @CBooksFree: $0.99—Imagine Your Life Without Fear—by Max Lucado https://t.co/pUVi2wWM1J https://t.co/7YakQTjzZf

@corund

RT @FearbySoftware: Securing an Ubuntu VM with a free LetsEncrypt SSL certificate in 1 Minute https://t.co/QWiyR2I9ur

#free #SSL #website #wordpress #nodejs

...

Search Twitter for “fearby” (max 10) and output at CSV

t search all "lang:en fearby" --csv

1	t search all "lang:en fearby" --csv

See more commands here: https://github.com/sferik/t

This is great, I can interact with Twitter from the command line and apps without having to go full REST API and OAUTH development.

Calling T from NodeJS

Read the guide here on calling T from NodeJS.

Calling T from php (Under construction)

Coming soon (this PHP section is under development)

You may need to exclude “pcntl_exec” from being blocked in “php.ini” under “disable_functions”

find your php.ini by typing

find / -iname "php.ini"

1	find / -iname "php.ini"

Restart php

sudo service php7.0-fpm restart
php7.0-fpm stop/waiting
php7.0-fpm start/running, process #####
user@servername:/www# service php7.0-fpm status
php7.0-fpm start/running, process #####

sudo service php7.0-fpm restart

php7.0-fpm stop/waiting

php7.0-fpm start/running, process #####

user@servername:/www# service php7.0-fpm status

php7.0-fpm start/running, process #####

This section is under development,

Todo: Security.

Need a server?

Set up a Server on Vultr here for as low as $2.5 a month or set up a Server on Digital Ocean (and get the first 2 months free ($5/m server)). I have a guide on setting up a Vultr server here or Digital Ocean server here. Don’t forget to add a free LetsEncrypt SSL Certificate and secure the server (read more here and here).

Still here, read more articles here or use the form below to ask a question or recommend an article.

Donate and make this blog better

Ask a question or recommend an article

v1.3 Querying T in NodeJS

How to use Sublime Text editor locally to edit code files on a remote server via SSH

September 16, 2017 by Simon Leave a Comment

This guide will show you how to use Sublime Text editor locally to edit code files on a remote server via SSH.

This guide assumes oy already have a working SSH connection between your Mac and your remote server (with no firewall issues) and have configured SSH keys via modifying to authorized_keys file to enable SSH access.

Need a server?

Setting up your local machine

Open Sublime Text 3 and press COMMAND+SHIFT+P to bring up the command bar and type Install and click Package Control: Install Package and click it.

Wait a few seconds for the packages list to show and type “rsub”

Ok let’s make an SSH alias to your server on your Mac by typing “sudo nano ~/.ssh/config”

Make these changes

File contents:

host mysrv
HostName www.myserver.com
User thesshuser
RemoteForward 52698 localhost:52698

host mysrv

HostName www.myserver.com

User thesshuser

RemoteForward 52698 localhost:52698

Now we can connect to the server via SSH by typing “ssh mysrv”

After typing the server’s password you will be connected to the ssh server

ssh mysrv
thesshuser@www.myserver.com's password: 
Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

0 packages can be updated.
0 updates are security updates.


You have new mail.
Last login: Sat Sep 16 12:51:35 2017 from xx.xx.xx.xx
thesshuser@myserver:~#

ssh mysrv

thesshuser@www.myserver.com's password:

Welcome to Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-87-generic x86_64)

* Documentation: https://help.ubuntu.com

* Management: https://landscape.canonical.com

* Support: https://ubuntu.com/advantage

0 packages can be updated.

0 updates are security updates.

You have new mail.

Last login: Sat Sep 16 12:51:35 2017 from xx.xx.xx.xx

thesshuser@myserver:~#

Now on your local Mac load the following page in a web browser (and review the code): https://raw.github.com/aurora/rmate/master/rmate and copy the contents to the clipboard.

On the remote server (the SSH one) type:

sudo nano /usr/local/bin/rmate

1	sudo nano /usr/local/bin/rmate

Now paste the contents or this page into nano editor and save it and exit nano.

Now run this chmod command to make the rmate file executable.

sudo chmod a+x /usr/local/bin/rmate

1	sudo chmod a+x /usr/local/bin/rmate

Now on the server, we can open any text file with rmate and have it open locally in Sublime via SSH. Yes, Open a file on a server and have it automatically open in locally 🙂

If you have many files to open then create a bash file to open files with rmate

sudo nano openfilesonmac.sh

1	sudo nano openfilesonmac.sh

Contents:

#!/bin/bash

rmate index.html 
rmate index1.html 
rmate index2.html 
rmate index3.html 
rmate index4.html 
rmate index5.html 
rmate index6.html 
rmate index7.html 
rmate index8.html 
rmate index9.html 
rmate index10.html

#!/bin/bash

rmate index.html

rmate index1.html

rmate index2.html

rmate index3.html

rmate index4.html

rmate index5.html

rmate index6.html

rmate index7.html

rmate index8.html

rmate index9.html

rmate index10.html

File permissions:

chmod +x openfilesonmac.sh

1	chmod +x openfilesonmac.sh

Now we can open may remote files locally by running the bash script.

All saves in Sublime locally are sent to the server 🙂

e.g

rmate /www/index.html
rmate /node/api/app01/app.js
rmate /www/dashboard/index.php

rmate /www/index.html

rmate /node/api/app01/app.js

rmate /www/dashboard/index.php

Still here, read more articles here or use the form below to ask a question or recommend an article.

Donate and make this blog better

Ask a question or recommend an article

v1.2 rmate examples

Installing and managing WordPress with WP-CLI from the command line on Ubuntu

September 10, 2017 by Simon 3 Comments

Here is a really simple way to manually manage WordPress installations on Ubuntu.

My previous WordPress (and related) guides

My Server Setup Guides

Go to https://make.wordpress.org/cli/handbook/installing/ and read the install instructions.

How to install wp-cli on Ubuntu

First read this post too install WordPress from the command line. You should install WordPress here.

Telnet to your server (SSH)

cd /www
mkdir wp-cli
cd wp-cli
pwd
/www/wp-cli
curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 4191k  100 4191k    0     0  4535k      0 --:--:-- --:--:-- --:--:-- 4536k

cd /www

mkdir wp-cli

cd wp-cli

pwd

/www/wp-cli

curl -O https://raw.githubusercontent.com/wp-cli/builds/gh-pages/phar/wp-cli.phar

% Total % Received % Xferd Average Speed Time Time Time Current

Dload Upload Total Spent Left Speed

100 4191k 100 4191k 0 0 4535k 0 --:--:-- --:--:-- --:--:-- 4536k

Did the file download?

ls wp-cli.phar -al
-rw-r--r-- 1 root root 4292472 Sep 10 13:29 wp-cli.phar

1 2	ls wp-cli.phar -al -rw-r--r-- 1 root root 4292472 Sep 10 13:29 wp-cli.phar

Execute

/usr/bin/php ./wp-cli.phar --info

1	/usr/bin/php ./wp-cli.phar --info

Make the file executable and move it to /usr/local/bin/wp

chmod +x wp-cli.phar
sudo mv wp-cli.phar /usr/local/bin/wp

1 2	chmod +x wp-cli.phar sudo mv wp-cli.phar /usr/local/bin/wp

Now you can run wp-cli by typing the following from the command line

wp --info

wp --info

Read the official wp-cli quick start guide https://make.wordpress.org/cli/handbook/quick-start/

Ever since I moved my WordPress to a self-managed server I have not been able to update my plugins and I have not installed an FTP server (by choice for security reasons).

Let’s see if we can update plugin with wp-cli

wp plugin update --all

1	wp plugin update --all

Yes, it worked

Enabling Maintenance mode...
Downloading update from https://downloads.wordpress.org/plugin/add-to-any.1.7.17.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/contact-form-7.4.9.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer.3.6.1.zip...
Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer.3.6.1.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer-cloud.3.6.1.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://downloads.wordpress.org/plugin/better-wp-security.6.6.0.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Downloading update from https://www.wponlinesupport.com/edd-sl/package_download/removedapikey...
Warning: Download failed. "Unauthorized"
Downloading update from https://downloads.wordpress.org/plugin/wordpress-seo.5.4.zip...
Unpacking the update...
Installing the latest version...
Removing the old version of the plugin...
Plugin updated successfully.
Disabling Maintenance mode...
+----------------------------+-------------+-------------+---------+
| name                       | old_version | new_version | status  |
+----------------------------+-------------+-------------+---------+
| add-to-any                 | 1.7.16      | 1.7.17      | Updated |
| contact-form-7             | 4.8.1       | 4.9         | Updated |
| ewww-image-optimizer       | 3.5.1       | 3.6.1       | Updated |
| ewww-image-optimizer-cloud | 3.5.1       | 3.6.1       | Updated |
| better-wp-security         | 6.3.0       | 6.6.0       | Updated |
| wp-news-and-widget-pro     | 1.1.9       | 2.1.1       | Updated |
| wordpress-seo              | 5.1         | 5.4         | Updated |
+----------------------------+-------------+-------------+---------+
Success: Updated 7 of 7 plugins.

Enabling Maintenance mode...

Downloading update from https://downloads.wordpress.org/plugin/add-to-any.1.7.17.zip...

Unpacking the update...

Installing the latest version...

Removing the old version of the plugin...

Plugin updated successfully.

Downloading update from https://downloads.wordpress.org/plugin/contact-form-7.4.9.zip...

Unpacking the update...

Installing the latest version...

Removing the old version of the plugin...

Plugin updated successfully.

Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer.3.6.1.zip...

Unpacking the update...

Installing the latest version...

Removing the old version of the plugin...

Plugin updated successfully.

Downloading update from https://downloads.wordpress.org/plugin/ewww-image-optimizer-cloud.3.6.1.zip...

Unpacking the update...

Installing the latest version...

Removing the old version of the plugin...

Plugin updated successfully.

Downloading update from https://downloads.wordpress.org/plugin/better-wp-security.6.6.0.zip...

Unpacking the update...

Installing the latest version...

Removing the old version of the plugin...

Plugin updated successfully.

Downloading update from https://www.wponlinesupport.com/edd-sl/package_download/removedapikey...

Warning: Download failed. "Unauthorized"

Downloading update from https://downloads.wordpress.org/plugin/wordpress-seo.5.4.zip...

Unpacking the update...

Installing the latest version...

Removing the old version of the plugin...

Plugin updated successfully.

Disabling Maintenance mode...

+----------------------------+-------------+-------------+---------+

+----------------------------+-------------+-------------+---------+

| add-to-any | 1.7.16 | 1.7.17 | Updated |

| contact-form-7 | 4.8.1 | 4.9 | Updated |

| ewww-image-optimizer | 3.5.1 | 3.6.1 | Updated |

| ewww-image-optimizer-cloud | 3.5.1 | 3.6.1 | Updated |

| better-wp-security | 6.3.0 | 6.6.0 | Updated |

| wp-news-and-widget-pro | 1.1.9 | 2.1.1 | Updated |

| wordpress-seo | 5.1 | 5.4 | Updated |

+----------------------------+-------------+-------------+---------+

Success: Updated 7 of 7 plugins.

I have now updated all active plugins from one command line.

I have been unable to update WordPress itself since I moved my website (without a FTP server).

Let’s try and update WordPress with wp-cli.

Run this command.

sudo wp core update

1	sudo wp core update

Success

Updating to version 4.8.1 (en_US)...
Downloading update from https://downloads.wordpress.org/release/wordpress-4.8.1-no-content.zip...
Unpacking the update...
Success: WordPress updated successfully.

Updating to version 4.8.1 (en_US)...

Downloading update from https://downloads.wordpress.org/release/wordpress-4.8.1-no-content.zip...

Unpacking the update...

Success: WordPress updated successfully.

Awesome 🙂

Now time to create a few executable bash files to update plugins and wordpress in future.

Update WordPress Plugins Bash Script

sudo nano /scripts/updatewordpressplugins.sh

1	sudo nano /scripts/updatewordpressplugins.sh

Contents

#!/bin/bash
wp plugin update --all

1 2	#!/bin/bash wp plugin update --all

Make Executable

chmod +X /scripts/updatewordpressplugins.sh

1	chmod +X /scripts/updatewordpressplugins.sh

Update WordPress Bash Script

sudo nano /scripts/updatewordpress.sh

1	sudo nano /scripts/updatewordpress.sh

Contents

!/bin/bash
sudo wp core update

1 2	!/bin/bash sudo wp core update

Make Executable

chmod +X /scripts/updatewordpress.sh

1	chmod +X /scripts/updatewordpress.sh

More Commands

More commands can be found here: https://developer.wordpress.org/cli/commands/

Information on exporting tables can be found here https://developer.wordpress.org/cli/commands/db/export/

Site Wide Search and Replace

wp search-replace 'http://fearby.com' 'https://fearby.com' --dry-run

1	wp search-replace 'http://fearby.com' 'https://fearby.com' --dry-run

Results

+------------------+-----------------------+--------------+------+
| Table            | Column                | Replacements | Type |
+------------------+-----------------------+--------------+------+
| wp_commentmeta   | meta_key              | 0            | SQL  |
| wp_commentmeta   | meta_value            | 0            | SQL  |
| wp_comments      | comment_author        | 0            | SQL  |
| wp_comments      | comment_author_email  | 0            | SQL  |
| wp_comments      | comment_author_url    | 72           | SQL  |
| wp_comments      | comment_author_IP     | 0            | SQL  |
| wp_comments      | comment_content       | 0            | SQL  |
| wp_comments      | comment_approved      | 0            | SQL  |
| wp_comments      | comment_agent         | 0            | SQL  |
| wp_comments      | comment_type          | 0            | SQL  |
| wp_links         | link_url              | 0            | SQL  |
| wp_links         | link_name             | 0            | SQL  |
| wp_links         | link_image            | 0            | SQL  |
| wp_links         | link_target           | 0            | SQL  |
| wp_links         | link_description      | 0            | SQL  |
| wp_links         | link_visible          | 0            | SQL  |
| wp_links         | link_rel              | 0            | SQL  |
| wp_links         | link_notes            | 0            | SQL  |
| wp_links         | link_rss              | 0            | SQL  |
| wp_options       | option_name           | 0            | SQL  |
| wp_options       | option_value          | 10           | PHP  |
| wp_options       | autoload              | 0            | SQL  |
| wp_postmeta      | meta_key              | 0            | SQL  |
| wp_postmeta      | meta_value            | 56           | PHP  |
| wp_posts         | post_content          | 2176         | SQL  |
| wp_posts         | post_title            | 0            | SQL  |
| wp_posts         | post_excerpt          | 0            | SQL  |
| wp_posts         | post_status           | 0            | SQL  |
| wp_posts         | comment_status        | 0            | SQL  |
| wp_posts         | ping_status           | 0            | SQL  |
| wp_posts         | post_password         | 0            | SQL  |
| wp_posts         | post_name             | 0            | SQL  |
| wp_posts         | to_ping               | 0            | SQL  |
| wp_posts         | pinged                | 26           | SQL  |
| wp_posts         | post_content_filtered | 0            | SQL  |
| wp_posts         | guid                  | 3928         | SQL  |
| wp_posts         | post_type             | 0            | SQL  |
| wp_posts         | post_mime_type        | 0            | SQL  |
| wp_term_taxonomy | taxonomy              | 0            | SQL  |
| wp_term_taxonomy | description           | 0            | SQL  |
| wp_termmeta      | meta_key              | 0            | SQL  |
| wp_termmeta      | meta_value            | 0            | SQL  |
| wp_terms         | name                  | 0            | SQL  |
| wp_terms         | slug                  | 0            | SQL  |
| wp_usermeta      | meta_key              | 0            | SQL  |
| wp_usermeta      | meta_value            | 0            | PHP  |
| wp_users         | user_login            | 0            | SQL  |
| wp_users         | user_nicename         | 0            | SQL  |
| wp_users         | user_email            | 0            | SQL  |
| wp_users         | user_url              | 0            | SQL  |
| wp_users         | user_activation_key   | 0            | SQL  |
| wp_users         | display_name          | 0            | SQL  |
+------------------+-----------------------+--------------+------+
Success: 6268 replacements to be made.

+------------------+-----------------------+--------------+------+

+------------------+-----------------------+--------------+------+

| wp_commentmeta | meta_key | 0 | SQL |

| wp_commentmeta | meta_value | 0 | SQL |

| wp_comments | comment_author | 0 | SQL |

| wp_comments | comment_author_email | 0 | SQL |

| wp_comments | comment_author_url | 72 | SQL |

| wp_comments | comment_author_IP | 0 | SQL |

| wp_comments | comment_content | 0 | SQL |

| wp_comments | comment_approved | 0 | SQL |

| wp_comments | comment_agent | 0 | SQL |

| wp_comments | comment_type | 0 | SQL |

| wp_links | link_url | 0 | SQL |

| wp_links | link_name | 0 | SQL |

| wp_links | link_image | 0 | SQL |

| wp_links | link_target | 0 | SQL |

| wp_links | link_description | 0 | SQL |

| wp_links | link_visible | 0 | SQL |

| wp_links | link_rel | 0 | SQL |

| wp_links | link_notes | 0 | SQL |

| wp_links | link_rss | 0 | SQL |

| wp_options | option_name | 0 | SQL |

| wp_options | option_value | 10 | PHP |

| wp_options | autoload | 0 | SQL |

| wp_postmeta | meta_key | 0 | SQL |

| wp_postmeta | meta_value | 56 | PHP |

| wp_posts | post_content | 2176 | SQL |

| wp_posts | post_title | 0 | SQL |

| wp_posts | post_excerpt | 0 | SQL |

| wp_posts | post_status | 0 | SQL |

| wp_posts | comment_status | 0 | SQL |

| wp_posts | ping_status | 0 | SQL |

| wp_posts | post_password | 0 | SQL |

| wp_posts | post_name | 0 | SQL |

| wp_posts | to_ping | 0 | SQL |

| wp_posts | pinged | 26 | SQL |

| wp_posts | post_content_filtered | 0 | SQL |

| wp_posts | guid | 3928 | SQL |

| wp_posts | post_type | 0 | SQL |

| wp_posts | post_mime_type | 0 | SQL |

| wp_term_taxonomy | taxonomy | 0 | SQL |

| wp_term_taxonomy | description | 0 | SQL |

| wp_termmeta | meta_key | 0 | SQL |

| wp_termmeta | meta_value | 0 | SQL |

| wp_terms | name | 0 | SQL |

| wp_terms | slug | 0 | SQL |

| wp_usermeta | meta_key | 0 | SQL |

| wp_usermeta | meta_value | 0 | PHP |

| wp_users | user_login | 0 | SQL |

| wp_users | user_nicename | 0 | SQL |

| wp_users | user_email | 0 | SQL |

| wp_users | user_url | 0 | SQL |

| wp_users | user_activation_key | 0 | SQL |

| wp_users | display_name | 0 | SQL |

+------------------+-----------------------+--------------+------+

Success: 6268 replacements to be made.

If you are updating past pages and post links do include https look for www alos

wp search-replace 'http://www.fearby.com' 'https://www.fearby.com' --dry-run

1	wp search-replace 'http://www.fearby.com' 'https://www.fearby.com' --dry-run

+------------------+-----------------------+--------------+------+
| Table            | Column                | Replacements | Type |
+------------------+-----------------------+--------------+------+
| wp_commentmeta   | meta_key              | 0            | SQL  |
| wp_commentmeta   | meta_value            | 0            | SQL  |
| wp_comments      | comment_author        | 0            | SQL  |
| wp_comments      | comment_author_email  | 0            | SQL  |
| wp_comments      | comment_author_url    | 0            | SQL  |
| wp_comments      | comment_author_IP     | 0            | SQL  |
| wp_comments      | comment_content       | 0            | SQL  |
| wp_comments      | comment_approved      | 0            | SQL  |
| wp_comments      | comment_agent         | 0            | SQL  |
| wp_comments      | comment_type          | 0            | SQL  |
| wp_links         | link_url              | 0            | SQL  |
| wp_links         | link_name             | 0            | SQL  |
| wp_links         | link_image            | 0            | SQL  |
| wp_links         | link_target           | 0            | SQL  |
| wp_links         | link_description      | 0            | SQL  |
| wp_links         | link_visible          | 0            | SQL  |
| wp_links         | link_rel              | 0            | SQL  |
| wp_links         | link_notes            | 0            | SQL  |
| wp_links         | link_rss              | 0            | SQL  |
| wp_options       | option_name           | 0            | SQL  |
| wp_options       | option_value          | 1            | PHP  |
| wp_options       | autoload              | 0            | SQL  |
| wp_postmeta      | meta_key              | 0            | SQL  |
| wp_postmeta      | meta_value            | 42           | PHP  |
| wp_posts         | post_content          | 729          | SQL  |
| wp_posts         | post_title            | 3            | SQL  |
| wp_posts         | post_excerpt          | 0            | SQL  |
| wp_posts         | post_status           | 0            | SQL  |
| wp_posts         | comment_status        | 0            | SQL  |
| wp_posts         | ping_status           | 0            | SQL  |
| wp_posts         | post_password         | 0            | SQL  |
| wp_posts         | post_name             | 0            | SQL  |
| wp_posts         | to_ping               | 0            | SQL  |
| wp_posts         | pinged                | 10           | SQL  |
| wp_posts         | post_content_filtered | 0            | SQL  |
| wp_posts         | guid                  | 0            | SQL  |
| wp_posts         | post_type             | 0            | SQL  |
| wp_posts         | post_mime_type        | 0            | SQL  |
| wp_term_taxonomy | taxonomy              | 0            | SQL  |
| wp_term_taxonomy | description           | 0            | SQL  |
| wp_termmeta      | meta_key              | 0            | SQL  |
| wp_termmeta      | meta_value            | 0            | SQL  |
| wp_terms         | name                  | 0            | SQL  |
| wp_terms         | slug                  | 0            | SQL  |
| wp_usermeta      | meta_key              | 0            | SQL  |
| wp_usermeta      | meta_value            | 0            | PHP  |
| wp_users         | user_login            | 0            | SQL  |
| wp_users         | user_nicename         | 0            | SQL  |
| wp_users         | user_email            | 0            | SQL  |
| wp_users         | user_url              | 0            | SQL  |
| wp_users         | user_activation_key   | 0            | SQL  |
| wp_users         | display_name          | 0            | SQL  |
+------------------+-----------------------+--------------+------+

+------------------+-----------------------+--------------+------+

+------------------+-----------------------+--------------+------+

| wp_commentmeta | meta_key | 0 | SQL |

| wp_commentmeta | meta_value | 0 | SQL |

| wp_comments | comment_author | 0 | SQL |

| wp_comments | comment_author_email | 0 | SQL |

| wp_comments | comment_author_url | 0 | SQL |

| wp_comments | comment_author_IP | 0 | SQL |

| wp_comments | comment_content | 0 | SQL |

| wp_comments | comment_approved | 0 | SQL |

| wp_comments | comment_agent | 0 | SQL |

| wp_comments | comment_type | 0 | SQL |

| wp_links | link_url | 0 | SQL |

| wp_links | link_name | 0 | SQL |

| wp_links | link_image | 0 | SQL |

| wp_links | link_target | 0 | SQL |

| wp_links | link_description | 0 | SQL |

| wp_links | link_visible | 0 | SQL |

| wp_links | link_rel | 0 | SQL |

| wp_links | link_notes | 0 | SQL |

| wp_links | link_rss | 0 | SQL |

| wp_options | option_name | 0 | SQL |

| wp_options | option_value | 1 | PHP |

| wp_options | autoload | 0 | SQL |

| wp_postmeta | meta_key | 0 | SQL |

| wp_postmeta | meta_value | 42 | PHP |

| wp_posts | post_content | 729 | SQL |

| wp_posts | post_title | 3 | SQL |

| wp_posts | post_excerpt | 0 | SQL |

| wp_posts | post_status | 0 | SQL |

| wp_posts | comment_status | 0 | SQL |

| wp_posts | ping_status | 0 | SQL |

| wp_posts | post_password | 0 | SQL |

| wp_posts | post_name | 0 | SQL |

| wp_posts | to_ping | 0 | SQL |

| wp_posts | pinged | 10 | SQL |

| wp_posts | post_content_filtered | 0 | SQL |

| wp_posts | guid | 0 | SQL |

| wp_posts | post_type | 0 | SQL |

| wp_posts | post_mime_type | 0 | SQL |

| wp_term_taxonomy | taxonomy | 0 | SQL |

| wp_term_taxonomy | description | 0 | SQL |

| wp_termmeta | meta_key | 0 | SQL |

| wp_termmeta | meta_value | 0 | SQL |

| wp_terms | name | 0 | SQL |

| wp_terms | slug | 0 | SQL |

| wp_usermeta | meta_key | 0 | SQL |

| wp_usermeta | meta_value | 0 | PHP |

| wp_users | user_login | 0 | SQL |

| wp_users | user_nicename | 0 | SQL |

| wp_users | user_email | 0 | SQL |

| wp_users | user_url | 0 | SQL |

| wp_users | user_activation_key | 0 | SQL |

| wp_users | display_name | 0 | SQL |

+------------------+-----------------------+--------------+------+

When you are ready to run the replace remove the

--dry-run

--dry-run

As always, backup your database and files before you make changes.

Success: Made 6270 replacements.
Success: Made 787 replacements.

1 2	Success: Made 6270 replacements. Success: Made 787 replacements.

This was the easiest way to update my WordPress site and force existing links use https links. Of course, you can downgrade https links if you don’t want to have https anymore.

Troubleshooting

All the following the wp command if you are logged in as root (not recommended for security reasons).

--allow-root

1	--allow-root

Adding “Advertisement” text above Google ads in all WordPress pages and posts, read this guide on setting up AdSense on your blog.

sudo wp search-replace '<script async src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'
sudo wp search-replace '<script async="" src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

sudo wp search-replace '<script async src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

sudo wp search-replace '<script async="" src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

Thank you Kerry Hoath for the tip on this awesome cli tool.

Donate and make this blog better

Ask a question or recommend an article

v1.2 Install WordPress.

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads

September 9, 2017 by Simon Leave a Comment

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads.

This guide is a shorter post around setting up SEO (Search Engine Optimization) and driving more traffic to your site without buying ADs. In a nutshell, to have better SEO you need to jump some technical hurdles in order to drive more traffic to your site from search engines along with understanding your customer’s needs and making things easier for them.

I have blogged about this topics before but these posts are too long in reflection.

Buying Ad’s?

Facebook, Google, Bing and advertising agencies will recommend you set goals around growth and site traffic and pay for those goals to succeed (usually by advertisements).

Don’t get me wrong Advertising works but it is a competitive market, Online sites can easily setup the display of Ad’s on their site (my guide here Add Google AdWords to your WordPress blog, https://fearby.com/article/add-google-adwords-wordpress-blog/ ). You can buy physical billboard ad’s on the side of roads (e.g http://www.buythisspace.com.au/). I tried to enquire about the costs of a physical billboard but the agencies robot verification rejected my enquiry submission so I gave up. Advertising is buying peoples times and people now how to avoid ad’s and not interact with them (7 Marketing Lessons from Eye-Tracking Studies https://blog.kissmetrics.com/eye-tracking-studies/)

Do more of what works

Spoiler: This guide will recommend you do more of what works over buying millions of ad’s and hoping for new and engaged customers and customer growth.

If you don’t already have Google Analytics setup on your site then do it, you cannot identify your customers or identify what is broken or in turn fix it (Setting up Google Analytics on your website, https://fearby.com/article/setting-up-google-analytics-on-your-website/ )
Monitor Data – Do review your logs and customer related data (review orders, customers and try and identify what works. Software like https://www.zoho.com/one/applications/web.html will help you connect the dots.
Adobe Audience Cloud: http://www.adobe.com/au/experience-cloud.html is a more expensive software suite for driving decisions based on data.
Benchmarks – Set goals and work toward them (e.g I want 10x more customers).

SEO Tip’s

This older article on How to boost your site’s SEO attempts to mention what you need to do it to get better SEO.

Do run a modern great site

I am a big fan of word of mouth over free/organic traffic over paid customers via advertising (Mostly because I am tight and realize advertising can be a bottomless pit). The single biggest thing you can do to have more organic traffic from search engines is run a modern and fast website, have valuable content and make it as easy for the customer as possible. This is why I moved my site and setup an SSL certificate (link to article).

Search engines like your site to be fast, updated frequently, have sitemaps to make their jobs easier and have an SSL certificate to keep the web safe etc.

Google, Bing and other search engines will not send traffic your way if you do not satisfy them that your site is liked or has valuable content. Google makes money from Google Analytics by helping people understand their site’s visitors then recommend you pay for ad’s to use on sites that have AdWords on their site ( WordPress to a new self-managed server away from CPanel ).

How to boost your site’s SEO https://fearby.com/article/how-to-boost-your-sites-seo/
Your website needs to be fast, use sites like https://www.webpagetest.org to measure how fast your site is (Aim for all A’s). Read this page for information on the impact of slow websites https://www.searchenginejournal.com/mobile-page-speed-benchmarks/194511/
Mobile friendly – Ensure your site is mobile friendly (or risk being dropped from search engine results)
SSL – Do have a secure SSL certificate on your website (view mine here https://www.ssllabs.com/ssltest/analyze.html?d=www.fearby.com&s=45.63.29.217&latest).
Incoming links – Having incoming links to your site tell search engines that your site is popular.

Traffic Source types

Organic – An organic visitor to your site is one who found your site by searching something that was relevant to their search term and not by clicking on an advertisement.
Paid – A paid user is someone who has clicked an ad to come to your site.
Social – A social visitor is one who is known to come from a social media site, using social media sites like Twitter, Facebook or Instagram is a must to driving organic traffic (go where the people are).

Engagement

How engaged are your customers? Have you asked your customers recently what they value or appreciate about your business or product? Have you asked for feedback recently?

User Engagement Levels

None – Do you have landing pages that quickly inform customers of your products or services?
Low – What do they need to know about your product or service?
Medium – Aware (engaged)
High – Can this person be an advocate for your business?
Gone – Did you get exit Feedback?

Ways to engage already engaged customers.

Setup a free MailChimp Newsletter to allow willing people to be alerted of new communication https://login.mailchimp.com/signup/?source=website&pid=GAW
Web Browser popup Alerts can be a great way to engage with users when new content is added to your site (Read the guide here https://documentation.onesignal.com/docs/web-push-setup )
Mobile apps or mobile friendly website are a no brainer given 2 billion people use mobile phones ( http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ ).

What can you do to help understand your customer’s needs and make their purchase processes easier?

Why are your customers leaving?

Understand more about your customers reasons for leaving and act upon preventing others from leaving.

Trying something new (Does your website need to be simpler?)
Are your products too expensive?.
Your site (or ordering) is not convenient (Do you need to setup online ordering/subscriptions and delivery?)
etc

Who are your customers

Personas – Do setup customer personas in order to focus on your customer segments (get a free customer persona template here https://blog.hubspot.com/blog/tabid/6307/bid/33491/everything-marketers-need-to-research-create-detailed-buyer-personas-template.aspx )
Does your website match these personas?

Are your customers.

Engaged
Informed
Advocates

Feedback

Do you have feedback loops (A simple feedback form can solve this)?

What do you know about your customers?

Product Satisfaction
Product Loyalty
Product Awareness

Paid Traffic (Ad’s)

Google Ad’s – Signup Here http://www.google.com.au/adwords/get-started/
Bing – Advertise on Bing here https://advertise.bingads.microsoft.com/
Facebook – Advertise on Facebook here https://www.facebook.com/business/products/ads

Free Traffic (SEO + Organic Ad’s)

Blog Posts (Sharing value/passion)
Social Media Posts (use hashtags)
Instagram (Post value/passion)

Most importantly Do what works (Measure and replicate).

Focus on Business Value

Generate a SWOT Analysis ( Free tool here https://xtensio.com/ )

What are your Strengths?
What are your Weaknesses?
What are your Opportunities?
What are your Threats?

Goals

Goals allow you to investigate, learn, act and measure I order to improve.

Investigate – Data.
Learn/Insight – Make Assumptions.
Act – Act and measure.

Read more about customer engagement here https://en.wikipedia.org/wiki/Customer_engagement

Bonus

Do ensure your website is compliant with accessibility and technical standards

Test our sites Accessibility – https://achecker.ca/checker/index.php
Test your sites HTML5 Compliance – https://validator.w3.org
Test your Google PageSpeed Test – https://developers.google.com/speed/pagespeed/insights/
Do A B testing to determine the statistical significance of changes to your site.

Conclusion

The more you know the better you can connect, Do set goals and as a minimum setup Google Analytics, SSL certificate and submit your site to search engines, then focus on a fast site that makes things simple for your customers.

Donate and make this blog better

Ask a question or recommend an article

v1.0 Initial version

Moving a CPanel domain with email to a self managed VPS and Gmail

August 3, 2017 by Simon 6 Comments

I have had www.fearby.com since 1999 on three CPanel hosts (superwerbhost in the US, Jumba in Australia, Uber in Australia (NetRegistry have acquired Uber and performance at the time of writing is terrible)). I was never informed by Uber of the sale but my admin portal was moved from one host to another and each time performance degraded. I tried to speed up WordPress by optimizing images, installing cache plugins but nothing worked, pages were loading in around 24 seconds on https://www.webpagetest.org.

I had issues with a CPanel domain on the hosts (Uber/Netregistry) as they were migrating domains and the NetRegstry chat rep said I needed to phone Uber for support. No thanks, I’m going self-managed and saving a dollar.

I decided to take ownership of my slow domain and setup my own VM and direct web traffic to it and redirect email to GMail (I have done this before). I have setup Digital Ocean VM’s (Ubuntu and Centos), Vultr VM’s and AWS VM’s.

I have also had enough of Resource Limit Reached messages with CPanel and I can’t wait to…

not have a slow WordPress.
setup my own server (not a slow hosts server).
spend $5 less (we currently pay $25 for a CPanel website with 20GB storage total)
get a faster website (sub 24 seconds load time).
larger email mailboxes (30GB each).
Generate my own “SSL Labs A+ rated” certificate for $10 a year instead of $150 a year for an “SSL Labs C rated” SSL certificate from my existing hosts.

Backup

I have about 10 email accounts on my CPanel domain (using 14GB) and 2x WordPress sites. I want to backup my emails with (Outlook Export and Thunderbird Profile backup) and backup my domain file(s) a few times before I do anything. Once DNS is set in motion no server waits.

The Plan

Once everything is backed up I intend to setup a $5 a month Vulr VM and redirect all mail to Google G Suite (I have redirected mail before).

I will setup a Vultr web server in Sydney (following my guide here), buy an SSL certificate from Namecheap and move my WordPress sites.

Rough Plan

Reduce email accounts from 10x to 3x
Backup emails (twice with ThunderBird and Outlook).
Setup A Ubuntu V on Vultr.
Signup for Google G Suite Trial.
Transfer my domain to Namecheap.
Link to domain DNS to Vultr
Link to domain MX records to Google Email.
Transfer website.
Setup emails on google.
Restore WordPress.
Go live.
Downgrade to personal G Suite before the trial expires
Close down the old server.

Signing up for Google G Suite

I visited https://gsuite.google.com/ and started creating an account.

Get 20% off your first year by signing up for Google G Suite using this link: https://goo.gl/6vpuMm

I created a link between G Suite and an existing GMail account.

Now I can create the admin account.

Tip: Don’t use any emails that are linked as secondary emails with any Google services (this won’t be allowed). It’s s a well-known issue that you cannot add users emails who are linked to Google services (even as backup emails for Gmail, detach the email before adding it). Read more here.

Final setup steps.

Now I can add email accounts to G Suite.

Adding email users to G Suite.

The next thing I had to do was upload a file to my domain to verify I own the domain (DNS verification is also an option).

I must say the setup and verify steps are quite easy to follow on G Suite.

Time to backup our existing CPanel site.

Backup Step 1 (hopefully I won’t need this)

I decided to grab a complete copy of my CPanel domain with domains, databases and email accounts. This took 24 hours.

Backup Step 2 (hopefully I won’t need this)

I download all mail via IMAP in Outlook and Mozilla Thunderbird and export it (Outlook Export and Thunderbird Profile backup). Google have IMAP instructions here.

DNS Changes at Namecheap

I obtained my domain EPP code from my CPanel hosts and transferred the domain name to Namecheap.

Namecheap was even nice enough to set my DNS point to my existing domain so I did not have to rush a move before DNS propagation.

P.S The Namecheap Chat Staff and Namecheap Mobile App is awesome.

Having backed up everything I logged into Namecheap and set my DNS to “NameCheap BasicDNS” and then went “Advanced DNS” and set appropriate DNS records for my domain. This assumes you have setup a VM with IPV4 and IPV6 (follow my guide here).

A Record @ IPV4_OF_MY_VULTR_SERVER
A Record www IPV4_OF_MY_VULTR_SERVER
A Record ftp IPV4_OF_MY_VULTR_SERVER
AAAA Record @ IPV6_OF_MY_VULTR_SERVER
AAAA Record www IPV6_OF_MY_VULTR_SERVER
AAAA Record ftp IPV6_OF_MY_VULTR_SERVER
C Name www fearby.com

The Google G Suite also asked me to add these following MX records to the DNS records.

MX Record @ ASPMX.L.GOOGLE.COM. 1
MX Record @ ASPMX1.L.GOOGLE.COM. 5
MX Record @ ASPMX2.L.GOOGLE.COM. 5
MX Record @ ASPMX3.L.GOOGLE.COM. 10
MX Record @ ASPMX4.L.GOOGLE.COM. 10

Then it was a matter of telling Google DNS changes were made (once DNS has replicated across the US).

My advice is to set DNS changes before bed as it can take 12 hours.

Sites like https://www.whatsmydns.net/ are great for keeping track of DNS replication.

Transferring WordPress

I logged into the CPanel and exported my WordPress Database (34MB SQL file).

I had to make the following PHP.ini changes to allow the larger file size restore uploads with the Adminer utility (default is 2mb). I could not get the server side adminer.sls.gz option to restore the database?

post_max_size = 50M
upload_max_filesize = 50M

# do change back to 2MB after you restore the files to prevent DOS attacks.

post_max_size = 50M

upload_max_filesize = 50M

# do change back to 2MB after you restore the files to prevent DOS attacks.

I had to make the following changes to nginx.conf (to prevent 404 errors on the database upload)

client_max_body_size 50M;
# client_max_body_size 2M; Reset when done

1 2	client_max_body_size 50M; # client_max_body_size 2M; Reset when done

I also had to make these changes to NGINX (sites-available/default) to allow WordPress to work

# Add index.php to the list if you are using PHP
	index index.php index.html index.htm;

location / {
        # try_files $uri $uri/ =404;
        try_files $uri $uri/ /index.php?q=$uri&$args;
        index index.php index.html index.htm;
        proxy
}

# Add index.php to the list if you are using PHP

index index.php index.html index.htm;

location / {

# try_files $uri $uri/ =404;

try_files $uri $uri/ /index.php?q=$uri&$args;

index index.php index.html index.htm;

proxy

}

I had a working MySQL (I followed my guide here).

Adminer is the best PHP MySQL management utility (beats PhpMyAdmin hands down).

Restart NGINX and PHP

nginx -t
nginx -s reload
sudo /etc/init.d/nginx restart
sudo service php7.0-fpm restart

nginx -t

nginx -s reload

sudo /etc/init.d/nginx restart

sudo service php7.0-fpm restart

I had an error on database import, a non-descript error in script line 1 (error hint here).

A simple search and replace in the SQL fixed it.

Once I had increased PHP uploads to 50M and Nginx I was able to upload my database backup with Adminer (just remember to import to the created database that matches. the wp-config.php. Also, ensure your WordPress content is in place too.

The only other problem I had was WordPress gave an “Error 500” so moved few plugins an all was good.

Importing Old Email

I was able to use the Google G Suite tools to import my old Mail (CPanel IMAP to Google IMAP).

I love root access on my own server now, goodbye CPanel “Usage Limit Exceeded” errors (I only had light traffic on my site).

My self-hosted WordPress is a lot snappier now, my server has plenty of space (and only costs $0.007c and hour for 1x CPU, 1GB ram, 25GB SSD storage and 1000GB data transfer quota). I use the htop command to view system processor and disk space usage.

I can now have more space for content and not be restricted by tight hosts disk quotas or slow shared servers. I use the pydf command to view dis space.

pydf
Filesystem Size  Used <strong>Avail</strong> Use%                                                    Mounted on
/dev/vda1   25G 3289M   <strong>20G</strong> 13.1 [######..........................................] /
/www/wp-content#

pydf

Filesystem Size Used <strong>Avail</strong> Use% Mounted on

/dev/vda1 25G 3289M <strong>20G</strong> 13.1 [######..........................................] /

/www/wp-content#

I use ncdu to view folder usage.

Installing ncdu

sudo apt-get install ncdu
Reading package lists... Done
Building dependency tree
Reading state information... Done
ncdu is already the newest version (1.11-1build1).
0 upgraded, 0 newly installed, 0 to remove and 58 not upgraded.

sudo apt-get install ncdu

Reading package lists... Done

Building dependency tree

Reading state information... Done

ncdu is already the newest version (1.11-1build1).

0 upgraded, 0 newly installed, 0 to remove and 58 not upgraded.

Type ncdu in the folder you want to browse under.

ncdu

ncdu

You can arrow up and down folder structures and view folder/file usage.

SSL Certificate

I am setting up a new multi year SS cert now, I will update this guide later. I had to read my SSL guide with Digital Ocean here.

I generated some certificate on my server

cd ~/
kdir sslcsrmaster4096
cd sslcsrmaster4096/
openssl req -new -newkey rsa:4096 -nodes -keyout domain.key -out domain.csr

cd ~/

kdir sslcsrmaster4096

cd sslcsrmaster4096/

openssl req -new -newkey rsa:4096 -nodes -keyout domain.key -out domain.csr

Sample output for a new certificate

openssl req -new -newkey rsa:4096 -nodes -keyout dummy.key -out dummy.csr
Generating a 4096 bit RSA private key
.................................................................................................++
......++
writing new private key to 'dummy.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]: AU
State or Province Name (full name) [Some-State]: NSW
Locality Name (eg, city) []:Tamworth
Organization Name (eg, company) [Internet Widgits Pty Ltd]: Dummy Org
Organizational Unit Name (eg, section) []: Dummy Org Dept
Common Name (e.g. server FQDN or YOUR name) []: DummyOrg
Email Address []: me@dummy.org

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []: password
An optional company name []: DummyCO
root@servernamehere:~/sslcsrmaster4096# cat dummy.csr
-----BEGIN CERTIFICATE REQUEST-----
MIIFAjCCAuoCAQAwgYsxCzAJBgNVBAYTAkFVMQwwCgYDVQQIDANOU1cxETAPBgNV
BAcMCFRhbXdvcnRoMRIwEAYDVQQKDAlEdW1teSBPcmcxFzAVBgNVBAsMDkR1bW15
IE9yZyBEZXB0MREwDwYDVQQDDAhEdW1teU9yZzEbMBkGCSqGSIb3DQEJARYMbWVA
ZHVtbXkub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA6PUtWkRl
+gL0Hx354YuJ5Sul2Xh+ljILSlFoHAxktKlE+OJDJAtUtVQpo3/F2rGTJWmmtef+
shortenedoutput
swrUzpBv8hjGziPoVdd8qdAA2Gh/Y5LsehQgyXV1zGgjsi2GN4A=
-----END CERTIFICATE REQUEST-----

openssl req -new -newkey rsa:4096 -nodes -keyout dummy.key -out dummy.csr

Generating a 4096 bit RSA private key

.................................................................................................++

......++

writing new private key to 'dummy.key'

-----

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]: AU

State or Province Name (full name) [Some-State]: NSW

Locality Name (eg, city) []:Tamworth

Organization Name (eg, company) [Internet Widgits Pty Ltd]: Dummy Org

Organizational Unit Name (eg, section) []: Dummy Org Dept

Common Name (e.g. server FQDN or YOUR name) []: DummyOrg

Email Address []: me@dummy.org

Please enter the following 'extra' attributes

to be sent with your certificate request

A challenge password []: password

An optional company name []: DummyCO

root@servernamehere:~/sslcsrmaster4096# cat dummy.csr

-----BEGIN CERTIFICATE REQUEST-----

MIIFAjCCAuoCAQAwgYsxCzAJBgNVBAYTAkFVMQwwCgYDVQQIDANOU1cxETAPBgNV

BAcMCFRhbXdvcnRoMRIwEAYDVQQKDAlEdW1teSBPcmcxFzAVBgNVBAsMDkR1bW15

IE9yZyBEZXB0MREwDwYDVQQDDAhEdW1teU9yZzEbMBkGCSqGSIb3DQEJARYMbWVA

ZHVtbXkub3JnMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA6PUtWkRl

+gL0Hx354YuJ5Sul2Xh+ljILSlFoHAxktKlE+OJDJAtUtVQpo3/F2rGTJWmmtef+

shortenedoutput

swrUzpBv8hjGziPoVdd8qdAA2Gh/Y5LsehQgyXV1zGgjsi2GN4A=

-----END CERTIFICATE REQUEST-----

I then uploaded the certificate to Namecheap for an SSL cert registration.

I selected DNS C Name record as a way to verify I own my domain.

I am now waiting for Namecheap to verify my domain

End of the Google G Suite Business Trial

Before the end of the 14-day trial, you will need to add billing details to keep the email working.

At this stage, you can downgrade from a $10/m business account per user to a $5/m per user account if you wish. The only loss would be storage and google app access.

Get 20% off your first year by signing up for Google G Suite using this link: https://goo.gl/6vpuMm

Before your trial ends, add your payment details and downgrade from $10/user a month business prices to $5/iser a month individual if needed.

G Suite Troubleshooting

I was able to access new G Suite email account via gmail.com but not via Outlook 2015? I reset the password, followed the google troubleshooting guide and used the official incoming and outgoing settings but nothing worked.

Google phone support suggested I enable less secure connection settings as Google firewall may be blocking Outlook. I know the IMAP RFC is many years old but I doubt Microsoft are talking to G Suite in a lazy manner.

Now I can view my messages and I can see one email that said I was blocked by the firewall. Google phone support and faqs don’t say why Outlook 2015 SSL based IMAP was blocked?

Conclusion

Thanks to my wife who put up with my continual updates over the entire domain move. Voicing the progress helped me a lot.

Donate and make this blog better

Ask a question or recommend an article

V1.7 added troubleshooting

Add Google AdWords to your WordPress blog

July 30, 2017 by Simon 3 Comments

Google says “Turn your passion into profit. AdSense is a free, simple way to make money online by placing ads on your website.” How hard is it to setup Google AdSense on your WordPress site?

Advertisement:</br />

First, you will need to create a Google AdSense account here and add your site to Google AdSense. Don’t forget to add your account and payment type here.

You will need to add a bit of HTML script from Google to your site to verify you own the domain you are adding ads too. Lucky for me my theme Genesis allows me to paste code straight into the header and footer sections for my site.

Google AdSense steps are quite clear and will let you know what you need to do (I monitor this from my phone).

After your site is verified (automatically) you will have to wait up to 1 week for Google to review your site. You will receive an email when Google has approved your site.

Then you are ready to log in to Google Adwords page and create an Advertisement style that matches your site. Now you can create your first ad type in the Google AdSense screen.

I created a “Text and Display Ads”.

I then entered a name and chose “Automatic Size” and “Responsive”, feel free to add your own colors here and customize.

From the Google Adsense page go to My Ads, Content then Ad Unit, Now you can click get the code.

I can use this code to place auto responsive ad’s on my web pages.

I should have embedded this code into my WordPress theme but I decided to place the code manually into each of my posts (in text mode).

This is how a Google AdWords Ad looks in this post (below the first paragraph). Very smart that Google chose to serve an AdWords company Ad in my post about Google Ad Words.

Here are my earnings 10 minutes after adding my first ad to my blog ($0 as expected). I plan on using any revenue to speed up this website as the host is very slow.

If you like this post please click my Ad’s. I will update my revenue numbers here in the future.

Thanks to Emma who runs a great blog and encouraged me to blog years ago and big thanks to my wife Alison who ran a blog for years (way before me with loads more visitors).

Payment

It appears you can’t add a payment method or add a payment method until you reach $100 earned.

It looks like earnings will be paid monthly if you meet the target. Payment placeholders are available at https://play.google.com/store/account# I guess new payment methods can be added here.

AdSense Mobile app.

I use the AdSense mobile app

I am not sure if it is accurate at forecasting?

Searching reveals that estimated revenue will be evaluated at the end of the month and the final amount will be reduced.

Content

I guess I need to look at Google Analytics and blog more of what people want.

SEO

Don’t forget to ensure your sites SEO is working as expected. Also, a slow website will have poor SEO so consider a faster private VM from Vultr or Digital Ocean.

Ad Glossary

Page RPM – Page revenue per thousand impressions (RPM) is calculated by dividing your estimated earnings by the number of page views you received, then multiplying by 1000.
Page CTR – The page click through rate (CTR) is the number of ad clicks divided by the number of page views.
Impression CTR – An impression is counted for each ad request that returns at least one ad to the site. It is the number of ad units (for content ads) or search queries (for search ads) that showed ads.
Cost Per Click – The cost-per-click (CPC) is the amount you earn each time a user clicks on your ad. The CPC for any ad is determined by the advertiser; some advertisers may be willing to pay more per click than others, depending on what they’re advertising.
Impression RPM – The impression revenue per thousand impressions (RPM) is the average earnings per one thousand impressions.
Page ROM – Page revenue per thousand impressions (RPM) is calculated by dividing your estimated earnings by the number of page views you received, then multiplying by 1000.
Coverage – Coverage is the percentage of ad requests that returned at least one ad. Generally, coverage can help you identify sites where AdSense isn’t able to provide targeted ads.
Maximum CPC Bid – A bid that you set to determine the highest amount that you’re willing to pay for a click on your ad.
CPM Bid (Cost Per 1000 Impressions) – CPM bidding means that you pay based on the number of impressions (times your ads are shown) that you receive on the Google Display Network. Starting this year, CPM bidding will be replaced by viewable CPM bidding.

How costs are calculated in AdWords.

Read the Google AdSense Glossary and help here.

Ad Sense Rules

Ad Sense Beginner’s Guide – Stay compliant with our policies.

Summary:

Don’t click your own ads.
Don’t ask others to click your ads.
Don’t include any prohibited site content, including adult content, violence or excessive profanity, drugs (including alcohol and tobacco) or copyrighted material.
Don’t modify the AdSense code.
Do follow our Webmaster Quality Guidelines.
Do provide a good user experience.
Don’t place more ads than content on any page.
Don’t place images near ads in a way that may mislead users into thinking that the images are associated with the ads.

Google Webmaster guidelines.Payment

Payment

You will be unable to receive payment until you verify your address.

If you are on a 14-day free trial you will need to setup billing before the trial expires

Tips

Do add the show ad’s code to WordPress over manually adding to pages or posts.
You can only add one script to your page once (e.g Header or Footer (not Both)) or multiple ad banners will be visible and only one will be clickable.

Bonus

Read my guide on managing WordPress via the command line where I automatically (with one command line added the word Advertisements) to over 80 post and pages.

sudo wp search-replace '<script async src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'
sudo wp search-replace '<script async="" src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

sudo wp search-replace '<script async src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

sudo wp search-replace '<script async="" src="//pagead2.googlesyndication.com' 'Advertisement:<br /> <script async="" src="//pagead2.googlesyndication.com'

Donate and make this blog better

Ask a question or recommend an article

Draft: v1.89 added wp-cli

How to boost your site’s SEO

July 21, 2017 by Simon 2 Comments

In these olden days of the Internet you just needed to submit your site to search engines and wait a few days and every thing was fine and dandy. Making your site appear high in search engines these days is next to impossible due to changing rules and formulas defined by search engines.

It is advisable to avoid so called experts that spam your inbox saying they can improve your site’s SEO for $99 (but strangely do not have a website themselves).

There is no golden rule or trade secret to getting pages ranked high on search engines. You will need to do a number of things to get higher in search results or even appear on search results at all.

Google is not the only search engine and you will need to do a number of things to appear on all of them. Other search engines include DuckDuckGo, Bing, Yahoo, Ask and 14 more search engines here.

Site maps

Google and other top search engines expect your website to declare pages on your site via a sitemap.xml file (specifications here). There is a number of ways you can generate a site map file. If you use the WordPress CMS there are a number of plugins that generate sitemap.xml files for you. Search Engines have automatic bots that will read known sitemap files in iorder to update thir search results.

Q1) Does your web site have a sitemap file?

Warnings

Most search engines will ignore your website if it is not trustworthy. You will be classified trustworthy if:

Your website is linked to from many other trustworthy sites.
Your website is not in an untrustworthy country or on an untrustworthy web site host.
Your website does not have banned words or linked banned media.
You don’t serve malicious files (or your Web servers IP address has never served malicious files).
Your website is updated frequently. Sites like http://web.archive.org/web/*/www.microsoft.com will let you know how frequently a site has been updated.
Your website needs to be secure (must have a secure SSL certificate installed and enabled). Google tells you HTTPS sites get better ranking here.
Your website needs to be fast, a search engine will not point people to slow sites period.
Your website also needs to have a lot of past and current traffic.

Content

Your site will also need to have good content in order to be seen as a valuable site. A blog is a good way to create new content and get new organic traffic to your site. You will need a desirable product or service to be high in the search engines result (e.g Chart generator or sell a good product or service ). Your site also needs to contain the matching keywords to be referred to when people search for a particular thing.

It is easy to appear high in the search results if you use feed google enough words that match your article but it is hard enough to get to the top of the rankings for the term “SEO” with “144,00,000” other matching pages in google coming up for “SEO”

If you are going to pay someone to improve your SEO ask for their track record on SEO ranks, ask for examples, adding the more words to search results to get higher is lazy. I think being high in google rankings with 2 words is near impossible unless you use a location in the search term. Google search is a trying to match the right results based on what you feed it and if you fail in any way your site is ranked down.

Analytics

Using search engines analytics like Google Analytics is a good way to ensure search engines know you exist (Google at least) and let you know what works and allows you to generate more content of what works.

Inbound Links

Most search engines prioritize your site over others if you have a high number of links to your site (inbound links), you can have the best site but with no inbound links to your site, you are in trouble. Creating fake links from non-important sites won’t cheat the system.

Site Speed

Your web site need needs to be fast in order to be deemed trustworthy from search engines. A slow site will fall down the search engine page listing. Do everything you can do to speed up your site (start with ensuring the server is in the right location on a non-shared server with a fast website utilizing cached content).

I have updated my post on Speeding up WordPress, my site used to load in 28 seconds, it now loads less than 9 seconds.

Secure sites

Google and other search engines will soon require your site to have a strong SSL certificate and high security (read my guide on SSL here and content security policy and public key pinning here).

You can check your site’s SSL certificate here. Check your site with shodan.io often for security flaws.

Blacklists

You will need to ensure your site is hosted on non-malicious serves and is using modern clean web technology. Search engines prefer to redirect people to sites that use modern web technologies like HTML5 over transitional HTML websites.

Metadata (Dublin Core)

Meta data is important to allow search engines know what is on a web page without remembering everything on each page. Read more on the latest meta data standards here.

Logs

Do check your logs (error and activity logs) for website crawlers activity (active and missed).

Search your website logs for known crawler activity. Are you missing search engines?

Your logs will tell you how many hits are coming from search engines. I have had 1300+ people visit my site from Google this month alone.

Set goals around search engine targets and work toward them (percentage or total traffic).

Campaigns

Although campaigns are not directly related to SEO they do allow you to combine stats around page loads and advertising campaigns. I prefer to track campaigns with a custom central redirect page (e.g /go/index.php) that logs the inbound clicks.

// Save this to a file called index.php (maybe under a folder called "/go/" at the top of your server (e.g "https://www.fearby.com/go/index.php")
header("Cache-Control: no-cache, must-revalidate");
header("Expires: Sat, 26 Jul 1997 05:00:00 GMT");
header('Content-Type: text/html; charset=utf-8');

// Insert campaign logging code here
// todo

/* Sample Usage
https://www.fearby.com/go/?seo
https://www.fearby.com/go/?aws
*/
switch ($_SERVER['QUERY_STRING']) { 

	case "seo":
		header( 'Location: https://fearby.com/article/how-to-boost-your-sites-seo/' ) ;
		break;

	case ("beyondssl"):
		header( 'Location: https://fearby.com/article/beyond-ssl-with-content-security-policy-public-key-pinning-etc/' ) ;
		break;

	case ("mysql"):
		header( 'Location: https://fearby.com/article/setting-up-a-fast-distributed-mysql-environment-with-ssl/' ) ;
		break;
	
	case ("aws"):
		header( 'Location: https://fearby.com/article/creating-an-aws-ec2-ubuntu-14-04-server-with-nginx-node-and-mysql-and-phpmyadmin/' ) ;
		break;

	case ("do"):
		header( 'Location: https://fearby.com/article/how-to-buy-a-new-domain-and-ssl-cert-from-namecheap-a-server-from-digital-ocean-and-configure-it/' ) ;
		break;

	default: 
		echo "I could not find a page for the URL you specified.";
		break;
}

// Save this to a file called index.php (maybe under a folder called "/go/" at the top of your server (e.g "https://www.fearby.com/go/index.php")

header("Cache-Control: no-cache, must-revalidate");

header("Expires: Sat, 26 Jul 1997 05:00:00 GMT");

header('Content-Type: text/html; charset=utf-8');

// Insert campaign logging code here

// todo

/* Sample Usage

https://www.fearby.com/go/?seo

https://www.fearby.com/go/?aws

switch ($_SERVER['QUERY_STRING']) {

case "seo":

header( 'Location: https://fearby.com/article/how-to-boost-your-sites-seo/' ) ;

break;

case ("beyondssl"):

header( 'Location: https://fearby.com/article/beyond-ssl-with-content-security-policy-public-key-pinning-etc/' ) ;

break;

case ("mysql"):

header( 'Location: https://fearby.com/article/setting-up-a-fast-distributed-mysql-environment-with-ssl/' ) ;

break;

case ("aws"):

header( 'Location: https://fearby.com/article/creating-an-aws-ec2-ubuntu-14-04-server-with-nginx-node-and-mysql-and-phpmyadmin/' ) ;

break;

case ("do"):

header( 'Location: https://fearby.com/article/how-to-buy-a-new-domain-and-ssl-cert-from-namecheap-a-server-from-digital-ocean-and-configure-it/' ) ;

break;

default:

echo "I could not find a page for the URL you specified.";

break;

}

Long Tail Titles.

I use long and descriptive titles to mention what my blog post is about. I find that this helps with matching what and Google search loves it.

Some of my descriptive post titles

Using long page titles can ensure people click on your links with confidence ( and they will know what your site is about ). Page views are not everything, engagement is.

My most recent popular posts.

I think I need to create some more technical guides as they seem popular.

Stats

Viewing statistics and reports can help you determine the popularity of your site.

Analytics

Reviewing Logs, viewing campaign data, viewing misc reports can help you identify incoming traffic to verify working acquisition sources.

I like looking at Google Analytics Behaviour reports for viewing inbound visits. I can see over 19,000 incoming visitors from google over the last 2.5 years with no sightmap.xml (this proves that content and long titles work).

Adding your site to bing searches

Go to https://www.bing.com/toolbox/submit-site-url and submit your site.

I tried to submit my XML files to Bing but the form stalled (my bad I did not see the organization size mandatory field).

I changed the domain above to just the domain (no sitemap).

I added my 6 sitemaps to the Bing Webmaster tools.

Submit your sitemp files in the Bing Webmaster tools.

Adding Your Site to the Google Search Console

I am trying the WordPress SEO plugin by Yoest but before I do I am adding my site to the Google Search Console (linked to my Gmail account). You will need to add your site to Google Search Console to have Google index your site.

Before Google trusts your domain you need to verify ownership of the domain ( for me I uploaded the verification files with the free FileZilla FTP program then loaded the file, then clicked verify at the Google Search Console page ).

Now my domain is verified and the Google Search console allows me to view the search status of my domain.

Installing a WordPress SEO Plugin – Yoast

Install the Yoast Plugin in WordPress (follow the install steps after you activate from step 1-8). You can add Yoast to your WordPress via the Add Plugins screen or download then upload it.

Don’t forget to activate then run the advanced Yoast configuration wizard.

After you get an authorization code it is important that you select your website from the profile list (if you don’t see this STOP and add your site to the Google Search Console (you cannot skip this step)).

Continue through steps 9-12 and finish installing the Yoast plugin.

Don’t forget to enable sitemaps (and review your settings). Now you can open the SEO plugin advanced options (/wp-admin/admin.php?page=wpseo_dashboard) and review the plugin options.

I installed the plugin but no sitemap.xml was created???

Twitter user https://www.twitter.com/atimmer10 said I may need to save the/wp-admin/options-permalink.php settings (no luck, this did not fix it and no sitemap.xml was created ).

I edited an existing published post and changed the short link and re-published a post ( no luck, this did not fix it and no sitemap.xml was created ).

I read this Yoast FAQ about the sitemap.xml not being created. I added the following to my .htaccess file ( no luck, this did not fix it and no sitemap.xml was created ).

# Yoast SEO - XML Sitemap Rewrite Fix
RewriteEngine On
RewriteBase /
RewriteRule ^sitemap_index.xml$ /index.php?sitemap=1 [L]
RewriteRule ^locations.kml$ /index.php?sitemap=wpseo_local_kml [L]
RewriteRule ^geo_sitemap.xml$ /index.php?sitemap=geo [L]
RewriteRule ^([^/]+?)-sitemap([0-9]+)?.xml$ /index.php?sitemap=$1&sitemap_n=$2 [L]
RewriteRule ^([a-z]+)?-?sitemap.xsl$ /index.php?xsl=$1 [L]
# END Yoast SEO - XML Sitemap Rewrite Fix

# Yoast SEO - XML Sitemap Rewrite Fix

RewriteEngine On

RewriteBase /

RewriteRule ^sitemap_index.xml$ /index.php?sitemap=1 [L]

RewriteRule ^locations.kml$ /index.php?sitemap=wpseo_local_kml [L]

RewriteRule ^geo_sitemap.xml$ /index.php?sitemap=geo [L]

RewriteRule ^([^/]+?)-sitemap([0-9]+)?.xml$ /index.php?sitemap=$1&sitemap_n=$2 [L]

RewriteRule ^([a-z]+)?-?sitemap.xsl$ /index.php?xsl=$1 [L]

# END Yoast SEO - XML Sitemap Rewrite Fix

I still cannot get Yoast to create a sitemap.xml???

It turns out Yoast was creating a hidden /sitemap_index.xml

I then added the 6 individual sitemaps to the Google Search Console and Google started indexing my site.

Now Google will index my site and Yoast will auto update my sitemaps when I add, Edit or delete a piece of content.

Do monitor your sitemap syncs from time to time.

Add your sitemaps(s) to Yahoo, Bing and Ask search engines

More on adding your site to search engines on the Yoast website here. Also, read this article.

robots.txt

robots.txt is also an older sitemap.xml file format, read more here.

I made a backward compatible robots.txt file

User-agent: *
Disallow: 
Disallow: /cgi-bin/
Disallow: /wp-admin/
Sitemap: https://fearby.com/post-sitemap.xml
Sitemap: https://fearby.com/page-sitemap.xml
Sitemap: https://fearby.com/news-sitemap.xml
Sitemap: https://fearby.com/category-sitemap.xml
Sitemap: https://fearby.com/post_tag-sitemap.xml
Sitemap: https://fearby.com/news-category-sitemap.xml

User-agent: *

Disallow:

Disallow: /cgi-bin/

Disallow: /wp-admin/

Sitemap: https://fearby.com/post-sitemap.xml

Sitemap: https://fearby.com/page-sitemap.xml

Sitemap: https://fearby.com/news-sitemap.xml

Sitemap: https://fearby.com/category-sitemap.xml

Sitemap: https://fearby.com/post_tag-sitemap.xml

Sitemap: https://fearby.com/news-category-sitemap.xml

Generate a robots.txt here and read more here.

Other Plugins

Generate sitemaps from Yoast sitemaps with WP SEO HTML Sitemaps.

Other Tools

Google has a Structured Data Testing Tool to allow you to diagnose various search issues.
Webpage accessibility checker.
Webpage speed checker

Top WordPress Plugins

Do More of What Works

Review pages and see what works, you may find old pages or posts are more popular than new ones.

Set Goals and Measure Them

Goals like page views are popular targets for SEO results but focusing on lowering bounce rates and increasing customer session time is just as important.

Through changes in the last 30 days, I have lowered my bounce rate from 94% to 80%. I have made some changes to my site to pick up chicks at the end of posts and hope to get a 50% or lower bounce rate.

Where are your visitors coming from?

Google Analytics will reveal where visitors come from. Twitter, Facebook, etc. Knowing where your readers are located allows you to target more users.

Time of Day

Knowing the time of day (from Google Analytics) will show you when it is best to post and promote.

Visitor Frequency

Google Analytics will tell you if your visitors are one-time visitors or regular visitors.

Conclusion

Ensure your sitemaps is up to date and keep adding new content.

Donate and make this blog better

Ask a question or recommend an article

V1.7 added info 1 month later

Development Blog

Coding for fun since 1996, Learn by doing and sharing.

Website

Setup Ruby, Rails, Gem and a command line twitter tool to query Twitter on Ubuntu 16.04 via a Twitter App

How to use Sublime Text editor locally to edit code files on a remote server via SSH

Installing and managing WordPress with WP-CLI from the command line on Ubuntu

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads

Moving a CPanel domain with email to a self managed VPS and Gmail

How to boost your site’s SEO

Popular

Security

Code

Tech

Wordpress

General

Main navigation

Website

Footer

Popular

Security

Code

Tech

Wordpress

General