• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

Analytics

HomePi – Raspberry PI powered touch screen showing information from house-wide sensors

March 14, 2022 by Simon

This post is a work in progress (14/3/2022, v0.9.63 – PCB’s v0.2 Designed and Ordered

Summary

After watching this video from Jeff Geerling (demonstrating how to build a Air Quality Sensor) I have decided to make 2. but why not build something bigger?

I want to make a RaspBerry Pi server with a touch screen to receive data from a dozen other WeMos Sensors that I will build.

The Plan

Below is a rough plan of what I am building

In a nutshell, it will be 20x WeMos Sensors recording

Picture of20x WeMoss Sensors, weather station and co2 sensors talking to an api that saves to MySQL then mysql being ready buy a webpage and touch screen panel

I ordered all the parts from Amazon, BangGood, AliExpress, eBay, Core Electronics and Kogan.

Fresh Bullseye Install (Buster upgrade failed)

On 21/11/2021 I tried to manually update Buster to Bullseye (without taking a backup first (bad idea)). I followed this guide to reinstall Rasbian from scratch (this with Bullseye)

Storage Type

Before I begin I need to decide on what storage media to use on the Raspberry Pi. I hate how unreliable and slow MicroSD cards. I tried using an old 128GB SATA SSD, a 1TB Magnetic Hard Drive, a SATA M.2 SSD and NVME M.2 in a USB caddy.

I decided to use a spare 250GB SATA based M.2 Solid State from my son’s PC in Geekworm X862 SATA M.21 Expansion board.

With this board I can bolt the M.2 Solid State Drive into a expansion board under the pi and Power it from the RaspBerry Pi USB Port.

Nice and tidy

I zip-tied a fan to the side of the boards to add a little extra airflow over the solid state drive

32Bit, 64Bit, Linux or Windows

Before I begin I set up Raspbian on an empty Micro SD card (just to boot it up and flash the firmware to the latest version). This is very easy and documented elsewhere. I needed the latest firmware to ensure boort from USB Drive (not Micro SD card was working).

I ran rpi-update and flashed the latest firmware onto my Raspberry Pi. Good, write up here.

When my Raspberry Pi had the latest firmware I used the Raspberry Pi Imager to install the 32 Bit Raspberry Pi OS.

I do have a 8GB Raspberry Pi 4 B, 64Bit Operating Systems do exist but I stuck with 32 bit for compatibility.

Ubuntu 64bit for Raspberry Pi Links

  • Install Ubuntu on a Raspberry Pi | Ubuntu
    • Server Setup Links
      • How to install Ubuntu Server on your Raspberry Pi | Ubuntu
    • Desktop Setup Links
      • How to install Ubuntu Desktop on Raspberry Pi 4 | Ubuntu

Windows 10 for Raspberry Pi Links
https://docs.microsoft.com/en-us/windows/iot-core/tutorials/quickstarter/prototypeboards
https://docs.microsoft.com/en-us/answers/questions/492917/how-to-install-windows-10-iot-core-on-raspberry-pi.html
https://docs.microsoft.com/en-us/windows/iot/iot-enterprise/getting_started

Windows 11 for Raspberry Pi Links
https://www.youtube.com/user/leepspvideo
https://www.youtube.com/watch?v=WqFr56oohCE
https://www.worproject.ml

Setting up the Raspberry Pi Server

These are the steps I used to setup my Pi

Dedicated IP

Before I began I ran ifconfig on my Pi to obtain my Raspberry Pi’s wireless cards mac address. I logged into my Router and setup a dedicated IP (192.168.0.50), this way I can have a IP address thta remains the same.

Hostname

I set my hostname here

sudo nano /etc/hosts
sudo nano /etc/hostname

I verified my hostname with this command

hostname

I verified my IP address with this command

hostname -I

Samba Share

I setup the Samba service to allow me to copy files to and from the Pi

sudo apt-get install samba samba-common-bin
sudo apt-get update

I made a folder to share files

 mkdir ~/share

I edited the Samba config file

sudo nano /etc/samba/smb.conf

In the config file I set my workgroup settings


workgroup = Hyrule
wins support = yes

I defined a share at the bottom of the config file (and saved)

[PiShare]
comment=Raspberry Pi Share
path=/home/pi/share
browseable=Yes
writeable=Yes
only guest=no
create mask=0777
directory mask=0777
public=no

I set a smb password

sudo smbpasswd -a pi
New SMB password: ********
Retype new SMB password: ********

I tested the share froma Windows PC

And the share is accessible on the Raspberry Pi

Great, now I can share files with drag and drop (instead of via SCP)

Mono

I know how to code C# Windows Executables, I have 25 years experince. I do nt want to learn Java or Python to code a GUI application for a touch screen if possible.

I setup Mono from Home | Mono (mono-project.com) to be anbe to run Windows C# EXE’s on Rasbian

sudo apt install apt-transport-https dirmngr gnupg ca-certificates

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF

echo "deb https://download.mono-project.com/repo/debian stable-raspbianbuster main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list

sudo apt update

sudo apt install mono-devel

I copied an EXE I wrote in C# on Windows and ran it with Mono

sudo mono ~/HelloWorld.exe
Exe Test OK

This worked.

Nginx Web Server

I Installed NginX and configured it

sudo apt-get install nginx

I created a /www folder for nginx

sudo mkdir /www

I created a place-holder file in the www root

sudo nano /wwww/index.html

I set permissions to allow Nginx to access /www

sudo chown -R www-data:www-data /www

I edited the NginX config as required

sudo nano /etc/nginx/sites-enabled/default
sudo nano /etc/nginx/nginx.conf 

I tested and reloaded the nginx config


sudo nginx -t
sudo nginx -s reload
sudo systemctl start nginx

I started NginX

sudo systemctl start nginx

I tested nginx in a web browser

NodeJS/NPM

I installed NodeJS

sudo apt update
sudo apt install nodejs npm -y

I verified Node was installed

nodejs --version
> v12.22.5

PHP

I installed PHP

sudo wget -O /etc/apt/trusted.gpg.d/php.gpg https://packages.sury.org/php/apt.gpg

echo "deb https://packages.sury.org/php/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/php.list

sudo apt update

sudo apt install -y php8.0-common php8.0-cli php8.0-xml

I verified PHP with this command

php --version

> PHP 8.0.13 (cli) (built: Nov 19 2021 06:40:53) ( NTS )
Copyright (c) The PHP Group
Zend Engine v4.0.13, Copyright (c) Zend Technologies
    with Zend OPcache v8.0.13, Copyright (c), by Zend Technologies

I installed PHP-FPM

sudo apt-get install php8.0-fpm

I verified the PHP FPM sock was available before adding it to the NGINX Config

sudo ls /var/run/php*/**.sock
> /var/run/php/php8.0-fpm.sock  /var/run/php/php-fpm.sock

I reviewed PHP Settings

sudo nano /etc/php/8.0/cli/php.ini 
sudo nano /etc/php/8.0/fpm/php.ini

I created a /www/ppp.php file with this contents

<?php
  phpinfo(); // all info
  // module info, phpinfo(8) identical
  phpinfo(INFO_MODULES);
?>

PHP is working

PHP Test OK

I changed these php.ini settings (fine for local development).

max_input_vars = 1000
memory_limit = 1024M
max_file_uploads = 20M
post_max_size = 20M
display_errors = on

MySQL Database

I installed MariaDB

sudo apt install mariadb-server

I updated my pi password

passwd

I ran the Secure MariaDB Program

sudo mysql_secure_installation

After setting each setting I want to run mysql as root to test mysql

PHPMyAdmin

I installed phpmyadmin to be able to edit mysql databases via the web

I followed this guide to setup phpmyadmin via lighthttp and then via nginx

I then logged into MySQL, set user permissions, create a test database and changes settings as required.

NginX to NodeJS API Proxy

I edited my NginX config to create a NodeAPI Proxy

Test Webpage/API

Todo

I installed PM2 the NodeJS agent software

sudo npm install -g pm2 

Node apps can be started as a service from cli

pm2 start api_v1.js

PM2 status

pm2 status

You can delete node apps from PM2 (if desired)

pm2 delete api_v1.js

Sending Email from CLI

I setup send email to allow emails to be sent from the cli with these commands

 sudo apt-get install libio-socket-ssl-perl libnet-ssleay-perl sendemail  

I logged into my GSuite account and setup an alias and app password to use.

Now I can send emails from the CLI

sudo sendemail -f [email protected] -t [email protected] -u "Test Email From PiHome" -m "Test Email From PiHome" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp **************

I added this to a Bash script (“/Scripts/Up.sh”) and added an event to send an email every 6 hours

7 Inch Full View LCD IPS Touch Screen 1024*600 

I purchased a 7″ Touch screen from Banggood. I got a head up from the following Video.

I plugged in the touch USB cable to my Pi’s USB3 port. I pliugged the HDMI adapter into the screen and the pi (with the supplied mini plug).

I turned on the pi and it work’s and looks amazing.

This is displaying a demo C# app I wrote. It’s running via mono.

I did have to add the following to config.txt to bet native resolution. The manual on the supplied CD was helpful (but I did not check it at first).

max_usb_current=1
hdmi_force_hotplug=1
config_hdmi_boost=7
hdmi_group=2
hdmi_mode=1
hdmi_mode=87 
hdmi_drive=1
display_rotate=0
hdmi_cvt 1024 600 60 6 0 0 0
framebuffer_width=1024
framebuffer_height=600

PiJuice UPS HAT

I purchased an external LiPi UPS to keep the raspberry pi fed with power (even when the power goes out)

The stock battery was not charged and was quite weak when I first installed it. Do fully charge the battery before testing.

PiJuice

Stock Battery = 3.7V @ 1820mAh

Stock Battery = 3.7V @ 1820mAh

Below are screenshots so the PIJuice Setup.

PiJuice HAT Settings

PiJuice General Settings

General Settings

There is an option to set events for every button

Extensive screen to set button events

LED Status color and function

Set LED status and color

IO for the PiJuice Input. I will sort this out later.

PiJuice IO settings

A new firmware was available. I had v1.4

Update firmware screen

I updated the firmware

Firmware update worked

Firmware flash success

Battery settings

Battery Settings

PiJuice Button Config

Button config

Wake Up Alarm time and RTC

Clock Settings

System Settings

System Settings

System Events

system settings page

User Scripts

Define user scripts

I ordered a bigger battery as my Screen, M.2, Fan and UPS consume near the maximum of the stock battery.

10,000mAh battery

After talking with the seller of the battery they advised I setup the 10,000mAh battery using the 1,000mAh battery setup in PiJuice but change the Capacity and Charge Current

  • Capacity = 10000C
  •  cutoff voltage

And for battery longevity set the 

  • Cutoff voltage: 3,250mv

Final Battery Setup

Battery settings based off 1000mAh battery profile , Capacity 10,000 mAh, Charge current 850 and Cutoff 3250mV

WeMos Setup

I orderd 20x Wemos Mini D1 Pro (16Mbit) clones to use to run the sensors. I soldered the legs on in batches of 8

WeMos installed on breadboards ready to solder pins

Soldering was not perfect but worked

20x soldered wemos

Soldering is not perfect but each joint was triple tested.

Close up of soldered joints

I only had one dead WeMos.

I will set up the final units on ProtoBoards.

Protoboard

20x Wemos ready for service and the external aerial is glued down. The hot glue was a bad idea, I had to rotate a resistor under the hot glue.

20x wemos ready.

Revision

I ended up reordering the WeMos Mini’s and soldering on Female headers so I can add OLED screens

air mon enclosure

I added female headers to allow an OLED screen

new wemos

I purchased a microscope tpo be able to see better.

microscope

Each sensor will have a mini OLED screen.

mini oled screen

0.66″ OLED Screens

oled screen

I designed a PCB in Photoshop and had it turned into a PCB via https://www.fiverr.com/syedzamin12. I ordered 30x bloards from https://jlcpcb.com/

Custom PCB

The PCB’s fit inside the new enclosure perfectly

I am waiting for smaller screws to arrive.

PCB v0.2

I decided to design a board with 2 switches (and a light sensor to turn the screen off at night)

Breadboard Prototype

Prototype

I spoke to https://www.fiverr.com/syedzamin12 and withing 24 hours a PCB was designed

I Layers

This time I will get a purple PCB from JLCPCB and add a dinosaur for my son

Top PCB View

TOP PCB View

Back PCB View

Back PCB View

3D PC View

3D PCB view

JLCPCB made the board in 3 days

3 days

Now I need to wait a few weeks for the new PCB to arrive

Also, I finsihed the firmware for v0.2 PCB

I ordered some switches

I also ordered some reset buttons

I might add a larger 0.96″ OLED screen

Wifi and Static IP Test

I uploaded a skepch to each WeMos and tested the Wifi and Static IP thta was given.

Sketch

#include <ESP8266WiFi.h>
#include <ESP8266HTTPClient.h>


#define SERVER_IP "192.168.0.50"

#ifndef STASSID
#define STASSID "wifi_ssid_name"
#define STAPSK  "************"
#endif

void setup() {

  Serial.begin(115200);

  Serial.println();
  Serial.println();
  Serial.println();

  WiFi.begin(STASSID, STAPSK);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected! IP address: ");
  Serial.println(WiFi.localIP());

}

void loop() {
  // wait for WiFi connection
  if ((WiFi.status() == WL_CONNECTED)) {

    WiFiClient client;
    HTTPClient http;

    Serial.print("[HTTP] begin...\n");
    // configure traged server and url
    http.begin(client, "http://" SERVER_IP "/api/v1/test"); //HTTP
    http.addHeader("Content-Type", "application/json");

    Serial.print("[HTTP] POST...\n");
    // start connection and send HTTP header and body
    int httpCode = http.POST("{\"hello\":\"world\"}");

    // httpCode will be negative on error
    if (httpCode > 0) {
      // HTTP header has been send and Server response header has been handled
      Serial.printf("[HTTP] POST... code: %d\n", httpCode);

      // file found at server
      if (httpCode == HTTP_CODE_OK) {
        const String& payload = http.getString();
        Serial.println("received payload:\n<<");
        Serial.println(payload);
        Serial.println(">>");
      }
    } else {
      Serial.printf("[HTTP] POST... failed, error: %s\n", http.errorToString(httpCode).c_str());
    }

    http.end();
  }

  delay(1000);
}

The Wemos booted, connected to WiFi, set and IP, and tried to post a request to a URL.

........................................................
Connected! IP address: 192.168.0.51
[HTTP] begin...
[HTTP] POST...
[HTTP] POST... failed, error: connection failed

The POST failed because my PI API Server was off.

Touch Screen Enclosure

I constructed a basic enclosure and screwed the touch screen to it. I need to find  aflexible black scrip to put around the screen and cover up the gaps.

Wooden box with the screen in it

The touch screen has been screwed in.

Screen screwed in

Over the Air Updating

I followed this guide and having the WeMos updatable over WiFi.

Basically, I installed the libraries “AsyncHTTPSRequest_Generic”, “AsyncElegantOTA”, “AsyncHTTPRequest_Generic”, “ESPAsyncTCP” and “ESPAsyncWebServer”.

Manage Libraries

A few libraries would not download so I manually downloaded the code from the GitHub repository from Confirm your account recovery settings (github.com) and then extracted them to my Documents\Arduino\libraries folder.

I then opened the exampel project “AsyncElegantOTA\ESP8266_Async_Demo”

I reviewed the code

#include <ESP8266WiFi.h>
#include <ESPAsyncTCP.h>
#include <ESPAsyncWebServer.h>
#include <AsyncElegantOTA.h>

const char* ssid = "........";
const char* password = "........";

AsyncWebServer server(80);


void setup(void) {
  Serial.begin(115200);
  WiFi.mode(WIFI_STA);
  WiFi.begin(ssid, password);
  Serial.println("");

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request) {
    request->send(200, "text/plain", "Hi! I am ESP8266.");
  });

  AsyncElegantOTA.begin(&server);    // Start ElegantOTA
  server.begin();
  Serial.println("HTTP server started");
}

void loop(void) {
  AsyncElegantOTA.loop();
}
I added my Wifi SSID and password, saved the project and compiled a the code and wrote it to my WeMos Mini D1

I added LED Blink Code

void setup(void) {
  ...
  pinMode(LED_BUILTIN, OUTPUT);     // Initialize the LED_BUILTIN pin as an output
  ...
}
void loop(void) {
 ...
  delay(1000);                      // Wait for a second
  digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
  delay(1000);                      // Wait for two seconds (to demonstrate the active low LED)
 ...
}

I compiled and tested the code

Now to get new code changes to the WeMos Mini via a binary, I edited the code (chnaged the LED blink speed) and clicked “Export Compiled Binary”

Compole Binary

When the binary compiled I opened the Sketch Folder

Show Sketch folder

I could see a bin file.

Bin File

I loaded the http://192.168.0.51/update and selected the bin file.

The new firmwaere applied.

Flashing

I navighated back to http://192.168.0.51

TIP: Ensure you add the starter sketch that has your wifi details in there.

Password Protection

I changed the code to add a basic passeord on access ad on OTA update

#include <ESP8266WiFi.h>
#include <ESPAsyncTCP.h>
#include <ESPAsyncWebServer.h>
#include <AsyncElegantOTA.h>


//Saved Wifi Credentials (Research Encruption Later or store in FRAM Module?
const char* ssid = "your-wifi-ssid";
const char* password = "********";

//Credentials for the regular user to access "http://{ip}:{port}/"
const char* http_username = "user";
const char* http_password = "********";

//Credentials for the admin user to access "http://{ip}:{port}/update/"
const char* http_username_admin = "admin";
const char* http_password_admin = "********";

//Define the Web Server Object
AsyncWebServer server(80);

void setup(void) {
  Serial.begin(115200);       //Serial Mode (Debug)
    
  WiFi.mode(WIFI_STA);        //Client Mode
  WiFi.begin(ssid, password); //Connect to Wifi
 
  Serial.println("");

  pinMode(LED_BUILTIN, OUTPUT);     // Initialize the LED_BUILTIN pin as an output

  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("");
  Serial.print("Connected to ");
  Serial.println(ssid);
  
  Serial.print("IP address: ");
  Serial.println(WiFi.localIP());

  // HTTP basic authentication on the root webpage
  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
    if(!request->authenticate(http_username, http_password))
        return request->requestAuthentication();
    request->send(200, "text/plain", "Login Success! ESP8266 #001.");
  });

  //This is the OTA Login
  AsyncElegantOTA.begin(&server, http_username_admin, http_password_admin);

  
  server.begin();
  Serial.println("HTTP server started");
}

void loop(void) {
  AsyncElegantOTA.loop();

  digitalWrite(LED_BUILTIN, LOW);
  delay(8000);                      // Wait for a second
  digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
  delay(8000);                      // Wait for two seconds (to demonstrate the active low LED)

}

Password prompt for users accessing the device.

Login scree

Password prompt for admin users accessing the device.

admin password protect

Later I will research encrypting the password and storing it on SPIFFS partition or a FRAM memory module.

Adding the DHT22 Sensors

I received my paxckl of DHT22 Sensors (AMT2302).

Specifications

  • Operating Voltage: 3.5V to 5.5V
  • Operating current: 0.3mA (measuring) 60uA (standby)
  • Output: Serial data
  • Temperature Range: 0°C to 50°C
  • Humidity Range: 20% to 90%
  • Resolution: Temperature and Humidity both are 16-bit
  • Accuracy: ±1°C and ±1%

I wired it up based on this Adafruit post.

DHT22 Wired Up on a breadboard.

DHT22 and Basic API Working

I will not bore you with hours or coding and debugging so here is my code thta

  • Allows the WeMos D1 Mini Prpo (ESP8266) to connect to WiFi
  • Web Server (with stats)
  • Admin page for OTA updates
  • Password Prpotects the main web folder and OTA admin page
  • Reading DHT Sensor values
  • Debug to serial Toggle
  • LED activity Toggle
  • Json Serialization
  • POST DHT22 data to an API on the Raspberry PI
  • Placeholder for API return values
  • Automatically posts data to the API ever 10 seconds
  • etc

Here is the work in progress ESP8288 Code

#include <ESP8266WiFi.h>        // https://github.com/esp8266/Arduino/blob/master/libraries/ESP8266WiFi/src/ESP8266WiFi.h
#include <ESPAsyncTCP.h>        // https://github.com/me-no-dev/ESPAsyncTCP
#include <ESPAsyncWebServer.h>  // https://github.com/me-no-dev/ESPAsyncWebServer
#include <AsyncElegantOTA.h>    // https://github.com/ayushsharma82/AsyncElegantOTA
#include <ArduinoJson.h>        // https://github.com/bblanchon/ArduinoJson
#include "DHT.h"                // https://github.com/adafruit/DHT-sensor-library
                                // Written by ladyada, public domain

//Todo: Add Authentication
//Fyi: https://api.gov.au/standards/national_api_standards/index.html

#include <ESP8266HTTPClient.h>  //POST Client

//Firmware Stats
bool bDEBUG = true;        //true = debug to Serial output
                           //false = no serial output
//Port Number for the Web Server
int WEB_PORT_NUMBER = 1337; 

//Post Sensor Data Delay
int POST_DATA_DELAY = 10000; 

bool bLEDS = true;         //true = Flash LED
                           //false =   NO LED's
//Device Variables
String sDeviceName = "ESP-002";
String sFirmwareVersion = "v0.1.0";
String sFirmwareDate = "27/10/2021 23:00";

String POST_SERVER_IP = "192.168.0.50";
String POST_SERVER_PORT = "";
String POST_ENDPOINT = "/api/v1/test";

//Saved Wifi Credentials (Research Encryption later and store in FRAM Module?
const char* ssid = "your_wifi_ssid";
const char* password = "***************";

//Credentials for the regular user to access "http://{ip}:{port}/"
const char* http_username = "user";
const char* http_password = "********";

//Credentials for the admin user to access "http://{ip}:{port}/update/"
const char* http_username_admin = "admin";
const char* http_password_admin = "********";

//Define the Web Server Object
AsyncWebServer server(WEB_PORT_NUMBER);    //Feel free to chnage the port number

//DHT22 Temp Sensor
#define DHTTYPE DHT22   // DHT 22  (AM2302), AM2321
#define DHTPIN 5
DHT dht(DHTPIN, DHTTYPE);

//Common Variables
String thisBoard = ARDUINO_BOARD;
String sHumidity = "";
String sTempC = "";
String sTempF = "";
String sJSON = "{ }";

//DHT Variables
float h;
float t;
float f;
float hif;
float hic;


void setup(void) {

  //Turn On PIN
  pinMode(LED_BUILTIN, OUTPUT);     // Initialize the LED_BUILTIN pin as an output
  
  //Serial Mode (Debug)
  //Debug LED Flash
  if (bLEDS) {
    digitalWrite(LED_BUILTIN, LOW);
    delay(100);                      // Wait for a second
    digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
    delay(100);                      // Wait for two seconds (to demonstrate the active low LED)    
  }

  if (bDEBUG) Serial.begin(115200);
  if (bDEBUG) Serial.println("Serial Begin");

  //Debug LED Flash
  if (bLEDS) {
    digitalWrite(LED_BUILTIN, LOW);
    delay(100);                      // Wait for a second
    digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
    delay(100);                      // Wait for two seconds (to demonstrate the active low LED)    
  }
  if (bDEBUG) Serial.println("Wifi Setup");
  if (bDEBUG) Serial.println(" - Client Mode");
  
  WiFi.mode(WIFI_STA);        //Client Mode
  
  if (bDEBUG) Serial.print(" - Connecting to Wifi: " + String(ssid));
  WiFi.begin(ssid, password); //Connect to Wifi
 
  if (bDEBUG) Serial.println("");
  // Wait for connection
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    if (bDEBUG) Serial.print(".");
  }
  if (bDEBUG) Serial.println("");
  if (bDEBUG) Serial.print("- Connected to ");
  if (bDEBUG) Serial.println(ssid);
  
  if (bDEBUG) Serial.print("IP address: ");
  if (bDEBUG) Serial.println(WiFi.localIP());

  //Debug LED Flash
  if (bLEDS) {
    digitalWrite(LED_BUILTIN, LOW);
    delay(100);                      // Wait for a second
    digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
    delay(100);                      // Wait for two seconds (to demonstrate the active low LED)    
  }

  
  // HTTP basic authentication on the root webpage
  server.on("/", HTTP_GET, [](AsyncWebServerRequest *request){
    if(!request->authenticate(http_username, http_password))
        return request->requestAuthentication();
    
        String sendHtml = "";
        sendHtml = sendHtml + "<html>\n";
        sendHtml = sendHtml + " <head>\n";
        sendHtml = sendHtml + " <title>ESP# 002</title>\n";
        sendHtml = sendHtml + " <meta http-equiv=\"refresh\" content=\"5\";>\n";
        sendHtml = sendHtml + " </head>\n";
        sendHtml = sendHtml + " <body>\n";
        sendHtml = sendHtml + " <h1>ESP# 002</h1>\n";
        sendHtml = sendHtml + " <u2>Debug</h2>";
        sendHtml = sendHtml + " <ul>\n";
        sendHtml = sendHtml + " <li>Device Name: " + sDeviceName + " </li>\n";
        sendHtml = sendHtml + " <li>Firmware Version: " + sFirmwareVersion + " </li>\n";
        sendHtml = sendHtml + " <li>Firmware Date: " + sFirmwareDate + " </li>\n";
        sendHtml = sendHtml + " <li>Board: " + thisBoard + " </li>\n";
        sendHtml = sendHtml + " <li>Auto Refresh Root: On </li>\n";
        sendHtml = sendHtml + " <li>Web Port Number: " + String(WEB_PORT_NUMBER) +" </li>\n";
        sendHtml = sendHtml + " <li>Serial Debug: " + String(bDEBUG) +" </li>\n";
        sendHtml = sendHtml + " <li>Flash LED's Debug: " + String(bLEDS) +" </li>\n";
        sendHtml = sendHtml + " <li>SSID: " + String(ssid) +" </li>\n";
        sendHtml = sendHtml + " <li>DHT TYPE: " + String(DHTTYPE) +" </li>\n";
        sendHtml = sendHtml + " <li>DHT PIN: " + String(DHTPIN) +" </li>\n";
        sendHtml = sendHtml + " <li>POST_DATA_DELAY: " + String(POST_DATA_DELAY) +" </li>\n";

        sendHtml = sendHtml + " <li>POST_SERVER_IP: " + String(POST_SERVER_IP) +" </li>\n";
        sendHtml = sendHtml + " <li>POST_ENDPOINT: " + String(POST_ENDPOINT) +" </li>\n";
        
        sendHtml = sendHtml + " </ul>\n";
        sendHtml = sendHtml + " <u2>Sensor</h2>";
        sendHtml = sendHtml + " <ul>\n";
        sendHtml = sendHtml + " <li>Humidity: " + sHumidity + "% </li>\n";
        sendHtml = sendHtml + " <li>Temp: " + sTempC + "c, " + sTempF + "f. </li>\n";
        sendHtml = sendHtml + " <li>Heat Index: " + String(hic) + "c, " + String(hif) + "f.</li>\n";
        sendHtml = sendHtml + " </ul>\n";
        sendHtml = sendHtml + " <u2>JSON</h2>";
        
        // Allocate the JSON document Object/Memory
        // Use https://arduinojson.org/v6/assistant to compute the capacity.
        StaticJsonDocument<250> doc;
        //JSON Values     
        doc["Name"] = sDeviceName;
        doc["humidity"] = sHumidity;
        doc["tempc"] = sTempC;
        doc["tempf"] = sTempF;
        doc["heatc"] = String(hic);
        doc["heatf"] = String(hif);
        
        sJSON = "";
        serializeJson(doc, sJSON);
        
        sendHtml = sendHtml + " <ul>" + sJSON + "</ul>\n";
        
        sendHtml = sendHtml + " <u2>Seed</h2>";
        long randNumber = random(100000, 1000000);
        sendHtml = sendHtml + " <ul>\n";
        sendHtml = sendHtml + " <p>" + String(randNumber) + "</p>\n";
        sendHtml = sendHtml + " </ul>\n";
       
        sendHtml = sendHtml + " </body>\n";
        sendHtml = sendHtml + "</html>\n";
        //Send the HTML   
        request->send(200, "text/html", sendHtml);
  });

  //This is the OTA Login
  AsyncElegantOTA.begin(&server, http_username_admin, http_password_admin);
  
  server.begin();
  if (bDEBUG) Serial.println("HTTP server started");
 
  if (bDEBUG) Serial.println("Board: " + thisBoard);

  //Setup the DHT22 Object
  dht.begin();
  
}

void loop(void) {

  AsyncElegantOTA.loop();

  //Debug LED Flash
  if (bLEDS) {
    digitalWrite(LED_BUILTIN, LOW);
    delay(100);                      // Wait for a second
    digitalWrite(LED_BUILTIN, HIGH);  // Turn the LED off by making the voltage HIGH
    delay(100);                      // Wait for two seconds (to demonstrate the active low LED)    
  }


  //Display Temp and Humidity Data

  h = dht.readHumidity();
  t = dht.readTemperature();
  f = dht.readTemperature(true);

  // Check if any reads failed and exit early (to try again).
  if (isnan(h) || isnan(t) || isnan(f)) {
    if (bDEBUG) Serial.println(F("Failed to read from DHT sensor!"));
    return;
  }
  
  hif = dht.computeHeatIndex(f, h);         // Compute heat index in Fahrenheit (the default)
  hic = dht.computeHeatIndex(t, h, false);  // Compute heat index in Celsius (isFahreheit = false)

  if (bDEBUG) Serial.print(F("Humidity: "));
  if (bDEBUG) Serial.print(h);
  if (bDEBUG) Serial.print(F("%  Temperature: "));
  if (bDEBUG) Serial.print(t);
  if (bDEBUG) Serial.print(F("°C "));
  if (bDEBUG) Serial.print(f);
  if (bDEBUG) Serial.print(F("°F  Heat index: "));
  if (bDEBUG) Serial.print(hic);
  if (bDEBUG) Serial.print(F("°C "));
  if (bDEBUG) Serial.print(hif);
  if (bDEBUG) Serial.println(F("°F"));

  //Save for Page Load
  sHumidity = String(h,2);
  sTempC = String(t,2);
  sTempF = String(f,2);

  //Post to Pi API
    // Allocate the JSON document Object/Memory
    // Use https://arduinojson.org/v6/assistant to compute the capacity.
    StaticJsonDocument<250> doc;
    //JSON Values     
    doc["Name"] = sDeviceName;
    doc["humidity"] = sHumidity;
    doc["tempc"] = sTempC;
    doc["tempf"] = sTempF;
    doc["heatc"] = String(hic);
    doc["heatf"] = String(hif);
    
    sJSON = "";
    serializeJson(doc, sJSON);

    //Post to API
    if (bDEBUG) Serial.println(" -> POST TO API: " + sJSON);

   //Test POST
  
    if ((WiFi.status() == WL_CONNECTED)) {
  
      WiFiClient client;
      HTTPClient http;
  
    
      if (bDEBUG) Serial.println(" -> API Endpoint: http://" + POST_SERVER_IP + POST_SERVER_PORT + POST_ENDPOINT);
      http.begin(client, "http://" + POST_SERVER_IP + POST_SERVER_PORT + POST_ENDPOINT); //HTTP


      if (bDEBUG) Serial.println(" -> addHeader: \"Content-Type\", \"application/json\"");
      http.addHeader("Content-Type", "application/json");
  
      // start connection and send HTTP header and body
      int httpCode = http.POST(sJSON);
      if (bDEBUG) Serial.print("  -> Posted JSON: " + sJSON);
  
      // httpCode will be negative on error
      if (httpCode > 0) {
        // HTTP header has been send and Server response header has been handled

  
        //See https://api.gov.au/standards/national_api_standards/api-response.html 
        // Response from Server
        if (bDEBUG) Serial.println("  <- Return Code: " + httpCode);
                
        //Get the Payload
        const String& payload = http.getString();
          if (bDEBUG) Serial.println("   <- Received Payload:");
          if (bDEBUG) Serial.println(payload);
          if (bDEBUG) Serial.println("   <- Payload (httpcode: 201):");
          

         //Hnadle the HTTP Code
        if (httpCode == 200) {
          if (bDEBUG) Serial.println("  <- 200: Invalid API Call/Response Code");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 201) {
          if (bDEBUG) Serial.println("  <- 201: The resource was created. The Response Location HTTP header SHOULD be returned to indicate where the newly created resource is accessible.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 202) {
          if (bDEBUG) Serial.println("  <- 202: Is used for asynchronous processing to indicate that the server has accepted the request but the result is not available yet. The Response Location HTTP header may be returned to indicate where the created resource will be accessible.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 400) {
          if (bDEBUG) Serial.println("  <- 400: The server cannot process the request (such as malformed request syntax, size too large, invalid request message framing, or deceptive request routing, invalid values in the request) For example, the API requires a numerical identifier and the client sent a text value instead, the server will return this status code.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 401) {
          if (bDEBUG) Serial.println("  <- 401: The request could not be authenticated.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 403) {
          if (bDEBUG) Serial.println("  <- 403: The request was authenticated but is not authorised to access the resource.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 404) {
          if (bDEBUG) Serial.println("  <- 404: The resource was not found.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 415) {
          if (bDEBUG) Serial.println("  <- 415: This status code indicates that the server refuses to accept the request because the content type specified in the request is not supported by the server");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 422) {
          if (bDEBUG) Serial.println("  <- 422: This status code indicates that the server received the request but it did not fulfil the requirements of the back end. An example is a mandatory field was not provided in the payload.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }
        if (httpCode == 500) {
          if (bDEBUG) Serial.println("  <- 500: An internal server error. The response body may contain error messages.");
          if (bDEBUG) Serial.println("  <- " + payload);
        }

        
      } else {
        if (bDEBUG) Serial.println("   <- Unknown Return Code (ERROR): " + httpCode);
        //if (bDEBUG) Serial.printf("    " + http.errorToString(httpCode).c_str());
        
      }

    }

    if (bDEBUG) Serial.print("\n\n");

    delay(POST_DATA_DELAY);
  }

Here is a screenshot of the Arduino IDE Serial Monitor debugging the code

Serial Monitor

Here is a screenshot of the NodeJS API on the raspberry Pi accepting the POSTed data from the ESP8266

API receiving data

Here is a sneak peek of the code accpeing the Posted Data

API COde

The final code will be open sourced.

API with 2x sensors (18x more soon)

I built 2 sensors (on Breadboards) to start hitting the API

2 sensors on a breadboard

18 more sensors are ready for action (after I get tempporary USB power sorted)

18x Sensors

PiJuice and Battery save the Day

I accidentally used my Pi for a few hours (to develop the API) and I realised the power to the PiJuice was not connected.

The PiJuice worked a treat and supplied the Pi from battery

Battery power was disconnected

I plugged in the battery after 25% was drained.

Power Restored/

Research and Setup TRIM/Defrag on the M.2 SSD

Todo: Research

Add a Buzzer to the RaspBerry Pi and Connect to Pi Juice No Power Event

Todo

Wire Up a Speaker to the PiJuice

Todo: Figure out cusrom scripts and add a Piezo Speaker to the PiJuice to alert me of issues in future.

Add buttons to the enclosure

Todo

Add email alerts from the system

I logged into Google G-Suite (my domain’s email provider) and set up an email alias for my domain “[email protected]”, I added this alias to GMail (logged in with my GSuite account.

I created an app-specific password at G-Suite to allow my poi to use a dedicated password to access my email.

I installed these packages on the Raspberry Pi

sudo apt-get install libio-socket-ssl-perl libnet-ssleay-perl sendemail    

I can run this command to send an email to my primary email

sudo sendemail -f [email protected] -t [email protected]_domain.com -u "Test Email From PiHome" -m "Test Email From PiHome" -s smtp.gmail.com:587 -o tls=yes -xu [email protected]_domain.com -xp ********************

The email arrives from the Raspberry Pi

Test Email Screenshot

PiJuice Alerts (email)

In created some python scripts and configured PiJuice to Email Me

user scripts

I assigned the scripts to Events

Added functions

Python Script (CronJob) to email the batteruy level every 6 hours

Todo

Building the Co2/PM2.5 Sensors

Todo: (Waiting for parts)

The AirGradient PCB’s have arrived

Air Gradient PCB's

NodeJS API writing to MySQL/Influx etc

Todo: Save Data to a Database

Setup 20x WeMos External Antennae’s (DONE, I ordered new factory rotated resistors)

I assumed the external antennae’s on the WeMos D1 Mini Pro’s were using the external antennae. Wrong.

I have to move 20x resistors (1 per WeMos) to switch over the the external antennae.

This will be fun as I added hot glue over the area where the resistior is to hold down the antennae.

Reading configuration files via SPIFFS

Todo

Power over Ethernet (PoE) (SKIP, WIll use plain old USB wall plugs)

Todo: Passive por PoE

Building a C# GUI for the Touch Panel

Todo (Started coding this)

Todo (Passive POE, 5v, 3.3v)?

Building the enclosures for the sensorsDesigned and ordered the PCB, FIrmware next.

Custom PCB?

Yes, See above

Backing up the Raspberry Pi M.2 Drive

This is quite easy as the M.2 Drive is connected to a USB Pliug. I shutdown the Pi and pugged int he M.2 board to my PC

I then Backed up the entire disk to my PC with Acronis Software (review here)

I now have a complete backup of my Pi on a remote network share (and my primary pc).

Version History

v0.9.63 – PCB v0.2 Designed and orderd.

v0.9.62 – 3/2/2022 Update

v0.9.61 – New Nginx, PHP, MySQL etc

v0.9.60 – Fresh Bullseye install (Buster upgrade failed)

v0.951 Email Code in PiJUice

v0.95 Added Email Code

v0.94 Added Todo Areas.

v0.93 2x Sensors hitting the API, 18x sensors ready, Air Gradient

v0.92 DHT22 and Basic API

v0.91 Password Protection

v0.9 Final Battery Setup

v0.8 OTA Updates

v0.7 Screen Enclosure

v0.6 Added Wifi Test Info

v0.5 Initial Post

Filed Under: Analytics, API, Arduino, Cloud, Code, GUI, IoT, Linux, MySQL, NGINX, NodeJS, OS Tagged With: api, ESP8266, MySQL, nginx, raspberry pi, WeMos

Setting web push notifications in WordPress with OneSignal

March 26, 2019 by Simon

Note: I will update this guide to include full WordPress push integration soon (this method works with any HTML website)

When visiting a friends website ( https://markontech.com/ ) I noticed he had push notifications setup.

Screenshot of push notification ability at https://markontech.com/

I was able to click a bell notification allow push notifications to be sent when he posts new content. Nice

When I clicked subscribe Google Chrome prompted me to allow or block notifications.

Screenshot showing Chrome asking for permissions to receive push notifications

A floating icon allows me to unsubscribe at anytime and manage subscription details.

Screehshot showing an unsubscribe icon for removing push notifications

Mark from https://markontech.com/ said he was using a WordPress plugin from OneSignal.

I checked out One Signal from a coding point of view in 2015 (read here) and apparently they now have a WordPress Plugin.

Who knew meme (girl with amazed look and hands up)

Price?

The One Signal blog post says is free under 30,000 subscribers then it jumps to $99/m for above 30,000 subscribers

Pricing table screenshot from https://onesignal.com/pricing

You will need to pay if you want full GDPR compliance (read more here).

Installing the OneSignal Plugin

I visited the OneSignal Plugin page at https://wordpress.org/plugins/onesignal-free-web-push-notifications/

Screenshot of https://onesignal.com Push plugin page at WordPress

I noted the latest plugin URL path

https://downloads.wordpress.org/plugin/onesignal-free-web-push-notifications.1.16.16.zip

I downloaded the zip file (from the command line)

wget https://downloads.wordpress.org/plugin/onesignal-free-web-push-notifications.1.16.16.zip
--2019-03-19 19:24:33--  https://downloads.wordpress.org/plugin/onesignal-free-web-push-notifications.1.16.16.zip
Resolving downloads.wordpress.org (downloads.wordpress.org)... 198.143.164.250
Connecting to downloads.wordpress.org (downloads.wordpress.org)|198.143.164.250|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 6764683 (6.5M) [application/octet-stream]
Saving to: 'onesignal-free-web-push-notifications.1.16.16.zip'

onesignal-free-web-push-notifications.1.1 100%[=====================================================================================>]   6.45M  --.-KB/s    in 0.1s

2019-03-19 19:24:33 (54.2 MB/s) - 'onesignal-free-web-push-notifications.1.16.16.zip' saved [6764683/6764683]

I unzipped the plugin

unzip onesignal-free-web-push-notifications.1.16.16.zip

I could now activate the One Signal Plugin in WordPress

Scrreenshot of the non activated WordPress plugin

A One Signal menu appeared in my WordPress Dashboard. Nice

Screenshot of the OneSignal menu in WordPress

I was prompted to add the API key etc to the plugins configuration.

Screesnhot of OneSignal asking for setup to continue.

I logged into OneSignal and created an app.

Screenshot of https://onesignal.com to create new app screen

I selected Web Push

Screenshot of https://onesignal.com choosing a push notification template (web push)

Major Issue

Long story, I at first selected WordPress Plugin or Website builder and finished the setup but my CDN was picking up the java script files and moving it to my CDN automatically and the One Push code was invoking a Web Service error. Upon investigation web services need to be run from the same origin.

I need top setup the One Signal separate to WordPress.

Later I will see if I can exclude my CDN from caching the JavaScript but for now I will delete the WordPress plugin I just downloaded (above) and instead setup a Typical Site (with HTML). My CDN (read here) is very fast so I will keep it.

I following on from the wizard above (after selecting Web Push) I selected Typical site for the OneSignal app. Having a typical site setup will force me to create a push notification manually after each post but that’s ok (gives me time to proof read).

Screenshot of a typical site template chosen

During the app creation process at One Signal I set my site name and URL.

Screenshot of me choosing a site name and description at https://onesignal.com

Now I need to create a prompt for the app.

Screenmshot of me setting up a prompt at https://onesignal.com

I set the prompt style (floating button)

Screenshot of me choosing the prompt style (floating button)

Permission prompt created.

Permission prompt created and saved

Now I copied the generated HTML code that One Signal created and added it to the <head> section of my WordPress theme.

Screnshot of html code to make the push notification work.

I opened the WordPress Dashboard and opened my theme “Header and Footer scripts” section and pasted the code from OneSignal.

Screenshot of me pasting code from OneSignal into WordPress head area.

I published the changes and loaded my site.

The button was not working and when I opened the Google Chrome Developer console I noticed I needed to upload some java script files to my site.

Screenshot of errors complaining of missing recources.

Oops: I should have read all the documentation (here)

I followed the prompts and downloaded the One Signal SDK files (from Github) to place on my site from https://github.com/OneSignal/OneSignal-Website-SDK/releases/download/https-integration-files/OneSignal-Web-SDK-HTTPS-Integration-Files.zip

Screenshot of OneSignal prompting me to download https://github.com/OneSignal/OneSignal-Website-SDK/releases/download/https-integration-files/OneSignal-Web-SDK-HTTPS-Integration-Files.zip

I downloaded the zip file on my server (over SSH command Line), I extracted the zip file contents into the root of my site (don’t forget to chown the files if the downloaded was from another account (I forgot and my nginx web server would not deliver the files)).

I reloaded my site and I now have Push notification subscription features for my readers.

My browser asked to to allow the notifications 🙂

Clicking the bell again allows you to unsubscribe.

Testing Push Notifications

Ok, lets send a notification for the last post I blogged about.

I logged into onesignal.com and clicked Messages. I clicked New Message and specified a Message title and body.

Screenshot of https://app.onesignal.com/apps

I set an icon, image, URL (when clicked) and priority. The icon needed to be 192×192 pixels.

Screen shot showing the ability to set an icon, image and priority.

I can sent instantly or at a scheduled time

Send Instantly or at a scheduled time and submit button.

A confirmation box appears before sending.

A confirmation box appears before sending.

A notification took a few seconds to appear on my Windows 10 machine.

Windows 10 push notification received

The notification also appeared on my android

Statistics of real time read receipts started coming in. I don’t think this is accurate as the android read was not shown

Statistics showing reads and not read

Nice

Adding an Auto Prompt

To enable auto prompt (to send notifications) on visitors page load I added a second prompt alongside the bell.

I re edited the app in One Signal and added an auto prompt (prompt).

One Signal can also ask the user  if they want to receive prompts automatically.

Done

One Signal prompts listing a) bell icon and b) auto push

This is what happens on a new Browser (Firefox), the prompt was not triggered on Google Chrome because I was already subscribed and unsubscribing would not trigger it.

Auto push notification prompt screenshot  in Firefox.

Mobile App

I did check for an Android App for One Signal but only poor third party apps were found.

Troubleshooting

  • OneSignal GitHub Repository
  • OneSignal GitHub Issues (don’t forget to watch and star the repository at GitHub to receive notifications)
  • OneSignal Help
  • OneSignal Support Chat

Update History

v1.1 Added “I will update this guide to include full WordPress push integration soon”

v1.2 added auto prompt information v1.0 Initial version

Filed Under: Analytics, One Signal, Push Tagged With: one signal, Web Push

Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution

November 18, 2018 by Simon

This is how I replacing Google Analytics with Piwik/Matomo for a locally hosted privacy-focused open source analytics solution

Aside

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. PHP is my programming language of choice.

Now on with the post

Google Analytics

I will fully admit Google Analytics is good. I posted this a while ago on how you can set up Google Analytics on your site.

Google Analytics has some great charts and graphs. Simple to set up and easy to use.

Analitics Home

My site traffic is growing and I would prefer to hold my own analytics on user data. Matomo is an analytics solution that stays on my server and not in the hands of Google.

Blog Growth

Google Analytics can be Slow

Sometimes the Google Analytics server is slow (affecting the speed of my server). I blogged recently about speeding up a WordPress site here and Google Servers were not adding expiry headers on assets.

I did log a ticket with Google to fix this and the experience was terrible.

Support for Google Analytics is terrible

Gogole Analytics support of terrible

GT Metrix scores show poor delivery of tracking assets.

Google Slow Assets

Privacy

After the Cambridge Analytica fiasco (that made me decide to delete facebook) sending analytics to Google is not a good idea.

  • Google Removes ‘Don’t Be Evil’ Clause From Its Code Of Conduct
  • FUTURE SOCIETY Three Signs Google Is Turning to the Dark Side
  • Top 10 Ways Google Does Evil

I am not saying Google is evil but I want my site’s visitors tracking data to remain local.

Website Speed Benchmark before installing Matomo

I can load my site in 1.3 seconds at best, 1.5 seconds on average and 2.0 seconds at worst. My site is loading 11 assets.

GTmetrix 1.3 second page load time

Page Speed Scores

GTMerix page speed load times

Y Slow Scores, Gogol Assets are reporting no expiry headers (slowing down scores)

GTMetrix yslow load times

Google Analytics tracking assets are slow.

Gmetrix waterfall list

Optimizations to be made

Browser caching is not possible with Google Analytics.

Gogole lacking browser caching

Missing Expiry Headers (I can see a Google Tag Manager server is slowing down my servers benchmark score)

Google lacking Expiry Headers

Why Mamoto (instead of Google Analytics)

I came across

Someone pointed out that @haveibeenpwned got a bunch of traction on Reddit today. With pretty much everything now either cached by @Cloudflare or served by @AzureFunctions, the first I know of a 28x traffic increase is no longer when something scales it’s when someone tells me 😎 pic.twitter.com/ifj7nQg3n4

— Troy Hunt (@troyhunt) November 5, 2018

Mamoto was mentioned

It’s an Open Source, self hostable, privacy friendly alternative to Google Analytics:https://t.co/NiK7A7uQAE

— Lukas Winkler (@lw1_at) November 5, 2018

I visited https://matomo.org/

Mamoto webpage

Snip

> Take care of running Matomo yourself by installing it on your own server. There is no cost for Matomo itself but you need a server and update Matomo & your server regularly to keep it fast and secure. Need help? The Matomo team provides free help resources and paid support.

Mamoto Setup Instruction Guide

Source Code

Source code is available.

> Matomo is the leading open alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites, apps & the IoT and visualise this data and extract insights. Privacy is built-in. We love Pull Requests! https://matomo.org/

https://github.com/matomo-org/matomo

Installation Guide

I read the installation guide here https://matomo.org/docs/installation/

You can view the changelog here https://matomo.org/changelog/

Downloading Mamoto

I logged into my server via SSH and downloaded the 18MB download to the desired folder

cd /www-root/matomo-folder/
wget https://builds.matomo.org/matomo.zip

I unzipped the zip file

unzip matomo.zip

I loaded the URL where Matoto was installed (e.g “https://fearby.com/folder/subfolder/matomo/”)

I received this well-crafted error.

Matomo File Permission Error

Raw Output

An error occurred
Matomo couldn't write to some directories (running as user 'www-usr').

Advertisement:





Try to Execute the following commands on your server, to allow Write access on these directories:

chown -R www-usr:www-usr /www-root/folder/subfolder/matomo
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/assets/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/cache/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/logs/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/tcpdf/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/templates_c/

If this doesn't work, you can try to create the directories with your FTP software, and set the CHMOD to 0755 (or 0777 if 0755 is not enough). To do so with your FTP software, right click on the directories then click permissions.

After applying the modifications, you can refresh the page.

I refreshed the page after running the commands above on my site (via SSH)

Matomo Setup Step 1

A system check was performed. I installed when PHP 7.2.11 was the latest, PHP 7.2.12 or higher might be available. Follow my guide to update PHP on Ubuntu.

System Check

I had one Issue with Freetype not being installed.

Install Freetype

I solved this error by installing FreeType

sudo apt-get install freetype*

Output

Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'freetype-tools' for glob 'freetype*'
Note, selecting 'freetype2-demos' for glob 'freetype*'
The following NEW packages will be installed:
  freetype2-demos
0 upgraded, 1 newly installed, 0 to remove and 66 not upgraded.
Need to get 123 kB of archives.
After this operation, 728 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 freetype2-demos amd64 2.8.1-2ubuntu2 [123 kB]
Fetched 123 kB in 0s (965 kB/s)
Selecting previously unselected package freetype2-demos.
(Reading database ... 122574 files and directories currently installed.)
Preparing to unpack .../freetype2-demos_2.8.1-2ubuntu2_amd64.deb ...
Unpacking freetype2-demos (2.8.1-2ubuntu2) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up freetype2-demos (2.8.1-2ubuntu2) ...

Then I installed “php-gd”

sudo apt-get install php-gd

Output:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libapache2-mod-php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-dev php7.2-fpm php7.2-gd php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml php7.2-zip
Recommended packages:
apache2
The following NEW packages will be installed:
php-gd php7.2-gd
The following packages will be upgraded:
libapache2-mod-php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-dev php7.2-fpm php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml php7.2-zip
13 upgraded, 2 newly installed, 0 to remove and 53 not upgraded.
Need to get 33.2 kB/6621 kB of archives.
After this operation, 150 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 php7.2-gd amd64 7.2.11-4+ubuntu18.04.1+deb.sury.org+1 [27.1 kB]
Get:2 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 php-gd all 2:7.2+68+ubuntu18.04.1+deb.sury.org+1 [6036 B]
Fetched 33.2 kB in 0s (75.9 kB/s)
Reading changelogs... Done
(Reading database ... 122597 files and directories currently installed.)
Preparing to unpack .../00-php7.2-mysql_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-mysql (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../01-php7.2-opcache_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-opcache (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../02-php7.2-json_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-json (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../03-php7.2-readline_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-readline (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../04-php7.2-mbstring_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-mbstring (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../05-php7.2-curl_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-curl (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../06-php7.2-zip_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-zip (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../07-php7.2-fpm_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-fpm (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../08-php7.2-xml_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-xml (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../09-php7.2-dev_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-dev (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../10-libapache2-mod-php7.2_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking libapache2-mod-php7.2 (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../11-php7.2-cli_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-cli (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../12-php7.2-common_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-common (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Selecting previously unselected package php7.2-gd.
Preparing to unpack .../13-php7.2-gd_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-gd (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Selecting previously unselected package php-gd.
Preparing to unpack .../14-php-gd_2%3a7.2+68+ubuntu18.04.1+deb.sury.org+1_all.deb ...
Unpacking php-gd (2:7.2+68+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-common (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Processing triggers for ureadahead (0.100.0-20) ...
Setting up php7.2-curl (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-mbstring (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-readline (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Processing triggers for systemd (237-3ubuntu10.4) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up php7.2-json (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-opcache (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-mysql (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-gd (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...

Creating config file /etc/php/7.2/mods-available/gd.ini with new version
Setting up php7.2-xml (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-zip (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-cli (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php-gd (2:7.2+68+ubuntu18.04.1+deb.sury.org+1) ...
Setting up libapache2-mod-php7.2 (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Warning: Could not load Apache 2.4 maintainer script helper.
Setting up php7.2-dev (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-fpm (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...

I refreshed the Matomo setup wizard page, Freetype is now installed 🙂

FreeType is installed

Database Settings

For the life of me, I could not get Matomo to talk to a database on another server so I set it up on my localhost.

I used this guide to help in mysql CLI to create the database and users.

Enter Matomo Database settings

Commands in mysql to create a database and user and assign the user to the database. If you are not comfortable with MySql CLI you can use Adminder GUI.

CREATE DATABASE tbdatabasename;
GRANT ALL PRIVILEGES ON tbdatabasename.* TO 'databaseuser'@'localhost' IDENTIFIED BY '#####################################';
GRANT SELECT ON tbdatabasename.* TO 'databaseuser'@'localhost';

I used this PHP code to test connecting to the dedicated server before using the localhost

<?php
$servername = "localhost";
$username = "databaseuser";
$password = "#################";
$dbname = "tbdatabasename";

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);

// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
} else {
        echo "Connection Success";
}

$conn->close();
?>

Database created ok

Database OK

I created a Matomo user then I grabbed the javascript tracking ID code so I could paste this into WordPress.

Matomo Tracking ID

I opened my WordPress theme settings and deleted the Google tracking tags and added the Matomo tracking code.

Delete Google Tracking Tags

I added the Matomo tracking javascript in the head section.

The dashboard is up and collecting data.

Matomo Dashboard

Some reports are missing data so I will come back later.

After 1 week I could see data

Matomo is not collecting daya

Securing Mamoto

I read this guide here to secure Matomo

Opt Out Tracking

I enabled Opt Out Tracking in the Mamoto settings and added the generated opt-out code to my front page and at the bottom or all existing articles.

I had to allow iframe tags on my site by adding this header in NGINX (previously I blocked iframes)

add_header X-Frame-Options sameorigin

Add Opt Out Tracking Code to WordPress.

Matomo Opt Out Added to WordPRess widgets

I updated my privacy page and my GDPR notification bar. Now visitors will see a opt-out of tracking on the front page and all article pages.

Opt out of tracking enabled

SMTP Settings

I added my GSuite mail server settings to enable sending of reports via email. I loaded my old guide here to get the GSuite SMTP settings.

GSuite SMTP Settings Added

I enabled force https on the Mamoto application (edited: config/config.ini.php file)

[General]
...
force_ssl = 1

Matomo Plugins (Marketplace)

I opened the System then Plugins section of Matomo to open the Marketplace

Plugins

I installed these plugins

  • Force SSL
  • HidePasswordReset
  • Google Authenticator
  • Device Pixel Ratio
  • Bandwidth
  • Js Tracker Force Async
  • Treemap Visualization
  • Security Info
  • Custom Alerts
  • IP Reports
  • Live Tab
  • etc

Updating PHP

Matomo Admin (Panel – Security/Diagnostics) section will report if your PHP gets out of date.

Matomo warning of PHP being out of date

Hardening Advice

I enabled 2fA Authorisation at logins (Google Analytics Plugin).

Matomo 2fa Login screenshot

Read my guide here on hardware 2FA YubiCo YubiKeys here.

php.ini hardening changes

Matomo also recommended some php.ini file changes.

> open_basedir – open_basedir is disabled. When this is enabled, only files that are in the given directory/directories and their subdirectories can be read by PHP scripts. You should consider turning this on. Keep in mind that other web applications not written in PHP will not be restricted by this setting.

> upload_tmp_dir – upload_tmp_dir is disabled, or is set to a common world-writable directory. This typically allows other users on this server to access temporary copies of files uploaded via your PHP scripts. You should set upload_tmp_dir to a non-world-readable directory

This may break your WordPress so enable at your own risk. I might move Mamoto to a dedicated “analytics” subdomain then enable these options.

Troubleshooting

I had to run this command when installing Device Pixel Ratio, Device Network Information, Bandwidth plugins

php /www-root/path/matomo/console core:update

Output:

    *** Update ***

    Database Upgrade Required

    Your Matomo database is out-of-date, and must be upgraded before you can continue.

    The following dimensions will be updated: log_visit.device_pixel_ratio.

    *** Note: this is a Dry Run ***

    ALTER TABLE `matomo_log_visit` ADD COLUMN `device_pixel_ratio` DECIMAL(5,2) DEFAULT NULL;

    *** End of Dry Run ***

A database upgrade is required. Execute update? (y/N) y

Starting the database upgrade process now. This may take a while, so please be patient.

    *** Update ***

    Database Upgrade Required

    Your Matomo database is out-of-date, and must be upgraded before you can continue.

    The following dimensions will be updated: log_visit.device_pixel_ratio.

    The database upgrade process may take a while, so please be patient.

  Executing ALTER TABLE `matomo_log_visit` ADD COLUMN `device_pixel_ratio` DECIMAL(5,2) DEFAULT NULL;... Done. [1 / 1]

Matomo has been successfully updated!

GTMetrix (After)

GT Metrix reports that my site is not slower (still 1.5 seconds)

GTMetrix After Pagespeed

I can see that some JavaScript is not being picked up by CDN.

GTMetrix After YSlow

Also 2 More files loading (when compared to Google Analytics)

2 More Files

Time to add the Mamoto files to my CDN.

Adding Matomo Resources to a CDN

I read this Matomo forum post.

I copied these 2 assets to my WordPress wp-content folder (my WordPress CDN ewww.io will then upload them to the CDN).

cd /www-root/wp-content/
cp /www-root/utils/matomo/piwik.js ./piwik.js
cp /www-root/utils/matomo/plugins/CoreAdminHome/javascripts/optOut.js ./optOut.js
chown www-data:www-data *.js

I have cache everything enabled in ewww.io and this will copy the javascript assets ot my CDN.  I will need to manually update these js files each time a Matomo update is installed.

I change my Matomo tracker code to include the new CDN location

<!-- Matomo -->
<script type="text/javascript">
  var _paq = _paq || [];
  /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
  (function() {
    var u="//fearby.com/utils/matomo/";
    _paq.push(['setTrackerUrl', u+'piwik.php']);
    _paq.push(['setSiteId', '1']);
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
    g.type='text/javascript'; g.async=true; g.defer=true; g.src='https://fearby-com.exactdn.com/wp-content/piwik.js'; s.parentNode.insertBefore(g,s);
  })();
</script>
<!-- End Matomo Code -->

I could not find out how to change the location of my (now CDN cached https://fearby-com.exactdn.com/wp-content/optOut.js) so I temporarily disabled the opt-out form on my front page.

todo: Find out how to change the CDN location of optOut.js and re-enabled the form.

All assets are loading from CDN.

GT Metrix shows my site loads in 1.4 seconds

Analytics Reporting

Graphs are not as pretty as Google Analytics but they are working.

Matomo is not collecting daya

Mobile Reporting

Mobile reporting is good too.

Screenshot of the Matomo Mobile app

Updating Matomo Plugins

Don’t forget to update your plugins from the Matomo dashboard.

Updating Matomo (Core)

Matomo has an official guide on how to update Matomo here.

I do not have FTP so I will perform the manual three step update.

But before I do that I will manually backup my web server and database server just in case.

I backed up my Matomo config (I SSH”ed to the server)

$ cd /www-root/matomo-root/

$ cp ./config.ini.php ./config.ini.3.x.x.php

I navigated to the folder above my Matomo folder

$ cd ..

$ cd ..

I downloaded Matomo

$ wget https://builds.matomo.org/matomo.zip

I unzipped the zip file

$ unzip -o matomo.zip

I removed the matomo.zip file

$ rm matomo.zip

I loaded the Matomo Login page again and was prompted to update the database.

Matomo Database Update Required

Matomo reported it was updated Successfully.

Matomo was updated message

Oops, and error in config error appeared when I tried to log in.

Matomo Error in config

Oh, Do I need to replace the config file with my backed up config file?

(edit: Yes Matomo say to do this, my bad)

Ten seconds later I accidentally deleted all my config files (I had zero backups), the next 2 minutes were spent shutting down my servers (web and db) and restoring them from backup. Thank goodness UpCloud are awesome hosts.

I now had to restore my servers and repeat the steps but this time restore my config file before logging back in.

I did this but had the same error

> An error occurred
> Authentication object cannot be found in the container. Maybe the Login plugin is not activated?
> You can activate the plugin by adding:
> Plugins[] = Login under the [Plugins] section in your config/config.ini.php

I checked my replaced config.ini.php and it did have

> [PluginsInstalled]
> PluginsInstalled[] = “Login”

I googled and found this page that said reset your password (this was not an option as Matomo was not loading)

I logged into mysql with my Matomo user

> mysql -u matomodbusername -p
> Enter password:
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Server version: 5.7.xxxx

> Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

> Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

> Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

> mysql> show databases;
+——————–+
| Database |
+——————–+
| information_schema |
| matomodb |
+——————–+
> 2 rows in set (0.00 sec)

The account and database seem ok.

I tried “FLUSH PRIVILEGES;” with no luck

I tried to sop mysql but it was locked

It was late so I rebooted my server (it did not come back up after a few minutes, I forced a reboot)

I still had an “Authentication object cannot be found in the container.” error when trying to login to Matomo???

I re-checked the “config.ini.php” file after reding threads at the Matomo Forums

$ sudo nano /www-root/matomo-root/config.ini.php

“Plugins[] = “Login”” was not in the “[Plugins]” area of the file???  I added it, saved the change and was able to reload the Matomo GUI.

I checked some key reports.

Visitors over time:

Visitors over time report

Visitor Location Map

Visitor Location Map

Visitor Overview

Visitor Overview

Out links Clicked

Out links Clicked

Nice

I subscribed to the Matomo newsletter here to keep up to date with Matomo update releases: https://matomo.org/newsletter/

Good luck and I hope this guide helps someone

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 Hardening info

v1.1 Updating Matomo

v1.0 Initial post

Filed Under: Analytics, Cloud, Free, Privacy Tagged With: a, analytics, focused, for, google, hosted, locally, Matomo, Open, Piwik, privacy, Replacing, solution, source, with

Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc

October 3, 2018 by Simon

This is a draft post showing how you can monitor the performance of a server (or servers) with NixStats and receive alerts by SMS, Push, Email, Telegram etc

fyi: This is not a paid post, this is just me using the NixStats software to monitor my servers and send alerts.

Finding a good host

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name here

Domain names for just 88 cents!

Monitoring Servers

The post below will show you how you can monitor servers online with https://nixstats.com/ and send alerts when resources reach limits or servers fail.

https://nixstats.com/

I signed up and started a Nixstats (14 day free trial).

Start Nixstats Trial

After I created an account I was emailed by Nixtsats with agent install instructions for Linux (1 line). I was also advised to add contacts and to set up alerts.

I logged into the Nixstats settings and set up…

  • My Timezone
  • Default reporting period
  • First name and Surname
  • Reporting emails
  • etc

Nixstats Subscription Upgrade

Subscription options

  • Free (5 monitors, 1 server, 24-hour data retention etc)
  • Founder (25 monitors, 10 servers, 30-day data retention etc)
  • Business (100 monitors, 15 servers, 60-day data retention etc)

Subscription Options

I enabled the limited founder subscription so I can monitor 10x servers (this deal is too good to miss). I tried creating a status page myself last year and it is terribly hard.

Subscriptiosn page

I am now out of the free trial period 🙂 Let’s start monitoring many servers.

Subscription Activated

I enabled two factor Auth to Nixstats logins

Nixstats Two Factor Auth

I created a Nixstats API key for future use (watch this space)

Create API Key

I installed the Nixstats agent (the dashboard gave a 1 line command you can run as root to install the agent (on Linux)).

Instal Nixstats Agent

FYI: Command (######################## is a number linked to your account)

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ########################

Output

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ########################
--2018-10-02 09:53:56--  https://www.nixstats.com/nixstatsagent.sh
Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8013, 2400:cb00:2048:1::6819:8113, 104.25.128.19, ...
Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8013|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 38708 (38K) [application/octet-stream]
Saving to: 'nixstatsagent.sh'

nixstatsagent.sh 100%[====================================================>] 37.80K –.-KB/s in 0.1s

2018-10-02 09:53:56 (338 KB/s) – ‘nixstatsagent.sh’ saved [38708/38708]

Found Ubuntu …
Installing …
Installing Python2-PIP …
Installing nixstatsagent …
Generation a server id …
Got server_id: ######################
Creating and starting service
Created symlink /etc/systemd/system/multi-user.target.wants/nixstatsagent.service -> /etc/systemd/system/nixstatsagent.service.
Created the nixstatsagent service

Server Dashboard
below is a summary of all connected servers ( https://nixstats.com/dashboard/servers ).

Server Sumamry

Monitor Setup

I set up a number of monitors to monitor ping replies and https traffic

Monitors

Advanced Monitoring

I can also set the monitor credentials, timeouts, retries, auth methods, max redirects and frequency. If you server blocks login or resource GET attempts you may need to whitelist IP’s. IP’s of monitoring servers are located here https://nixstats.com/whitelist.php

Monitoring advanced options

Monitor Summary

The default dashboard is very informative. Feel free to create your own dashboards that focus on your own infrastructure or apps.

Monitor Summary

Individual Server Reports

You can click on a server and monitor it in detail.

Nixstats Graphs

Server Memory Graphs

Long-term memory graphs.

Memory Graph

Install Optional Nixstats Plugins

Nixstats offers many plugins to monitor software that is installed on your server (e.g NGINX, MySQL, PHP etc).

1) NGINX Monitoring (Plugin)

To enable NGINX monitoring I read https://help.nixstats.com/en/article/monitoring-nginx-50nu7f/

I edited my NGINX sites-enabled config.

sudo nano /etc/nginx/sites-enabled/default

I added the following

server {
    listen 127.0.0.1:8080;
    server_name localhost;
    location /status_page {
        stub_status on;
        allow 127.0.0.1;
        deny all;
    }
}

I tested, reloaded and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restart

The status page will only be available on the local machine, I tested the page on the local machine

wget -qO- http://127.0.0.1:8080/status_nginx
Active connections: 3
server accepts handled requests
 15 15 31
Reading: 0 Writing: 1 Waiting: 2

It’s Working.

I edit /etc/nixstats.ini

sudo nano /etc/nixstats.ini

I remove comments before these lines to enable the plugin

[nginx]
enabled = yes
status_page_url = http://127.0.0.1:8080/status_nginx

I ran the following command to see if NGINX monitoring is possible

nixstatsagent --test nginx

Output

nginx:
{
    "accepts": 39,
    "accepts_per_second": 0.0,
    "active_connections": 6,
    "handled": 39,
    "handled_per_second": 0.0,
    "reading": 0,
    "requests": 119,
    "requests_per_second": 0.0,
    "waiting": 5,
    "writing": 1
}

It’s Working

I restarted the nixstatsagent

service nixstatsagent restart

I can now view NGINX properties like active_connections in my dashboard. 🙂

2) Enable PHP-FPM Monitoring (Plugin)

Looks like a PHP-FPM monitoring was recently added lets set that up too. Read my guide on setting up PHP child workers here.

We’ve added a premade dashboard for PHP-FPM. If you’re not yet monitoring PHP-FPM take a look at the integration guide https://t.co/X4ywRHw9hX pic.twitter.com/aag1fTsr3R

— Nixstats (@nixstats) September 6, 2018

To enable PHP-FPM monitoring I read https://help.nixstats.com/en/article/monitoring-php-fpm-1tlyur6/

I edited my PHP-FPM ini file

sudo nano /etc/php/7.2/fpm/pool.d/www.conf

I added the following line

pm.status_path = /status_phpfpm

Restart PHP

sudo service php7.2-fpm restart

I added the following to /etc/nginx/sites-enabled/default localhost server block added above Note I use php 7.2 below (read more here).

server {
    listen 127.0.0.1:8080;
    server_name localhost;

location /status_phpfpm {
access_log off;
allow 127.0.0.1;
deny all;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass unix:/run/php/php7.2-fpm.sock;
log_not_found off;
}
}

I tested, reloaded and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restart

I restart PHP-FPM

sudo systemctl restart php7.2-fpm

Enabled the plugin in /etc/nixstats.ini

[phpfpm]
enabled = yes
status_page_url = http://127.0.0.1:8080/status_phpfpm?json

I tested the status page

wget -qO- http://127.0.0.1:8080/status_phpfpm?json

Output:

{
	"pool":"www",
	"process manager":"static",
	"start time":1538654543,
	"start since":178,
	"accepted conn":28,
	"listen queue":0,
	"max listen queue":0,
	"listen queue len":0,
	"idle processes":49,
	"active processes":1,
	"total processes":50,
	"max active processes":2,
	"max children reached":0,
	"slow requests":0
}

I tested the agent

 nixstatsagent --test phpfpm

Output:

phpfpm:
{
    "accepted_conn": 51,
    "accepted_conn_per_second": 0.0,
    "active_processes": 1,
    "idle_processes": 49,
    "listen_queue": 0,
    "listen_queue_len": 0,
    "max_active_processes": 2,
    "max_children_reached": 0,
    "max_listen_queue": 0,
    "pool": "www",
    "process_manager": "static",
    "slow_requests": 0,
    "start_since": 318,
    "start_time": 1538654543,
    "total_processes": 50
}

I can now query PHP-FPM status values in Nixstats 🙂

Query PHP-FPM

Enable MySQL Monitoring (Plugin)

To enable MySQL monitoring I read https://help.nixstats.com/en/article/monitoring-mysql-1frskd8/

I edited the nixstats.ini

sudo nano /etc/nixstats.ini

I enabled the mysql section in nixstats.ini and added my mysql credentials

[mysql]
enabled=yes
username=mysqluser
password=#######################
host=127.0.0.1
database=mysql
port=3306
socket=null

I ran this command to test MySQL querying

nixstatsagent --test mysql
mysql:
Load error: No module named MySQLdb

I had an error.

A quick Google revealed I had to install a mysql python module.

sudo apt-get install python-mysqldb

Allow localhost to connect to MySQL

Edit /etc/mysql.cnf and allow all localhost and external connections (I could not bind to localhost and an external IP at the same time)

bind-address    = 0.0.0.0

TIP: Ensure you have firewalled access to your MySQL server, never open it up without protection.

Let’s try again

nixstatsagent --test mysql

Output

mysql:
{
    "aborted_clients": 0,
    "aborted_connects": 0,
    "binlog_cache_disk_use": 0,
    "binlog_cache_use": 0,
    "bytes_received": 0,
    "bytes_sent": 0,
    "com_delete": 0,
    "com_delete_multi": 0,
    "com_insert": 0,
    "com_insert_select": 0,
    "com_load": 0,
    "com_replace": 0,
    "com_replace_select": 0,
    "com_select": 0,
    "com_update": 0,
    "com_update_multi": 0,
    "connections": 0,
    "created_tmp_disk_tables": 0,
    "created_tmp_files": 0,
    "created_tmp_tables": 0,
    "key_read_requests": 0,
    "key_reads": 0,
    "key_write_requests": 0,
    "key_writes": 0,
    "max_used_connections": 3.0,
    "open_files": 14.0,
    "open_tables": 316.0,
    "opened_tables": 0,
    "qcache_free_blocks": 1.0,
    "qcache_free_memory": 16760152.0,
    "qcache_hits": 0,
    "qcache_inserts": 0,
    "qcache_lowmem_prunes": 0,
    "qcache_not_cached": 0,
    "qcache_queries_in_cache": 0,
    "qcache_total_blocks": 1.0,
    "questions": 0,
    "select_full_join": 0,
    "select_full_range_join": 0,
    "select_range": 0,
    "select_range_check": 0,
    "select_scan": 0,
    "slave_open_temp_tables": 0.0,
    "slow_launch_threads": 0,
    "slow_queries": 0,
    "sort_range": 0,
    "sort_rows": 0,
    "sort_scan": 0,
    "table_locks_immediate": 0,
    "table_locks_waited": 0,
    "threads_cached": 2.0,
    "threads_connected": 1.0,
    "threads_created": 0,
    "threads_running": 1.0,
    "uptime": 35837.0
}

Nice,

I restart MySQL

sudo systemctl restart mysql

I restart my Nixstats service

service nixstatsagent restart

Now let’s monitor MySQL in Nixstats

I can now view MySQL metrix

MySQL MEtrix

Status Page

Nixstats allows you to create a status page ( https://nixstats.com/pages/overview ) where you can add any servers or monitors to that page. This stats page is truly awesome, it builds a live status page based on data coming from your installed agents.

You can even set up a custom subdomain that points to a Nixstats hosted status page too (e.g https://status.yourdomain.com).

FYI: An SSL certificate on your staus page may take a few hours to set up. Don’t panic if it is not instantatly available.

Custom Status Page

Nice.

This saves doing it yourself. The status page will look like it running on your server.

Status Page

You can create a status page that automatically aggregates collected data from monitors and displays them in a nice layout.

Status Page

This is great, I used to do my own status pages but not anymore.

Alerts

I added a contact so I could receive alerts. I could then add my mobile, email and PushOver key (to receive push notifications) and Telegram Bot API token.

Contact

Test Alerts

I sent a test alert to each service against the contact.

Test Alerts

I activated a Pushover licence on my Android device for about $7.49 AUD (one off) to ensure I keep getting Push Notifications.

Bought licence for PushOver

Nixstats have links that show you how you can create a Telegram Bot and Pushover.net account.

Pushover will cost about $5 USD one off per device (see faq).

I created the following alerts

Alert: Disk Usage higher than 90%

Alert Disk Usage higher than 90%

Alert: Load greater than 90 per cent for 1 minute

Alert load greater than 90 percent for 1 minute

Alert: Less than 5 percent memory free.

Alert less than 5 percent memory free

Summary of alerts.

Alert Sumamry

I also added a CPU reached 95% one for 5 mins alert too (but it’s not pictured above)

I forgot to specify alert recipients and methods for each alert so I edited each alert and added the contact.

Selected Alert Recipients and methods

Now it’s time to test the alerts.

I shut down a server to test alerts

shutdown -h now

Alerts to my defined Email, SMS, Push and Telegram are working a treat 🙂

Alerts Working

After I rebooted the server I also received alerts about the server being back up.

The status page showed the server that was offline too.

Server Offline

Nice

Troubleshooting

I had an issue instaling the agent on Debian

I ran the following command

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh #######################
--2018-10-02 00:41:38--  https://www.nixstats.com/nixstatsagent.sh
Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8113, 2400:cb00:2048:1::6819:8013, 104.25.129.19, ...
Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8113|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File 'nixstatsagent.sh' not modified on server. Omitting download.

nixstatsagent.sh: line 508: [: Installer exited with error code 0. See nixstatsagent.log for details.: integer expression expected

An error occurred, please check the install log file (nixstatsagent.log)!

Contents of nixstatsagent.log

cat nixstatsagent.log
Ign:1 http://deb.debian.org/debian stretch InRelease
Hit:2 http://deb.debian.org/debian-security stretch/updates InRelease
Hit:3 http://deb.debian.org/debian stretch-updates InRelease
Hit:4 http://deb.debian.org/debian stretch Release
Ign:5 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic InRelease
Ign:7 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Hit:9 https://packages.sury.org/php stretch InRelease
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Err:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
  404  Not Found
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Reading package lists...
W: The repository 'http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release' does not have a Release file.
E: Failed to fetch http://ppa.launchpad.net/ondrej/php/ubuntu/dists/cosmic/main/binary-amd64/Packages  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
nixstatsagent.sh: line 118: apt-get upgrade returned error code 100. Please see nixstatsagent.log for details.: command not found

I asked the Nixstats chat help and I was advised I had a dead repository (I removed this (editing the dead repo in the appropriate file in /etc/apt/) and all was ok)

I had trouble testing my Telegram alerts but it was my fault as I forgot to follow the bot account I created. Telegram does not allow message from a user (bot) unless you follow them.

A chat with the Nixstats staff sorted me out. Thanks, Nixstats chat team.

Nixstats chat

I had an issue with a missing python mysql package

Load error: No module named MySQLdb

I solved it by instaling python-mysqldb

sudo apt-get install python-mysqldb

Nixstats Help

Nixstats have a help subdomain: https://help.nixstats.com/en/

Nixstats Help

Error Logs Plugin

I did ask Nixstats on Twitter and they said they are working on a logging plugin, I can’t wait for that.

We’re launching a closed beta for Logging at Nixstats. Contact us to get setup! You can search and tail log files across all your servers! pic.twitter.com/FIeip2SOUw

— Nixstats (@nixstats) October 4, 2018

I now have access to beta log features and can see log tabs in Nixstats

I had or check the version of my rsyslogd

rsyslogd -v
rsyslogd 8.32.0, compiled with:
        PLATFORM:                               x86_64-pc-linux-gnu
        PLATFORM (lsb_release -d):
        FEATURE_REGEXP:                         Yes
        GSSAPI Kerberos 5 support:              Yes
        FEATURE_DEBUG (debug build, slow code): No
        32bit Atomic operations supported:      Yes
        64bit Atomic operations supported:      Yes
        memory allocator:                       system default
        Runtime Instrumentation (slow code):    No
        uuid support:                           Yes
        systemd support:                        Yes
        Number of Bits in RainerScript integers: 64

I edited: /etc/rsyslog.d/31-nixstats.conf

I pasted

##########################################################
### Rsyslog Template for Nixstats ###
##########################################################

$WorkDirectory /var/spool/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down

template(name=”NixFormat” type=”string”
string=”<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [[email protected] tag=\”rsyslog\”] %msg%\n”
)

action(type=”omfwd” protocol=”tcp” target=”log.nixstats.com” port=”514″ template=”NixFormat”)
#################END CONFIG FILE#########################

I restarted the rsyslog service

sudo service rsyslog restart

Live Log Output

I can see a live log from (unknown) logs.

I can see the firewall blocking access to certain ports.

Live Log

Search logs

Search

Blacklist Checking (Beta)

Nixstats tweeted “We just launched a new blacklist check feature. Monitor your IP and hostname reputation. Free during the Beta!”

I enabled it.

I received a Blacklist notification

I requested removal at junkmailfilter.com

Thanks, Nixstats

Conclusion

This is one of the best software packages I have seen in a while. I have developed status

pages in the past (no more). Great work Nixstats.

Please check out Nixstats today, they are awesome. Signup for a free account and consider the limited time founder plan (it’s a bargain).

Nixstats Live chat support is awesome

Server Plug

If you need a server, consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.3 Blacklist beta

V1.2 Logs beta

v1.1 Logging Tweet

v1.0 Initial Post

Filed Under: Alert, Analytics, Cloud, Domain, Monitor Tagged With: alerts, and, by, email, etc, monitor, NixStats, Performance, push, receive, server, SMS, Telegram, with

Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

March 13, 2018 by Simon

This guide will show how you can set up a website to use Cloudflare on a VM hosted on Vultr and Namecheap

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. This post will show how to let Cloudflare handle the DNS for the domain.

Update 2018: For the best performing VM host (UpCloud) read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Snip from here “Cloudflare’s enterprise-class web application firewall (WAF) protects your Internet property from common vulnerabilities like SQL injection attacks, cross-site scripting, and cross-site forgery requests with no changes to your existing infrastructure.”

Buy a Domain 

Buy a domain name from Namecheap here.

Domain names for just 88 cents!

Cloudflare Benefits (Free Plan)

  • DDoS Attack Protection (Huge network to absorb attacks DDoS attacks over 600Gbps are no problem for our 15 Tbps networks)
  • Global CDN
  • Shared SSL certificate (I disabled this and opted to use my own)
  • Access to audit logs
  • 3 page rules (maximum)

View paid plan options here.

Cloudflare CDN map

Cloudflare CDN says it can load assets up to 2x faster, 60% less bandwidth from your servers by delivering assets from 127 data centres.

Cloudflare Global Network

Setup

You will need to sign up at cloudflare.com

Cloudflare

After you create an account you will be prompted to add a siteAdd SiteCloudflare will pull your public DNS records to import.

Query DNS

You will be prompted to select a plan (I selected free)

Plan Select

Verify DNS settings to import.

DNS Import

You will now be asked to change your DNS nameservers with your domain reseller

DNS Nameservers

TIP: If you have an SSL cert (e.g Lets Encrypt) already setup head to the crypto section and select ” Full (Strict)” to prevent ERR_TOO_MANY_REDIRECTS errors.

Strict SSL

Cloudflare UI

I asked Twitter if they could kindly load my site so I could see if Cloudflare dashboard/stats were loading.

Could I kindly ask if you are reading this that you visit https://t.co/9x5TFARLCt, I am writing a @Cloudflare blog post and need to screenshot stats. Thanks in advance

— Simon Fearby (Developer) (@FearbySoftware) March 13, 2018

The Cloudflare CTO responded.  🙂

Sure thing 🙂

— John Graham-Cumming (@jgrahamc) March 13, 2018

Confirm Cloudflare link to a domain from the OSX Comand line

host -t NS fearby.com
fearby.com name server dane.ns.cloudflare.com.
fearby.com name server nora.ns.cloudflare.com.

Caching Rule

I set up the following caching rule to cache everything for 8 hours instead of WordPress pages

Page Rules

“fearby.com.com/wp-*” Cache level: Bypass

“fearby.com.com/wp-admin/post.php*” Cache level: Bypass

“fearby.com/*” Cache Everything, Edge Cache TTL: 8 Hours

Cache Results

Cache appears to be sitting at 50% after 12 hours.  having cache os dynamic pages out there is ok unless I need to fix a typo, then I need to login to Cloudflare and clear the cache manually (or wait 8 hours)

Performance after a few hours

DNS times in gtmetrix have now fallen to a sub 200ms (Y Slow is now a respectable A, it was a C before).  I just need to wait for caching and minification to kick in.

DNS Improved

webpagetest.org results are awesome

See here: https://www.webpagetest.org/result/180314_PB_7660dfbe65d56b94a60d7a604ca250b3/

  • Load Time: 1.80s
  • First Byte 0.176s
  • Start Render 1.200s

webpagetest

Google Page Speed Insights Report

Mobile: 78/100

Desktop: 87/100

Check with https://developers.google.com/speed/pagespeed/insights/

Update 24th March 2018 Attacked?

I noticed a spike in and traffic (incoming and threats) on the 24th of March 2018.

I logged into Cloudflare on my mobile device and turned on Under Attack Mode.

Under Attack Flow

Cloudflare was now adding a delay screen in the middle of my initial page load. Read more here.  A few hours after the Attach started it was over.

After the Attack

I looked at the bandwidth and found no increase in traffic from my initial host VM. Nice.

cloudflare-attack-001

Thanks, Cloudflare.

Cloudflare Pros

  • Enabling Attack mode was simple.
  • Soaked up an attack.
  • Free Tier
  • Many Reports
  • Option to force HTTPS over HTTP
  • Option to ban/challenge suspicious IP’s and set challenge timeframes.
  • Ability to setup IP firewall rules and Application Firewalls.
  • User-agent blocking
  • Lockdown URL’s to IP’s (pro feature)
  • Option to minify Javascript, CSS and HTML
  • Option to accelerate mobile links
  • Brotli compression on assets served.
  • Optio to enable BETA Rocket loader for Javascript performance tweaks.
  • Run Javascript service workers from the 120+ CDN’s
  • Page/URL rules o perform custom actions (redirects, skip cache, Encryption etc)
  • HTTP/2 on, IPV6 ON
  • Option to setup load balancing/failover
  • CTO of Cloudflare responded in Twitter 🙂
  • Option to enable rate limiting (charged at 10,000 hits for $0.05c)
  • Option to block countries (pro feature)
  • Option to install apps in Cloudflare like(Goole Analytics,

Cloudflare Cons

  • No more logging into NameCheap to perform DNS management (I now goto Cloudflare, Namecheap are awesome).
  • Cloudflare Support was slow/confusing (I ended up figuring out the redirect problem myself).
  • Some sort of verify Cloudflare Setup/DNS/CDN access would be nice. After I set this up my gtmetrix load times were the same and I was not sure if DNS needs to replicate? Changing minify settings in Cloudflare did not seem to happen.
  • WordPress draft posts are being cached even though page riles block wp-admin page caching.
  • Would be nice to have ad automatic Under Attack mode
  • Now all sub-domains were transferred in the setup ( id did not know for weeks)

Cloudflare status

Check out https://www.cloudflarestatus.com/ for status updates.

Don’t forget to install the CloudFlare Plugin for WordPress if you use WordPress.

More Reading

Check out my OWASP Zap and Kali Linux self-application Penetration testing posts.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.8 host Command from the OSX CLI

v1.7 Subdomain error

v1.6 Cloudflare Attack

v1.5 WordPress Plugin

v1.4 More Reading

v1.3 added WAF snip

v1.2 Added Google Page Speed Insights and webpage rest results

v1.1 Added Y-Slow

v1.0 Initial post

Filed Under: Analytics, App, Cache, CDN, Cloud, Cloudflare, DNS, Domain, Hosting, LetsEncrypt, Marketing, Secure, Security, SEO, Server, VM, Vultr, Website, Wordpress Tagged With: a, and, Cloudflare, hosted, namecheap, on, Setting, to, up, use, vm, vultr, website

Manage Social Media posts with Buffer

October 10, 2017 by Simon

Here is a quick setup guide for Buffer.com where you can connect to and post (manually or scheduled) to multiple social media platforms.

You can view pricing here. You can signup for a  free Buffer Individual plan: https://buffer.com/signup. Signup to Buffer (Free, Limited)

Post Signup Setup

Connect Buffer to Social Media Platforms

Buffer SIgnup

Type post content

Schedule

Change the default image

Define an Image

Choose the platforms and images

Choose Platforms

Schedule the Post

Schedule

You can manually share to a platform at any time.

Share Now

TIP: If you share now you will need to manually share on each platform separately.

Results

Buffer Results

Buffer features I like

  • Good Free Plan
  • Post Scheduling
  • Image Creation Integration (Paid)
  • Reply integration (Paid)
  • Manage all your social accounts from one simple dashboard
  • Ability to set custom posting slots.

Buffer features I Don’t like

  • Manual Share to all feature missing.
  • Timezones earlier than US Timezones appear to be untouchable (my Timezone is set)
    Timezone

Buffer FAQ’s: https://faq.buffer.com/

Tip: Create custom posting slots

Custom Slots

More soon (reply automation and image creation).

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Added Posting Slots Info

etc

Filed Under: Analitics, Analytics, Automation, Blog, Business, Marketing, Networking, SEO Tagged With: Buffer, Manage, Media, posts, Social

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads

September 9, 2017 by Simon

How to optimize your sites Search Engine Optimization (SEO) and grow customers without paying for Ads.

This guide is a shorter post around setting up SEO (Search Engine Optimization) and driving more traffic to your site without buying ADs.  In a nutshell, to have better SEO you need to jump some technical hurdles in order to drive more traffic to your site from search engines along with understanding your customer’s needs and making things easier for them.

I have blogged about these topics before but these posts are too long in reflection.

  • Setting up Google Analytics on your website
  • How to boost your site’s SEO
  • Improving the speed of WordPress
  • Digital marketing and user engagement 101
  • Add Google AdWords to your WordPress blog
  • etc

Buying Ad’s?

Facebook, Google, Bing and advertising agencies will recommend you set goals around growth and site traffic and pay for those goals to succeed (usually by advertisements).

Don’t get me wrong Advertising works but it is a competitive market, Online sites can easily setup the display of Ad’s on their site (my guide here Add Google AdWords to your WordPress blog, https://fearby.com/article/add-google-adwords-wordpress-blog/ ). You can buy physical billboard ad’s on the side of roads (e.g http://www.buythisspace.com.au/). I tried to enquire about the costs of a physical billboard but the agencies robot verification rejected my enquiry submission so I gave up.  Advertising is buying peoples times and people now how to avoid ad’s and not interact with them (7 Marketing Lessons from Eye-Tracking Studies https://blog.kissmetrics.com/eye-tracking-studies/)

Do more of what works

Spoiler: This guide will recommend you do more of what works over buying millions of ad’s and hoping for new and engaged customers and customer growth.

  • If you don’t already have Google Analytics setup on your site then do it, you cannot identify your customers or identify what is broken or in turn fix it (Setting up Google Analytics on your website, https://fearby.com/article/setting-up-google-analytics-on-your-website/ )
  • Monitor Data – Do review your logs and customer related data (review orders, customers and try and identify what works. Software like https://www.zoho.com/one/applications/web.html will help you connect the dots.
  • Adobe Audience Cloud: http://www.adobe.com/au/experience-cloud.html is a more expensive software suite for driving decisions based on data.
  • Benchmarks – Set goals and work toward them (e.g I want 10x more customers).

SEO Tip’s

This older article on  How to boost your site’s SEO  attempts to mention what you need to do it to get better SEO.

Do run a modern great site

I am a big fan of word of mouth over free/organic traffic over paid customers via advertising (Mostly because I am tight and realize advertising can be a bottomless pit). The single biggest thing you can do to have more organic traffic from search engines is run a modern and fast website, have valuable content and make it as easy for the customer as possible. This is why I moved my site and setup an SSL certificate (link to article).

Search engines like your site to be fast, updated frequently, have sitemaps to make their jobs easier and have an SSL certificate to keep the web safe etc.

Google, Bing and other search engines will not send traffic your way if you do not satisfy them that your site is liked or has valuable content.  Google makes money from Google Analytics by helping people understand their site’s visitors then recommend you pay for ad’s to use on sites that have AdWords on their site ( WordPress to a new self-managed server away from CPanel ).

  • How to boost your site’s SEO https://fearby.com/article/how-to-boost-your-sites-seo/
  • Your website needs to be fast, use sites like https://www.webpagetest.org to measure how fast your site is (Aim for all A’s). Read this page for information on the impact of slow websites https://www.searchenginejournal.com/mobile-page-speed-benchmarks/194511/
  • Mobile friendly – Ensure your site is mobile friendly (or risk being dropped from search engine results)
  • SSL – Do have a secure SSL certificate on your website (view mine here https://www.ssllabs.com/ssltest/analyze.html?d=www.fearby.com&s=45.63.29.217&latest).
  • Incoming links – Having incoming links to your site tell search engines that your site is popular. 

Traffic Source types

  • Organic – An organic visitor to your site is one who found your site by searching something that was relevant to their search term and not by clicking on an advertisement.
  • Paid – A paid user is someone who has clicked an ad to come to your site.
  • Social – A social visitor is one who is known to come from a social media site, using social media sites like Twitter, Facebook or Instagram is a must to driving organic traffic (go where the people are).

Engagement

How engaged are your customers?  Have you asked your customers recently what they value or appreciate about your business or product? Have you asked for feedback recently?

User Engagement Levels

  • None – Do you have landing pages that quickly inform customers of your products or services?
  • Low – What do they need to know about your product or service?
  • Medium – Aware (engaged)
  • High – Can this person be an advocate for your business?
  • Gone – Did you get exit Feedback?

Ways to engage already engaged customers.

  • Setup a free MailChimp Newsletter to allow willing people to be alerted of new communication https://login.mailchimp.com/signup/?source=website&pid=GAW
  • Web Browser popup Alerts can be a great way to engage with users when new content is added to your site (Read the guide here https://documentation.onesignal.com/docs/web-push-setup )
  • Mobile apps or mobile friendly website are a no brainer given 2 billion people use mobile phones ( http://www.smartinsights.com/mobile-marketing/mobile-marketing-analytics/mobile-marketing-statistics/ ).

What can you do to help understand your customer’s needs and make their purchase processes easier?

Why are your customers leaving?

Understand more about your customers reasons for leaving and act upon preventing others from leaving.

  • Trying something new (Does your website need to be simpler?)
  • Are your products too expensive?.
  • Your site (or ordering) is not convenient (Do you need to setup online ordering/subscriptions and delivery?)
  • etc

Who are your customers

  • Personas – Do setup customer personas in order to focus on your customer segments (get a free customer persona template here https://blog.hubspot.com/blog/tabid/6307/bid/33491/everything-marketers-need-to-research-create-detailed-buyer-personas-template.aspx )
  • Does your website match these personas?

Are your customers.

  • Engaged
  • Informed
  • Advocates

Feedback

  • Do you have feedback loops (A simple feedback form can solve this)?

What do you know about your customers?

  • Product Satisfaction
  • Product Loyalty
  • Product Awareness

Paid Traffic (Ad’s)

  • Google Ad’s – Signup Here http://www.google.com.au/adwords/get-started/
  • Bing – Advertise on Bing here https://advertise.bingads.microsoft.com/
  • Facebook – Advertise on Facebook here https://www.facebook.com/business/products/ads

Free Traffic (SEO + Organic Ad’s)

  • Blog Posts (Sharing value/passion)
  • Social Media Posts (use hashtags)
  • Instagram (Post value/passion)

Most importantly Do what works (Measure and replicate).

Focus on Business Value

Generate a  SWOT Analysis ( Free tool here https://xtensio.com/ )

  • What are your Strengths?
  • What are your Weaknesses?
  • What are your Opportunities?
  • What are your Threats?

Goals

Goals allow you to investigate, learn, act and measure I order to improve.

  • Investigate – Data.
  • Learn/Insight – Make Assumptions.
  • Act – Act and measure.

Read more about customer engagement here https://en.wikipedia.org/wiki/Customer_engagement

Bonus

 Do ensure your website is compliant with accessibility and technical standards

  • Test our sites Accessibility – https://achecker.ca/checker/index.php
  • Test your sites HTML5 Compliance – https://validator.w3.org
  • Test your Google PageSpeed Test – https://developers.google.com/speed/pagespeed/insights/
  • Do A B testing to determine the statistical significance of changes to your site.

Conclusion

The more you know the better you can connect, Do set goals and as a minimum setup Google Analytics, SSL certificate and submit your site to search engines, then focus on a fast site that makes things simple for your customers.

Donate and make this blog better



Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.0 Initial version

Filed Under: Ads, Analytics, Business, LetsEncrypt, SEO, ssl, Website Tagged With: analytics, seo, ssl

Setting up Google Analytics on your website

August 22, 2017 by Simon Fearby

Google Analytics is a popular easy to install and use statistics and reporting tool that you can add to your website (and it’ free)

To setup, Google Analytics go to https://analytics.google.com/analytics/web/ and create an account. From here you can add a site and generate a tracking ID.

Google Analitics Geenrate ID

The website tracking code was (I changed the code to 555555555).

<script>
  (function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
  (i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
  m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
  })(window,document,'script','https://www.google-analytics.com/analytics.js','ga');

  ga('create', 'UA-555555555-1', 'auto');
  ga('send', 'pageview');

</script>

I opened WordPress and went to Appearance then Editor and selected header.php and added the tracking code under the <head> HTML tag.

edit file

This tracking ID allows Google to generate stats from your visitors.

I was unable to update the file in WordPress until I set permissions in Ubuntu in (Read these guide to setup an Ubuntu Server on Vultr for as low as $2.5 a month of setup a $5 a month with Digital Ocean or AWS).  I have guides on moving WordPress here or setting up WordPress from the command line here). If you update WordPress you may need to re add the tracking ID.

sudo chmod 666 /www/wp-content/themes/twentyseventeen/header.php

I loaded my WordPress website and verified that the tracking code was loading in the HTML source.  You can also embed the tracking code in static HTML websites.

html source

After a few days, you can view your sites statistics. from the Googe Analytics home portal.  This will allow you know when to publish, know how popular your content is, know what new content to create etc.

Page Hits

The best feature of Google Analytics is page hit information. To me, the total number of hits is less important than Avg. Time on Page and Bounce Rate.

page hits

Dashboard

The Google Analytics dashboard home is very informative.

Analitics Home

Google Analytics Terms

Google has a glossary for terms here.

  • Users – The unique user that visited your site.
  • Bounce Rate – The percentage or users who loaded your site and left after viewing the initial page.
  • Active Users – The total number of active users reading your site.
  • User Retention – The percentage of users who have returned to your site.
  • Device – The device (Desktop, tablet or mobile device) that was used to read your site.
  • Organic Search – The number of users who found your site via a search engine.  Having a highly efficient SEO will see a higher Organic search percentage.
  • Sessions – The number of unique sessions that your users have accessed your site.
  • Direct – The times a user has directly typed your website URL (or have visited your site in incognito/privacy mode).
  • Referral – The percentage or know referrals from other websites.
  • Social – Known number of visits to our site from social media platforms.

Overview

You can watch in real-time users accessing your site. This is important when you send out mailing list to users when new content is posted, will 1,000 visitors take down your site? Are you posting at the right time for your sites visitors timezone?

active usersAudience Overview

This report will tell you a lot about who and where people are visiting your site form and what language they speak, OS they use, what browser they use and what city they are from.

audience stats

Google Analytics allows you to drill down on most captured data.

City breakdown

I can see Apple devices are the most popular mobile devices accessing my site (but mobile devices in total only take up 12 % of my site’s traffic).

Devices

The User Flow report is a great way to see how people interact with your site (where they come from, what they do and where they drop out).

user flow

Google Analytics has a handy page speed tool that you can use to identify what you need to do to speed up your site.

Page speed

Google Analytics have goals that allow you to set targets to meet. Usually, Google encourages you to assign a monetary value to a goal then suggest you buy Google Ad’s to achieve these goals (this is why Google Analytics is free). Read my guide on setting up Google AdWords on your WordPress blog.

goals

You can set email alerts on key stats.

alerts

More to come later.

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.1 added page hits information

Filed Under: Ads, Analytics, Business, Marketing, Planning, Wordpress Tagged With: analytics

Securing Ubuntu in the cloud

August 9, 2017 by Simon

It is easy to deploy servers to the cloud within a few minutes, you can have a cloud-based server that you (or others can use). ubuntu has a great guide on setting up basic security issues but what do you need to do.

If you do not secure your server expects it to be hacked into. Below are tips on securing your cloud server.

First, read more on scanning your server with Lynis security scan.

Always use up to date software

Always use update software, malicious users can detect what software you use with sites like shodan.io (or use port scan tools) and then look for weaknesses from well-published lists (e.g WordPress, Windows, MySQL, node, LifeRay, Oracle etc). People can even use Google to search for login pages or sites with passwords in HTML (yes that simple).  Once a system is identified by a malicious user they can send automated bots to break into your site (trying millions of passwords a day) or use tools to bypass existing defences (Security researcher Troy Hunt found out it’s child’s play).

Portscan sites like https://mxtoolbox.com/SuperTool.aspx?action=scan are good for knowing what you have exposed.

You can also use local programs like nmap to view open ports

Instal nmap

sudo apt-get install nmap

Find open ports

nmap -v -sT localhost

Starting Nmap 7.01 ( https://nmap.org ) at 2017-08-08 23:57 AEST
Initiating Connect Scan at 23:57
Scanning localhost (127.0.0.1) [1000 ports]
Discovered open port 80/tcp on 127.0.0.1
Discovered open port 3306/tcp on 127.0.0.1
Discovered open port 22/tcp on 127.0.0.1
Discovered open port 9101/tcp on 127.0.0.1
Discovered open port 9102/tcp on 127.0.0.1
Discovered open port 9103/tcp on 127.0.0.1
Completed Connect Scan at 23:57, 0.05s elapsed (1000 total ports)
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00020s latency).
Not shown: 994 closed ports
PORT     STATE SERVICE
22/tcp   open  ssh
80/tcp   open  http
3306/tcp open  mysql
9101/tcp open  jetdirect
9102/tcp open  jetdirect
9103/tcp open  jetdirect

Read data files from: /usr/bin/../share/nmap
Nmap done: 1 IP address (1 host up) scanned in 0.17 seconds
           Raw packets sent: 0 (0B) | Rcvd: 0 (0B)

Limit ssh connections

Read more here.

Use ufw to set limits on login attempts

sudo ufw limit ssh comment 'Rate limit hit for openssh server'

Only allow known IP’s access to your valuable ports

sudo ufw allow from 123.123.123.123/32 to any port 22

Delete unwanted firewall rules

sudo ufw status numbered
sudo ufw delete 8

Only allow known IP’s to certain ports

sudo ufw allow from 123.123.123.123 to any port 80/tcp

Also, set outgoing traffic to known active servers and ports

sudo ufw allow out from 123.123.123.123 to any port 22

Don’t use weak/common Diffie-Hellman key for SSL certificates, more information here.

openssl req -new -newkey rsa:4096 -nodes -keyout server.key -out server.csr
 
Generating a 4096 bit RSA private key
...

More info on generating SSL certs here and setting here and setting up Public Key Pinning here.

Intrusion Prevention Software

Do run fail2ban: Guide here https://www.linode.com/docs/security/using-fail2ban-for-security

I use iThemes Security to secure my WordPress and block repeat failed logins from certain IP addresses.

iThemes Security can even lock down your WordPress.

You can set iThemes to auto lock out users on x failed logins

Remember to use allowed whitelists though (it is so easy to lock yourself out of servers).

Passwords

Do have strong passwords and change the root password provided by the hosts. https://howsecureismypassword.net/ is a good site to see how strong your password is from brute force password attempts. https://www.grc.com/passwords.htm is a good site to obtain a strong password.  Do follow Troy Hunt’s blog and twitter account to keep up to date with security issues.

Configure a Firewall Basics

You should install a firewall on your Ubuntu and configure it and also configure a firewall with your hosts (e.g AWS, Vultr, Digital Ocean).

Configure a Firewall on AWS

My AWS server setup guide here. AWS allow you to configure the firewall here in the Amazon Console.

Type Protocol Port Range Source Comment
HTTP TCP 80 0.0.0.0/0 Opens a web server port for later
All ICMP ALL N/A 0.0.0.0/0 Allows you to ping
All traffic ALL All 0.0.0.0/0 Not advisable long term but OK for testing today.
SSH TCP 22 0.0.0.0/0 Not advisable, try and limit this to known IP’s only.
HTTPS TCP 443 0.0.0.0/0 Opens a secure web server port for later

Configure a Firewall on Digital Ocean

Configuring a firewall on Digital Ocean (create a $5/m server here).  You can configure your Digital Ocean droplet firewall by clicking Droplet, Networking then Manage Firewall after logging into Digital Ocean.

Configure a Firewall on Vultr

Configuring a firewall on Vultr (create a $2.5/m server here).

Don’t forget to set IP rules for IPV4 and IPV6, Only set the post you need to allow and ensure applications have strong passwords.

Ubuntu has a firewall built in (documentation).

sudo ufw status

Enable the firewall

sudo ufw enable

Adding common ports

sudo ufw allow ssh/tcp
sudo ufw logging on
sudo ufw allow 22
sudo ufw allow 80
sudo ufw allow 53
sudo ufw allow 443
sudo ufw allow 873
sudo ufw enable
sudo ufw status
sudo ufw allow http
sudo ufw allow https

Add a whitelist for your IP (use http://icanhazip.com/ to get your IP) to ensure you won’t get kicked out of your server.

sudo ufw allow from 123.123.123.123/24 to any port 22

More help here.  Here is a  good guide on ufw commands. Info on port numbers here.

https://en.wikipedia.org/wiki/List_of_TCP_and_UDP_port_numbers

If you don’t have a  Digital Ocean server for $5 a month click here and if a $2.5 a month Vultr server here.

Backups

rsync is a good way to copy files to another server or use Bacula

sudo apt install bacula

Basics

Initial server setup guide (Digital Ocean).

Sudo (admin user)

Read this guide on the Linux sudo command (the equivalent if run as administrator on Windows).

Users

List users on an Ubuntu OS (or compgen -u)

cut -d: -f1 /etc/passwd

Common output

cut -d: -f1 /etc/passwd
root
daemon
bin
sys
sync
games
man
lp
mail
news
uucp
proxy
www-data
backup
list
irc
gnats
nobody
systemd-timesync
systemd-network
systemd-resolve
systemd-bus-proxy
syslog
_apt
lxd
messagebus
uuidd
dnsmasq
sshd
pollinate
ntp
mysql
clamav

Add User

sudo adduser new_username

e.g

sudo adduser bob
Adding user `bob' ...
Adding new group `bob' (1000) ...
Adding new user `bob' (1000) with group `bob' ...
Creating home directory `/home/bob' ...
etc..

Add user to a group

sudo usermod -a -G MyGroup bob

Show users in a group

getent group MyGroup | awk -F: '{print $4}'

This will show users in a group

Remove a user

sudo userdel username
sudo rm -r /home/username

Rename user

usermod -l new_username old_username

Change user password

sudo passwd username

Groups

Show all groups

compgen -ug

Common output

compgen -g
root
daemon
bin
sys
adm
tty
disk
lp
mail
proxy
sudo
www-data
backup
irc
etc

You can create your own groups but first, you must be aware of group ids

cat /etc/group

Then you can see your systems groups and ids.

Create a group

groupadd -g 999 MyGroup

Permissions

Read this https://help.ubuntu.com/community/FilePermissions

How to list users on Ubuntu.

Read more on setting permissions here.

Chmod help can be found here.

Install Fail2Ban

I used this guide on installing Fail2Ban.

apt-get install fail2ban

Check Fail2Ban often and add blocks to the firewall of known bad IPs

fail2ban-client status

Best practices

Ubuntu has a guide on basic security setup here.

Startup Processes

It is a good idea to review startup processes from time to time.

sudo apt-get install rcconf
sudo rcconf

Accounts

  • Read up on the concept of least privilege access for apps and services here.
  • Read up on chmod permissions.

Updates

Do update your operating system often.

sudo apt-get update
sudo apt-get upgrade

Minimal software

Only install what software you need

Exploits and Keeping up to date

Do keep up to date with exploits and vulnerabilities

  • Follow 0xDUDE on twitter.
  • Read the GDI.Foundation page.
  • Visit the Exploit Database
  • Vulnerability & Exploit Database
  • Subscribe to the Security Now podcast.

Secure your applications

  • NodeJS: Enable logging in applications you install or develop.

Ban repeat Login attempts with FailBan

Fail2Ban config

sudo nano /etc/fail2ban/jail.conf
[sshd]

enabled  = true
port     = ssh
filter   = sshd
logpath  = /var/log/auth.log
maxretry = 3

Hosts File Hardening

sudo nano /etc/host.conf

Add

order bind,hosts
nospoof on

Add a whitelist with your ip on /etc/fail2ban/jail.conf (see this)

[DEFAULT]
# "ignoreip" can be an IP address, a CIDR mask or a DNS host. Fail2ban will not                          
# ban a host which matches an address in this list. Several addresses can be                             
# defined using space separator.
                                                                         
ignoreip = 127.0.0.1 192.168.1.0/24 8.8.8.8

Restart the service

sudo service fail2ban restart
sudo service fail2ban status

Intrusion detection (logging) systems

Tripwire will not block or prevent intrusions but it will log and give you a heads up with risks and things of concern

Install Tripwire.

sudo apt-get install tiger tripwire

Running Tripwire

sudo tiger

This will scan your system for issues of note

sudo tiger
Tiger UN*X security checking system
   Developed by Texas A&M University, 1994
   Updated by the Advanced Research Corporation, 1999-2002
   Further updated by Javier Fernandez-Sanguino, 2001-2015
   Contributions by Francisco Manuel Garcia Claramonte, 2009-2010
   Covered by the GNU General Public License (GPL)

Configuring...

Will try to check using config for 'x86_64' running Linux 4.4.0-89-generic...
--CONFIG-- [con005c] Using configuration files for Linux 4.4.0-89-generic. Using
           configuration files for generic Linux 4.
Tiger security scripts *** 3.2.3, 2008.09.10.09.30 ***
20:42> Beginning security report for simon.
20:42> Starting file systems scans in background...
20:42> Checking password files...
20:42> Checking group files...
20:42> Checking user accounts...
20:42> Checking .rhosts files...
20:42> Checking .netrc files...
20:42> Checking ttytab, securetty, and login configuration files...
20:42> Checking PATH settings...
20:42> Checking anonymous ftp setup...
20:42> Checking mail aliases...
20:42> Checking cron entries...
20:42> Checking 'services' configuration...
20:42> Checking NFS export entries...
20:42> Checking permissions and ownership of system files...
--CONFIG-- [con010c] Filesystem 'fuse.lxcfs' used by 'lxcfs' is not recognised as a valid filesystem
20:42> Checking for indications of break-in...
--CONFIG-- [con010c] Filesystem 'fuse.lxcfs' used by 'lxcfs' is not recognised as a valid filesystem
20:42> Performing rootkit checks...
20:42> Performing system specific checks...
20:46> Performing root directory checks...
20:46> Checking for secure backup devices...
20:46> Checking for the presence of log files...
20:46> Checking for the setting of user's umask...
20:46> Checking for listening processes...
20:46> Checking SSHD's configuration...
20:46> Checking the printers control file...
20:46> Checking ftpusers configuration...
20:46> Checking NTP configuration...
20:46> Waiting for filesystems scans to complete...
20:46> Filesystems scans completed...
20:46> Performing check of embedded pathnames...
20:47> Security report completed for simon.
Security report is in `/var/log/tiger/security.report.simon.170809-20:42'.

My Output.

sudo nano /var/log/tiger/security.report.username.170809-18:42

Security scripts *** 3.2.3, 2008.09.10.09.30 ***
Wed Aug  9 18:42:24 AEST 2017
20:42> Beginning security report for username (x86_64 Linux 4.4.0-89-generic).

# Performing check of passwd files...
# Checking entries from /etc/passwd.
--WARN-- [pass014w] Login (bob) is disabled, but has a valid shell.
--WARN-- [pass014w] Login (root) is disabled, but has a valid shell.
--WARN-- [pass015w] Login ID sync does not have a valid shell (/bin/sync).
--WARN-- [pass012w] Home directory /nonexistent exists multiple times (3) in
         /etc/passwd.
--WARN-- [pass012w] Home directory /run/systemd exists multiple times (2) in
         /etc/passwd.
--WARN-- [pass006w] Integrity of password files questionable (/usr/sbin/pwck
         -r).

# Performing check of group files...

# Performing check of user accounts...
# Checking accounts from /etc/passwd.
--WARN-- [acc021w] Login ID dnsmasq appears to be a dormant account.
--WARN-- [acc022w] Login ID nobody home directory (/nonexistent) is not
         accessible.

# Performing check of /etc/hosts.equiv and .rhosts files...

# Checking accounts from /etc/passwd...

# Performing check of .netrc files...

# Checking accounts from /etc/passwd...

# Performing common access checks for root (in /etc/default/login, /securetty, and /etc/ttytab...
--WARN-- [root001w] Remote root login allowed in /etc/ssh/sshd_config

# Performing check of PATH components...
--WARN-- [path009w] /etc/profile does not export an initial setting for PATH.
# Only checking user 'root'

# Performing check of anonymous FTP...

# Performing checks of mail aliases...
# Checking aliases from /etc/aliases.

# Performing check of `cron' entries...
--WARN-- [cron005w] Use of cron is not restricted

# Performing check of 'services' ...
# Checking services from /etc/services.
--WARN-- [inet003w] The port for service ssmtp is also assigned to service
         urd.
--WARN-- [inet003w] The port for service pipe-server is also assigned to
         service search.

# Performing NFS exports check...

# Performing check of system file permissions...
--ALERT-- [perm023a] /bin/su is setuid to `root'.
--ALERT-- [perm023a] /usr/bin/at is setuid to `daemon'.
--ALERT-- [perm024a] /usr/bin/at is setgid to `daemon'.
--WARN-- [perm001w] The owner of /usr/bin/at should be root (owned by daemon).
--WARN-- [perm002w] The group owner of /usr/bin/at should be root.
--ALERT-- [perm023a] /usr/bin/passwd is setuid to `root'.
--ALERT-- [perm024a] /usr/bin/wall is setgid to `tty'.

# Checking for known intrusion signs...
# Testing for promiscuous interfaces with /bin/ip
# Testing for backdoors in inetd.conf

# Performing check of files in system mail spool...

# Performing check for rookits...
# Running chkrootkit (/usr/sbin/chkrootkit) to perform further checks...
--WARN-- [rootkit004w] Chkrootkit has detected a possible rootkit installation
Possible Linux/Ebury - Operation Windigo installetd

# Performing system specific checks...
# Performing checks for Linux/4...

# Checking boot loader file permissions...
--WARN-- [boot02] The configuration file /boot/grub/menu.lst has group
         permissions. Should be 0600
--FAIL-- [boot02] The configuration file /boot/grub/menu.lst has world
         permissions. Should be 0600
--WARN-- [boot06] The Grub bootloader does not have a password configured.

# Checking for vulnerabilities in inittab configuration...

# Checking for correct umask settings for init scripts...
--WARN-- [misc021w] There are no umask entries in /etc/init.d/rcS

# Checking Logins not used on the system ...

# Checking network configuration
--FAIL-- [lin013f] The system is not protected against Syn flooding attacks
--WARN-- [lin017w] The system is not configured to log suspicious (martian)
         packets

# Verifying system specific password checks...

# Checking OS release...
--WARN-- [osv004w] Unreleased Debian GNU/Linux version `stretch/sid'

# Checking installed packages vs Debian Security Advisories...

# Checking md5sums of installed files

# Checking installed files against packages...
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.dep' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.alias.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.devname' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.softdep' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.alias' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.symbols.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.builtin.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.symbols' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-87-generic/modules.dep.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.dep' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.alias.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.devname' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.softdep' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.alias' does not
         belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.symbols.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.builtin.bin'
         does not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.symbols' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/modules/4.4.0-89-generic/modules.dep.bin' does
         not belong to any package.
--WARN-- [lin001w] File `/lib/udev/hwdb.bin' does not belong to any package.

# Performing check of root directory...

# Checking device permissions...
--WARN-- [dev003w] The directory /dev/block resides in a device directory.
--WARN-- [dev003w] The directory /dev/char resides in a device directory.
--WARN-- [dev003w] The directory /dev/cpu resides in a device directory.
--FAIL-- [dev002f] /dev/fuse has world permissions
--WARN-- [dev003w] The directory /dev/hugepages resides in a device directory.
--FAIL-- [dev002f] /dev/kmsg has world permissions
--WARN-- [dev003w] The directory /dev/lightnvm resides in a device directory.
--WARN-- [dev003w] The directory /dev/mqueue resides in a device directory.
--FAIL-- [dev002f] /dev/rfkill has world permissions
--WARN-- [dev003w] The directory /dev/vfio resides in a device directory.

# Checking for existence of log files...
--FAIL-- [logf005f] Log file /var/log/btmp permission should be 660
--FAIL-- [logf007f] Log file /var/log/messages does not exist

# Checking for correct umask settings for user login shells...
--WARN-- [misc021w] There is no umask definition for the dash shell
--WARN-- [misc021w] There is no umask definition for the bash shell

# Checking symbolic links...

# Performing check of embedded pathnames...
20:47> Security report completed for username.

More on Tripwire here.

Hardening PHP

Hardening PHP config (and backing the PHP config it up), first create an info.php file in your website root folder with this info

<?php
phpinfo()
?>

Now look for what PHP file is loadingPHP Config

Back that your PHP config file

TIP: Delete the file with phpinfo() in it as it is a security risk to leave it there.

TIP: Read the OWASP cheat sheet on using PHP securely here and securing php.ini here.

Some common security changes

file_uploads = On
expose_php = Off
error_reporting = E_ALL
display_errors          = Off
display_startup_errors  = Off
log_errors              = On
error_log = /php_errors.log
ignore_repeated_errors  = Off

Don’t forget to review logs, more config changes here.

Antivirus

Yes, it is a good idea to run antivirus in Ubuntu, here is a good list of antivirus software

I am installing ClamAV as it can be installed on the command line and is open source.

sudo apt-get install clamav

ClamAV help here.

Scan a folder

sudo clamscan --max-filesize=3999M --max-scansize=3999M --exclude-dir=/www/* -i -r /

Setup auto-update antivirus definitions

sudo dpkg-reconfigure clamav-freshclam

I set auto updates 24 times a day (every hour) via daemon updates.

tip: Download manual antivirus update definitions. If you only have a 512MB server your update may fail and you may want to stop fresh claim/php/nginx and mysql before you update to ensure the antivirus definitions update. You can move this to a con job and set this to update at set times over daemon to ensure updates happen.

sudo /etc/init.d/clamav-freshclam stop

sudo service php7.0-fpm stop
sudo /etc/init.d/nginx stop
sudo /etc/init.d/mysql stop

sudo freshclam -v
Current working dir is /var/lib/clamav
Max retries == 5
ClamAV update process started at Tue Aug  8 22:22:02 2017
Using IPv6 aware code
Querying current.cvd.clamav.net
TTL: 1152
Software version from DNS: 0.99.2
Retrieving http://db.au.clamav.net/main.cvd
Trying to download http://db.au.clamav.net/main.cvd (IP: 193.1.193.64)
Downloading main.cvd [100%]
Loading signatures from main.cvd
Properly loaded 4566249 signatures from new main.cvd
main.cvd updated (version: 58, sigs: 4566249, f-level: 60, builder: sigmgr)
Querying main.58.82.1.0.C101C140.ping.clamav.net
Retrieving http://db.au.clamav.net/daily.cvd
Trying to download http://db.au.clamav.net/daily.cvd (IP: 193.1.193.64)
Downloading daily.cvd [100%]
Loading signatures from daily.cvd
Properly loaded 1742284 signatures from new daily.cvd
daily.cvd updated (version: 23644, sigs: 1742284, f-level: 63, builder: neo)
Querying daily.23644.82.1.0.C101C140.ping.clamav.net
Retrieving http://db.au.clamav.net/bytecode.cvd
Trying to download http://db.au.clamav.net/bytecode.cvd (IP: 193.1.193.64)
Downloading bytecode.cvd [100%]
Loading signatures from bytecode.cvd
Properly loaded 66 signatures from new bytecode.cvd
bytecode.cvd updated (version: 308, sigs: 66, f-level: 63, builder: anvilleg)
Querying bytecode.308.82.1.0.C101C140.ping.clamav.net
Database updated (6308599 signatures) from db.au.clamav.net (IP: 193.1.193.64)

sudo service php7.0-fpm restart
sudo /etc/init.d/nginx restart
sudo /etc/init.d/mysql restart 

sudo /etc/init.d/clamav-freshclam start

Manual scan with a bash script

Create a bash script

mkdir /script
sudo nano /scripts/updateandscanav.sh

# Include contents below.
# Save and quit

chmod +X /scripts/updateandscanav.sh

Bash script contents to update antivirus definitions.

sudo /etc/init.d/clamav-freshclam stop

sudo service php7.0-fpm stop
sudo /etc/init.d/nginx stop
sudo /etc/init.d/mysql stop

sudo freshclam -v

sudo service php7.0-fpm restart
sudo /etc/init.d/nginx restart
sudo /etc/init.d/mysql restart

sudo /etc/init.d/clamav-freshclam start

sudo clamscan --max-filesize=3999M --max-scansize=3999M -v -r /

Edit the crontab to run the script every hour

crontab -e
1 * * * * /bin/bash /scripts/updateandscanav.sh > /dev/null 2>&1

Uninstalling Clam AV

You may need to uninstall Clamav if you don’t have a lot of memory or find updates are too big.

sudo apt-get remove --auto-remove clamav
sudo apt-get purge --auto-remove clamav

Setup Unattended Ubuntu Security updates

sudo apt-get install unattended-upgrades
sudo unattended-upgrades -d

At login, you should receive

0 updates are security updates.

Other

  • Read this awesome guide.
  • install Fail2Ban
  • Do check your log files if you suspect suspicious activity.

Check out the extensive Hardening a Linux Server guide at thecloud.org.uk: https://thecloud.org.uk/wiki/index.php?title=Hardening_a_Linux_Server

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.92 added hardening a linux server link

Filed Under: Ads, Advice, Analitics, Analytics, Android, API, App, Apple, Atlassian, AWS, Backup, BitBucket, Blog, Business, Cache, Cloud, Community, Computer, CoronaLabs, Cost, CPI, DB, Development, Digital Ocean, DNS, Domain, Email, Feedback, Firewall, Free, Git, GitHub, GUI, Hosting, Investor, IoT, JIRA, LetsEncrypt, Linux, Malware, Marketing, mobile app, Monatization, Monetization, MongoDB, MySQL, Networking, NGINX, NodeJS, NoSQL, OS, Planning, Project, Project Management, Psychology, push notifications, Raspberry Pi, Redis, Route53, Ruby, Scalability, Scalable, Security, SEO, Server, Share, Software, ssl, Status, Strength, Tech Advice, Terminal, Transfer, Trello, Twitter, Ubuntu, Uncategorized, Video Editing, VLOG, VM, Vultr, Weakness, Web Design, Website, Wordpress Tagged With: antivirus, brute force, Firewall

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) HTTPS (6) IoT (9) LetsEncrypt (7) Linux (20) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) PHP (13) Scalability (12) Scalable (14) Security (44) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (44) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT