• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar
  • Skip to footer
  • Home
  • Create a VM ($25 Credit)
  • Buy a Domain
  • 1 Month free Back Blaze Backup
  • Other Deals
    • Domain Email
    • Nixstats Server Monitoring
    • ewww.io Auto WordPress Image Resizing and Acceleration
  • About
  • Links

IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

  • Cloud
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to buy a new domain and SSL cert from NameCheap, a Server from Digital Ocean and configure it.
    • Setting up a Vultr VM and configuring it
    • All Cloud Articles
  • Dev
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • How to setup pooled MySQL connections in Node JS that don’t disconnect
    • NodeJS code to handle App logins via API (using MySQL connection pools (1000 connections) and query parameters)
    • Infographic: So you have an idea for an app
    • All Development Articles
  • MySQL
    • Using the free Adminer GUI for MySQL on your website
    • All MySQL Articles
  • Perf
    • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Measuring VM performance (CPU, Disk, Latency, Concurrent Users etc) on Ubuntu and comparing Vultr, Digital Ocean and UpCloud – Part 1 of 4
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Setting up a website to use Cloudflare on a VM hosted on Vultr and Namecheap
    • All Performance Articles
  • Sec
    • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
    • Using OWASP ZAP GUI to scan your Applications for security issues
    • Setting up the Debian Kali Linux distro to perform penetration testing of your systems
    • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
    • PHP implementation to check a password exposure level with Troy Hunt’s pwnedpasswords API
    • Setting strong SSL cryptographic protocols and ciphers on Ubuntu and NGINX
    • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
    • All Security Articles
  • Server
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All Server Articles
  • Ubuntu
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • Useful Linux Terminal Commands
    • All Ubuntu Articles
  • VM
    • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
    • All VM Articles
  • WordPress
    • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
    • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
    • How to backup WordPress on a host that has CPanel
    • Moving WordPress to a new self managed server away from CPanel
    • Moving a CPanel domain with email to a self managed VPS and Gmail
    • All WordPress Articles
  • All

for

Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution

November 18, 2018 by Simon

This is how I replacing Google Analytics with Piwik/Matomo for a locally hosted privacy-focused open source analytics solution

Aside

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. PHP is my programming language of choice.

Now on with the post

Google Analytics

I will fully admit Google Analytics is good. I posted this a while ago on how you can set up Google Analytics on your site.

Google Analytics has some great charts and graphs. Simple to set up and easy to use.

Analitics Home

My site traffic is growing and I would prefer to hold my own analytics on user data. Matomo is an analytics solution that stays on my server and not in the hands of Google.

Blog Growth

Google Analytics can be Slow

Sometimes the Google Analytics server is slow (affecting the speed of my server). I blogged recently about speeding up a WordPress site here and Google Servers were not adding expiry headers on assets.

I did log a ticket with Google to fix this and the experience was terrible.

Support for Google Analytics is terrible

Gogole Analytics support of terrible

GT Metrix scores show poor delivery of tracking assets.

Google Slow Assets

Privacy

After the Cambridge Analytica fiasco (that made me decide to delete facebook) sending analytics to Google is not a good idea.

  • Google Removes ‘Don’t Be Evil’ Clause From Its Code Of Conduct
  • FUTURE SOCIETY Three Signs Google Is Turning to the Dark Side
  • Top 10 Ways Google Does Evil

I am not saying Google is evil but I want my site’s visitors tracking data to remain local.

Website Speed Benchmark before installing Matomo

I can load my site in 1.3 seconds at best, 1.5 seconds on average and 2.0 seconds at worst. My site is loading 11 assets.

GTmetrix 1.3 second page load time

Page Speed Scores

GTMerix page speed load times

Y Slow Scores, Gogol Assets are reporting no expiry headers (slowing down scores)

GTMetrix yslow load times

Google Analytics tracking assets are slow.

Gmetrix waterfall list

Optimizations to be made

Browser caching is not possible with Google Analytics.

Gogole lacking browser caching

Missing Expiry Headers (I can see a Google Tag Manager server is slowing down my servers benchmark score)

Google lacking Expiry Headers

Why Mamoto (instead of Google Analytics)

I came across

Someone pointed out that @haveibeenpwned got a bunch of traction on Reddit today. With pretty much everything now either cached by @Cloudflare or served by @AzureFunctions, the first I know of a 28x traffic increase is no longer when something scales it’s when someone tells me 😎 pic.twitter.com/ifj7nQg3n4

— Troy Hunt (@troyhunt) November 5, 2018

Mamoto was mentioned

It’s an Open Source, self hostable, privacy friendly alternative to Google Analytics:https://t.co/NiK7A7uQAE

— Lukas Winkler (@lw1_at) November 5, 2018

I visited https://matomo.org/

Mamoto webpage

Snip

> Take care of running Matomo yourself by installing it on your own server. There is no cost for Matomo itself but you need a server and update Matomo & your server regularly to keep it fast and secure. Need help? The Matomo team provides free help resources and paid support.

Mamoto Setup Instruction Guide

Source Code

Source code is available.

> Matomo is the leading open alternative to Google Analytics that gives you full control over your data. Matomo lets you easily collect data from websites, apps & the IoT and visualise this data and extract insights. Privacy is built-in. We love Pull Requests! https://matomo.org/

https://github.com/matomo-org/matomo

Installation Guide

I read the installation guide here https://matomo.org/docs/installation/

You can view the changelog here https://matomo.org/changelog/

Downloading Mamoto

I logged into my server via SSH and downloaded the 18MB download to the desired folder

cd /www-root/matomo-folder/
wget https://builds.matomo.org/matomo.zip

I unzipped the zip file

unzip matomo.zip

I loaded the URL where Matoto was installed (e.g “https://fearby.com/folder/subfolder/matomo/”)

I received this well-crafted error.

Matomo File Permission Error

Raw Output

An error occurred
Matomo couldn't write to some directories (running as user 'www-usr').

Advertisement:





Try to Execute the following commands on your server, to allow Write access on these directories:

chown -R www-usr:www-usr /www-root/folder/subfolder/matomo
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/assets/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/cache/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/logs/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/tcpdf/
chmod -R 0755 /www-root/folder/subfolder/matomo/tmp/templates_c/

If this doesn't work, you can try to create the directories with your FTP software, and set the CHMOD to 0755 (or 0777 if 0755 is not enough). To do so with your FTP software, right click on the directories then click permissions.

After applying the modifications, you can refresh the page.

I refreshed the page after running the commands above on my site (via SSH)

Matomo Setup Step 1

A system check was performed. I installed when PHP 7.2.11 was the latest, PHP 7.2.12 or higher might be available. Follow my guide to update PHP on Ubuntu.

System Check

I had one Issue with Freetype not being installed.

Install Freetype

I solved this error by installing FreeType

sudo apt-get install freetype*

Output

Reading package lists... Done
Building dependency tree
Reading state information... Done
Note, selecting 'freetype-tools' for glob 'freetype*'
Note, selecting 'freetype2-demos' for glob 'freetype*'
The following NEW packages will be installed:
  freetype2-demos
0 upgraded, 1 newly installed, 0 to remove and 66 not upgraded.
Need to get 123 kB of archives.
After this operation, 728 kB of additional disk space will be used.
Get:1 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 freetype2-demos amd64 2.8.1-2ubuntu2 [123 kB]
Fetched 123 kB in 0s (965 kB/s)
Selecting previously unselected package freetype2-demos.
(Reading database ... 122574 files and directories currently installed.)
Preparing to unpack .../freetype2-demos_2.8.1-2ubuntu2_amd64.deb ...
Unpacking freetype2-demos (2.8.1-2ubuntu2) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up freetype2-demos (2.8.1-2ubuntu2) ...

Then I installed “php-gd”

sudo apt-get install php-gd

Output:

Reading package lists... Done
Building dependency tree
Reading state information... Done
The following additional packages will be installed:
libapache2-mod-php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-dev php7.2-fpm php7.2-gd php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml php7.2-zip
Recommended packages:
apache2
The following NEW packages will be installed:
php-gd php7.2-gd
The following packages will be upgraded:
libapache2-mod-php7.2 php7.2-cli php7.2-common php7.2-curl php7.2-dev php7.2-fpm php7.2-json php7.2-mbstring php7.2-mysql php7.2-opcache php7.2-readline php7.2-xml php7.2-zip
13 upgraded, 2 newly installed, 0 to remove and 53 not upgraded.
Need to get 33.2 kB/6621 kB of archives.
After this operation, 150 kB of additional disk space will be used.
Do you want to continue? [Y/n] Y
Get:1 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 php7.2-gd amd64 7.2.11-4+ubuntu18.04.1+deb.sury.org+1 [27.1 kB]
Get:2 http://ppa.launchpad.net/ondrej/php/ubuntu bionic/main amd64 php-gd all 2:7.2+68+ubuntu18.04.1+deb.sury.org+1 [6036 B]
Fetched 33.2 kB in 0s (75.9 kB/s)
Reading changelogs... Done
(Reading database ... 122597 files and directories currently installed.)
Preparing to unpack .../00-php7.2-mysql_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-mysql (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../01-php7.2-opcache_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-opcache (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../02-php7.2-json_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-json (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../03-php7.2-readline_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-readline (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../04-php7.2-mbstring_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-mbstring (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../05-php7.2-curl_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-curl (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../06-php7.2-zip_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-zip (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../07-php7.2-fpm_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-fpm (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../08-php7.2-xml_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-xml (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../09-php7.2-dev_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-dev (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../10-libapache2-mod-php7.2_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking libapache2-mod-php7.2 (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../11-php7.2-cli_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-cli (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Preparing to unpack .../12-php7.2-common_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-common (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) over (7.2.11-2+ubuntu18.04.1+deb.sury.org+1) ...
Selecting previously unselected package php7.2-gd.
Preparing to unpack .../13-php7.2-gd_7.2.11-4+ubuntu18.04.1+deb.sury.org+1_amd64.deb ...
Unpacking php7.2-gd (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Selecting previously unselected package php-gd.
Preparing to unpack .../14-php-gd_2%3a7.2+68+ubuntu18.04.1+deb.sury.org+1_all.deb ...
Unpacking php-gd (2:7.2+68+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-common (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Processing triggers for ureadahead (0.100.0-20) ...
Setting up php7.2-curl (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-mbstring (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-readline (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Processing triggers for systemd (237-3ubuntu10.4) ...
Processing triggers for man-db (2.8.3-2) ...
Setting up php7.2-json (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-opcache (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-mysql (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-gd (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...

Creating config file /etc/php/7.2/mods-available/gd.ini with new version
Setting up php7.2-xml (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-zip (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-cli (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php-gd (2:7.2+68+ubuntu18.04.1+deb.sury.org+1) ...
Setting up libapache2-mod-php7.2 (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Warning: Could not load Apache 2.4 maintainer script helper.
Setting up php7.2-dev (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...
Setting up php7.2-fpm (7.2.11-4+ubuntu18.04.1+deb.sury.org+1) ...

I refreshed the Matomo setup wizard page, Freetype is now installed 🙂

FreeType is installed

Database Settings

For the life of me, I could not get Matomo to talk to a database on another server so I set it up on my localhost.

I used this guide to help in mysql CLI to create the database and users.

Enter Matomo Database settings

Commands in mysql to create a database and user and assign the user to the database. If you are not comfortable with MySql CLI you can use Adminder GUI.

CREATE DATABASE tbdatabasename;
GRANT ALL PRIVILEGES ON tbdatabasename.* TO 'databaseuser'@'localhost' IDENTIFIED BY '#####################################';
GRANT SELECT ON tbdatabasename.* TO 'databaseuser'@'localhost';

I used this PHP code to test connecting to the dedicated server before using the localhost

<?php
$servername = "localhost";
$username = "databaseuser";
$password = "#################";
$dbname = "tbdatabasename";

// Create connection
$conn = new mysqli($servername, $username, $password, $dbname);

// Check connection
if ($conn->connect_error) {
    die("Connection failed: " . $conn->connect_error);
} else {
        echo "Connection Success";
}

$conn->close();
?>

Database created ok

Database OK

I created a Matomo user then I grabbed the javascript tracking ID code so I could paste this into WordPress.

Matomo Tracking ID

I opened my WordPress theme settings and deleted the Google tracking tags and added the Matomo tracking code.

Delete Google Tracking Tags

I added the Matomo tracking javascript in the head section.

The dashboard is up and collecting data.

Matomo Dashboard

Some reports are missing data so I will come back later.

After 1 week I could see data

Matomo is not collecting daya

Securing Mamoto

I read this guide here to secure Matomo

Opt Out Tracking

I enabled Opt Out Tracking in the Mamoto settings and added the generated opt-out code to my front page and at the bottom or all existing articles.

I had to allow iframe tags on my site by adding this header in NGINX (previously I blocked iframes)

add_header X-Frame-Options sameorigin

Add Opt Out Tracking Code to WordPress.

Matomo Opt Out Added to WordPRess widgets

I updated my privacy page and my GDPR notification bar. Now visitors will see a opt-out of tracking on the front page and all article pages.

Opt out of tracking enabled

SMTP Settings

I added my GSuite mail server settings to enable sending of reports via email. I loaded my old guide here to get the GSuite SMTP settings.

GSuite SMTP Settings Added

I enabled force https on the Mamoto application (edited: config/config.ini.php file)

[General]
...
force_ssl = 1

Matomo Plugins (Marketplace)

I opened the System then Plugins section of Matomo to open the Marketplace

Plugins

I installed these plugins

  • Force SSL
  • HidePasswordReset
  • Google Authenticator
  • Device Pixel Ratio
  • Bandwidth
  • Js Tracker Force Async
  • Treemap Visualization
  • Security Info
  • Custom Alerts
  • IP Reports
  • Live Tab
  • etc

Updating PHP

Matomo Admin (Panel – Security/Diagnostics) section will report if your PHP gets out of date.

Matomo warning of PHP being out of date

Hardening Advice

I enabled 2fA Authorisation at logins (Google Analytics Plugin).

Matomo 2fa Login screenshot

Read my guide here on hardware 2FA YubiCo YubiKeys here.

php.ini hardening changes

Matomo also recommended some php.ini file changes.

> open_basedir – open_basedir is disabled. When this is enabled, only files that are in the given directory/directories and their subdirectories can be read by PHP scripts. You should consider turning this on. Keep in mind that other web applications not written in PHP will not be restricted by this setting.

> upload_tmp_dir – upload_tmp_dir is disabled, or is set to a common world-writable directory. This typically allows other users on this server to access temporary copies of files uploaded via your PHP scripts. You should set upload_tmp_dir to a non-world-readable directory

This may break your WordPress so enable at your own risk. I might move Mamoto to a dedicated “analytics” subdomain then enable these options.

Troubleshooting

I had to run this command when installing Device Pixel Ratio, Device Network Information, Bandwidth plugins

php /www-root/path/matomo/console core:update

Output:

    *** Update ***

    Database Upgrade Required

    Your Matomo database is out-of-date, and must be upgraded before you can continue.

    The following dimensions will be updated: log_visit.device_pixel_ratio.

    *** Note: this is a Dry Run ***

    ALTER TABLE `matomo_log_visit` ADD COLUMN `device_pixel_ratio` DECIMAL(5,2) DEFAULT NULL;

    *** End of Dry Run ***

A database upgrade is required. Execute update? (y/N) y

Starting the database upgrade process now. This may take a while, so please be patient.

    *** Update ***

    Database Upgrade Required

    Your Matomo database is out-of-date, and must be upgraded before you can continue.

    The following dimensions will be updated: log_visit.device_pixel_ratio.

    The database upgrade process may take a while, so please be patient.

  Executing ALTER TABLE `matomo_log_visit` ADD COLUMN `device_pixel_ratio` DECIMAL(5,2) DEFAULT NULL;... Done. [1 / 1]

Matomo has been successfully updated!

GTMetrix (After)

GT Metrix reports that my site is not slower (still 1.5 seconds)

GTMetrix After Pagespeed

I can see that some JavaScript is not being picked up by CDN.

GTMetrix After YSlow

Also 2 More files loading (when compared to Google Analytics)

2 More Files

Time to add the Mamoto files to my CDN.

Adding Matomo Resources to a CDN

I read this Matomo forum post.

I copied these 2 assets to my WordPress wp-content folder (my WordPress CDN ewww.io will then upload them to the CDN).

cd /www-root/wp-content/
cp /www-root/utils/matomo/piwik.js ./piwik.js
cp /www-root/utils/matomo/plugins/CoreAdminHome/javascripts/optOut.js ./optOut.js
chown www-data:www-data *.js

I have cache everything enabled in ewww.io and this will copy the javascript assets ot my CDN.  I will need to manually update these js files each time a Matomo update is installed.

I change my Matomo tracker code to include the new CDN location

<!-- Matomo -->
<script type="text/javascript">
  var _paq = _paq || [];
  /* tracker methods like "setCustomDimension" should be called before "trackPageView" */
  _paq.push(['trackPageView']);
  _paq.push(['enableLinkTracking']);
  (function() {
    var u="//fearby.com/utils/matomo/";
    _paq.push(['setTrackerUrl', u+'piwik.php']);
    _paq.push(['setSiteId', '1']);
    var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
    g.type='text/javascript'; g.async=true; g.defer=true; g.src='https://fearby-com.exactdn.com/wp-content/piwik.js'; s.parentNode.insertBefore(g,s);
  })();
</script>
<!-- End Matomo Code -->

I could not find out how to change the location of my (now CDN cached https://fearby-com.exactdn.com/wp-content/optOut.js) so I temporarily disabled the opt-out form on my front page.

todo: Find out how to change the CDN location of optOut.js and re-enabled the form.

All assets are loading from CDN.

GT Metrix shows my site loads in 1.4 seconds

Analytics Reporting

Graphs are not as pretty as Google Analytics but they are working.

Matomo is not collecting daya

Mobile Reporting

Mobile reporting is good too.

Screenshot of the Matomo Mobile app

Updating Matomo Plugins

Don’t forget to update your plugins from the Matomo dashboard.

Updating Matomo (Core)

Matomo has an official guide on how to update Matomo here.

I do not have FTP so I will perform the manual three step update.

But before I do that I will manually backup my web server and database server just in case.

I backed up my Matomo config (I SSH”ed to the server)

$ cd /www-root/matomo-root/

$ cp ./config.ini.php ./config.ini.3.x.x.php

I navigated to the folder above my Matomo folder

$ cd ..

$ cd ..

I downloaded Matomo

$ wget https://builds.matomo.org/matomo.zip

I unzipped the zip file

$ unzip -o matomo.zip

I removed the matomo.zip file

$ rm matomo.zip

I loaded the Matomo Login page again and was prompted to update the database.

Matomo Database Update Required

Matomo reported it was updated Successfully.

Matomo was updated message

Oops, and error in config error appeared when I tried to log in.

Matomo Error in config

Oh, Do I need to replace the config file with my backed up config file?

(edit: Yes Matomo say to do this, my bad)

Ten seconds later I accidentally deleted all my config files (I had zero backups), the next 2 minutes were spent shutting down my servers (web and db) and restoring them from backup. Thank goodness UpCloud are awesome hosts.

I now had to restore my servers and repeat the steps but this time restore my config file before logging back in.

I did this but had the same error

> An error occurred
> Authentication object cannot be found in the container. Maybe the Login plugin is not activated?
> You can activate the plugin by adding:
> Plugins[] = Login under the [Plugins] section in your config/config.ini.php

I checked my replaced config.ini.php and it did have

> [PluginsInstalled]
> PluginsInstalled[] = “Login”

I googled and found this page that said reset your password (this was not an option as Matomo was not loading)

I logged into mysql with my Matomo user

> mysql -u matomodbusername -p
> Enter password:
> Welcome to the MySQL monitor. Commands end with ; or \g.
> Server version: 5.7.xxxx

> Copyright (c) 2000, 2019, Oracle and/or its affiliates. All rights reserved.

> Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

> Type ‘help;’ or ‘\h’ for help. Type ‘\c’ to clear the current input statement.

> mysql> show databases;
+——————–+
| Database |
+——————–+
| information_schema |
| matomodb |
+——————–+
> 2 rows in set (0.00 sec)

The account and database seem ok.

I tried “FLUSH PRIVILEGES;” with no luck

I tried to sop mysql but it was locked

It was late so I rebooted my server (it did not come back up after a few minutes, I forced a reboot)

I still had an “Authentication object cannot be found in the container.” error when trying to login to Matomo???

I re-checked the “config.ini.php” file after reding threads at the Matomo Forums

$ sudo nano /www-root/matomo-root/config.ini.php

“Plugins[] = “Login”” was not in the “[Plugins]” area of the file???  I added it, saved the change and was able to reload the Matomo GUI.

I checked some key reports.

Visitors over time:

Visitors over time report

Visitor Location Map

Visitor Location Map

Visitor Overview

Visitor Overview

Out links Clicked

Out links Clicked

Nice

I subscribed to the Matomo newsletter here to keep up to date with Matomo update releases: https://matomo.org/newsletter/

Good luck and I hope this guide helps someone

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.2 Hardening info

v1.1 Updating Matomo

v1.0 Initial post

Filed Under: Analytics, Cloud, Free, Privacy Tagged With: a, analytics, focused, for, google, hosted, locally, Matomo, Open, Piwik, privacy, Replacing, solution, source, with

Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins

July 23, 2018 by Simon

This is a quick post that shows how I upgraded to Wordfence Premium to get real-time defence feeds, malware scanner and two-factor authentication for WordPress logins

Aside

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name here

Domain names for just 88 cents!

Now on with the post.

What is Wordfence

WordFence is a free WordPress plugin (install guide here) that helps protect your WordPress site by logging and blocking bad events.  I was a big fan of the Wordfence sister program called GravityScan (before it was retired)

Read my review of the free Wordfence plugin here.

I was using Wordfence free to

  • Whitelist logins for known IP’s (read my guide on whitelisting IPs here)
  • Block known bad IPs from the Wordfence global network (but with a 30-day delay)
  • Create a firewall
  • Rate limiting page requests
  • Scan my site for malware
  • Ability to see past failed logins (and ban them)
  • Ability to block/ban users who try and login form new IP’s
  • Force strong WordPress account passwords
  • Set ban thresholds
  • Have I Been Pwned breached password checks
  • Much more

Install and set up Wordfence (Free)

Read my guide here to learn how to setup Wordfence (Free).

Malware Infections

Your website is often scanned and ranked for safety by sites like Norton Safe Web, Google, Trend Micro, Kaspersky Virus Desk, SiteGuarding etc along with search engines. Having malicious files on your site will affect your site Search EnginOptimizationio (SEO).

I had a 5-year-old scan of a subdomain (that was hosted on a CPanel Host). The subdomain had false positives for malware.

Screenshot of a scan of https://sitecheck.sucuri.net/

Working to remove the false positive was a lengthy process.

Clean Site Scan Results

You should aim to stay off the radar or many site scanning, check VirusTotal often to keep your self-updated as to the status of your website. Wordfence will hopefully detect real malware issues automatically in the future.

Screenshot of https://www.virustotal.com/

https://sitecheck.sucuri.net/ is a good site that can aggregate your sites safety ratings.

WordfFence Free v Premium 

Wordfence Premium

ScreenShot showing https://www.wordfence.com/#get-plugin

Prices (USD)

Wordfence Prices

WordFence Premium

Read about some benefits of Wordfence Premium here.

  • Real-time firewall rules and malware signatures
  • Global Wordfence premium IP blacklist
  • Priority server processing for premium customers
  • Two Factor Authentication (only if you don’t use whitelisting I found out)

Read more about getting the most from Wordfence Premium

Buying a Wordfence Premium API Key

  1. Login to https://www.wordfence.com/dashboard/
  2. Click Buy More API Keys
  3. Enter your Payment Details

>Thanks, your card information has been updated. You can now go to your API Key Manager and create and manage your Wordfence API keys.

Now you can buy an API key and copy and paste the API ey o to your Wordfence plugin.

Wordfence Firewall

Wordfence does a great job at showing failed/successful, top blocked IP’s

Wordfence Firewall Stats

Wordfence Malware Scanner

Wordfence premium has schedulable scans with real-time malware signatures

Schrrnshot of Wordfence scan scheduler

Scan Progress

Malware Scanner

Testing the scanner

Wordfence says “A Wordfence scan examines all files on your WordPress website looking for malicious code, backdoors, shells that hackers have installed, known malicious URLs and known patterns of infections.”

I created an eicar.txt test file (information on eicar here (slightly modified so I don’t get tagged again b virus scanners)) to test the Wordfence malware scanner

echo 'X5O!P#removed#X54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' > /www-root/eicar.txt
sudo chown www-data:www-data eicar.txt

I enable scanning of files outside of WordPress

eicar test file

I rescanned my site with Wordfence

Result: Nothing??

Wordfence clean results

I logged a support ticket to see if this is right?

Update: Wordfence support replied and said “Thanks for writing in. We do detect the EICAR test file, but scans don’t scan file types that aren’t dangerous on a site by default, since scans would waste a lot of time on files that aren’t exploitable.“

I disagree a virus is a virus.

Wordfence says “A Wordfence scan examines all files on your WordPress website looking for malicious code, backdoors, shells that hackers have installed, known malicious URLs and known patterns of infections.”

wordfence stating is scans all filesI guess “all” does not mean “all”?

Wordfence support said EICAR files are detected if I rename the file to php. I renamed the file and to enabled “Scan images, binary, and other files as if they were executable“.

EICAR needs to be a php file

I started a new scan

> Scan Failed
>The scan has failed because we received an unexpected response from the Wordfence servers. This may be a temporary error, though some sites may need adjustments to run scans reliably

🙁

I scanned my system with ClamAV and it found the EICAR file.

clamscan -r --bell -i /www-root

Result:

/www-root/eicar.txt: Eicar-Test-Signature FOUND

ClamAV found the virus.

Setting up Two Factor Authentication (work in progress)

Add your desired user and number

Enable Two Factor Authentication

Click Enable User

Wait for the text message and activation code (on your phone)

Enter the activation code and press Activate

Activate TFA

The two-factor authentication should be activated

Enabled TFA

List of two-factor authorization enabled users.

List of TFA enabled users

I logged out of WordPress and logged back in but the two-factor auth did not work, I logged a support Ticket with my theme maker and WordFence.

Update: Wordfence Support “Wordfence > Tools > Two Factor Authentication options there is an option for Enable Separate Prompt for Two Factor Code which you could disable and try.“

This fix did not work.  I sent a 2nd diagnostics report to Wordfence.

Wordfence support said

>When our two-factor authentication feature allows you to login bypassing the need to enter the authentication code it is typically because of these possible reasons:
> 1)  The user has whitelisted their IP address in the advanced firewall option “Whitelisted IP addresses that bypass all rules“.
>2)  Another plugin, or possibly a theme, that creates non-standard WordPress behaviour such as user role and capabilities modification, or that modifies the login flow process in some way.
It appears my IP whitelist was disabling the two-factor auth feature 🙁
I’d rather keep the two-factor auth along with keeping the whitelist (just in case my whitelist IP is known and used).
Refund
I asked Wordfence for a refund (given)
Conclusion

Pros

  • Protects and blocks bad logins
  • Real-time blocked IP and malware feeds

Cons

  • Almost $140 Australian dollars a year
  • A scan does not detect eicar.txt test virus files (ticked logged), renamed to eicar.php and still no luck.
  • Two-factor auth (authenticator and SMS) does not work (ticket logged)
  • Wordfence support resolve/close support tickets with no confirmation from the user.
  • Two Factor Auth is disabled if you whitelist IPs 🙁

Is Premium worth it?  Yes if you want “Real-time firewall rules and malware signatures” (and don’t whitelist your IP).

I hope this guide helps someone.

Please consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.

https://www.upcloud.com/register/?promo=D84793

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.4 Updated conclusion and Wordfence refund

v1.3 added whitelist 2FA info

v1.2 added replied from Wordfence support re EICAR and Two Factor Auth.

v1.1 Added Pros and Cons section

v1.0 Initial Post

Filed Under: Firewall, Security, Wordfence, Wordpress, WP Security Tagged With: -time, and, authentication, defence, for, get, login, logins, malware, Premium, real, Scanner, to, two-factor, Upgrading, Wordfence, wordpress

Privacy, General Data Protection Regulation (GDPR) information for WordPress bloggers.

May 25, 2018 by Simon

This is a short post with General Privacy, Data Protection Regulation (GDPR) information for WordPress bloggers.

Note: This is not legal advice, just late minute information on current opinions and facts around GDPR.

fyi: Read my guide on the awesome UpCloud VM hosts (get $25 free credit by signing up here).

Facebook, Google, Whatsapp and Instagram are facing lawsuits for failing to comply with GDPR, Europe’s sweeping new data protection law.

Facebook, Google, Whatsapp and Instagram are facing lawsuits for failing to comply with GDPR, Europe’s sweeping new data protection law https://t.co/o7FyX0fspI

— CNN (@CNN) May 25, 2018

It is GPRD Compliance Eve and there are loads of last-minute GDPR activity.

Official European Commission resources on GRDP

What are your new #dataprotection rights? What is the right to be forgotten?
Our official website provides you with more information → https://t.co/h0rqJaHqJt #GDPR pic.twitter.com/VLhWzOUzR6

— European Commission ?? (@EU_Commission) May 25, 2018

Some US News sites are blocking Europeans

GDPR: US news sites blocked to EU users over data protection rules https://t.co/G0g5U0eqM1

— BBC Technology (@BBCTech) May 25, 2018

Legal Teams are up late

shout out to the legal teams pushing their GDPR-driven privacy policy updates out at the last minute pic.twitter.com/afSAT2egyf

— Patrick Donahue (@prdonahue) May 25, 2018

First Lawsuits are filed

Under #GDPR, Schrems files legal cases worth €7bn against Facebook https://t.co/eQtbptLl09

— Irish Times Business (@IrishTimesBiz) May 25, 2018

Should you panic?

No.

If you want a good summary for GDPR for bloggers – does it apply to you and how to comply then read this.

Also, Wikipedia has a good article.

wpbeginner.com has an Ultimate Guide to WordPress and GDPR Compliance – Everything You Need to Know

Read wpbeginners.com’s summary of what GDPR is?

The General Data Protection Regulation (GDPR) is a European Union (EU) law taking effect on May 25, 2018. The goal of GDPR is to give EU citizens control over their personal data and change the data privacy approach of organizations across the world.

Are there fines?

Basically after May 25th, 2018, businesses that are not in compliance with GDPR’s requirement can face large fines up to 4% of a company’s annual global revenue OR €20 million (whichever is greater). This is enough reason to cause wide-spread panic among businesses around the world.

First, there will be warnings, then reprimands then Suspension then Fines and more.

Does GDPR apply to my WordPress site?

The answer is YES. It applies to every business, large and small, around the world (not just in the European Union).

If your website has visitors from European Union countries, then this law applies to you.

But don’t panic, this isn’t the end of the world.

While GDPR has the potential to escalate to those high level of fines, it will start with a warning, then a reprimand, then a suspension of data processing, and if you continue to violate the law, then the large fines will hit.

Read more at Ultimate Guide to WordPress and GDPR Compliance – Everything You Need to Know

But warning are issued before fines are given.

What can you do?

  • Check your web server hosts GDPR Compliance (my blog host is Vultr, their GDPR compliance summary on their blog, Vultr data processing guide).  Read my guide on setting up a server on Vultr here.
  • Know your mail servers GDPR status – I use  GSuite  (e.g Google GDPR Reference Center, Whitepaper and Resource Center).
  • Ubuntu GDPR Auditing and compliance information (e.g Purging old data, use strong passwords, be accountable, perform audits (with Lynis, Qualsys and Zap), running virus scanners, use secure protocols and security (like TLS 1.3)).
  • Securing and protecting users private data (e.g using SPF, DKIM and DMARC on your mail server).
  • Review the Google AdSense Compiance Information (If you are using  AdSense )
  • Read WordPress Core GDPR v4.9.6 changes
  • Search each of your WordPress plugins and see what you need to be aware of in relation to GDPR.
  • Review Mailchimp GDPR data.
  • Cookie Consent (I use GDPR Cookie Consent Plugin).
  • etc.

But the takeaway is, don’t create a website (then be lazy) and abuse users private data or be lazy with security.

My blog hosts (Vultr) GDPR information

Vultr GDPR

I instaled a GDPR Cookie Consent WordPress Plugin

I used the WP-CLI plugin install GDPR Cookie plugin for the command line. View the developer site here.

# Visited the WP Plugin page and got the URL for the latest plugin version
# https://wordpress.org/plugins/cookie-law-info/
# Connect to my server via SSH
cd /www-root
cd wp-content/plugins/
wget https://downloads.wordpress.org/plugin/cookie-law-info.1.5.5.zip
unzip cookie-law-info.1.5.5.zip
unzip -r cookie-law-info.1.5.5.zip
rm -R cookie-law-info.1.5.5.zip

I then activated the plugin and configured it.

Configure COokie Concent Plugin

Cookie bot also has a great page on GDRP here.

I edited the following Privacy/GDRP placeholder files.

cd /www-root
# Made a reject cookies placeholder
sudo nano rejectcookies.html

# Made a privacy placeholder
sudo nano privacy.html

I should have skipped creating a privacy.htm page as WordPress v4.9.6 has a Privacy Page Generator. Nice

Create Privacy Page

Goto tour sites Dashboard, click Settings then Privacy. Create a new page, fill in the blanks and publish it.

Privacy Page Generator

I read MailChimp GDPR Advice as I had a few lists with private data

  • General Data Protection Regulation FAQs: http://eepurl.com/dufEZb
  • About MailChimp, the EU/Swiss Privacy Shield, and the GDPR: http://eepurl.com/c567FL

More to come. Let’s get back to those GDPR emails

Trying to delete #GDPR emails like… pic.twitter.com/eZpqSS2OVF

— H3roes&Vi1lains (@H3roesVi1lains) May 25, 2018

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Cookie Bot GDPR Link

v1.0 Initial post

Filed Under: GDPR, Privacy Tagged With: bloggers privacy, data, for, GDPR, General, information, Protection, Regulation, wordpress

Using the Qualys FreeScan Scanner to test your website for online vulnerabilities

March 23, 2018 by Simon

It is possible to deploy a server in minutes to hours but it can take days to secure.  What tools can you use to help identify what to secure on your website?

I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line, installing a Free SSL certificate and setting up SSL security.

Security Tools

  • https://asafaweb.com/ is a good tool for quick scanning
  • Kali Linux has a number of security tools you can use.
  • You can run a system audit Lynis Audit.
  • Checking your site for vulnerabilities with Zap.
  • Run a Gravity Scan malware and supply chain scan
  • Use Qualys SSL scan to test your SSL certificate: https://www.ssllabs.com/ssltest/

Qualys

Qualys SSL Labs SSL Tester is the best tool for checking an SSL certificate strength

Most people don’t know Qualys also has another free (limited to 10 scans) vulnerability scanner for websites.

Goto https://freescan.qualys.com/ and click Start your free account.

Complete the signup form

Now check your email to login and confirm your email account

Login now from the email.

Create a password (why the 25 char max Qualys?)

Enter your website URL and click Scan

The scan can take hours

While the scan was being performed I noticed that Qualys offers alerts (I’ll check this out later): https://www.qualys.com/research/security-alerts/

Yes, the scan can take hours, take a walk or read other posts here.

The scan is almost complete

Yay, my latest scan revealed 0 High, 0 Medium and 0 Low-risk vulnerabilities.

It did report 23 informational alerts like “Firewall Detected“.

Threat Report Results

Patch Report Results

This report was empty (probably because I don’t run Windows)

Threat Report Results

The OWASP report contained partial scan results (maybe the full report is available to pro users)

Previous Scan Results

The Qualys dashboard will show all past scans.

My first scan showed a Low priority issue with the /wp-login.php page as the input fields did not have “autocomplete=”off””, I fixed this by adding “autocomplete=”off”” the removing the page (safer).

The second scan found two issues with cookies (possibly ad banner cookies) and 2 subfolders that I created in past development exercises. I deleted the two sub-folders that were not needed.

The third scan was clean.

Here is a scan of a static website of a friends server (static can be less secure if the server underneath is old or unpatched).

Static Website

Happy scanning. I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.1 Static Web Server Scan

v1.0 Initial post

Filed Under: Firewall, LetsEncrypt, Linux, Malware, Security, Server, Ubuntu, Vulnerabilities, Vulnerability, WP Security Tagged With: for, FreeScan, online, Qualys, Scanner, test, the, to, Using, Vulnerabilities, website, your

Using OWASP ZAP GUI to scan your Applications for security issues

March 17, 2018 by Simon

OWASP is a non-profit that lists the Top Ten Most Critical Web Application Security Risks, they also have a GUI Java tool called OWASP Zap that you can use to check your apps for security issue.

I have a number of guides on moving hosting away form CPanel , Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. It is important that you always update your site and software and test your sites and software for vulnerabilities. Zap is free and completely open source.

Disclaimer, I am not an expert (this Zap post and my past Kali Linux guide will be updated as I learn more).

OWASP Top 10

OWASP has a top 10 list of things to review.

OWASP Top 10

Download the OWASP 10 10 Application security risks PDF here form here.

Using the free OWASP Zap Tool

Snip from https://www.owasp.org/index.php/OWASP_Zed_Attack_Proxy_Project

“The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by hundreds of international volunteers*. It can help you automatically find security vulnerabilities in your web applications while you are developing and testing your applications. It’s also a great tool for experienced pentesters to use for manual security testing.”

Zap Overview

Here is a quick demo of Zap in action.

Do check out the official Zap videos on youtube: https://www.youtube.com/user/OWASPGLOBAL/videos if you want to learn more.

Installing Zap

Download Zap from here.

Download Zap

Download Options

Download

Download contents

Run Install

Copy to the app to the OSX Application folder

Installing

App Installed

App Insatalled

Open OSX’s Privacy and Security screen and click Open Anyway

Open Anwway

OWASP Zap is now Installed

Insallled

Ready for a Scan

Blind Scan

But before we do let’s check out the Options

Options

OWASP Zap allows you to label reports to ad from anyone you want.

Report Label Options

Now let’s update the program and plugins, Click Manage Add-ons

Manage Adons

Click Update All to Update addons

Updates

I clicked Update All

Plugins

Installed some plugins

Marketplace

Zap is Ready

Zap

Add a site and right click on the site and you can perform an active scan or port scan.

Right click Zap

First Scan (https failed)

https failed

I enabled unsafe SSL/TLS Renegotiation.

Allow Unsafe HTTPS

This did not work and this guide said I needed to install the “Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files” from here.

Cryptography Files OSX

The extract files to /Library/Java/JavaVirtualMachines/%your_jdk%/Contents/Home/jre/lib/security

Extract

I restarted OWASP Zap and tried to scan my site buy it appears Cloudflare (that I recently set up) was blocking my scans and reported error 403. I decided to scan another site of mine that was not on Cloudflare but had the same Lets Encrypt style SSL cert.

fyi: I own and set up the site I queried below.

Zap Results

OWASP Zap scan performed over 800 requests and tried traversal exploits and many other checks. Do repair any major failures you find.

Zan Scan

Generating a Report

To generate a report click Report then the appropriate generation menu of choice.

Generate Report

FYI: The High Priority Alert is a false positive with an HTML item being mistaken for a CC number.

I hope this guide helps someone. Happy software/server hardening and good luck.

More Reading

Check out my Kali Linux guide.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V1.3 fixed hasting typo.

v1.2 False Positive

v1.1 updated main features

v1.0 Initial post

Filed Under: Cloud, Cloudflare, Code, DNS, Exploit, Firewall, LetsEncrypt, MySQL, owasp, Secure, Security, ssl, Ubuntu Tagged With: Applications, for, gui, issues, OWASP, scan, security, to, Using, your, ZAP

Using Fritzing to draw electronic schematics for Arduino, Raspberry Pi and ESP8266

February 10, 2018 by Simon

This guide will show how you can create an electronics schematic to represent elements of a circuit for Arduino, Raspberry Pi and ESP8266 micro-controllers.

Wikipedia states here: “A schematic, or schematic diagram, is a representation of the elements of a system using abstract, graphic symbols rather than realistic pictures. A schematic usually omits all details that are not relevant to the information the schematic is intended to convey and may add unrealistic elements that aid comprehension.” This guide will help you create schematics using Fritzing on OSX, you may still want to document your system requirements etc in Github or Bitbucket.

I am creating some small personal weather station on Arduino, Raspberry Pi and ESP8266 to submit data to cloud servers on Vutr (or AWS or Digital Ocean), read my guide on using Adminer to create and manage MySQL databases.

What are sensors

A sensor could detect temperature, humidity, barometric pressure, light and just about anything else.

Sensor

Temperature, Humidity and Barometric sensor (bme280)

GY-BME280

Most sensors work on low voltage have analogue or digital outputs (I2C or SPI) and require minimal wires. The real problem is having multiple sensors and wiring gets out of hand.

What is Arduino

Arduino is a low power 8bit and 32bit hardware/software product that you can rapidly wire up circuits and sensors. Sites like Adafruit sell loads of sensors and develop a lot of software libraries to drive sensors.

Non-genuine Arduino boards age mega cheap on eBay.

Arduino has slide-on shields that can be pushed onto an Arduino board to expand them.

Pros

  • Cheap
  • Loads of support
  • Good analogue sensor support

Cons

  • Limited monitor support

Sensors or expansions can come in shields or be connected to pins with wires.

What is Raspberry Pi

Raspberry Pi is also a single board computer but with a more powerful ARM processor that runs a Linux operating system from an SD card. Raspberry Pi’s are essentially small desktop computers. Raspberry Pi’s have USB, can plug into a monitor, have a bootable desktop have Arduino like GPIO pins that can talk to sensors (but not as many analogue sensors).

Pros

  • Very Powerful.
  • Desktop Operating System included.
  • Can talk to sensors.
  • Good GPIO pins for controlling the real world.
  • Has HDMI monitor output.

Cons

  • Full kits are expensive.
  • Low analogue read pins.
  • Failure to shut down correctly can corrupt the OS.

Pi

What is ESP8266

Pros

  • Cheap
  • Good Analog and digital puns
  • Code in Arduino IDE
  • Has WiFi

Cons

  • Limited monitor support

ss

Installing Fritzing

Fritzing is a software package that can allow you to design and learn electronic schematics (filter by kid level, amateurs, master or higher).

Goto: http://fritzing.org/

Fritzing site

Click Download and Donate

Download Fritzing

Click Download and choose the right version for your Operating System

Download

Copy the app to your Application folder

Copy the app to your Application folder

Open the app a few times (open, wait, proceed, close, open wait proceed etc) to ensure parts updates are installed.

Open the app a few times to instal updates

Fritzing Introduction on YouTube

Watch the Introduction video for Fritzing here

Fritzing Killer Tips Series on YouTube

Watch Fritzing Killer Tips 001 The generic IC

Extending Fritzing

Fritzing allows you to import new parts and design your own parts or ask Fritzing to design a part for you.

Import

I was able to import a bme280 sensor within seconds.

bme280

If a part does not exist in the forums search google for “partname” and “.fzpz”

I was able to find an ESP8266 Node MCU from https://github.com/squix78/esp8266-fritzing-parts/tree/master/nodemcu-v1.0

NodeMCUV1.0

It looks like Fritzing has all the parts I need to work with (Pi, Arduino and Node MCU’s).

All Parts

Fritzing Views

Coming soon (Views: Breadboard, schematic, PCB, code).

Designing and Ordering a PCB

Coming soon.

Find Fritzing Projects

You can find all community-created Raspberry Pi Fritzing Projects below.

http://fritzing.org/projects/by-tag/raspberry%20pi/

Arduino Fritzing Projets:

http://fritzing.org/projects/by-tag/arduino/

ESP8266 Fritzing Projets:

http://fritzing.org/projects/by-tag/esp8266/

Happy coding and I hope this helps someone.

More to come.

Making a PCB

Bonus: It is possible to order your own physical PCB from say
https://easyeda.com but you will need to export your schematic from Fritzing to Eagle (or manually create your PCB in Easy EDA online IDE).

Download Eagle:
https://www.autodesk.com/products/eagle/free-download

Easy EDA guides here:

How to make a custom PCB in Easy EDA (Part 1):

How to make a custom PCB in Easy EDA (Part 2):

I will try and find a PCB manufacturer that accepts orders from Fritzing exports.

More Reading

https://www.raspberrypi.org/magpi/

Free MagPi magazine Issues.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V1.1 Making a PCB info

v1.0 Initial Draft

Filed Under: Schematics Tagged With: and, Arduino, draw, electronic, ESP8266, for, Fritzing, raspberry pi, schematic, to, Using

Using the free Adminer GUI for MySQL on your website

February 8, 2018 by Simon

Adminer is a free GUI tool that can you can easily install on a PHP web server. Adminer allows you to easily connect to your MySQL instance, create databases/tables/indexes/rows and backup/import databases and much more.

You can read my other posts on Useful Linux Terminal Commands and Useful OSX Terminal Commands.

I used to use phpMyAdmin to manage MySQL databases on AWS, Digital Ocean and Vultr but switched to Adminer due to forgotten issues.  You can always manage MySQL via command line but that is quite boring.

The below screenshots were taken on my local Development Mac Laptop (with optional OSX Apache SSL Setup (that reports “Not Secure” (but it is good enough to use locally)). I prefer to code in SSL and warn when SSL is not detected.

Downloading and Installing Adminer

Navigate to https://www.adminer.org/ and click Download.

Adminer GUIClick English only (.php file)

Adminer

Save the Adminder for MySQL (.php) file to your web server and give it a random name and put in a folder also with a random name (I use https://www.grc.com/passwords.htm to generate strong password).

Tip: Uploading this file to a live serve offers hackers and unauthorized people potential access to your MySQL server.  I would remove this file from live serves when you are not using it not to be sure.

Tip: Read my guide here on setting up NGINX, MySQL and PHP here.  Basically, I did this to setup MySQL on Ubuntu 16.04.

sudo apt-get install mysql-common
sudo apt-get install mysql-server
mysql --version
>mysql Ver 14.14 Distrib 5.7.19, for Linux (x86_64) using EditLine wrapper
sudo mysql_secure_installation
>Y (Valitate plugin)
>2 (Strong passwords)
>N (Don't chnage root password)
>Y (Remove anon accounts)
>Y (No remote root login)
>Y (Remove test DB)
>Y (Reload)
service mysql status
> mysql.service - MySQL Community Server

TIP: Ensure MySQL is secure and has a good root password, also consider setting up Ubuntu Firewalls and Securing Ubuntu. Also, ensure the Server is patched and does not have exploits like Spectre and meltdown.

Now you can access your Admirer php file on your Web Server (hopefully with an obfuscated name).

Randomize

Login to Adminer with your MySQL root password.

Login

Click Create databaase

Create Database

Give the database a name and choose the character coding standard (e.g UTF8 general ci). Different standards have different performance impacts too.

Save

Now that you have a database you can create a table.

Adminer

Consider adding an auto-incrementing ID and say a Key and Value varchar column.

Adminer

When the table is created you can add a row to the table.

Adminer

I created one with a “TestKey” and “TestValue” row.

Adminer

The row was inserted.

Adminer

The final thing to do is add a database user that code can connect to the database with. Click Privileges.

Adminer

Click Create user

Adminer

Tick All privileges and click Save

Adminer

Now the user is added to the database

Adminer

Let’s create a PHP file and talk to the database. Let’s use parameterized queries

<?php

date_default_timezone_set('Australia/Sydney');
echo "Last modified: " . date ("F d Y H:i:s.", getlastmod()) . "<br /><br />";

// Turn on if you need to see errors
// error_reporting(E_ALL);
// ini_set('display_errors', 0);

$dbhost = '127.0.0.1';
$dbname = 'dbtest';
$dbusername = 'dbtestuser';
$dbpassword = '*****************************************'';

$con = mysqli_connect($dbhost, $dbusername, $dbpassword, $dbname);
 
// Turn on debug stuff if you need it
// echo var_dump($con);
// printf(" - Error: %s.n", $stmt->error);
 
if($con->connect_errno > 0){

    printf(" - Error: %s.n", $stmt->error);
    die("Error: Unable to connect to MySQL");

} else {

    echo "Charset set to utf8<br />";
    mysqli_set_charset($con,"utf8");
}
 
if (!$con) {

    echo "Error: Unable to connect to MySQL (E002)" . PHP_EOL;
    echo "Debugging errno: " . mysqli_connect_errno() . PHP_EOL;
    echo "Debugging error: " . mysqli_connect_error() . PHP_EOL;
    exit;

} else {

    echo "Database Connection OK<br />";
 
    echo "&nbsp; Success: A proper connection to MySQL was made! The $dbname database is great." . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Host information: " . mysqli_get_host_info($con) . PHP_EOL . "<br />";
    echo "&nbsp; &nbsp;- Server Info: '" . mysqli_get_server_info($con) . "'<br />";
    echo "&nbsp; &nbsp;- Server Protocol Info : ". mysqli_get_proto_info($con) . "<br />";
    echo "&nbsp; &nbsp;- Server Version: " . mysqli_get_server_version($con) . "<br />";
    //echo " - Server Connection Stats: " . print_r(vmysqli_get_connection_stats($con)) . "<br />";
    echo "&nbsp; &nbsp;- Client Version: " . mysqli_get_client_version($con) . "<br />";
    echo "&nbsp; &nbsp;- Client Info: '" . mysqli_get_client_info() . "'<br />";
 
    echo "Ready to Query the database '$dbname'.<br />";
 
    // Input Var's that are parameterized/bound into the query statement
    $in_key = mysqli_real_escape_string($con, 'TestKey');
 
    // Output Var's that the query fills after querying the database
    // These variables will be filled with data from the current returned row
    $out_id = "";
    $out_key = "";
    $out_value = "";
 
    echo "1. About to query the database: '$dbname'<br />";
    $stmt = mysqli_stmt_init($con);

    $sql = "SELECT testid, testkey, testvalue FROM tbtest WHERE testkey = ?";
    echo "SQL: $sql (In = $in_key)<br /";

    if (mysqli_stmt_prepare($stmt, $sql)) {

            echo "2. Query Returned<br />";
            /*
                Type specification chars
                Character   Description
                i   corresponding variable has type integer
                d   corresponding variable has type double
                s   corresponding variable has type string
                b   corresponding variable is a blob and will be sent in packets
            */
            mysqli_stmt_bind_param($stmt, 's', $in_key);
            mysqli_stmt_execute($stmt);
            mysqli_stmt_bind_result($stmt, $out_id, $out_key, $out_value);
            mysqli_stmt_fetch($stmt);
     
            // Do something with the 1st returned row        
            echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//

            // Do we have more rows to process
            while($stmt->fetch()) { 
                
                    // Output returned values
                    echo " - Row: ID: $out_id, KEY: $out_key, VAL: $out_value <br />";//
            
            }
            mysqli_stmt_close($stmt);
            
            echo "Done<br />";
        
        } else {
        
            echo "3. Error Querying<br/>";
            printf(" - Error: %s.n", $stmt->error);
        
        }
}    
?>

Result

Adminer Results

If you don’t have a server check out my guides on AWS, Digital Ocean and Vultr.

Happy coding and I hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial Version

Filed Under: MySQLGUI Tagged With: Adminer, for, free, gui, MySQL, on, the, Using, website, your

Scanning WordPress with Gravity scan for free to detect Supply Chain Attacks and WordPress malware

January 5, 2018 by Simon

A recent trend with some WordPress Plugins (and Google Chrome Extensions) is malicious parties will purchase existing plugins (extensions) and inject malicious code into new versions to infect sites and software, this is called “Supply Chain Attacks”. This is a personal unpaid review of Gravity Scan.

Update Feb 2018: Gravity Scan is shutting down 🙁

Recently WordFence wrote a blog post about Supply Chain Attacks found cases where older plugins are being purchased by malicious people in order to infect WordPress sites. WordPress CMS apparently runs 29% of the websites on the internet. Wordfence is a firewall and Gravity Scan is a vulnerability scanner, they complement each other.

I have blogged here about setting up WordPress via Command line and setting up an Ubuntu server for as low as 42.5 a month on Vultr.

What can you do to protect your WordPress sites from “Supply Chain Attacks”? First, install the WordFence plugin (I blogged about it here). Wordfence gives you a great set of security settings and reports to keep your site safe. The Wordfence dashboard page on your site is a good place to stay up to date.

WordFence is a Firewall, Gravity scan is a vulnerability and malware scanner. Read more here.

wordfence dashboard

Gravity Scan

Gravity scan is also made by the WordFence people to enable external audits and reports.

Gravity Scan Website

Sign up at https://www.gravityscan.com/ Verify your email and log in.

At login, you will be prompted to add a domain.

Add A Site

Scan

tip: You may want to whitelist Gravity scan servers. Read my guide about securing Ubuntu in the cloud.

sudo ufw allow from 68.64.48.0/27 to any port 443

A site scan will automatically be started.

Scan Started

Scan Results

Scan Results

Post Scan Actions

Speed Up future scans by downloading the Install Gravity Scan Accelerator (by clicking “Not instaled” under “Accelerator” in scan results) and follow the instructions to download, upload and verify the accelerator.

Install Gravityscan Accelerator

Read the Gravity Scan Accelerator Install Instructions here.

tip: I had to run the following command to make the

sudo chown www-data.www-data /www/gravityscan-agent-#############################################.php

Accelerator Installed

I also clicked “Trust Badge” link and added the script code to my site and verified it.

Trust Badle

I now have a scan badge in my site footer.

Future Scans

Future scans are all good to go.

Ready

New Scan Options

New Scan Options

It looks as if the accelerator gives more server-side verifications of checks of WordPress and PHP versions etc.

Go Pro

Gravity Scan also offers a non-free (paid) version where you can enable more options, enable scan schedules and set up SMS alerts and more for $4.95 a month per site.

Go Pro

To be honest I am happy with performing manual scans and I’d rather pay for a premium WordFence subscription first.

The Catch

Hang on Gravity scan requires a Pro membership to see High and Critical issues 🙁

Critical Issues

I decided not to go pro to reveal issues.

A few months later

I started receiving scan results with severity Critical (but I can’t see results until I start a trail (and enter payment details)).

Issues

Time to start a trial

I started a trial and full details were shown, the critical error was my fault

Gone Pro

Critical Issue

This was my fault, I left a previous version of WordPress in a subfolder from when I moved the site to a self-managed server. A quick few Linux commands later (removed) and this was fixed.

old files visible

High Issue

Publically accessible file (fixed with a chmod command)

File Visible

Current Scan Results

Current

Remote Scan Options

Daily Scans, Alter levels, Malware, Vulnerability and status checks. Definitely, install the Accelerator as it found my local backup of WordPress.

Manage Remote

Pros

  • Found a publically readable file
  • Found a past copy of my WordPress site (and all known issues with the old WordPress backup).
  • Can setup daily remote scans.

Cons

  • You have to go pro.
  • Can’t read my NGINX version (“Nginx version not detected, Gravityscan is unable to detect any associated vulnerabilities.“). I logged a ticket. Surely they can add a shell to”nginx -v” to the scan accelerator.
  • No word fence discount bundle?
  • Gravity scan and Word fence on twitter are slow to respond.

More to come.

More Reading

  • Run and Ubuntu Security scan with Lynis
  • WordFence security plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Setting up additional server storage on cloud servers (block storage on Vultr)

Hope this helps someone.

Donate and make this blog better

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

V.1.5 Gravity Scan shutting down

v1.4 Added remote scan options

v1.3 Pros and Cons and current results.

v1.2 Added more

v1.1 Fixed a few issues

v1.0 Initial Version

Filed Under: WP Security Tagged With: and, Attacks, Chain, detect, for, free, Gravity, malware, scan, Scanning, Supply, to, with, wordpress

Security checklist for securing a self-managed Ubuntu server in the cloud

November 2, 2017 by Simon

Below is a (perpetually updated) security checklist for securing a self-managed Ubuntu server. Recently WordPress released patch v4.8.3  that fixed some SQL injection issues.  Is your OS, Database, Web Server, OS and software up to date?

Although I have recently blogged about securing Ubuntu in the cloud, and running a server Audit with Lynus,  this new post is really about obtaining a mindset change and allocating time (each week) to ensure your self-managed servers and software is kept up to date. You can easily list down the actions you need to follow but keeping a system up to date is hard work. Sites like www.shodan.io will reveal what servers or services are vulnerable, let software updates lapse long enough and an open exploit may open a hole to your system.  It only takes minutes to set up a $2.5  a month Ubuntu server with Vultr, $5 a month Digital Ocean Server or AWS server but you need to maintain it.

I highly recommend that you watch the following video that highlights the need for even minor vulnerabilities to be patched asap. If you leave one minor vulnerability open you will give hackers a foothold into your system.

Follow @jawache on twitter.

Troy Hunt has a great post about the simplicity of hacking. Hacking is child’s play.

General Security Checklist

  • Do Setup a Firewall and only allow needed ports to accept data (use tools like Portscan and Shodan.io to find open ports).
  • Use least access permissions (on NGINX, PHP and MySQL processes).
  • Use strong unique passwords for every service (1Password and sites like Gibson Research Corp have password generators, use www.howsecureismypassword.net to check tour passwords strength)
  • Enable logging.

Find log files on your system:

cd /
find -iname "*.log"

Output (handy logs to review):

./var/log/mongodb/mongod.log
./var/log/fail2ban.log
./var/log/mysql/error.log
./var/log/ufw.log
./var/log/lynis.log
./var/log/dpkg.log
./var/log/nginx/error.log
./var/log/nginx/nginxcriterror.log
./var/log/nginx/access.log
./var/log/audit/audit.log
./var/log/php7.0-fpm.log
./var/log/mail.log
./backup/backup.log
./scripts/boot.log
etc
  • Enable brute force detection and banning (fail2ban etc) Read more here.
  • Secure folders with service accounts.
  • Do secure software (e.g WordPress Wordfence)
  • Do use SSL Certificates (and use modern cyphers and test with https://www.ssllabs.com/ssltest/)
  • Monitor SSL vulnerabilities.
  • Do a Lynis security report.
  • Install a Virus scanner (read here).
  • Secure MySQL/Databases.

First, find the version of MySQL

mysql --version
mysql  Ver 14.14 Distrib 5.7.19, for Linux (x86_64)

Read the official MySQL manual here and security guidelines here.

Read this Digital Ocean guide on securing MySQL.

  • Other: _______

Application (coding) checklist

Retain and protect information.

  • Disable errors (PHP: turn off or here)
  • Enable logging (web server, PHP and or node)
  • Sanitize data (never trust uses data) in code (see how to do this in PHP 7)
  • Do no develop on production boxes (use parameterised queries and follow OWASP application security procedures.
  • Read the OWASP Secure Coding Practices – Quick Reference Guide

Infrastructure

Plan for the worst, hope for the best.

  • Use the latest Long Term Support (LTS) version or Ubuntu
  • Update packages

View app packages (Ubuntu 16.04) with updates

sudo /usr/lib/update-notifier/apt-check -p

View app packages (Ubuntu 16.04) with updates

apt list --upgradable

To update packages type (remember to backup data and config files first)

sudo apt-get update && sudo apt-get upgrade

Among other things, you will see the following information

The following packages will be upgraded:
  binutils certbot cracklib-runtime curl distro-info-data grub-common grub-pc grub-pc-bin grub2-common initramfs-tools initramfs-tools-bin initramfs-tools-core libapache2-mod-php7.0
  libcrack2 libcurl3 libcurl3-gnutls libgnutls-openssl27 libgnutls30 libicu55 libpam-systemd libsystemd0 libudev1 linux-firmware linux-libc-dev lshw mdadm mysql-client-5.7
  mysql-client-core-5.7 mysql-common mysql-server mysql-server-5.7 mysql-server-core-5.7 nodejs php7.0 php7.0-cli php7.0-common php7.0-curl php7.0-dev php7.0-fpm php7.0-gd php7.0-imap
  php7.0-intl php7.0-json php7.0-ldap php7.0-mbstring php7.0-mysql php7.0-opcache php7.0-pgsql php7.0-phpdbg php7.0-pspell php7.0-readline php7.0-recode php7.0-snmp php7.0-tidy
  php7.0-xml php7.0-zip python-acme python-certbot python-certbot-nginx python-cffi-backend python-chardet python-idna python-six python3-chardet python3-distupgrade python3-six
  python3-update-manager systemd systemd-sysv ubuntu-release-upgrader-core udev update-manager-core wget

Show available updates

/usr/lib/update-notifier/apt-check --human-readable
0 packages can be updated.
0 updates are security updates.
  • Only work on code checked into GitHub or BitBucket (You will thank me when data or servers disappear).
  • Backup configuration files or backup to remote servers (my rsync guide here)
  • Use snapshots of VM’s.
  • Use Green/Blue server deployments (toggle one server a Prod and the other and Dev/Test and have one ready for a hot spare). Digital Ocean has a good guide here.
  • Consider forcing Content Security Polic and Public Key Pinning or at least using LetsEncrypt SSL certificates.
  • Take Snapshots of VM’s (automate)
  • Backup MySQL databases:
sudo mysqldump --all-databases > /backup/dump-$( date '+%Y-%m-%d_%H-%M-%S' ).sql -u root -p

Other Useful Linus Terminal Commands.

Mindset/Culture

Dedicate time to securing your site.

  1. Spend one day a week (or automate) the updating of the OS/Software (no excuses).
  2. Follow people on twitter and subscribe to newsletters of those that are security conscious

Don’t forget to read securing Ubuntu in the cloud blog post here.

And check out the extensive Hardening a Linux Server guide at thecloud.org.uk: https://thecloud.org.uk/wiki/index.php?title=Hardening_a_Linux_Server

More to come..

Donate and make this blog better


Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.2 added link to Hardening Linux Server link

v1.1 added @jawache link

Short (Article):

Filed Under: OS, Secure, Security, VM Tagged With: a, checklist, cloud, for, in, securing, security, self managed, server, the, ubuntu

Infographic: So you have an idea for an app

October 31, 2017 by Simon

I created this graphic as I was asked by multiple people how to develop an app. This does not include tips on coding but many people with the non-technical prerequisites to building an app.

I hope this graphic helps someone (It’s my first infographic/decision flow image, feedback welcome).

So You Have an Idea For An App: Graphic

Click for a larger version.

Infographic-So-you-have-an-idea-for-an-app-v1-3

Standalone Image URL’s

v1.3 (22nd November 2017)
  https://fearby.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-3.jpg
v1.2 (4th Nov 2017, Added requirements and MoSCoW): 
  https://fearby.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-2.jpg
v1.1 (1st Nov 2017, Fixed Typos): 
  https://fearby.com/wp-content/uploads/2017/10/Infographic-So-you-have-an-idea-for-an-app-v1-1.jpg

todo: Things to add Issues to fix in 1.4:
 - Add user personas and Epic, Story and Task stages.
 - How to capture good stories (and validated ideas (landing pages/interviews/problems/value/painpoints)

Define the problem(s) (pain points)

Before you start coding, do list your app requirements (problem’s to solve (pain points)).

Atlassian JIRA or Trello can help with this. I personally use (and like) Atlaz.io (now Hygger), I reviewed the BETA here).

Using Trello lists are also a simple way to capture tasks/ideas.

ListMore on these Read more here also read my Atlaz.io BETA Preview here.

Nothing beats pen and paper too.

Notepad

Moscow Prioritization

Must-Have Should-Have, Could-Have and Won’t-have are buckets you should sort ideas into. If you have trouble moving items away from Must to Should, Could or Won’t then assign a fictitious monetary value to spend on each item and that will help you decide what is more important.

Read this MoSCoW Method article at Wikipedia: https://en.wikipedia.org/wiki/MoSCoW_method

Managing MoSCoW tasks on paper is OK if you do not want to use planning software.

More

Read my guide on how to prototype apps with Adobe XD guide here.  You can also Prototype a Web app with Platforma (review here).

Read my post on how to develop software and stay on track.

Research

Do research your idea for market fit/need, competition, complexity, legal and validate ideas early. It’s best to find out early that Google will quote $60,000+ TAX a year to allow you to use Google map’s in your app early, then you can use https://www.mapbox.com for $499 a year.

Do you have competition?

Some people say “don’t develop an app that already exists”. Why would you develop a new Uber app? Henry Ford did make a new transportation mode when people were happy with horses, other car manufacturers like Tesla are moving in on the space so don’t be discouraged.

Landing Page

A landing page with a signup form (Newsletter and Register Interest) form is a good way to validate ideas and get feedback early (I would suggest you use a free Mainchimp signup form, a generated website with Platforma on a $5/m server for quick results). There is no point coding and launching to crickets.

Do you have an app Prototype or Mock-Up?

This is very important and easy step.  Programs like Adobe XD CC  (read my guide here) and Balsamiq can help you prototype an app, Platforma can help you prototype web apps.

Wire up a prototype

Drag and Drop

Have you validated your idea (app) with end-users?

If you don’t do this you are mad.  Watch this video to see lessons learned from Trades Cloud.

Is this app idea a hobby (passion)?

This can help you limit costs and expectations.  Cheap serves exist (read here and here).

Do you have time to develop/manage this?

Developing and managing an app and planning (paying for) development cycle can be time-consuming and mentally draining.

Can you code?

Do you need to hire developers or learn to code?  Blog post coming soon on how to hire coders.

Do you have funds?

Having funds on hand to set up and build an app is very important.

Do you want to hide developers (or get Venture Capital)?

This can help you get moving but you will have to give away a slice of the profits and or IP, managing mentors and VC’s can be tiresome.

Have you set failure criteria (post-mortem)?

Read this page on lessons learned from over 200 startup failures, save your favourites.  Having realistic goals and limits is a wise idea, do stop when you reach preset limits.

Do you have a business case?

There is plenty of business case generator template’s,  you will want to document some of the following.

  • What is your apps Purpose – App X will be..
  • What is your Mission Statement – App X will..
  • Who are your Target Customers – Retail..
  • Who are the Early Adopters – Retail..
  • What Problems does your app solve – App X will..
  • What Milestones will your app go through – iOS, Android, Apple TV, Web etc..
  • What Existing solutions exist – App: A, B and C..
  • How does your app Solve your customer’s problems (pain points) – App X will..
  • How will your app Find customers – Word of Mouth, Referrals, Advertisements?
  • What is your Revenue model – Sales, Ad’s, Subscriptions?
  • What is your apps Goal statement – App X will hit X users in X?
  • What are your apps Failure points – If app X does not reach X or monthly costs reach Y….
  • What is your Marketing message – App X will..
  • What is your apps Metrics – iOS, Android, Apple TV apps..
  • What is your Unfair Advantage – Why will you succeed over others?

Are you using a project management methodology?

Proven Methodology can help you develop software and stay on track, software like Atlaz, JIRA or Trello are highly recommended tools. Capturing ideas and processing feedback in tools is very important.

Before you code (or hire coders) use source code versioning software like GitHub and Bitbucket (guides here and here).  You want to retain the code and insist on owning it.

Product Goal

Simon Sinek has a good video on companies (or Products) being in a finite or infinite game.

Are you in full control of your development stack?

If you are not a developer you may not care if you are in control, but you will if there are issues with hired developers or issues with service providers.  I moved from CPanel to self-managed servers, moved from IBM Cloudant to Digital Ocean to AWS then Vultr servers where I can have full control or scalability, features, security and costs.

Can you forecast the costs?

Lowering cost and boosting performance is important and having spare money is a good thing.

I read recently that  Telsla is burning through $6,000 a minute and is forecast to need something like 2 billion dollars in the next 2 years. Software as Service platforms will drain your budget quick (they do take on some risk and maintenance tasks), is this worth it?

Mark Fedin (CEO and Co-founder at Atlaz) has a great post on the topic of viability Stop Dabbling At Startups .

Are you using the right tech?

Don’t be afraid of changing tech along the way, you may start with MySQL and move to MongoDB, Redis, Oracle ot MSSQL database servers etc.

Do you have systems to capture customer feedback?

Self-explanatory, you are solving customer problems, right? You will pivot in the first year (trust me).

What is your revenue/sales model?

If you don’t know how to make money then don’t make an app (apps are expensive to code and maintain).

Are you prioritizing task?

I have blogged about this before, do use the tools to stay on track.

Funny Bit

Project Mangement LolProject Mangement Lol

Donate and make this blog better


Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]

v1.5 Fixed typos and fixed CDN link issue.

v1.4 Updated the graphic to version v1.3.

Short (Article): https://fearby.com/go2/so/

Short (Image): https://fearby.com/go2/so-img/

Filed Under: Advice, Android, App, Atlassian, AWS, Cost, Development, Digital Ocean, Feedback, Git, GitHub, JIRA, Marketing, MongoDB, MySQL, Project Management, Redis, Scalable, Software, Tech Advice, Trello, VM, Vultr Tagged With: an, app, for, have, idea, Infographic, So, you

Primary Sidebar

Poll

What would you like to see more posts about?
Results

Support this Blog

Create your own server today (support me by using these links

Create your own server on UpCloud here ($25 free credit).

Create your own server on Vultr here.

Create your own server on Digital Ocean here ($10 free credit).

Remember you can install the Runcloud server management dashboard here if you need DevOps help.

Advertisement:

Tags

2FA (9) Advice (17) Analytics (9) App (9) Apple (10) AWS (9) Backup (21) Business (8) CDN (8) Cloud (49) Cloudflare (8) Code (8) Development (26) Digital Ocean (13) DNS (11) Domain (27) Firewall (12) Git (7) Hosting (18) HTTPS (6) IoT (9) LetsEncrypt (7) Linux (20) Marketing (11) MySQL (24) NGINX (11) NodeJS (11) OS (10) PHP (13) Scalability (12) Scalable (14) Security (44) SEO (7) Server (26) Software (7) SSH (7) ssl (17) Tech Advice (9) Ubuntu (39) Uncategorized (23) UpCloud (12) VM (44) Vultr (24) Website (14) Wordpress (25)

Disclaimer

Terms And Conditions Of Use All content provided on this "www.fearby.com" blog is for informational purposes only. Views are his own and not his employers. The owner of this blog makes no representations as to the accuracy or completeness of any information on this site or found by following any link on this site. Never make changes to a live site without backing it up first.

Advertisement:

Footer

Popular

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • I moved my domain to UpCloud (on the other side of the world) from Vultr (Sydney) and could not be happier with the performance.
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Add Google AdWords to your WordPress blog

Security

  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Setup two factor authenticator protection at login on Ubuntu or Debian
  • Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software
  • Setting up DNSSEC on a Namecheap domain hosted on UpCloud using CloudFlare
  • Set up Feature-Policy, Referrer-Policy and Content Security Policy headers in Nginx
  • Securing Google G Suite email by setting up SPF, DKIM and DMARC with Cloudflare
  • Enabling TLS 1.3 SSL on a NGINX Website (Ubuntu 16.04 server) that is using Cloudflare
  • Using the Qualys FreeScan Scanner to test your website for online vulnerabilities
  • Beyond SSL with Content Security Policy, Public Key Pinning etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Run an Ubuntu VM system audit with Lynis
  • Securing Ubuntu in the cloud
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider

Code

  • How to code PHP on your localhost and deploy to the cloud via SFTP with PHPStorm by Jet Brains
  • Useful Java FX Code I use in a project using IntelliJ IDEA and jdk1.8.0_161.jdk
  • No matter what server-provider you are using I strongly recommend you have a hot spare ready on a different provider
  • How to setup PHP FPM on demand child workers in PHP 7.x to increase website traffic
  • Installing Android Studio 3 and creating your first Kotlin Android App
  • PHP 7 code to send object oriented sanitised input data via bound parameters to a MYSQL database
  • How to use Sublime Text editor locally to edit code files on a remote server via SSH
  • Creating your first Java FX app and using the Gluon Scene Builder in the IntelliJ IDEA IDE
  • Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

Tech

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Is OSX Mojave on a 2014 MacBook Pro slower or faster than High Sierra
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • The case of the overheating Mac Book Pro and Occam’s Razor
  • Useful Linux Terminal Commands
  • Useful OSX Terminal Commands
  • Useful Linux Terminal Commands
  • What is the difference between 2D, 3D, 360 Video, AR, AR2D, AR3D, MR, VR and HR?
  • Application scalability on a budget (my journey)
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.

Wordpress

  • Replacing Google Analytics with Piwik/Matomo for a locally hosted privacy focused open source analytics solution
  • Setting web push notifications in WordPress with OneSignal
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..
  • Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker
  • Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App
  • Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc
  • Upgraded to Wordfence Premium to get real-time login defence, malware scanner and two-factor authentication for WordPress logins
  • Wordfence Security Plugin for WordPress
  • Speeding up WordPress with the ewww.io ExactDN CDN and Image Compression Plugin
  • Installing and managing WordPress with WP-CLI from the command line on Ubuntu
  • Moving WordPress to a new self managed server away from CPanel
  • Moving WordPress to a new self managed server away from CPanel

General

  • Backing up your computer automatically with BackBlaze software (no data limit)
  • How to back up an iPhone (including photos and videos) multiple ways
  • US v Huawei: The battle for 5G
  • Using the WinSCP Client on Windows to transfer files to and from a Linux server over SFTP
  • Connecting to a server via SSH with Putty
  • Setting web push notifications in WordPress with OneSignal
  • Infographic: So you have an idea for an app
  • Restoring lost files on a Windows FAT, FAT32, NTFS or Linux EXT, Linux XFS volume with iRecover from diydatarecovery.nl
  • Building faster web apps with google tools and exceed user expectations
  • Why I will never buy a new Apple Laptop until they fix the hardware cooling issues.
  • Telstra promised Fibre to the house (FTTP) when I had FTTN and this is what happened..

Copyright © 2023 · News Pro on Genesis Framework · WordPress · Log in

Some ads on this site use cookies. You can opt-out if of local analytics tracking by scrolling to the bottom of the front page or any article and clicking "You are not opted out. Click here to opt out.". Accept Reject Read More
GDPR, Privacy & Cookies Policy

Privacy Overview

This website uses cookies to improve your experience while you navigate through the website. Out of these cookies, the cookies that are categorized as necessary are stored on your browser as they are essential for the working of basic functionalities of the website. We also use third-party cookies that help us analyze and understand how you use this website. These cookies will be stored in your browser only with your consent. You also have the option to opt-out of these cookies. But opting out of some of these cookies may have an effect on your browsing experience.
Necessary
Always Enabled
Necessary cookies are absolutely essential for the website to function properly. This category only includes cookies that ensures basic functionalities and security features of the website. These cookies do not store any personal information.
Non-necessary
Any cookies that may not be particularly necessary for the website to function and is used specifically to collect user personal data via analytics, ads, other embedded contents are termed as non-necessary cookies. It is mandatory to procure user consent prior to running these cookies on your website.
SAVE & ACCEPT