and

How to backup and restore a MySQL database on Windows and Linux

April 21, 2019 by Simon

Why backup and restore

This is a quick guide demonstrating how you can backup and restore a MySQL database on Windows and Linux using Adminer.

You may need to know how to backup a restore a database for a number of reasons..

e.g

Send the database to someone to debug or give feedback while learning.
Move the database from a local machine to the cloud
Move the database from cloud vendor A to cloud vendor B
etc.

Having a backup of the VM is good but having a backup of the database too is better. I use UpCloud for hosting my VM’s and setting backups is easy. But I cannot download those backups.

Murphy’s Law

“If anything can go wrong, it will”

The most important reason for taking a backup and knowing how to restore it is for disaster recovery reasons.

Backup (the easiest way) with Adminer

Adminer is a free PHP based IDE for MySQL and other databases. Simply install Adminer and save the file on your local computer or remote web server directory.

FYI: The Adminer author Jakub Vrana has a patron page, I am a patron of this awesome software.

Snip from Adminers website. “Adminer (formerly phpMinAdmin) is a full-featured database management tool written in PHP. Conversely to phpMyAdmin, it consist of a single file ready to deploy to the target server. Adminer is available for MySQL, MariaDB, PostgreSQL, SQLite, MS SQL, Oracle, Firebird, SimpleDB, Elasticsearch andMongoDB.”

TIP: The file would be publicly accessible to anyone so don’t save it to a common area, obfuscate the file, protect it of delete the file when you are done using it.

Once Adminer is installed load it in a web browser, login with your MySQL credentials. Once you login you will see all databases and an Import and Export menu.

Adminer main screen, all databases and import and export menu.

tbtest is a simple database with one table and 4 fields (ID, Key, Value and Modified)

.Click Export to open the export screen.

Export screen showing a list of databases and export options

Click Export, a SQL file will be generated (this is the export of the database).

Here is a save of the file:
https://fearby.com/wp-content/uploads/export.txt

Exported view of https://dev.mysql.com/doc/workbench/en/wb-admin-export-import-management.html

Its that simple.

If I add a binary blob file to the table and upload a PNG file lets see how the export looks.

Screenshot o the new table with a blog field in Adminer UI

Let export the database again in Adminer and check out the output. I used Sublime Text editor to view the export file.

New Export shows the binary file in the Backup SQL file

Restore (the easiest way) with Adminer

OK lets delete the tbtest database and then restore it with Adminer. I used Adminer to delete (DROP) the database.

Database “dbtest” deleted.

Now lets create a blank database to restore to (same name).

Database created.

Now lets import the database backup using Adminer.

Click Import, select the backup file and un-tick Stop on errors.

Import screenshot, dxtest selectded, Restore file selected, stop on errors disabled

TIP: The 2MB next the the choose file button is defined by your web server and PHP configuration. If you are trying to import a larger database (e.g 80MB) first increase the limits in your web server and PHP (via php.ini).

The Import (restore should take seconds)

The database was imported from a backup, all tables and records imported just fine.

Bonus methods.

On Ubuntu use this guide to backup from the command line. If you use the Oracle MySQL Workbench read this.

I hope this helps someone.

Updating NGINX to the development branch to get more frequent updates and features over the stable branch

November 20, 2018 by Simon

Here is how I set NGINX to the development branch to get more frequent updates and features over the stable branch

Read the full article here: https://fearby.com/article/updating-nginx-to-the-development-branch-to-get-more-frequent-updates-and-features-over-the-stable-branch/

Updating NGINX to the development branch to get more frequent updates and features over the stable branch

November 20, 2018 by Simon

Updating NGINX to the development branch (on Ubuntu) to get more frequent updates and features over the stable branch

Aside

I have a number of guides on moving away from CPanel, Setting up VM’s on UpCloud, AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. View all recent posts here https://fearby.com/all/

Now on with the post

Warning

Backup your Nginx and Server before making any changes. The Nginx development branch is quite stable but anything can happen. If your site is mission critical then stay on the stable branch.

Nginx Branches

By default, you will most likely get the stable branch of Nginx when instaling and updating Nginx. I have been running the stable version for the last few years but was made aware of a DDoS vulnerability in Nginx.

Here is a good write-up on development merges into the stable branch.

Nginx Updates

Widely-used #Nginx server releases versions 1.15.6 and 1.14.1 to patch two HTTP/2 implementation vulnerabilities that might cause excessive memory consumption (CVE-2018-16843) & CPU usage (CVE-2018-16844), allowing a remote attacker to perform #DoS attackhttps://t.co/1Z3JoghoBr pic.twitter.com/qQ3pOFD1Lk
— The Hacker News (@TheHackersNews) November 9, 2018

I was aware recently of a DDoS bug affecting Nginx and the recommendation was to update ot Nginx 1.15.6 development branch (or 1.14.1 stable branch).

A few days ago no 1.14.1 update was available but a 1.15.6 was, should I switch to the development branch to get updates earlier?

Reminder to update your #nginx installations to the 1.14.1 stable or the 1.15.6 mainline versions for critical security patches released this week. #NGINXPlus customers, see instructions for updating based on the patch released 10/30 https://t.co/KitsOWIJkb
— NGINX, Inc. (@nginx) November 8, 2018

Recent Nginx Changes

Here are the recent changes to Nginx: http://nginx.org/en/CHANGES

Changes with nginx 1.15.6                                        06 Nov 2018

    *) Security: when using HTTP/2 a client might cause excessive memory
       consumption (CVE-2018-16843) and CPU usage (CVE-2018-16844).

    *) Security: processing of a specially crafted mp4 file with the
       ngx_http_mp4_module might result in worker process memory disclosure
       (CVE-2018-16845).

    *) Feature: the "proxy_socket_keepalive", "fastcgi_socket_keepalive",
       "grpc_socket_keepalive", "memcached_socket_keepalive",
       "scgi_socket_keepalive", and "uwsgi_socket_keepalive" directives.

    *) Bugfix: if nginx was built with OpenSSL 1.1.0 and used with OpenSSL
       1.1.1, the TLS 1.3 protocol was always enabled.

    *) Bugfix: working with gRPC backends might result in excessive memory
       consumption.


Changes with nginx 1.15.5                                        02 Oct 2018

    *) Bugfix: a segmentation fault might occur in a worker process when
       using OpenSSL 1.1.0h or newer; the bug had appeared in 1.15.4.

    *) Bugfix: of minor potential bugs.


Changes with nginx 1.15.4                                        25 Sep 2018

    *) Feature: now the "ssl_early_data" directive can be used with OpenSSL.

    *) Bugfix: in the ngx_http_uwsgi_module.
       Thanks to Chris Caputo.

    *) Bugfix: connections with some gRPC backends might not be cached when
       using the "keepalive" directive.

    *) Bugfix: a socket leak might occur when using the "error_page"
       directive to redirect early request processing errors, notably errors
       with code 400.

    *) Bugfix: the "return" directive did not change the response code when
       returning errors if the request was redirected by the "error_page"
       directive.

    *) Bugfix: standard error pages and responses of the
       ngx_http_autoindex_module module used the "bgcolor" attribute, and
       might be displayed incorrectly when using custom color settings in
       browsers.
       Thanks to Nova DasSarma.

    *) Change: the logging level of the "no suitable key share" and "no
       suitable signature algorithm" SSL errors has been lowered from "crit"
       to "info".


Changes with nginx 1.15.3                                        28 Aug 2018

    *) Feature: now TLSv1.3 can be used with BoringSSL.

    *) Feature: the "ssl_early_data" directive, currently available with
       BoringSSL.

    *) Feature: the "keepalive_timeout" and "keepalive_requests" directives
       in the "upstream" block.

    *) Bugfix: the ngx_http_dav_module did not truncate destination file
       when copying a file over an existing one with the COPY method.

    *) Bugfix: the ngx_http_dav_module used zero access rights on the
       destination file and did not preserve file modification time when
       moving a file between different file systems with the MOVE method.

    *) Bugfix: the ngx_http_dav_module used default access rights when
       copying a file with the COPY method.

    *) Workaround: some clients might not work when using HTTP/2; the bug
       had appeared in 1.13.5.

    *) Bugfix: nginx could not be built with LibreSSL 2.8.0.


Changes with nginx 1.15.2                                        24 Jul 2018

    *) Feature: the $ssl_preread_protocol variable in the
       ngx_stream_ssl_preread_module.

    *) Feature: now when using the "reset_timedout_connection" directive
       nginx will reset connections being closed with the 444 code.

    *) Change: a logging level of the "http request", "https proxy request",
       "unsupported protocol", and "version too low" SSL errors has been
       lowered from "crit" to "info".

    *) Bugfix: DNS requests were not resent if initial sending of a request
       failed.

    *) Bugfix: the "reuseport" parameter of the "listen" directive was
       ignored if the number of worker processes was specified after the
       "listen" directive.

    *) Bugfix: when using OpenSSL 1.1.0 or newer it was not possible to
       switch off "ssl_prefer_server_ciphers" in a virtual server if it was
       switched on in the default server.

    *) Bugfix: SSL session reuse with upstream servers did not work with the
       TLS 1.3 protocol.


Changes with nginx 1.15.1                                        03 Jul 2018

    *) Feature: the "random" directive inside the "upstream" block.

    *) Feature: improved performance when using the "hash" and "ip_hash"
       directives with the "zone" directive.

    *) Feature: the "reuseport" parameter of the "listen" directive now uses
       SO_REUSEPORT_LB on FreeBSD 12.

    *) Bugfix: HTTP/2 server push did not work if SSL was terminated by a
       proxy server in front of nginx.

    *) Bugfix: the "tcp_nopush" directive was always used on backend
       connections.

    *) Bugfix: sending a disk-buffered request body to a gRPC backend might
       fail.


Changes with nginx 1.15.0                                        05 Jun 2018

    *) Change: the "ssl" directive is deprecated; the "ssl" parameter of the
       "listen" directive should be used instead.

    *) Change: now nginx detects missing SSL certificates during
       configuration testing when using the "ssl" parameter of the "listen"
       directive.

    *) Feature: now the stream module can handle multiple incoming UDP
       datagrams from a client within a single session.

    *) Bugfix: it was possible to specify an incorrect response code in the
       "proxy_cache_valid" directive.

    *) Bugfix: nginx could not be built by gcc 8.1.

    *) Bugfix: logging to syslog stopped on local IP address changes.

    *) Bugfix: nginx could not be built by clang with CUDA SDK installed;
       the bug had appeared in 1.13.8.

    *) Bugfix: "getsockopt(TCP_FASTOPEN) ... failed" messages might appear
       in logs during binary upgrade when using unix domain listen sockets
       on FreeBSD.

    *) Bugfix: nginx could not be built on Fedora 28 Linux.

    *) Bugfix: request processing rate might exceed configured rate when
       using the "limit_req" directive.

    *) Bugfix: in handling of client addresses when using unix domain listen
       sockets to work with datagrams on Linux.

    *) Bugfix: in memory allocation error handling.

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

Changes with nginx 1.15.6 06 Nov 2018

*) Security: when using HTTP/2 a client might cause excessive memory

consumption (CVE-2018-16843) and CPU usage (CVE-2018-16844).

*) Security: processing of a specially crafted mp4 file with the

ngx_http_mp4_module might result in worker process memory disclosure

(CVE-2018-16845).

*) Feature: the "proxy_socket_keepalive", "fastcgi_socket_keepalive",

"grpc_socket_keepalive", "memcached_socket_keepalive",

"scgi_socket_keepalive", and "uwsgi_socket_keepalive" directives.

*) Bugfix: if nginx was built with OpenSSL 1.1.0 and used with OpenSSL

1.1.1, the TLS 1.3 protocol was always enabled.

*) Bugfix: working with gRPC backends might result in excessive memory

consumption.

Changes with nginx 1.15.5 02 Oct 2018

*) Bugfix: a segmentation fault might occur in a worker process when

using OpenSSL 1.1.0h or newer; the bug had appeared in 1.15.4.

*) Bugfix: of minor potential bugs.

Changes with nginx 1.15.4 25 Sep 2018

*) Feature: now the "ssl_early_data" directive can be used with OpenSSL.

*) Bugfix: in the ngx_http_uwsgi_module.

Thanks to Chris Caputo.

*) Bugfix: connections with some gRPC backends might not be cached when

using the "keepalive" directive.

*) Bugfix: a socket leak might occur when using the "error_page"

directive to redirect early request processing errors, notably errors

with code 400.

*) Bugfix: the "return" directive did not change the response code when

returning errors if the request was redirected by the "error_page"

directive.

*) Bugfix: standard error pages and responses of the

ngx_http_autoindex_module module used the "bgcolor" attribute, and

might be displayed incorrectly when using custom color settings in

browsers.

Thanks to Nova DasSarma.

*) Change: the logging level of the "no suitable key share" and "no

suitable signature algorithm" SSL errors has been lowered from "crit"

to "info".

Changes with nginx 1.15.3 28 Aug 2018

*) Feature: now TLSv1.3 can be used with BoringSSL.

*) Feature: the "ssl_early_data" directive, currently available with

BoringSSL.

*) Feature: the "keepalive_timeout" and "keepalive_requests" directives

in the "upstream" block.

*) Bugfix: the ngx_http_dav_module did not truncate destination file

when copying a file over an existing one with the COPY method.

*) Bugfix: the ngx_http_dav_module used zero access rights on the

destination file and did not preserve file modification time when

moving a file between different file systems with the MOVE method.

*) Bugfix: the ngx_http_dav_module used default access rights when

copying a file with the COPY method.

*) Workaround: some clients might not work when using HTTP/2; the bug

had appeared in 1.13.5.

*) Bugfix: nginx could not be built with LibreSSL 2.8.0.

Changes with nginx 1.15.2 24 Jul 2018

*) Feature: the $ssl_preread_protocol variable in the

ngx_stream_ssl_preread_module.

*) Feature: now when using the "reset_timedout_connection" directive

nginx will reset connections being closed with the 444 code.

*) Change: a logging level of the "http request", "https proxy request",

"unsupported protocol", and "version too low" SSL errors has been

lowered from "crit" to "info".

*) Bugfix: DNS requests were not resent if initial sending of a request

failed.

*) Bugfix: the "reuseport" parameter of the "listen" directive was

ignored if the number of worker processes was specified after the

"listen" directive.

*) Bugfix: when using OpenSSL 1.1.0 or newer it was not possible to

switch off "ssl_prefer_server_ciphers" in a virtual server if it was

switched on in the default server.

*) Bugfix: SSL session reuse with upstream servers did not work with the

TLS 1.3 protocol.

Changes with nginx 1.15.1 03 Jul 2018

*) Feature: the "random" directive inside the "upstream" block.

*) Feature: improved performance when using the "hash" and "ip_hash"

directives with the "zone" directive.

*) Feature: the "reuseport" parameter of the "listen" directive now uses

SO_REUSEPORT_LB on FreeBSD 12.

*) Bugfix: HTTP/2 server push did not work if SSL was terminated by a

proxy server in front of nginx.

*) Bugfix: the "tcp_nopush" directive was always used on backend

connections.

*) Bugfix: sending a disk-buffered request body to a gRPC backend might

fail.

Changes with nginx 1.15.0 05 Jun 2018

*) Change: the "ssl" directive is deprecated; the "ssl" parameter of the

"listen" directive should be used instead.

*) Change: now nginx detects missing SSL certificates during

configuration testing when using the "ssl" parameter of the "listen"

directive.

*) Feature: now the stream module can handle multiple incoming UDP

datagrams from a client within a single session.

*) Bugfix: it was possible to specify an incorrect response code in the

"proxy_cache_valid" directive.

*) Bugfix: nginx could not be built by gcc 8.1.

*) Bugfix: logging to syslog stopped on local IP address changes.

*) Bugfix: nginx could not be built by clang with CUDA SDK installed;

the bug had appeared in 1.13.8.

*) Bugfix: "getsockopt(TCP_FASTOPEN) ... failed" messages might appear

in logs during binary upgrade when using unix domain listen sockets

on FreeBSD.

*) Bugfix: nginx could not be built on Fedora 28 Linux.

*) Bugfix: request processing rate might exceed configured rate when

using the "limit_req" directive.

*) Bugfix: in handling of client addresses when using unix domain listen

sockets to work with datagrams on Linux.

*) Bugfix: in memory allocation error handling.

Development branch changes are made every few weeks and stable branch changes are made less often.

Updating Nginx

Normally you update Nginx bu running an update and upgrade

apt-get update && apt-get upgrade

1	apt-get update && apt-get upgrade

Restart Nginx for good measure

/etc/init.d/nginx restart

1	/etc/init.d/nginx restart

Checking NGINX Version

nginx -v
nginx version: nginx/1.14.1

1 2	nginx -v nginx version: nginx/1.14.1

Changing your repository to the development branch

I changed ot the development branch by running

sudo add-apt-repository ppa:nginx/development

1	sudo add-apt-repository ppa:nginx/development

Update and upgrade Nginx

apt-get update && apt-get upgrade

1	apt-get update && apt-get upgrade

Restart Nginx for good measure

/etc/init.d/nginx restart

1	/etc/init.d/nginx restart

Checking NGINX Version

nginx -v
nginx version: nginx/1.16.6

1 2	nginx -v nginx version: nginx/1.16.6

Removing the stable Nginx repository

Run this command to remove the stable branch of Nginx

sudo add-apt-repository -r ppa:nginx/stable

1	sudo add-apt-repository -r ppa:nginx/stable

Check to see if the development branch is listed

grep -r --include '*.list' '^deb ' /etc/apt/sources.list* |grep nginx
/etc/apt/sources.list.d/nginx-ubuntu-development-bionic.list:deb http://ppa.launchpad.net/nginx/development/ubuntu bionic main

1 2	grep -r --include '.list' '^deb ' /etc/apt/sources.list \|grep nginx /etc/apt/sources.list.d/nginx-ubuntu-development-bionic.list:deb http://ppa.launchpad.net/nginx/development/ubuntu bionic main

Good luck and I hope this guide helps someone

Ask a question or recommend an article

Revision History

v1.0 Initial post

Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker

November 7, 2018 by Simon

This is how I checked the compatibility of my WordPress theme and plugin (code) with PHP Compatibility Checker

Read the full article here: https://fearby.com/article/check-the-compatibility-of-your-wordpress-theme-and-plugin-code-with-php-compatibility-checker/

Check the compatibility of your WordPress theme and plugin code with PHP Compatibility Checker

November 7, 2018 by Simon

This is how I checked the compatibility of my WordPress theme and plugin (code) with PHP Compatibility Checker

Aside

I have a number of guides on moving away from CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. PHP is my programming language of choice.

Now on with the post

Snip from: https://wordpress.org/plugins/php-compatibility-checker/

What is PHP Compatibility Checker

> The WP Engine PHP Compatibility Checker can be used by any WordPress website on any web host to check PHP version compatibility.

> This plugin will lint theme and plugin code inside your WordPress file system and give you back a report of compatibility issues for you to fix. Compatibility issues are categorized into errors and warnings and will list the file and line number of the offending code, as well as the info about why that line of code is incompatible with the chosen version of PHP. The plugin will also suggest updates to themes and plugins, as a new version may offer compatible code.

> This plugin does not execute your theme and plugin code, as such this plugin cannot detect runtime compatibility issues.
Please note that linting code is not perfect. This plugin cannot detect unused code-paths that might be used for backwards compatibility, and thus might show false positives. We maintain a whitelist of plugins that can cause false positives. We are continuously working to ensure the checker provides the most accurate results possible.
This plugin relies on WP-Cron to scan files in the background. The scan will get stuck if the site’s WP-Cron isn’t running correctly. Please see the FAQ for more information.

Install PHP Compatibility Checker

PHP Compatibility Checker

I instaled by SSH’ing to my server and opening my WP Plugins folder

cd /www-root/wp-content/plugins/

1	cd /www-root/wp-content/plugins/

I grabbed the latest download URL from here (hover over the download button), at the time of writing this was the latest version: https://downloads.wordpress.org/plugin/php-compatibility-checker.1.4.6.zip

I downloaded the plugin on my server (then unzipped it and deleted the zip)

wget https://downloads.wordpress.org/plugin/php-compatibility-checker.1.4.6.zip
unzip php-compatibility-checker.1.4.6.zip
rm php-compatibility-checker.1.4.6.zip

wget https://downloads.wordpress.org/plugin/php-compatibility-checker.1.4.6.zip

unzip php-compatibility-checker.1.4.6.zip

rm php-compatibility-checker.1.4.6.zip

Enable PHP Compatibility Checker Plugin

I enabled the plugin

I clicked on the following message

> You have just activated the PHP Compatibility Checker. Start scanning your plugins and themes for compatibility with the latest PHP versions now!

I already have PHP 7.2 installed so let’s scan my site. PHP 7.3 will be available in December and it is already being tested in beta.

PHP Versions

Site Scanning

PHP Compatibility Checker site scanning is very business like

PHP Compatability Checker Scan Results

2 of 22 plugins I use were not PHP 7.2 compatible (WordFence and WP Meta SEO)?

I read on twitter that Wordfence may be a false positive.

Clicking toggle details reveal why the scan failed. A Two Factor Auth plugin was all OK.

Your results will hopefully be…

> PHP 7.2 compatible

Of if errors exist it should explain why it did not pass.

FILE: /www-root/wp-content/plugins/wp-meta-seo/jutranslation/jutranslation.php
> —————————————————————————————-
> FOUND 1 ERROR AFFECTING 1 LINE
> —————————————————————————————-
> 251 | ERROR | The function is_countable() is not present in PHP version 7.2 or earlier
> —————————————————————————————-

I can’t wait for PHP 7.3 scanning. I will update this post in December 2018 after PHP 7.3 is released.

Good luck and I hope this guide helps someone

Ask a question or recommend an article

Revision History

v1.0 Initial post

Add two factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA Authenticator App

October 28, 2018 by Simon

Here is a quick guide to show you how to add two-factor auth login protection to WordPress with YubiCo hardware YubiKeys and or 2FA authenticator app

Using the Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and software

October 4, 2018 by Simon

This post aims to show you how you can use a Yubico YubiKey NEO hardware-based two-factor authentication device to improve authentication and logins to OSX and other software and services.

Background

Although I am a developer I do like security related topics and I try and do as much as I can to secure my systems and applications. Reading the Multi-Factor Authentication Wikipedia page has all the details on Multi-Factor authentication.

I have been a big fan of 1Password to generate strong and unique passwords for separate accounts for a while now. Read my guide on upgrading from a standalone 1Password licence to a 1Password subscription. I love generating unique and complex passwords with 1Password.

But what happens if someone gets access to my 1password vault? Yubico has a catalogue of support services that I can use Yubikeys with to have, 1password is one supported service 🙂

I want to add Yubico protections with these services.

macOS Logins (DONE)
macOS Screensavers (DONE)
1Password (DONE)
Dropbox (DONE)
Twitter (DONE)
Google (DONE)
Google GSuite (DONE, WAITING TO VERIFY)
Google GMail (DONE)
Google Analytics and AdSense (DONE)
Github (DONE)
Thunderbird Email (DONE)
Debian servers in the cloud (SSH) (DONE)
Ubuntu servers in the cloud (SSH) (DONE)
Securing WordPress (DONE)

Etc

Final Warning

Do not attempt to activate Two Factor Authentication on a system unless you…

A) Have backups of your data
B) Have backup methods of getting into your account(s)

Murphy’s Law: “Anything that can go wrong will go wrong”

You never know when a Two Factor Authentication Key may die or an Authenticator app or a Mac/PC may stop working so always have a backup method just in case.

General

General Yubico YubiKey Setup guides https://www.yubico.com/setup/

Buying a Yubico YubiKey

International visitors can buy a YubiKey from the official store here. Australian readers can buy a key locally here. I grabbed 2x YubiKey YubiKey Neo 4 (with NFC) for $50 USD (about $75 AUD) each.

This blog post will aim to show how you can set up a primary key and backup key for use on macOS and other apps to add hardware-based two-factor authentication to logins.

Authenticator Apps

You can use Google Authenticator, Yubico Authenticator or freeOTP from https://freeotp.github.io

Plugging the YubiKey into macOS Mojave

First I read this guide: https://www.yubico.com/works-with-yubikey/catalog/macos/

1) I plugged in my Yubico Neo key into my USB slot.
2) I closed the Keyboard setup window that appeared (I guess the YubiKey is a kind of a keyboard to allow inserting of challenge-response character streams into apps and websites).

3) I followed the basic troubleshooting page and confirmed that the key was being detected (yes it was.)

4) I followed this guide to test U2F functionality and this guide to test OTP functionality. Web pages and Google Chome can talk to the plugged-in YubiKey(s).

I was prompted to register a UTF deice (and create an account)

I was prompted to (insert) and touch my Yubico key.

Google Chome asked for some permissions first.

FYI: Chrome 67 is recommended to securely allow the reading of UbiKey’s from web pages. Only allow sites you trust access to your USB devices and use a modern browser.

Success, Chrome could now see my YubiKey and my device was now verified.

Technical data is available to let you know what is going on in the background. I am not going to break down how this works but Yubico has in-depth whitepapers and documentation if you are interested.

Nice

Configuring OSX

I logged into my Mac with the account that I was going to secure.

I performed a complete time machine backup before proceeding. If you lock yourself out you will need to restore OSX from a Time Machine backup.

I Read the “Using Yubico Pluggable Authentication Module (PAM) with Challenge-Response” login guide: https://www.yubico.com/wp-content/uploads/2016/07/yubico_YubiKeyMacLoginGuide_en.pdf

I downloaded the Download the YubiKey Manager

I downloaded the yubikey-manager from here so I could configure the keys to use “HMAC-SHA1 Challenge-Response”.

Oops, I downloaded the wrong tool, good to know this one exists though.

I will update what this tool does in future (update firmware?)

I Downloaded the Yubikey Personalization Tool

I went back to the Yubico download page and downloaded the Personalization tool.

Many options are available here.

It’s time to configure a primary and backup (duplicate YubiKey) for use with macOS etc.

Enable Challenge Response

I opened the YubiKey Personalization Tool, Inserted my primary key, clicked the Settings tab, and in the Logging Settings group, selected Log configuration output and Yubico format.

I then clicked on the Challenge Response Tab, clicked the HMAC-SHA1 button, selected Configuration Slot 2, ticked “Program Multiple YubiKeys“, changed the “Parameter Generation Scheme = Same for all Keys“, Selected “Fixed 64 byte input” under “HMAC-SHA1 Parameters” and generated a new key (wrote it down).

Under “Configuration Protection” then I selected Enable Protection” I then visited here and generated a 6 digit string to convert to hex array (with spaces (e.g: “70 61 73 73 77 64”)).

Warning: If you set an access code and later forget it, you cannot make any programming changes to this YubiKey. You would need to buy another YubiKey.

I clicked on Write Configuration

If you chose Configuration Slot 1 you will receive a warning about not saving over Configuration Slot 1 due to Yubico VIP/Symantec, I personally do not trust Symantec or the https://vip.symantec.com/ service due to Symantec issuing non-compliant certificates for use on websites. Yubico allows you to swap configuration slots if want to keep the configuration data.

On the output of the first write, I was prompted to save a file. I saved this to “secretkey.csv” onto the Desktop.

When the write to my primary key was successful, I ejected it then inserted my backup key and wrote the same configuration data to it too (on Configuration Slot 2).

Testing the HMAX-SHA1 Challenge

I open the YubiKey Personalization Tool, then click the Tools tab and click Challenge Response. Choose Configuration Slot 2, I selected HMAC-SHA1. I typed a sample input challenge (e.g “hello world”) and clicked Perform.

I noticed the Yubico key touch panel was flashing. I pressed the button, then a response appeared below the input textbox. I copied this response text then insert your second key and perform the same test so I could compare the responses (they should be the same). They were.

If the responses don’t match rewrite the configuration to your primary and secondary keys and ensure the same key and secret was used for both keys.

FYI: I rewrote configuration a few times until I got it right.

Installing the Pluggable Authentication Module (PAM) on macOS

I re-read the Mac login guide here as I don’t want to lock myself out of my Mac.

I opened the Yubico Software Download page here and clicked Computer Login Tools and downloaded the PAM for Mac.

I installed the PAM package and verified the package installation with this command.

ls -al /usr/local/lib/security

1	ls -al /usr/local/lib/security

Output:

Text Output:

> drwxr-xr-x 3 root wheel 96 9 Oct 10:29 .

> drwxrwxr-x 74 simon admin 2368 9 Oct 10:29 ..

> -rwxr-xr-x 1 root wheel 143172 20 Apr 21:13 pam_yubico.so

Backup macOS

Again I ensured my Mac was backed up with Time Machine.

I logged in to my Mac with the account I wanted to be protected with the Yubico YubiKeys.

I ran the following command in terminal

mkdir –m0700 –p ~/.yubico

1	mkdir –m0700 –p ~/.yubico

I double checked that my Yubico key(s) were set up for challenge response (above).

I inserted my Uubico key and ran this command

ykpamcfg -2

1	ykpamcfg -2

Feel free to read the “ykpamcfg” manual here. The yubico-pam source code is located here.

Output:

The contents of “/Users/simon/.yubico/challenge-#######” looked like (I replaced 232 random chars with #’s below). The filename ended with my keys serial number.

v2:########################################################################################################################################################################################################################################:10000:2

Next, I was supposed to copy the challenge output from ykpamcfg to /var/root/.yubico/challenge-[YUBIKEY SERIAL NUMBER] with this command..

sudo cp /var/root/.yubico/challenge-[YUBIKEY SERIAL NUMBER] /Users/[USERNAME]/.yubico

1	sudo cp /var/root/.yubico/challenge-[YUBIKEY SERIAL NUMBER] /Users/[USERNAME]/.yubico

But I had this error.

No such file or directory

1	No such file or directory

Weird as the source file existed?? macOS issues?

I Opened /Users/[USERNAME]/.yubico/challenge-[YUBIKEY SERIAL NUMBER] in the nano editor (sudo elevated process) and saved the file to /var/root/.yubico/challenge-[YUBIKEY SERIAL NUMBER].

I reopened my terminal and verified the contents of /var/root/.yubico/challenge-[YUBIKEY SERIAL NUMBER]. The file is now there.

Permissions on the file is “-rw——-“. Good.

I inserted my second backuP key and re-ran “ykpamcfg -2” and copied the file to “/Users/simon/.yubico”

I verified the file contents

sudo cd /var/root/.yubico/
ls -al

1 2	sudo cd /var/root/.yubico/ ls -al

Output

Text Output:

> drwxr-xr-x 4 root wheel 128 9 Oct 09:50 .
> drwxr-x— 12 root wheel 384 9 Oct 09:39 ..
> -rw-r–r– 1 root wheel 244 9 Oct 09:50 challenge-#######
> -rw-r–r– 1 root wheel 244 9 Oct 09:42 challenge-#######

Snip from: https://www.yubico.com/wp-content/uploads/2016/07/yubico_YubiKeyMacLoginGuide_en.pdf

“Program at least two YubiKeys when implementing a requirement for authentication with a YubiKey on your Mac. If you configure only one YubiKey and something happens to the YubiKey, you must restore the Mac from a Time Machine backup that you created before editing the authorization file before you can log back in to your account. ”

Reading the guide regarding multiple accounts (setting up a Key for each login). I have 5 logins on my Mac but when this works I will disable the other accounts from logging in.

Enable the use of the Yubico key when the screensaver is deactivated on macOS

I opened a terminal and edited “/etc/pam.d/screensaver ” (I use the easier nano editor)

sudo nano /etc/pam.d/screensaver

1	sudo nano /etc/pam.d/screensaver

I added this line

auth       required       /usr/local/lib/security/pam_yubico.so mode=challenge-response

1	auth required /usr/local/lib/security/pam_yubico.so mode=challenge-response

auth[7 spaces]required[7 spaces]/usr/local/lib/security/pam_yubico.so mode=challenge-response

I saved the file ( [CTRL+O], [CTRL+X] ) and exited nano.

I tested my screensaver and no extra protection was provided (the screensaver just exited).

I rebooted, still no change?

I reinstalled the PAM module.

Silly me, I needed to enable the password on the screensaver to then activate the /etc/pam.d/screensaver entries.

I enabled the screensaver passwords

I am now prompted to enter my password and inset and tap my Yubico Key on screensaver exit (on both keys). Awesome.

Next, I need to enable this at macOS login.

Enable the use of the Yubico key at macOS Login

I edited /etc/pam.d/authorization file with nano in the terminal

sudo nano /etc/pam.d/authorization

1	sudo nano /etc/pam.d/authorization

I added the same line as was added to the file /etc/pam.d/screensaver

auth       required       /usr/local/lib/security/pam_yubico.so mode=challenge-response

1	auth required /usr/local/lib/security/pam_yubico.so mode=challenge-response

auth[7 spaces]required[7 spaces]/usr/local/lib/security/pam_yubico.so mode=challenge-response

I saved the file ( [CTRL+O], [CTRL+X] ) and exited nano.

Now let’s log out and test this.

It’s working.

Excellent

Add Two Factor Authentication to 1Password

Here is a guide on using the Yubico YubiKey with 1Password. This directed me to https://support.1password.com/yubikey/

I downloaded the Yubico Authenticator app on macOS and installed it.

After I inserted my primary Key I received a “No Credentials Found”message.

I logged into https://my.1password.com/signin and clicked My Profile.

I clicked More Actions then Turn On Two-Factor Authentication

I added the generated QR code details to the Android Authenticator and macOS Yubico Authenticator app. At first, I could not scan the QR code in macOS (was Mojave blocking this?), I manually entered the details (after confirming them from the Android app QR code scan).

Details:

Issuer: 1Password
Account Name: my.1password.com
Secret Key: ###################
Time: 30
Algorithm: SHA-1
Period: 30
Digits: 6

Now, 1Password web and the desktop app are asking for the 2-factor code (generated in the Yubico Authenticator app after I insert my YubioKey).

Nice

I logged off and I was not prompted for my Two Factor code?

Snip from: https://support.1password.com/two-factor-authentication/

“Your 1Password account is now protected by two-factor authentication. From now on, you’ll need to enter a six-digit authentication code from your authenticator app when you sign in to 1Password on a new device.”

I logged in to 1Password from Google Chrome on Android and indeed I was prompted for a two-factor auth code form the Yubico Authenticator app (with a KubiKey inserted).

Add Two Factor Authentication to Dropbox
I read https://www.yubico.com/works-with-yubikey/catalog/dropbox-personal/. Dropbox also has setup instructions here.

I logged into Dropbox and went to Settings then Security then clicked Add next to Security Keys

I started the Wizard, entered my Dropbox password, then inserted my YubiKey.

Name the Key

I added my Primary and Backup Key(s)

I logged out and back in and no Security Key prompt?

I am using Chrome and had cleared past browsers from the Dropbox list of web browsers at https://www.dropbox.com/account/security

I discovered that I need to set the primary authentication method to Use Mobile App (My Bad, it would be nice if Dropbox set this as default after I added the keys).

I added the Dropbox QR code to the Yuboico Authenticator app

I was asked to enter a 6 digit code from my Yubico Authenticator app to verify the working link. I inserted my YubiKey into my machine to show the code.

Now Dropbox is configured 🙂

Success

I now have to insert my primary key when logging into Dropbox

I need to find a way to copy my Authenticator credentials to my Backup Key from my Primary key

Add Two Factor Authentication to Twitter

I read https://www.yubico.com/works-with-yubikey/catalog/twitter/ (Setup Instructions)

1) Login to Twitter

2) Open your Settings and Look For Security

3) Click Start

4) Enter Your Password

5) Accept and enter any SMS codes if you set up SMS Two Factor codes via SMS

6) Click “Review your login verification methods”

7) Click “Setup Key”

8) Insert Your YubiKey and follow the prompts to activate it.

9) Now the key will be requoted to log in to Twitter

Testing Two Factor Login to Twitter

I logged out of and back into Twitter but the SMS Two Factor Authentication method was still active?

I tried to disable the SMS method in Twitter but two factor was disabled altogether and the registered key was deleted. I re-added my key 🙁

I solved this by choosing “Choose a different verification method” when logging in then choosing “Use your security key“, Twitter then accessed my YubiKey and further login attempts used the key instead of SMS 🙂 I could use an Authenticator code but they YubiKey touch method is quicker.

Done

It would be nice if Twitter allowed multiple keys to be used to log in?

Add Two Factor Authentication to Google, Google cloud, Gsuite etc

I read https://www.yubico.com/works-with-yubikey/catalog/google-accounts (Instructions https://myaccount.google.com/).

Adding two Factor authentication details to Google was not easily accessible at Google so I Googled (lol) this https://support.yubico.com/support/solutions/articles/15000006418-using-your-yubikey-with-google

I loaded: https://myaccount.google.com/signinoptions/two-step-verification/enroll-welcome

I clicked Get Start

I clicked Choose Another Option (not SMS Two factor)

Clicked Security Key

As prompted I inserted my key and allowed access to it.

I named the Key

I repeated the steps and added my 2nd key.

Done

I logged out my https://myaccount.google.com and logged back in and I was prompted to insert my YubiKey

Nice

I did try and login to my google GSuite account at https://admin.google.com but it did not prompt me to insert a key. I will do this next.

Add Two Factor Authentication to GSuite

I logged into the GSuite admin interface at https://admin.google.com/ I generated some backup codes in case I need them in the future.

I checked my main admin user account and I could see the 2 google security keys synced through from Google.

I then searched GSuite for “Two Factor” and loaded the “Enforcement” Page

I enabled “Turn On Enforcement Now”

I enabled “Only Security Keys”

I logged out and back into https://gsuite.google.com/ TWICE and no security key prompt.

Silly me: I forgot to click save at the bottom of the screen and it appears there is a 24-hour delay?

Add Two Factor Authentication to GMail

This is already done (above), GSuite email takes up to 24 hours to become active, GMail is instant.

Add Two Factor Authentication to Google Analytics

I can’t see an option to turn Two Factor Auth on in Google Analytics 🙂

I did send feedback to the Google Analytics team.

Add Two Factor Authentication to Google Adsense

I can’t see an option to turn Two Factor Auth on in Google Adsense either 🙂

I did send feedback to the Google AdSense team.

Add Two Factor Authentication to Github

I logged into Github, opened my Settings and clicked Security then Enable two-factor authentication

Click Setup using an app save the recovery codes.

Open the Yubico Authenticator app (ensure you can see the QR Code in GitHub)

In the Yubico Authenticator, App click File then Scan QR Code

The GitHub details should be added to the Authenticator

Two Factor via authenticator tokens is enabled and now I can see a Keys options,

I clicked Add next to security keys then Register New Device, I gave the key a name then clicked Add.

I added both keys then I Logged out and back in and two factor was enabled by YubiKey 🙂

Add Two Factor Authentication to Debian servers in the cloud (SSH)

Read Setup two-factor authenticator protection at login on Ubuntu or Debian

Add Two Factor Authentication to Ubuntu servers in the cloud (SSH)

Read Setup two-factor authenticator protection at login on Ubuntu or Debian

YubiKey Support

There are loads of Yubico support articles here: https://support.1password.com/yubikey/

Yubico Developer Info

A GitHub repository of source code is located here: https://github.com/Yubico

Other developer related pages here

Securing WordPress

Read this guide on Securing WordPress with 2FA (YubiKey insertion or Authenticator app).

I found a good WordPress plugin to handle 2FA logn methods.

I am prompted to insert my YubiKey after logging into WordPress.

Nice

Java Code to use the Yubico YubiKey in software (challenge mode)

todo: I will add this section soon.

Yubico has Java repository that contains a Java library with an accompanying demo server, as well as a JAAS module, to validate YubiKey OTP’s (One-Time Passwords).

https://developers.yubico.com/yubico-java-client/

PHP Code to use the Yubico YubiKey in software (challenge mode)

todo: I will add this section soon.

Yubico has PHP library ad source code but it has not been updated in 3 years. I cannot get this working on PHP 7.2.

https://github.com/Yubico/php-yubico

Using Yubico YubiKeys as 2fA with one-time Passwords.

The YubiKeys can be used to store and generate one time passwords.

Monitor server performance with NixStats and receive alerts by SMS, Push, Email, Telegram etc

October 3, 2018 by Simon

This is a draft post showing how you can monitor the performance of a server (or servers) with NixStats and receive alerts by SMS, Push, Email, Telegram etc

fyi: This is not a paid post, this is just me using the NixStats software to monitor my servers and send alerts.

Finding a good host

If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.

Buy a domain name here

Monitoring Servers

The post below will show you how you can monitor servers online with https://nixstats.com/ and send alerts when resources reach limits or servers fail.

I signed up and started a Nixstats (14 day free trial).

After I created an account I was emailed by Nixtsats with agent install instructions for Linux (1 line). I was also advised to add contacts and to set up alerts.

I logged into the Nixstats settings and set up…

My Timezone
Default reporting period
First name and Surname
Reporting emails
etc

Nixstats Subscription Upgrade

Subscription options

Free (5 monitors, 1 server, 24-hour data retention etc)
Founder (25 monitors, 10 servers, 30-day data retention etc)
Business (100 monitors, 15 servers, 60-day data retention etc)

I enabled the limited founder subscription so I can monitor 10x servers (this deal is too good to miss). I tried creating a status page myself last year and it is terribly hard.

I am now out of the free trial period 🙂 Let’s start monitoring many servers.

I enabled two factor Auth to Nixstats logins

I created a Nixstats API key for future use (watch this space)

I installed the Nixstats agent (the dashboard gave a 1 line command you can run as root to install the agent (on Linux)).

FYI: Command (######################## is a number linked to your account)

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ########################

1	wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ########################

Output

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ########################
--2018-10-02 09:53:56--  https://www.nixstats.com/nixstatsagent.sh
Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8013, 2400:cb00:2048:1::6819:8113, 104.25.128.19, ...
Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8013|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 38708 (38K) [application/octet-stream]
Saving to: 'nixstatsagent.sh'

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh ########################

--2018-10-02 09:53:56-- https://www.nixstats.com/nixstatsagent.sh

Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8013, 2400:cb00:2048:1::6819:8113, 104.25.128.19, ...

Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8013|:443... connected.

HTTP request sent, awaiting response... 200 OK

Length: 38708 (38K) [application/octet-stream]

Saving to: 'nixstatsagent.sh'

nixstatsagent.sh 100%[====================================================>] 37.80K –.-KB/s in 0.1s

2018-10-02 09:53:56 (338 KB/s) – ‘nixstatsagent.sh’ saved [38708/38708]

Found Ubuntu …
Installing …
Installing Python2-PIP …
Installing nixstatsagent …
Generation a server id …
Got server_id: ######################
Creating and starting service
Created symlink /etc/systemd/system/multi-user.target.wants/nixstatsagent.service -> /etc/systemd/system/nixstatsagent.service.
Created the nixstatsagent service

Server Dashboard
below is a summary of all connected servers ( https://nixstats.com/dashboard/servers ).

Monitor Setup

I set up a number of monitors to monitor ping replies and https traffic

Advanced Monitoring

I can also set the monitor credentials, timeouts, retries, auth methods, max redirects and frequency. If you server blocks login or resource GET attempts you may need to whitelist IP’s. IP’s of monitoring servers are located here https://nixstats.com/whitelist.php

Monitor Summary

The default dashboard is very informative. Feel free to create your own dashboards that focus on your own infastructure or apps.

Individual Server Reports

You can click on a server and monitor it in detail.

Server Memory Graphs

Long-term memory graphs.

Install Optional Nixstats Plugins

Nixstats offers many plugins to monitor software that is installed on your server (e.g NGINX, MySQL, PHP etc).

1) NGINX Monitoring (Plugin)

To enable NGINX monitoring I read https://help.nixstats.com/en/article/monitoring-nginx-50nu7f/

I edited my NGINX sites-enabled config.

sudo nano /etc/nginx/sites-enabled/default

1	sudo nano /etc/nginx/sites-enabled/default

I added the following

server {
    listen 127.0.0.1:8080;
    server_name localhost;
    location /status_page {
        stub_status on;
        allow 127.0.0.1;
        deny all;
    }
}

server {

listen 127.0.0.1:8080;

server_name localhost;

location /status_page {

stub_status on;

allow 127.0.0.1;

deny all;

}

I tested, reloaded and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restart

nginx -t

nginx -s reload

/etc/init.d/nginx restart

The status page will only be available on the local machine, I tested the page on the local machine

wget -qO- http://127.0.0.1:8080/status_nginx
Active connections: 3
server accepts handled requests
 15 15 31
Reading: 0 Writing: 1 Waiting: 2

wget -qO- http://127.0.0.1:8080/status_nginx

Active connections: 3

server accepts handled requests

15 15 31

Reading: 0 Writing: 1 Waiting: 2

It’s Working.

I edit /etc/nixstats.ini

sudo nano /etc/nixstats.ini

1	sudo nano /etc/nixstats.ini

I remove comments before these lines to enable the plugin

[nginx]
enabled = yes
status_page_url = http://127.0.0.1:8080/status_nginx

[nginx]

enabled = yes

status_page_url = http://127.0.0.1:8080/status_nginx

I ran the following command to see if NGINX monitoring is possible

nixstatsagent --test nginx

1	nixstatsagent --test nginx

Output

nginx:
{
    "accepts": 39,
    "accepts_per_second": 0.0,
    "active_connections": 6,
    "handled": 39,
    "handled_per_second": 0.0,
    "reading": 0,
    "requests": 119,
    "requests_per_second": 0.0,
    "waiting": 5,
    "writing": 1
}

nginx:

{

"accepts": 39,

"accepts_per_second": 0.0,

"active_connections": 6,

"handled": 39,

"handled_per_second": 0.0,

"reading": 0,

"requests": 119,

"requests_per_second": 0.0,

"waiting": 5,

"writing": 1

}

It’s Working

I restarted the nixstatsagent

service nixstatsagent restart

1	service nixstatsagent restart

I can now view NGINX properties like active_connections in my dashboard. 🙂

2) Enable PHP-FPM Monitoring (Plugin)

Looks like a PHP-FPM monitoring was recently added lets set that up too. Read my guide on setting up PHP child workers here.

We’ve added a premade dashboard for PHP-FPM. If you’re not yet monitoring PHP-FPM take a look at the integration guide https://t.co/X4ywRHw9hX pic.twitter.com/aag1fTsr3R
— Nixstats (@nixstats) September 6, 2018

To enable PHP-FPM monitoring I read https://help.nixstats.com/en/article/monitoring-php-fpm-1tlyur6/

I edited my PHP-FPM ini file

sudo nano /etc/php/7.2/fpm/pool.d/www.conf

1	sudo nano /etc/php/7.2/fpm/pool.d/www.conf

I added the following line

pm.status_path = /status_phpfpm

1	pm.status_path = /status_phpfpm

Restart PHP

sudo service php7.2-fpm restart

1	sudo service php7.2-fpm restart

I added the following to /etc/nginx/sites-enabled/default localhost server block added above Note I use php 7.2 below (read more here).

server {
    listen 127.0.0.1:8080;
    server_name localhost;

server {

listen 127.0.0.1:8080;

server_name localhost;

location /status_phpfpm {
access_log off;
allow 127.0.0.1;
deny all;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass unix:/run/php/php7.2-fpm.sock;
log_not_found off;
}
}

I tested, reloaded and restarted NGINX

nginx -t
nginx -s reload
/etc/init.d/nginx restart

nginx -t

nginx -s reload

/etc/init.d/nginx restart

I restart PHP-FPM

sudo systemctl restart php7.2-fpm

1	sudo systemctl restart php7.2-fpm

Enabled the plugin in /etc/nixstats.ini

[phpfpm]
enabled = yes
status_page_url = http://127.0.0.1:8080/status_phpfpm?json

[phpfpm]

enabled = yes

status_page_url = http://127.0.0.1:8080/status_phpfpm?json

I tested the status page

wget -qO- http://127.0.0.1:8080/status_phpfpm?json

1	wget -qO- http://127.0.0.1:8080/status_phpfpm?json

Output:

{
	"pool":"www",
	"process manager":"static",
	"start time":1538654543,
	"start since":178,
	"accepted conn":28,
	"listen queue":0,
	"max listen queue":0,
	"listen queue len":0,
	"idle processes":49,
	"active processes":1,
	"total processes":50,
	"max active processes":2,
	"max children reached":0,
	"slow requests":0
}

{

"pool":"www",

"process manager":"static",

"start time":1538654543,

"start since":178,

"accepted conn":28,

"listen queue":0,

"max listen queue":0,

"listen queue len":0,

"idle processes":49,

"active processes":1,

"total processes":50,

"max active processes":2,

"max children reached":0,

"slow requests":0

}

I tested the agent

 nixstatsagent --test phpfpm

1	nixstatsagent --test phpfpm

Output:

phpfpm:
{
    "accepted_conn": 51,
    "accepted_conn_per_second": 0.0,
    "active_processes": 1,
    "idle_processes": 49,
    "listen_queue": 0,
    "listen_queue_len": 0,
    "max_active_processes": 2,
    "max_children_reached": 0,
    "max_listen_queue": 0,
    "pool": "www",
    "process_manager": "static",
    "slow_requests": 0,
    "start_since": 318,
    "start_time": 1538654543,
    "total_processes": 50
}

phpfpm:

{

"accepted_conn": 51,

"accepted_conn_per_second": 0.0,

"active_processes": 1,

"idle_processes": 49,

"listen_queue": 0,

"listen_queue_len": 0,

"max_active_processes": 2,

"max_children_reached": 0,

"max_listen_queue": 0,

"pool": "www",

"process_manager": "static",

"slow_requests": 0,

"start_since": 318,

"start_time": 1538654543,

"total_processes": 50

}

I can now query PHP-FPM status values in Nixstats 🙂

Enable MySQL Monitoring (Plugin)

To enable MySQL monitoring I read https://help.nixstats.com/en/article/monitoring-mysql-1frskd8/

I edited the nixstats.ini

sudo nano /etc/nixstats.ini

1	sudo nano /etc/nixstats.ini

I enabled the mysql section in nixstats.ini and added my mysql credentials

[mysql]
enabled=yes
username=mysqluser
password=#######################
host=127.0.0.1
database=mysql
port=3306
socket=null

[mysql]

enabled=yes

username=mysqluser

password=#######################

host=127.0.0.1

database=mysql

port=3306

socket=null

I ran this command to test MySQL querying

nixstatsagent --test mysql
mysql:
Load error: No module named MySQLdb

nixstatsagent --test mysql

mysql:

Load error: No module named MySQLdb

I had an error.

A quick Google revealed I had to install a mysql python module.

sudo apt-get install python-mysqldb

1	sudo apt-get install python-mysqldb

Allow localhost to connect to MySQL

Edit /etc/mysql.cnf and allow all localhost and external connections (I could not bind to localhost and an external IP at the same time)

bind-address    = 0.0.0.0

1	bind-address = 0.0.0.0

TIP: Ensure you have firewalled access to your MySQL server, never open it up without protection.

Let’s try again

nixstatsagent --test mysql

1	nixstatsagent --test mysql

Output

mysql:
{
    "aborted_clients": 0,
    "aborted_connects": 0,
    "binlog_cache_disk_use": 0,
    "binlog_cache_use": 0,
    "bytes_received": 0,
    "bytes_sent": 0,
    "com_delete": 0,
    "com_delete_multi": 0,
    "com_insert": 0,
    "com_insert_select": 0,
    "com_load": 0,
    "com_replace": 0,
    "com_replace_select": 0,
    "com_select": 0,
    "com_update": 0,
    "com_update_multi": 0,
    "connections": 0,
    "created_tmp_disk_tables": 0,
    "created_tmp_files": 0,
    "created_tmp_tables": 0,
    "key_read_requests": 0,
    "key_reads": 0,
    "key_write_requests": 0,
    "key_writes": 0,
    "max_used_connections": 3.0,
    "open_files": 14.0,
    "open_tables": 316.0,
    "opened_tables": 0,
    "qcache_free_blocks": 1.0,
    "qcache_free_memory": 16760152.0,
    "qcache_hits": 0,
    "qcache_inserts": 0,
    "qcache_lowmem_prunes": 0,
    "qcache_not_cached": 0,
    "qcache_queries_in_cache": 0,
    "qcache_total_blocks": 1.0,
    "questions": 0,
    "select_full_join": 0,
    "select_full_range_join": 0,
    "select_range": 0,
    "select_range_check": 0,
    "select_scan": 0,
    "slave_open_temp_tables": 0.0,
    "slow_launch_threads": 0,
    "slow_queries": 0,
    "sort_range": 0,
    "sort_rows": 0,
    "sort_scan": 0,
    "table_locks_immediate": 0,
    "table_locks_waited": 0,
    "threads_cached": 2.0,
    "threads_connected": 1.0,
    "threads_created": 0,
    "threads_running": 1.0,
    "uptime": 35837.0
}

mysql:

{

"aborted_clients": 0,

"aborted_connects": 0,

"binlog_cache_disk_use": 0,

"binlog_cache_use": 0,

"bytes_received": 0,

"bytes_sent": 0,

"com_delete": 0,

"com_delete_multi": 0,

"com_insert": 0,

"com_insert_select": 0,

"com_load": 0,

"com_replace": 0,

"com_replace_select": 0,

"com_select": 0,

"com_update": 0,

"com_update_multi": 0,

"connections": 0,

"created_tmp_disk_tables": 0,

"created_tmp_files": 0,

"created_tmp_tables": 0,

"key_read_requests": 0,

"key_reads": 0,

"key_write_requests": 0,

"key_writes": 0,

"max_used_connections": 3.0,

"open_files": 14.0,

"open_tables": 316.0,

"opened_tables": 0,

"qcache_free_blocks": 1.0,

"qcache_free_memory": 16760152.0,

"qcache_hits": 0,

"qcache_inserts": 0,

"qcache_lowmem_prunes": 0,

"qcache_not_cached": 0,

"qcache_queries_in_cache": 0,

"qcache_total_blocks": 1.0,

"questions": 0,

"select_full_join": 0,

"select_full_range_join": 0,

"select_range": 0,

"select_range_check": 0,

"select_scan": 0,

"slave_open_temp_tables": 0.0,

"slow_launch_threads": 0,

"slow_queries": 0,

"sort_range": 0,

"sort_rows": 0,

"sort_scan": 0,

"table_locks_immediate": 0,

"table_locks_waited": 0,

"threads_cached": 2.0,

"threads_connected": 1.0,

"threads_created": 0,

"threads_running": 1.0,

"uptime": 35837.0

}

Nice,

I restart MySQL

sudo systemctl restart mysql

1	sudo systemctl restart mysql

I restart my Nixstats service

service nixstatsagent restart

1	service nixstatsagent restart

Now let’s monitor MySQL in Nixstats

I can now view MySQL metrix

Status Page

Nixstats allows you to create a status page ( https://nixstats.com/pages/overview ) where you can add any servers or monitors to that page. This stats page is truly awesome, it builds a live status page based on data coming from your installed agents.

You can even set up a custom subdomain that points to a Nixstats hosted status page too (e.g https://status.yourdomain.com).

FYI: An SSL certificate on your staus page may take a few hours to set up. Don’t panic if it is not instantatly available.

Nice.

This saves doing it yourself. The status page will look like it running on your server.

Status Page

You can create a status page that automatically aggregates collected data from monitors and displays them in a nice layout.

This is great, I used to do my own status pages but not anymore.

Alerts

I added a contact so I could receive alerts. I could then add my mobile, email and PushOver key (to receive push notifications) and Telegram Bot API token.

Test Alerts

I sent a test alert to each service against the contact.

I activated a Pushover licence on my Android device for about $7.49 AUD (one off) to ensure I keep getting Push Notifications.

Nixstats have links that show you how you can create a Telegram Bot and Pushover.net account.

Pushover will cost about $5 USD one off per device (see faq).

I created the following alerts

Alert: Disk Usage higher than 90%

Alert: Load greater than 90 per cent for 1 minute

Alert: Less than 5 percent memory free.

Summary of alerts.

I also added a CPU reached 95% one for 5 mins alert too (but it’s not pictured above)

I forgot to specify alert recipients and methods for each alert so I edited each alert and added the contact.

Now it’s time to test the alerts.

I shut down a server to test alerts

shutdown -h now

1	shutdown -h now

Alerts to my defined Email, SMS, Push and Telegram are working a treat 🙂

After I rebooted the server I also received alerts about the server being back up.

The status page showed the server that was offline too.

Nice

Troubleshooting

I had an issue instaling the agent on Debian

I ran the following command

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh #######################
--2018-10-02 00:41:38--  https://www.nixstats.com/nixstatsagent.sh
Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8113, 2400:cb00:2048:1::6819:8013, 104.25.129.19, ...
Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8113|:443... connected.
HTTP request sent, awaiting response... 304 Not Modified
File 'nixstatsagent.sh' not modified on server. Omitting download.

wget --no-check-certificate -N https://www.nixstats.com/nixstatsagent.sh && bash nixstatsagent.sh #######################

--2018-10-02 00:41:38-- https://www.nixstats.com/nixstatsagent.sh

Resolving www.nixstats.com (www.nixstats.com)... 2400:cb00:2048:1::6819:8113, 2400:cb00:2048:1::6819:8013, 104.25.129.19, ...

Connecting to www.nixstats.com (www.nixstats.com)|2400:cb00:2048:1::6819:8113|:443... connected.

HTTP request sent, awaiting response... 304 Not Modified

File 'nixstatsagent.sh' not modified on server. Omitting download.

nixstatsagent.sh: line 508: [: Installer exited with error code 0. See nixstatsagent.log for details.: integer expression expected

An error occured, please check the install log file (nixstatsagent.log)!

Contents of nixstatsagent.log

cat nixstatsagent.log
Ign:1 http://deb.debian.org/debian stretch InRelease
Hit:2 http://deb.debian.org/debian-security stretch/updates InRelease
Hit:3 http://deb.debian.org/debian stretch-updates InRelease
Hit:4 http://deb.debian.org/debian stretch Release
Ign:5 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic InRelease
Ign:7 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Hit:9 https://packages.sury.org/php stretch InRelease
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Ign:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Ign:8 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main all Packages
Err:10 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main amd64 Packages
  404  Not Found
Ign:11 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic/main Translation-en
Reading package lists...
W: The repository 'http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release' does not have a Release file.
E: Failed to fetch http://ppa.launchpad.net/ondrej/php/ubuntu/dists/cosmic/main/binary-amd64/Packages  404  Not Found
E: Some index files failed to download. They have been ignored, or old ones used instead.
nixstatsagent.sh: line 118: apt-get upgrade returned error code 100. Please see nixstatsagent.log for details.: command not found

cat nixstatsagent.log

Ign:1 http://deb.debian.org/debian stretch InRelease

Hit:2 http://deb.debian.org/debian-security stretch/updates InRelease

Hit:3 http://deb.debian.org/debian stretch-updates InRelease

Hit:4 http://deb.debian.org/debian stretch Release

Ign:5 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic InRelease

Ign:7 http://ppa.launchpad.net/ondrej/php/ubuntu cosmic Release