This is a draft post showing how you can monitor the performance of a server (or servers) with NixStats and receive alerts by SMS, Push, Email, Telegram etc
fyi: This is not a paid post, this is just me using the NixStats software to monitor my servers and send alerts.
Finding a good host
If you have not read my previous posts I have now moved my blog to the awesome UpCloud host (signup using this link to get $25 free UpCloud VM credit). I compared Digital Ocean, Vultr and UpCloud Disk IO here and UpCloud came out on top by a long way (read the blog post here). Here is my blog post on moving from Vultr to UpCloud.
Buy a domain name here
Monitoring Servers
The post below will show you how you can monitor servers online with https://nixstats.com/ and send alerts when resources reach limits or servers fail.
I signed up and started a Nixstats (14 day free trial).
After I created an account I was emailed by Nixtsats with agent install instructions for Linux (1 line). I was also advised to add contacts and to set up alerts.
I logged into the Nixstats settings and set up…
- My Timezone
- Default reporting period
- First name and Surname
- Reporting emails
- etc
Nixstats Subscription Upgrade
Subscription options
- Free (5 monitors, 1 server, 24-hour data retention etc)
- Founder (25 monitors, 10 servers, 30-day data retention etc)
- Business (100 monitors, 15 servers, 60-day data retention etc)
I enabled the limited founder subscription so I can monitor 10x servers (this deal is too good to miss). I tried creating a status page myself last year and it is terribly hard.
I am now out of the free trial period 🙂 Let’s start monitoring many servers.
I enabled two factor Auth to Nixstats logins
I created a Nixstats API key for future use (watch this space)
I installed the Nixstats agent (the dashboard gave a 1 line command you can run as root to install the agent (on Linux)).
FYI: Command (######################## is a number linked to your account)
Output
nixstatsagent.sh 100%[====================================================>] 37.80K –.-KB/s in 0.1s
2018-10-02 09:53:56 (338 KB/s) – ‘nixstatsagent.sh’ saved [38708/38708]
Found Ubuntu …
Installing …
Installing Python2-PIP …
Installing nixstatsagent …
Generation a server id …
Got server_id: ######################
Creating and starting service
Created symlink /etc/systemd/system/multi-user.target.wants/nixstatsagent.service -> /etc/systemd/system/nixstatsagent.service.
Created the nixstatsagent service
Server Dashboard
below is a summary of all connected servers ( https://nixstats.com/dashboard/servers ).
Monitor Setup
I set up a number of monitors to monitor ping replies and https traffic
Advanced Monitoring
I can also set the monitor credentials, timeouts, retries, auth methods, max redirects and frequency. If you server blocks login or resource GET attempts you may need to whitelist IP’s. IP’s of monitoring servers are located here https://nixstats.com/whitelist.php
Monitor Summary
The default dashboard is very informative. Feel free to create your own dashboards that focus on your own infrastructure or apps.
Individual Server Reports
You can click on a server and monitor it in detail.
Server Memory Graphs
Long-term memory graphs.
Install Optional Nixstats Plugins
Nixstats offers many plugins to monitor software that is installed on your server (e.g NGINX, MySQL, PHP etc).
1) NGINX Monitoring (Plugin)
To enable NGINX monitoring I read https://help.nixstats.com/en/article/monitoring-nginx-50nu7f/
I edited my NGINX sites-enabled config.
I added the following
server {
listen 127.0.0.1:8080;
server_name localhost;
location /status_page {
stub_status on;
allow 127.0.0.1;
deny all;
}
}
I tested, reloaded and restarted NGINX
The status page will only be available on the local machine, I tested the page on the local machine
It’s Working.
I edit /etc/nixstats.ini
I remove comments before these lines to enable the plugin
I ran the following command to see if NGINX monitoring is possible
Output
nginx:
{
"accepts": 39,
"accepts_per_second": 0.0,
"active_connections": 6,
"handled": 39,
"handled_per_second": 0.0,
"reading": 0,
"requests": 119,
"requests_per_second": 0.0,
"waiting": 5,
"writing": 1
}
It’s Working
I restarted the nixstatsagent
I can now view NGINX properties like active_connections in my dashboard. 🙂
2) Enable PHP-FPM Monitoring (Plugin)
Looks like a PHP-FPM monitoring was recently added lets set that up too. Read my guide on setting up PHP child workers here.
We’ve added a premade dashboard for PHP-FPM. If you’re not yet monitoring PHP-FPM take a look at the integration guide https://t.co/X4ywRHw9hX pic.twitter.com/aag1fTsr3R
— Nixstats (@nixstats) September 6, 2018
To enable PHP-FPM monitoring I read https://help.nixstats.com/en/article/monitoring-php-fpm-1tlyur6/
I edited my PHP-FPM ini file
I added the following line
Restart PHP
I added the following to /etc/nginx/sites-enabled/default localhost server block added above Note I use php 7.2 below (read more here).
server {
listen 127.0.0.1:8080;
server_name localhost;
location /status_phpfpm {
access_log off;
allow 127.0.0.1;
deny all;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
fastcgi_pass unix:/run/php/php7.2-fpm.sock;
log_not_found off;
}
}
I tested, reloaded and restarted NGINX
I restart PHP-FPM
Enabled the plugin in /etc/nixstats.ini
I tested the status page
Output:
{
"pool":"www",
"process manager":"static",
"start time":1538654543,
"start since":178,
"accepted conn":28,
"listen queue":0,
"max listen queue":0,
"listen queue len":0,
"idle processes":49,
"active processes":1,
"total processes":50,
"max active processes":2,
"max children reached":0,
"slow requests":0
}
I tested the agent
Output:
phpfpm:
{
"accepted_conn": 51,
"accepted_conn_per_second": 0.0,
"active_processes": 1,
"idle_processes": 49,
"listen_queue": 0,
"listen_queue_len": 0,
"max_active_processes": 2,
"max_children_reached": 0,
"max_listen_queue": 0,
"pool": "www",
"process_manager": "static",
"slow_requests": 0,
"start_since": 318,
"start_time": 1538654543,
"total_processes": 50
}
I can now query PHP-FPM status values in Nixstats 🙂
Enable MySQL Monitoring (Plugin)
To enable MySQL monitoring I read https://help.nixstats.com/en/article/monitoring-mysql-1frskd8/
I edited the nixstats.ini
I enabled the mysql section in nixstats.ini and added my mysql credentials
[mysql] enabled=yes username=mysqluser password=####################### host=127.0.0.1 database=mysql port=3306 socket=null
I ran this command to test MySQL querying
nixstatsagent --test mysql mysql: Load error: No module named MySQLdb
I had an error.
A quick Google revealed I had to install a mysql python module.
Allow localhost to connect to MySQL
Edit /etc/mysql.cnf and allow all localhost and external connections (I could not bind to localhost and an external IP at the same time)
bind-address = 0.0.0.0
TIP: Ensure you have firewalled access to your MySQL server, never open it up without protection.
Let’s try again
nixstatsagent --test mysql
Output
Nice,
I restart MySQL
I restart my Nixstats service
service nixstatsagent restart
Now let’s monitor MySQL in Nixstats
I can now view MySQL metrix
Status Page
Nixstats allows you to create a status page ( https://nixstats.com/pages/overview ) where you can add any servers or monitors to that page. This stats page is truly awesome, it builds a live status page based on data coming from your installed agents.
You can even set up a custom subdomain that points to a Nixstats hosted status page too (e.g https://status.yourdomain.com).
FYI: An SSL certificate on your staus page may take a few hours to set up. Don’t panic if it is not instantatly available.
Nice.
This saves doing it yourself. The status page will look like it running on your server.
Status Page
You can create a status page that automatically aggregates collected data from monitors and displays them in a nice layout.
This is great, I used to do my own status pages but not anymore.
Alerts
I added a contact so I could receive alerts. I could then add my mobile, email and PushOver key (to receive push notifications) and Telegram Bot API token.
Test Alerts
I sent a test alert to each service against the contact.
I activated a Pushover licence on my Android device for about $7.49 AUD (one off) to ensure I keep getting Push Notifications.
Nixstats have links that show you how you can create a Telegram Bot and Pushover.net account.
Pushover will cost about $5 USD one off per device (see faq).
I created the following alerts
Alert: Disk Usage higher than 90%
Alert: Load greater than 90 per cent for 1 minute
Alert: Less than 5 percent memory free.
Summary of alerts.
I also added a CPU reached 95% one for 5 mins alert too (but it’s not pictured above)
I forgot to specify alert recipients and methods for each alert so I edited each alert and added the contact.
Now it’s time to test the alerts.
I shut down a server to test alerts
Alerts to my defined Email, SMS, Push and Telegram are working a treat 🙂
After I rebooted the server I also received alerts about the server being back up.
The status page showed the server that was offline too.
Nice
Troubleshooting
I had an issue instaling the agent on Debian
I ran the following command
nixstatsagent.sh: line 508: [: Installer exited with error code 0. See nixstatsagent.log for details.: integer expression expected
An error occurred, please check the install log file (nixstatsagent.log)!
Contents of nixstatsagent.log
I asked the Nixstats chat help and I was advised I had a dead repository (I removed this (editing the dead repo in the appropriate file in /etc/apt/) and all was ok)
I had trouble testing my Telegram alerts but it was my fault as I forgot to follow the bot account I created. Telegram does not allow message from a user (bot) unless you follow them.
A chat with the Nixstats staff sorted me out. Thanks, Nixstats chat team.
I had an issue with a missing python mysql package
I solved it by instaling python-mysqldb
Nixstats Help
Nixstats have a help subdomain: https://help.nixstats.com/en/
Error Logs Plugin
I did ask Nixstats on Twitter and they said they are working on a logging plugin, I can’t wait for that.
We’re launching a closed beta for Logging at Nixstats. Contact us to get setup! You can search and tail log files across all your servers! pic.twitter.com/FIeip2SOUw
— Nixstats (@nixstats) October 4, 2018
I now have access to beta log features and can see log tabs in Nixstats
I had or check the version of my rsyslogd
I edited: /etc/rsyslog.d/31-nixstats.conf
I pasted
########################################################## ### Rsyslog Template for Nixstats ### ##########################################################
$WorkDirectory /var/spool/rsyslog # where to place spool files
$ActionQueueFileName fwdRule1 # unique name prefix for spool files
$ActionQueueMaxDiskSpace 1g # 1gb space limit (use as much as possible)
$ActionQueueSaveOnShutdown on # save messages to disk on shutdown
$ActionQueueType LinkedList # run asynchronously
$ActionResumeRetryCount -1 # infinite retries if host is down
template(name=”NixFormat” type=”string”
string=”<%pri%>%protocol-version% %timestamp:::date-rfc3339% %HOSTNAME% %app-name% %procid% %msgid% [5bb8825c3f0bc16aa068f144@5fJR7vuYTJOX6cEiyZlLv5By tag=\”rsyslog\”] %msg%\n”
)
action(type=”omfwd” protocol=”tcp” target=”log.nixstats.com” port=”514″ template=”NixFormat”)
#################END CONFIG FILE#########################
I restarted the rsyslog service
Live Log Output
I can see a live log from (unknown) logs.
I can see the firewall blocking access to certain ports.
Search logs
Blacklist Checking (Beta)
Nixstats tweeted “We just launched a new blacklist check feature. Monitor your IP and hostname reputation. Free during the Beta!”
I enabled it.
I received a Blacklist notification
I requested removal at junkmailfilter.com
Thanks, Nixstats
Conclusion
This is one of the best software packages I have seen in a while. I have developed status
pages in the past (no more). Great work Nixstats.
Please check out Nixstats today, they are awesome. Signup for a free account and consider the limited time founder plan (it’s a bargain).
Nixstats Live chat support is awesome
Server Plug
If you need a server, consider using my referral code and get $25 UpCloud VM credit if you need to create a server online.
https://www.upcloud.com/register/?promo=D84793
Ask a question or recommend an article
[contact-form-7 id=”30″ title=”Ask a Question”]
Revision History
v1.3 Blacklist beta
V1.2 Logs beta
v1.1 Logging Tweet
v1.0 Initial Post
