IoT, Code, Security, Server Stuff etc

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View all of my posts.

monitoring

Monitoring cronjobs on Linux

by

I recently created a number of cron jobs to backup databases, backup files offsite, virus scan and perform a number of tasks (at various times) I put a call out on Twitter as to the best program to monitor these jobs.

George Liu from https://community.centminmod.com/ kindly replied with this advice.

I visited https://healthchecks.io/. A quick Google reveals this software is open source too, nice.

Screenshot of https://healthchecks.io/

Create a healthchecks.io account

I created an account at https://healthchecks.io/ by visiting https://healthchecks.io/accounts/login/ and clicking the signup link then entering my email address.

I click the link that was sent to my email account to log in.

I then logged in and reviewed my account settings at https://healthchecks.io/accounts/profile and set a password (I then click the link in my email to set a password).

After my password was setup I visited https://healthchecks.io/

Pricing

The service is free to use up to 20 checks, then it’s $20/mo for 100 cron job checks. I am using the free version.

View the pricing page here.

Creating Monitors (Checks)

I logged into https://healthchecks.io and clicked the project icon (the one with my email)

Screenshot of the dashboard at This post will.

I deleted the sample item then clicked ‘Add Check‘

Add Check Button

I was prompted to give a name to the job and configure it.

If you click the ‘Name’ (unnamed) item a form will appear.

Add check job to https://healthchecks.io screenshot (naming it)

TIP: Name the jobs something memorable or similar to the job in “crontab”

https://healthchecks.io form asking for a job name, tag and description

I completed the form and clicked ‘Save‘

I click the gear icon for the job to configure it

https://healthchecks.io - new job settings screenshot

After clicking the Settings icon I could see the full details of the job

  • Name
  • Description
  • How to ping it (HTTP or email)
  • Current Status (Pause or Ping Now)
  • Timezone (Important, make sure this is the same timezone as your server)
  • Expected Schedule
  • Notification Method
  • Log
https://healthchecks.io screenshot of a jobs settings

I could see the custom URL I needed to request in my cronjob to make the job pass the success test (a random GUI will be created for each job)

https://hc-ping.com/######-####-####-####-############

I added the following to the end of my desired cron job entry (I have removed my GUID for security reasons and replaced it with #’s)

&& curl -fsS --retry 3 https://hc-ping.com/########-####-####-####-######## > /dev/null

The complete cron entry looks like this (not sure what “12 12 * * *” means, check here)

12 12 * * *  /bib/bash /scripts/thescript.sh && curl -fsS --retry 3 https://hc-ping.com/########-####-####-####-######## > /dev/null

I can also configure the job to expect when to receive an indication when the job has been run.

Click Change Schedule to set a schedule

You can set a schedule with sliders..

Screenshot of when https://healthchecks.io expects a job to run

Or you can set a schedule by pasting in the cron job format and see when the job will be expected to run in the future.

Visit https://crontab.guru/ to learn how to set crontab schedules.

Crontab schedule for 12 12 * * *

After you save the job settings you will be returned to the main page.

Screenshot pf the portal once logged into https://healthchecks.io/

At any time you can reopen the job’s to adjust settings.

You can even see details about the requested clients are accessing the URLs, nice.

Screenshot of past jobs run

I created 17 jobs on 2x servers

Screenshot of 17 jobs added to https://healthchecks.io/

I setup 2 jobs that will fail just to see what failed jobs look like.

Badges

You can also use badges to quickly display the status of cron jobs in other apps or webpages (by embedding an image)

Badges available at https://healthchecks.io/projects/

I created a quick HTML (PHP) file that linked to the badge images. I have swapped out my badge images for security but paste in yours (from above)

HTML

Cron Jobs

Cron Jobs

  • Server 1: f
  • Server 2: d
  • Health Checks: h

    This is what the simple html file looks like in a browser.

    View of the html file showing cron status

    Reports

    I logged into the portal and it did not take long for me to start seeing errors.

    I had an invalid cron job entry (the wrong user was assigned and the job was failing).

    I had an error all this time and did not know it.

    Screenshot of a report of a bad cron job entry.

    My dummy job that was setup to fail did fail.

    My job to purposely fail did fail.

    An email alert was sent to my defined email address, nice.

    Email is failing report

    I could also see the email with the failing event in the log (and an initial test call made with a browser)

    Failing log table

    I was able to pause the reporting of this job in future.

    Conclusion

    I really like https://healthchecks.io/ and will continue using it.

    Sending Emails

    I have added a send email command to the end of my cron job called bach script(s) to be alerted of the jobs running.

    sendemail -f [email protected] -t [email protected] -u "CRON: xxx Run (server.com)" -m "/scripts/script.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp YourEmailPasswordGoesHere -a /folder/file-to-attach.log

    Read my post here How to send email via G Suite from Ubuntu in the cloud

    If you have a G Suite account with 2FA enabled you will need to create a App Specific Password for this to work in your G Suite admin panel.

    Links

    Related links can be found here

    Version:

    1.2 Added send email Info

    1.1 Initial Post

Deploying nodejs apps in the background and monitoring them with PM2 from keymetrics.io

by

This guide will help you install and setup the pm2 NodejJS process monitor PM2 from Keymetrics.io for free and manage your node apps performance and exceptions.

What is PM2?

PM2 is a production process manager for Node.js applications with a built-in load balancer. It allows you to keep applications alive forever, to reload them without downtime and to facilitate common system admin tasks. This is the steps I used on Ubuntu 16.04. This is NOT a paid endorsement (just self-documenting).

Key Features of PM2

PM2 offers web-based monitoring dashboard, exception reporting, load balancer, CPU and memory monitoring, transaction tracer and much more for NodeJS apps.

pm2-features

What is PM2?

Official page: http://pm2.keymetrics.io/

More info https://www.npmjs.com/package/pm2

Install PM2

Install Output

PM2 Pricing

PM2 appears to be for high-end apps but I am only using the free version or PM2 (thanks KeyMetrics)

pm2-pricing

Create a bucket for your node app

Login to keymetrics.io,

Click Generate New Bucket

Create New Bucket

Give the bucket a name etc.

Node Bucket Name

You can now link your bucket with your local pm2 installation (keep the keys private (this one no longer exists))

pm2-link

Linking your local pm2 installation with your keymetrics bucket

To add an existing node app to PM2 type the following.

You can view node apps that pm2 is managing by typing

I had a two CPU VM and I found that the app I added was added to each of the two CPU (I only needed one) so I needed to delete the second app on my second core

Restart the API

You can add a single node apps one 1, 3 or max available CPU’s

Again, to add an existing node app to PM2 type the following.

Now you can view node app data online. If you don’t have a node app ready you can use the test app.

monitor output

You can monitor your node app locally too from the CLI.

local monitoring

You can also view a demo bucket at keymetrix.io

pm2-demo-bucket

PM2’s one age documentation can be found here.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 id=”30″ title=”Ask a Question”]

Revision History

v1.0 Initial post