Code, Security and Server Stuff

Views are my own and not my employer's.

Personal Development Blog...

Coding for fun since 1996, Learn by doing and sharing.

Buy a domain name, then create your own server (get $25 free credit)

View All Posts.

Linux

Monitoring cronjobs on Linux

by

I recently created a number of cron jobs to backup databases, backup files offsite, virus scan and perform a number of tasks (at various times) I put a call out on Twitter as to the best program to monitor these jobs.

George Liu from https://community.centminmod.com/ kindly replied with this advice.

I visited https://healthchecks.io/. A quick Google reveals this software is open source too, nice.

Screenshot of https://healthchecks.io/

Advertisement:



Create a healthchecks.io account

I created an account at https://healthchecks.io/ by visiting https://healthchecks.io/accounts/login/ and clicking the signup link then entering my email address.

I click the link that was sent to my email account to log in.

I then logged in and reviewed my account settings at https://healthchecks.io/accounts/profile and set a password (I then click the link in my email to set a password).

After my password was setup I visited https://healthchecks.io/

Pricing

The service is free to use up to 20 checks, then it’s $16/mo for 100 cron job checks and $64/mo for 100 checks. I am using the free version.

View the pricing page here.

Creating Monitors (Checks)

I logged into https://healthchecks.io and clicked the project icon (the one with my email)

Screenshot of the dashboard at This post will.

I deleted the sample item then clicked ‘Add Check‘

Add Check Button

I was prompted to give a name to the job and configure it.

If you click the ‘Name’ (unnamed) item a form will appear.

Add check job to https://healthchecks.io screenshot (naming it)

TIP: Name the jobs something memorable or similar to the job in “crontab”

https://healthchecks.io form asking for a job name, tag and description

I completed the form and clicked ‘Save‘

Advertisement:



I click the gear icon for the job to configure it

https://healthchecks.io - new job settings screenshot

After clicking the Settings icon I could see the full details of the job

  • Name
  • Description
  • How to ping it (HTTP or email)
  • Current Status (Pause or Ping Now)
  • Timezone (Important, make sure this is the same timezone as your server)
  • Expected Schedule
  • Notification Method
  • Log
https://healthchecks.io screenshot of a jobs settings

I could see the custom URL I needed to request in my cronjob to make the job pass the success test (a random GUI will be created for each job)

https://hc-ping.com/######-####-####-####-############

I added the following to the end of my desired cron job entry (I have removed my GUID for security reasons and replaced it with #’s)

&& curl -fsS --retry 3 https://hc-ping.com/########-####-####-####-######## > /dev/null

The complete cron entry looks like this (not sure what “12 12 * * *” means, check here)

12 12 * * *  /bib/bash /scripts/thescript.sh && curl -fsS --retry 3 https://hc-ping.com/########-####-####-####-######## > /dev/null

I can also configure the job to expect when to receive an indication when the job has been run.

Click Change Schedule to set a schedule

You can set a schedule with sliders..

Screenshot of when https://healthchecks.io expects a job to run

Or you can set a schedule by pasting in the cron job format and see when the job will be expected to run in the future.

Visit https://crontab.guru/ to learn how to set crontab schedules.

Crontab schedule for 12 12 * * *

After you save the job settings you will be returned to the main page.

Screenshot pf the portal once logged into https://healthchecks.io/

Advertisement:



At any time you can reopen the job’s to adjust settings.

You can even see details about the requested clients are accessing the URLs, nice.

Screenshot of past jobs run

I created 17 jobs on 2x servers

Screenshot of 17 jobs added to https://healthchecks.io/

I setup 2 jobs that will fail just to see what failed jobs look like.

Badges

You can also use badges to quickly display the status of cron jobs in other apps or webpages (by embedding an image)

Badges available at https://healthchecks.io/projects/

I created a quick HTML (PHP) file that linked to the badge images. I have swapped out my badge images for security but paste in yours (from above)

HTML

Cron Jobs

Cron Jobs

  • Server 1: f
  • Server 2: d
  • Health Checks: h

    Advertisement:



    This is what the simple html file looks like in a browser.

    View of the html file showing cron status

    Reports

    I logged into the portal and it did not take long for me to start seeing errors.

    I had an invalid cron job entry (the wrong user was assigned and the job was failing).

    I had an error all this time and did not know it.

    Screenshot of a report of a bad cron job entry.

    My dummy job that was setup to fail did fail.

    My job to purposely fail did fail.

    An email alert was sent to my defined email address, nice.

    Email is failing report

    I could also see the email with the failing event in the log (and an initial test call made with a browser)

    Failing log table

    I was able to pause the reporting of this job in future.

    Conclusion

    I really like https://healthchecks.io/ and will continue using it.

    Advertisement:



    Sending Emails

    I have added a send email command to the end of my cron job called bach script(s) to be alerted of the jobs running.

    sendemail -f [email protected] -t [email protected] -u "CRON: xxx Run (server.com)" -m "/scripts/script.sh" -s smtp.gmail.com:587 -o tls=yes -xu [email protected] -xp YourEmailPasswordGoesHere -a /folder/file-to-attach.log

    Read my post here How to send email via G Suite from Ubuntu in the cloud

    If you have a G Suite account with 2FA enabled you will need to create a App Specific Password for this to work in your G Suite admin panel.

    Links

    Related links can be found here

    Version:

    1.2 Added send email Info

    1.1 Initial Post

Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

by

This guide will show how you can automatically backup files to a Backblaze B2 clouds bucket with the Duplicati software on Windows and Linux.

I have blogged about how you can use the Backblaze personal backup program ( here, Use my link and get your first month of Backblaze backups free). The personal application from Backblaze is cool but what if you wanted to store 10GB for free (the first 10GB is free) or setup different buckets for different computers or applications.

Welcome to Backblaze B2 Cloud Storage.

In Australia B2 cloud costs (+GST).

  • First 10GB is free.
  • $0.005c per GB stored per month.
  • To download, 1x GB costs $0.01c.

Check our my other related posts

This works alongside the private Backblaze backup mentioned here.

Screenshot: Duplicati Downalods page https://www.duplicati.com/download

Backblaze B2 Pricing

How does Backblaze B2 Compare that to other cloud data storage provides.

Screenshot of https://www.backblaze.com/b2/cloud-storage.html

Comparison chart.

ProviderStorage ($/GB/Month)Download ($/GB)
Backblaze$0.005$0.01
Amazon S3$0.021$0.05+
Microsoft Azure$0.018+$0.05+
Google cloud$0.020$0.08

Vendor cost breakdown (source).

Read here to view the official page on pricing.

Snip:

Storage:

The first 10 GB of total storage is free, and is $0.005 per GB per month for additional storage beyond 10 GB.   

Download:

The first 1 GB of downloads per day are free.  The price to download stored data costs $0.01 per GB beyond the free daily 1 GB. 

Transactions:

A 'transaction' refers to any time an api call is made with your account, such as listing the contents of your bucket or downloading a file.   B2 will charge for some transactions however, for the average user, these charges are largely negligible unless you are making an extremely high volume of api calls.

Class A transactions are free.

The first 2,500 Class B transactions are free each day. Additional Class B API calls are charged at $0.004 per 10,000 calls.  

The first 2,500 Class C transactions are free each day. Additional Class C API calls are charged is $0.004 per 1,000 calls.

Vendor Relability

https://downdetector.com is  a great site for viewing is  a vendor is down or has issues, lets compare, Backblaze with Amazon, Azure and Google Cloud

Backblaze have has far fewer downtime issues during the great outage of May/June 2019 compared to its competitors. What I like is Backblaze tell us how they do it and tell us what hard drives are good or bad.

Creating a Backblaze B2 Account

Login/SignUp to backblaze at https://secure.backblaze.com/user_signin.htm

You can have a sneak peek at the personal Backblaze backup signup steps in this guide (why not take up a free 1 month back free by clicking this link when you create an account).

From your account page click billing and add your account details

Screenshot of https://secure.backblaze.com/billing.htm?billing_page=b2

Read the B2 Strater Guide: https://www.backblaze.com/b2/docs/

Also review your review the following under ‘My Settings‘

  • Phone Numer (mobile) for usage alerts.
  • Two Factor Authorization for extra security (don’t forget to çopy your backup codes)

Setup Billing Alerts (just in case)

In the Backblaze portal, under ‘Caps and Alerts‘ you should set..

  • Alerts to your mobile phone (for Daily storage caps,bandwidth caps and class B/C caps)
  • Under ‘Daily Storage Caps’ click ‘edit’ and set monetary alerts (you can increase alert limits later)
Added $5 and $10 daily storage alerts

Click ‘Edit Caps’ and set daily limits on all categories

Edit Caps button

I set alert limits of 10c (I can increase this later)

Daily 10c alerts should see is costs creep above $3 a month.

Screenshot of cap alerts, I set 10c warnings on everyitng

Create a cloud bucket to store files

Go to https://secure.backblaze.com/b2_buckets.htm and click ‘Create Bucket’

Screenshot of https://secure.backblaze.com/b2_buckets.htm (create bucket button)

You can create upto 100 buckets per account.

I created a descriptive bucket name that reminded me what server, drive and folder this was used to back up.

"#######-drive-backup-folder" (####### was the name of my computer)
Screenshot of bucket being created an named.

I created a number of buckets (all private).

The Backblaze portal lists all of my new storage buckets.

Screenshot of my 4 new storage buckets

Bucket Storage (Lifestyle Settings)

You can click the ‘Lifecycle Settings‘ button and set the desired buckets file retention policy (overwrite or keep all versions of a file etc).

Read the official Lifestyle settings faq here.

The default is to keep all versions of a file. Choose a setting that is required for each bucket(for me ‘Keep only the latest version of the file’ is best and lowers potential cost blowouts. Having all versions of a file may be handy for a Documents folder.

Screenshot of Lifestyle settings at https://secure.backblaze.com/b2_buckets.htm

You could opt to keep all versions of a file (forever) but that would be expensive.

You tell Backblaze to delete older versions of a file after xx days (this is a good option)

You can also setup custom lifecycle rules to hide a file with a certain prefix after xx days then delete after xx days.

Create a Master Application Key for all buckets

You will need to create a Master Application Key that will grant you full access (Permissions: listKeys, writeKeys, deleteKeys, listBuckets, writeBuckets, deleteBuckets, listFiles, readFiles, shareFiles, writeFiles, deleteFiles ) to each bucket.

Snip from: https://secure.backblaze.com/app_keys.htm

Application keys are used as a pair: Key ID and Application Key. This allows B2 to communicate securely with different devices or apps. Once you generate your Master Application Key, this key has full capabilities. Create your own Application Keys to limit features like read/write. Learn more.

Do write down this “KeyID” and “applicationKey”

Screenshot of https://secure.backblaze.com/app_keys.htm (generated "KeyID" and "applicationKey")

Don’t lose the Master Application Key.

Advertisement:



Create an App Key for each bucket

A master key is like an administrator account, its great for complete access but you should create a separate key per bucket for better security.

Back under the App Keys you can click “Add a New Application Key” for each bucket.

Screenshot of Add a new application key button

Read the official guide on Application Keys here

Screenshot of add application key (add name, option to choose a bucket and assign read/write acess

Do Set..

  • “Name of Key” (similar to your bucket (e.g “my-bucket-name-key”))
  • “Allow access to Bucket(s)” (choose a bucket to grant access to).
  • “Type of Access” (Read and Write, Read Only, Write Only)

Optional..

  • Set a “File name prefix” (snip from here)
"The file name prefix of files the download authorization token will allow b2_download_file_by_name to access. For example, if you have a private bucket named "photos" and generate a download authorization token for the fileNamePrefix "pets/" you will be able to use the download authorization token to access: https://f345.backblazeb2.com/file/photos/pets/kitten.jpg but not: https://f345.backblazeb2.com/file/photos/vacation.jpg."
  • Set a “Duration” (snip from here)
"The number of seconds before the authorization token will expire. The minimum value is 1 second. The maximum value is 604800 which is one week in seconds."

I now has 1x Master Application key and 4x Application Keys for my 4x Buckets

Screenshot of https://secure.backblaze.com/app_keys.htm (1 master key and 4x app keys)

TIP: Make sure you save the keys, if you did not grab the keys delete the keys and buckets (if there is not data and try again). If you don;t have the keys you are locked out of the buckets

OK, Let’s fill those buckets with data with Duplicati

Go to https://www.duplicati.com/ and familiarise yourself with the site

Setup Duplicati on Windows (10)

Go to https://www.duplicati.com/download and download the latest Windows client. At the time of writing Duplicati 2.0.4.5 beta was the latest version. Thew instal was 1MB (the download serve was not that fast)

Warning: Beta software may have bugs, feel free to check out the Github page for Issues etc.

Duplicati 2.0.4.5 download button

Open the install and Click ‘Next‘

Duplicati install wizard start, click Next

Agree to the ‘Licence Agreement‘ by clicking ‘Next‘

Duplicati licence agreement screenshot

Select Duplicati components to install and click ‘Next‘

Install an components screenshot.

Tick ‘Launch Duplicati‘ an click ‘Finish‘

Duplicati has finished installing screenshot.

Using Duplicati Software

Upon first open you will be prompted to set a password (do this even if you are the only user of thew computer)

First run screenshot

Set a strong and unique password

Screenshot of setting a duplicati password

Click ‘Ok‘

Ok Button

When the page reloads it will report it is not logged in.

Screenshot of user is not logged in error.

Enter your password

Enter password screenshot

The main screen of Duplicati reports that there are no scheduled set.

Screenshot no scheduled tasks screenshot.

OK, lets create a backup schedule.

Creating a Backup on Windows

If your screen is small you will need to click the menu at the top right and click ‘Add Backup’, if your screen is large there will be an ‘Add Backup’ menu on the left hand side of the screen.

Duplicati menu screenshot

Select ‘Configure a new backup‘ and click ‘Next‘

Configure a new backup screenshot

You will be able to name and describe the backup schedule name and be able to set up an encryption password.

Then click ‘Next‘

Screenshot of create backup schedule with name, description and encryption password

Under ‘Storage Type‘ choose ‘B2 Cloud Storage‘

When you choosing ‘B2 Cloud Storage‘ you will be directed to the screen below where you stecify..

  • B2 Cloud bucket name
  • Path in the bucket to store the files (leave blank)
  • Enter your Backblaze Account ID (e.g. ##########################)
  • Enter the Buckey ApplicationKey (e.g. ###############################)

Click ‘Next‘

Screenshot of enter B2 credentials

Advertisement:



I entered my credentials

I entered my B2 bucket credentials

Click ‘Test Connection‘, if you entered everything OK you should see ‘Connection Worked‘

Connection worked Screenshot

Click ‘Next‘

Next button screenshot

Now you can select the folders to backup on your local computer

Duplicati, select folders to backup.

Click ‘Next‘

Next button

Now you can choose a schedule to backup

Screenshot choose time and days to run the backup.

Click ‘Next‘

Click Next button screenshot

Now you can set the remote backup size and backup retention mode.

Read the official guide on setting the volume size here

  • Increase the ‘Remote Volume Size’ if you are worried about consuming your daily API quota in Backblaze (this may happen if your upload speed is fast and your backup is very large).
  • Lower the ‘Remote Volume Size’ size if backups are small or your upload speed is low.
Screenshot of remote volume size and backup retention

Choose a desirable backup retention mode. Duplicati only offers full backups (not incremental or differential backups). This will be an issue if your backups are large.

  • Keep all backups (“Nothing will be deleted. The backup size will grow with each change.“)
  • Delete backups that are older than (“If at least one newer backup is found, all backups older than this date are deleted.“)
  • Keep a specific number of backups (“Once there are more backups than the specified number, the oldest backups are deleted.“)
  • Smart Backup Retention (“Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup.“)
  • Custom backup retention (“Enter a retention strategy manually. Placeholders are D/W/Y for days/weeks/years and U for unlimited. The syntax is: 7D:1D,4W:1W,36M:1M. This example keeps one backup for each of the next 7 days, one for each of the next 4 weeks, and one for each of the next 36 months. This can also be written as 1W:1D,1M:1W,3Y:1M.“)

Each storage option has pro’s and cons with potential cost implications. I would not set 300GB to “Keep all backups” and backup daily. That would be expensive.

Screenshot of the 5 backup retention modes

Click ‘Save‘

Save button screenshot

TIP: Don”t forget to save your encryption key if you set one

Screenshot of a duplicati warning saying have we saved our encryption ley

Now you will see the backup schedule on your Duplicati screen.

You can run, edit or delete the backup status here.

Screenshot of the backup schedule on the home screen.

I clicked ‘Run‘ and the backup was under way.

A progress bar updates as the backup happens.

Screenshot of backup underway

A slight delay happens at the end of the backup.

Screenshot of backup waiting to finish screen.

I logged into the Backblaze portal to see how many files and usage credits were used.

Note it seems?

Screenshot of backblaze portal

When the files are backed up a verification processes happens.

screenshot of verify files screen

Now the backup has completed.

Screenshot of the backup completed screen

I waited 2 minutes and finally Backblaze updated it’s status, Duplicati used 3 files and 87MB

Screenshot of B2 Cloud uploads stats

I logged into the view the Duplicati files that were uploaded and I could see the three files were there and they were encrypted.

Screenshot of B2 cloud encrypted Duplicati files.

Awesome

Advertisement:



Restoring a Backup on Windows

Having a backup is useless you can restore the files (especially when they are encrypted).

Lets restore a few files and see if it works.

I opened Duplicati and clicked ‘Restore‘

Screenshot: Duplicati restore menu item

I clicked the bottom option that listed my bucket name/backup name and clicked ‘Next‘

Screenshot restore from the backup

Before I ran the restore I deleted some location files that were just backed up.

Screenshot of me deleting 2 files.

Duplicati now asked me to choose files to restore.

Screenshot restore files treeview with selections

I chose the original location to restore to and clicked ‘Restore‘

Screenshot restore to.

Duplicati took a few minutes to restore the files.

Screenshot: Duplicati downloading files

The restore was a success.

Screenshot restore success

Nice

Backblaze Costs?

Beware, make sure you calculate the costs to upload and download files.

Do set alarms on cap’s limits and review them often.

Screenshot of data usage and caps.

Second Backup Schedule on Windows

I quickly set a second backup schedule of a different folder to a different bucket.

This backup was 300GB so I will only backup once a month

Schedule once a month backup schedule.

The backup is listed on the front screen of Duplicati.

Do create multiple buckets and set multiple backup schedules to break up the backups into smaller chunks.

Screenshot of multiple backup schedules.

Setup Duplicati on Linux (Debian or Ubuntu)

Lets setup B2 Cloud Storage backup on Linux.

I visited the Duplicati forums here. Duplicati informed me that the Linux client needs mono installed

Duplicati depends on other software. For Windows, Microsoft .NET Framework 4.5 or higher needs to be installed. Linux and Mac OS X require Mono to be installed.

Mono is a Cross Platform open source .NET that runs on Linux (read here)

On Ubuntu 1.08 I installed mono with these commands

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu stable-bionic main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

On Debian 9.9 I installed Mono with these commands (from here)

sudo apt install apt-transport-https dirmngr gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/debian stable-stretch main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

I rebooted the serves and the CPU and memory went crazy after I installed Mono. My Nixstats Service reports were going off telling me my servers CPU was busy.

If you don’s have reports on Linux server CPU/MEM/Disk reports coming to you via Email and Telegram then read this.

Screensot of Nixstats Reports

I felt sorry for my web and database server server(s) so I rebooted them, wow, My website came back up in about 10 seconds thanks to UpClouds fast Max IOPS disks.

Installing Duplicati on Linux

I visited the download Duplicati page here and copied the Ubuntu and Debian download URL.

At the time of writing it was..

https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I downloaded the deb package

cd /utils/duplicati
wget https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I then ran this command to install the package

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

I received this error even thpogh libmono-2.0-1 was installed

dpkg: dependency problems prevent configuration of duplicati:
duplicati depends on libmono-2.0-1; however:
Package libmono-2.0-1 is not installed.

This forum post helped me, I ran

sudo apt install -f

Output

sudo apt install -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
Suggested packages:
  indicator-application
The following NEW packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
0 upgraded, 6 newly installed, 0 to remove and 7 not upgraded.
1 not fully installed or removed.
Need to get 143 kB of archives.
After this operation, 541 kB of additional disk space will be used.
Do you want to continue? [Y/n]

Advertisement:



And I ran the following

sudo apt dist-upgrade --fix-missing

I now ran the following to install the duplicati package.

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

It worked on Ubuntu 18.08 bt not Debian 9.9

Debian produced the following error

sudo dpkg -i duplicati_2.0.4.5-1_all.deb                (Reading database ... 70082 files and directories currently installed.)
Preparing to unpack duplicati_2.0.4.5-1_all.deb ...
Unpacking duplicati (2.0.4.5-1) over (2.0.4.5-1) ...
dpkg: dependency problems prevent configuration of duplicati:
 duplicati depends on libappindicator0.1-cil | libappindicator3-0.1-cil; however:
  Package libappindicator0.1-cil is not installed.
  Package libappindicator3-0.1-cil is not installed.

dpkg: error processing package duplicati (--install):
 dependency problems - leaving unconfigured
Processing triggers for mime-support (3.60) ...
Errors were encountered while processing:
 duplicati

On Debian I installed the stretch version of the package

sudo apt-get install libappindicator1

Then in Debian I fixed broken packages

sudo apt --fix-broken install

This I was able to install Duplicati on Debian

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

Running Duplicati CLI

I can now Run Duplicati CLI (it works)

duplicati-cli

See duplicati.commandline.exe help <topic> for more information.
  General: example, changelog
  Commands: backup, find, restore, delete, compact, test, compare, purge, vacuum
  Repair: repair, affected, list-broken-files, purge-broken-files
  Debug: debug, logging, create-report, test-filters, system-info, send-mail
  Targets: aftp, amzcd, azure, b2, box, cloudfiles, dropbox, ftp, file,
  googledrive, gcs, hubic, jottacloud, mega, msgroup, onedrive, onedrivev2,
  sharepoint, openstack, rclone, s3, ssh, od4b, mssp, sia, tahoe, webdav
  Modules: aes, gpg, zip, 7z, console-password-input, mssql-options,
  hyperv-options, http-options, sendhttp, sendmail, runscript, sendxmpp,
  check-mono-ssl
  Formats: date, time, size, encryption, compression
  Advanced: mail, advanced, returncodes, filter, filter-groups, <option>

http://www.duplicati.com/              Version:  - 2.0.4.5_beta_2018-11-28

TIP: Don’t run “duplicati” that’s the desktop verison.

Creating a Duplicati Backup on Linux

I read this page and this page to get an understanding of how to back up from the command line. I was not sure if I needed to run a service or run the CLI from a bach script so I asked on the forums here.

Within minutes I had this reply

kenkendk
1h
Hi @feartec, welcome to the forum!

The CLI and Web-UI are not the same, and operate independently.

If you use the CLI, you need to use cron or something similar to run the backups periodically. You can use duplicati-cli help to get started. You do not need the server running for this.

For the web-UI we do not currently have any tools for manipulating the backups other than the main WebUI itself. You need to somehow forward/tunnel access to the server, and then access the WebUI from a local browser.

Backing up a folder with Duplicati to a Backblaze B2 Bucket

I created this folder structure

/test/
/test/testfile.txt < hello
/test/subfolder/
/test/subfolder/file.txt < world

I ran this command to back up the folder structure above to backblaze (official guide). See advanced switches.

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=**********

TIP: read more about “dblock” size here

Output

Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  4 files need to be examined (8 bytes)
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1021 bytes) ...
  Downloading file (1.06 KB) ...
  Downloading file (2.00 KB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:06
  Remote files: 3
  Remote size: 4.05 KB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 0
  Files deleted: 0
  Files changed: 0
  Data uploaded: 0 bytes
  Data downloaded: 4.05 KB
Backup completed successfully!

I logged into Backblaze to check the backups

Duplicati backup success

I created a second test file and added 2 test files

mkdir /test2
cd /test2
wget http://ipv4.download.thinkbroadband.com/10MB.zip
wget http://ipv4.download.thinkbroadband.com/5MB.zip

ls -al
total 15368
...
-rw-r--r--  1 user user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

I ran the backup with the second folder in the command line

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=********** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/07/2019 23:05:58
Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  7 files need to be examined (15.00 MB)
  Uploading file (15.03 MB) ...
  Uploading file (11.43 KB) ...
  Uploading file (1.29 KB) ...
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1.29 KB) ...
  2 files need to be examined (8 bytes)
  Downloading file (11.43 KB) ...
  Downloading file (15.03 MB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:10
  Remote files: 6
  Remote size: 15.05 MB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 2
  Files deleted: 0
  Files changed: 0
  Data uploaded: 15.05 MB
  Data downloaded: 15.05 MB

Success

Restoring a Duplicati Backup on Linux

O’k lets delete the local “/test2/” folder then restore it

rm -R /test2

Now lets restore it (official guide)

/usr/bin/duplicati-cli restore "b2://bucket-name-goes-here/test" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --passphrase=************* --overwrite=true --restore-path="/test2"

Output

Restore started at 06/07/2019 23:21:51
Checking remote backup ...
  Listing remote folder ...
Checking existing target files ...
  2 files need to be restored (15.00 MB)
Scanning local files for needed data ...
  Downloading file (15.03 MB) ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 0 (0 bytes) files to /test2
Duration of restore: 00:00:05

Did it work

cd /test2
ls -al
total 15368
...
-rw-r--r--  1 use user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

Yes it worked

Backup 2 folders an emailing the results

I use Google GSuite for email and have an app password setup just for duplicati. Here is the official guide for sending an email after a backup operation

I downloaded a 100MB file

cd /test2
wget http://ipv4.download.thinkbroadband.com/100MB.zip

I ran this command to backup an emailthe progress

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=***************** [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/08/2019 00:02:28
Checking remote backup …
Listing remote folder …
Scanning local files …
8 files need to be examined (115.00 MB)
7 files need to be examined (15.00 MB)
Uploading file (49.99 MB) …
Uploading file (18.01 KB) …
Uploading file (49.99 MB) …
Uploading file (17.97 KB) …
Uploading file (233.33 KB) …
Uploading file (33.17 KB) …
Uploading file (1.42 KB) …
Compacting remote backup …
Checking remote backup …
Listing remote folder …
Verifying remote backup …
Remote backup verification completed
Downloading file (1.42 KB) …
4 files need to be examined (15.00 MB)
Downloading file (18.01 KB) …
Downloading file (49.99 MB) …
0 files need to be examined (0 bytes)
Duration of backup: 00:00:15
Remote files: 13
Remote size: 115.33 MB
Total remote quota: 0 bytes
Available remote quota: 0 bytes
Files added: 1
Files deleted: 0
Files changed: 0
Data uploaded: 100.28 MB
Data downloaded: 50.01 MB
Backup completed successfully!

Email

Screenshot of and email from Duplicati software

Backup Types and Versions

Read up here on backup retention policies here

TIP: Use the desktop Duplicati software to generate a backup then use the export to command line option to get the command line parameters to use in the command line version.

Screenshot of the desktop version of Duplicati and the export to command line link

After you click Commandline you can view each separate command liene option and parameter.

This was handy for finding this backup retention parameter

--retention-policy="1W:1D,4W:1W,12M:1M

“This will give me “one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup. “

Setup Auto Backup Cron Scripts

Login to your linux machines and edit a bash file (e.g in “/scripts/backupnow.sh” folder).

Make the script executable

chmod +x /scripts/backupnow.sh

Add the following

#!/bin/bash

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=123 [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Now you can add the script to your crontab by running the following.

crontab -e

Make sure you have the following in your crontab

SHELL=/bin/bash

Add a cron entry to run the bash script

#Backup Database
0 22 * * * /bin/bash /scripts/backupnow.sh

Use https://crontab.guru/ to check the time you want to run thew script

https://crontab.guru/ screenshot

Links

 

 

 

Version

v1.1 Fixed typo (”downalod” thanks to @NicolasWolf for reporting.)

v1.0 Initial Post (added retention polity and Linux crontabs

v0.95 Backup, Restore and Send Email

v0.91 Added Duplicati service info

v0.9 Working Copy

How to backup and restore a MySQL database on Windows and Linux

by

Why backup and restore

This is a quick guide demonstrating how you can backup and restore a MySQL database on Windows and Linux using Adminer.

You may need to know how to backup a restore a database for a number of reasons..

e.g

  • Send the database to someone to debug or give feedback while learning.
  • Move the database from a local machine to the cloud
  • Move the database from cloud vendor A to cloud vendor B
  • etc.

Advertisement:



Having a backup of the VM is good but having a backup of the database too is better. I use UpCloud for hosting my VM’s and setting backups is easy. But I cannot download those backups.

UpCloud Backup Screen

Murphy’s Law

“If anything can go wrong, it will”

The most important reason for taking a backup and knowing how to restore it is for disaster recovery reasons.

Backup (the easiest way) with Adminer

Adminer is a free PHP based IDE for MySQL and other databases. Simply install Adminer and save the file on your local computer or remote web server directory.

FYI: The Adminer author Jakub Vrana has a patron page, I am a patron of this awesome software.

Snip from Adminers website. “Adminer (formerly phpMinAdmin) is a full-featured database management tool written in PHP. Conversely to phpMyAdmin, it consist of a single file ready to deploy to the target server. Adminer is available for MySQL, MariaDB, PostgreSQL, SQLite, MS SQL, Oracle, Firebird, SimpleDB, Elasticsearch andMongoDB.”

adminer.php file icon screenshot

TIP: The file would be publicly accessible to anyone so don’t save it to a common area, obfuscate the file, protect it of delete the file when you are done using it.

Advertisement:



Once Adminer is installed load it in a web browser, login with your MySQL credentials. Once you login you will see all databases and an Import and Export menu.

Adminer main screen, all databases and import and export menu.

tbtest is a simple database with one table and 4 fields (ID, Key, Value and Modified)

.Click Export to open the export screen.

Export screen showing a list of databases and export options

Click Export, a SQL file will be generated (this is the export of the database).

Here is a save of the file:
https://fearby.com/wp-content/uploads/export.txt

Exported view of https://dev.mysql.com/doc/workbench/en/wb-admin-export-import-management.html

Its that simple.

If I add a binary blob file to the table and upload a PNG file lets see how the export looks.

Screenshot o the new table with a blog field in Adminer UI

Advertisement:



Let export the database again in Adminer and check out the output. I used Sublime Text editor to view the export file.

New Export shows the binary file in the Backup SQL file

Restore (the easiest way) with Adminer

OK lets delete the tbtest database and then restore it with Adminer. I used Adminer to delete (DROP) the database.

Database dropped with Adminer

Database “dbtest” deleted.

Now lets create a blank database to restore to (same name).

Create database screen.

Database created.

dbtest created.

Advertisement:



Now lets import the database backup using Adminer.

Click Import, select the backup file and un-tick Stop on errors.

Import screenshot, dxtest selectded, Restore file selected, stop on errors disabled

TIP: The 2MB next the the choose file button is defined by your web server and PHP configuration. If you are trying to import a larger database (e.g 80MB) first increase the limits in your web server and PHP (via php.ini).

The Import (restore should take seconds)

Import Success

The database was imported from a backup, all tables and records imported just fine.

The database was imported from a backup

Bonus methods.

On Ubuntu use this guide to backup from the command line. If you use the Oracle MySQL Workbench read this.

I hope this helps someone.

Advertisement:



Setting up the Debian Kali Linux distro to perform penetration testing of your systems

by

This post will show you how to setup the Kali Linux distro to perform penetration testing of your systems

Advertisement:



I have a number of guides on moving hasting away form CPanel, Setting up VM’s on AWS, Vultr or Digital Ocean along with installing and managing WordPress from the command line. Securing your systems is very important (don’t stop) and keep learning (securing ubuntu in the cloud, securing checklist, run a Lynis system audit etc)

snip from: https://www.kali.org/about-us/

“Kali Linux is an open source project that is maintained and funded by Offensive Security, a provider of world-class information security training and penetration testing services. In addition to Kali Linux, Offensive Security also maintains the Exploit Database and the free online course, Metasploit Unleashed.”

Download Kali

I downloaded the torrent version (as the HTTP version kept stopping (even on 50/20 NBN).

Download Kali

After the download finished I checked the SHA sum to verify it’a integrity

A least it matched the known (or hacked) hash here.

Installing Parallels in a VM on OSX

I use Parallels 11 on OSX to set up a VM os Demina Kali, you can use VirtualBox, VMWare etc.

VM Setup in Parallels

Hardware: 2x CPU, 2048MB Ram, 32MB Graphics, 64GB Disk.

I selected Graphical Install (English, Australia, American English, host: kali, network: hyrule, New South Wales, Partition: Guided – entire disk, Default, Default, Default, Continue, Yes, Network Mirror: Yes, No Proxy, Installed GRUB bootloader on VM HD.

Post Install

Install Parallel Tools

Official Guide: https://kb.parallels.com/en/123968

I opened the VM then selected the Actions then Install Parallels Tools, this mounted /media/cdrom/, I copied all contents to /temp/

As recommended by the Parallels instal bash script I updated headers.

Then the following from https://kb.parallels.com/en/123968

Parallels will not install, I think I need to upgrade to parallel 12 or 12 as the printer driver detection is not detecting (even though it is installed).

Installing Google Chrome

I used the video below

I have to run chrome with

It works.

Chrome

Running your first remote vulnerability scan in Kali

I found this video useful in helping me scan and check my systems for exploits

Simple exploit search in Armitage (metasploit)

Armitage Scan

A quick scan of my server revealed three ports open and (22, 80 and 443). Port 80 redirects to 443 and port 22 is firewalled.  I have WordPress and exploits I rued failed to work thanks to patching (always stay ahead of patching and updating of software and the OS.

k006-ports

Without knowing what I was doing I was able to check my WordPress against known exploits. 

If you open the Check Exploits menu at the end of the Attacks menu you can do a bulk exploit check.

kali_bulk

WP Scan

Kali also comes with a WordPress scanner

This will try and output everything from your web server and WordPress plugins.

/xmlrpc.php was found and I was advised to deny access to that file in NGINX. xmlrpc.php is ok but can be used in denial of service attacks.

I had a hit for a vulnerability in a Youtube Embed plugin but I had a patched version.

k007-wpscan

TIP: Check your WordPress often.

More to come (Draft Post).

  • OWASP scanner
  • WPSCAN
  • Ethical Hacker modules
  • Cybrary training
  • Sent tips to @FearbySoftware

Tips

Don’t have unwanted ports open, securely installed software, Use unattended security updates in Ubuntu, update WordPress frequently and limit plugins and also consider running more verbose audit tools like Lynis.

More Reading

Read my OWASP Zap guide on application testing and Cloudflare guide.

I hope this guide helps someone.

Ask a question or recommend an article

[contact-form-7 404 "Not Found"]

Revision History

v1.2 added More Reading links.

v1.1 Added bulk exploit check.

v1.0 Initial post

Useful Linux Terminal Commands

by

Below are Ubuntu Linux commands I use often to setup, debug maintain servers.

Advertisement:



Read this guide for Useful OSX Commands (for setting up Apache, PHP, MySQL, Adminer etc on OSX)

I recently moved my domain from a C-Panel hosted domain (and Email to Google G Suite (my guide here)) to a self-managed Digital Ocean domain (my LetsEncrypt Guide here, my Digital Ocean guide here, my AWS setup guide here, my Vultr setup guide here) and needed to transfer my WordPress site. Setup your own Digital Ocean Ubuntu server form $5  a month (get the first 2 months free by clicking here) or setup your own Vultr Ubuntu server for as low as $2.5/month by clicking here.

How to Reboot (from time to time when prompted with *** system restart required ***” messages appear).

How to set up a  bash file (*.sh) as Executable.

The file will now be executable.

Viewing your crontab (Windows Task Scheduler equiv)

Ping a port

Rename a folder

Set the owner of a folder

Check the rsync port

Run a file every 1 minute

Show server name

How to verify patch status for Meltdown and Spectre

Read my guide here to install the patch here.

Verify Spectre and Meltdown patch status

or

Restart network

More here.

Show Operating System Name

hostnamectl ! grep "Operating System"
 Operating System: Ubuntu 10.04.3 LTS

Show installed Packages

Show all packages with “PHP” in the name.

Send Messages Other Logged In Users (CLI)

Show all user logged in

Sent a message to usr1

Sent a message to the root console

Messages will appear at the bottom of the user’s console.

Processor

List processes in a tree view (… = removed)

View major processes by usage/memory

Read more on ps here.

How big is a folder

Change FIle Create/Modify time’s

Change File Creation Date

Change file modify/accessed time

Tree

Tree needs to be installed first

Show an ASCII representation of a folder structure

Show files in a  structure

Show directories

List all files and folders in a structure

Backup a  www folder

Common (Digital Ocean) Debugging commands

Networking

Display all TCP connections

Display all UDP connections

List all Listening Connections

Show all Network stats

Show all UDP Network stats

Show all TCP Network stats

Show network packets

Displaying raw info

Show open ports

Upload a file to a remote server over ssh

Zip files

Install zip

Zip a  whole directory (with high compression)

Zip a  whole directory (excluding *.tmp, *temp, *.bak and *.zip file types)

Zip Help

Backup NGINX

Unzip

Installing Unzip

Unzip a  file

unzip filename.zip

Updates

Setup Unattended Ubuntu Security updates

At login, you should receive

Show Configured NGINX server names

Services

Reload daemon services

Verify the MongoDB service file exists

View the status of services

Local Dump

Show All Defined Ports

Show defined rsync ports

Show listening ports (active connections)

Show service status

View your bash history for a past command (e.g “openssl”).

grep "openssl" ~/.bash_history
openssl req -new -newkey rsa:4096 -nodes -keyout fearby.key -out fearby.csr

View last 1 line of a file.

Ping a server

How to check system uptime (and load average)

Uptime in friendly format

The load averages at the end are the last 1, 5 min and 15 minutes.

The w command is handy for showing uptime information as-well as logged in users (The ‘w -i’  parameter -i is handy for seeing what IP people are logged in from).

How to download a file

How to edit NGINX configuration

sudo nano /etc/nginx/nginx.conf

and

How to find a file

also

locate php.ini

Find contents in files (recursive)

Find files by name and run a command on each

Show differences in files

Show contents of file (e.g urls.txt)

cat urls.txt
http://www.server1.com
http://www.server2.com
http://www.server3.com
http://www.server4.com
http://www.server5.com

Download all files mentioned in a text file.

View all packages with updates

Output:

or (on ubuntu 16.04)

Updates

Always backup your server’s configuration before updating.

Backup MySQL

Crontab

Add this to crontab -e to backup at 1 am every day

/scripts/shrinkmysql.sh  script to shrink SQL files

I had to add this to crontab

Cron job to shrink SQL dumps at 2am every day

Write to a single log file from the crontab at 2 am every day.

todo

Query a package (e.g. siege package)

How to set up a free SSL certificate (see my guide here).

How to set your timezone.

How to restart PHP

How to show the time on the server

sudo hwclock --show

Reload and restart the NGINX configuration and web server.

JSON viewing program

Installing

Using

or

output from json tool

Below are utilities I use a lot.

ncdu file size utility

Installing

Using

pydf disk checking utility

Installing

Using

output from pydf tool

ntp timezone service

Installing

Using

Displaying startup processes

Installing

Using

htop process manager.

Installing

Using

output from htop tool

Network Benchmarking (between two servers)

I use iperf to measure total bandwidth between two servers. You will need to allow port 5001 (TCP IPV4 and TCP IPV6 in and out) in any local firewalls and hosts GUIs.

Allow port 5001 on an ufw firewall (IN and OUT)

Deny IP

Allow port 22 access to known IP

Deny Outgoing Port

Allow out on port to known IP

More securing Ubuntu in the cloud here.

Install iperf on the target and source Ubuntu server.

Run this on the listening server.

Run this on benchmarking server (and add the IP of the listening server).

iperf -c 123.123.123.123

Results

Screen dump of ipref -c ip
iperf benchmarking output

Testing concurrent connections to a web server with siege.

Install siege

Benchmarking a HTTP server

./siege -t1m c10 'https://fearby.com'

#10 Concurent users

Benchmarking HTTPS sites

Install siege 4.0.2 ( steps here ).

Verify siege 4.0.2 is installed by running

Now can benchmark https sites

View incoming connections on the target server

I always increase my history size and tell it not to store duplicates.

Viewing your typed terminal history

Increasing your history size

How to Update Ubuntu

or

Edit SSH authorized keys

Search file and show lines where text matches

View packages with updates

View Boot Text

Automatic Monitoring (ever 1 second)

Active network connections

Network Packets

Free memory

Memory breakdown

or

Monitor NGINX memory

more soon…

Read this guide for Useful OSX Commands (for setting up Apache, PHP, MySQL, Adminer etc on OSX)

50 most useful Linux commands (view here).

View other Linux command informing sites here, here and here.

 

Donate and make this blog better




Ask a question or recommend an article
[contact-form-7 404 "Not Found"]

v1.8 Show users and senda messages to other users

v.71 Show Operating System Name

v1.7 added Meltdown and Spectre patch information

v1.6 Removed -x fom zip directory

V1.5 Restart Network

v1.4 added Useful OSX Commands

Setting up a Raspberry Pi Zero W

by

fyi: Here is my guide that I created while I set up a Raspberry Pi Zero W. My previous Raspberry PI 2 Setup Guide here and I wanted to try the single core Raspberry Pi Zero W. Eventually, want to run my Raspberry Pi from batteries (my older guide here).

Advertisement:



I plugged in a  2Amp 5v micro USB powder pack, micro HDMI cable, micro USB keyboard and expected to see a light but nothing.  This guide says the Raspberry Pi has no power LED. Unfortunately, I thought the power pack was faulty and I plugged it in a few times and corrupted the installed SD-card.

Failed Setup

After reading this guide I remembered I forgot to switch the monitor to the HDMI input (oops).

Yep, I realized I corrupted the OS on the SD Card.

Old Noobs Operating System

I checked the SD-Card (that I purchased from eBay) I noticed the SD-Card had an older NOOBS 2.0.0 installation in an in an invalid subfolder (that’s why it would not boot).

I plugged the Micro SD card into an SD-Card adapter and plugged it into my Mac (time to install a newer OS).

 

Downloading the latest Raspian Operating System

I visited https://www.raspberrypi.org/downloads/raspbian/ and download raspbian

I downloaded a 1.66GB Zip File.

The file took 30 mins to download.

I expanded the zip file to a 4.66 img file.

Formatting the SD-Card

I formatted my SD Card with the SD Card Associations SD formatting tool from https://www.sdcard.org/downloads/formatter_4/ here.

Setting up the SD-Card

The Raspberry Pi Foundation has a great guide on how to install the Raspberry Pi Raspian image onto an SD-Card here.

I downloaded the free 64MB Etcher program for Mac OS here https://etcher.io/ in order to copy (flash) Raspian onto the SD-Card.

I opened Etcher and selected the Raspian Jesse image file and clicked flash.

The flash will take between 5-10 minutes.

SD Card Contents

The Raspian image is now ready for reinsertion back into the Raspberry Pi to setup

1st Raspberry Pi (Raspian) boot.

The raspberry pi does, in fact, have a  power/activity led with the latest operating system 🙂

The desktop loaded quite swiftly too. My Bay special keyboard detected just fine. You can manually connect to your wifi by entering the following command in the terminal (thanks to this guide).

Pi

 

Generic Config

I am not using the desktop much so I am happy setting 16MB for the video card.

Get the Mac Address and set a static IP Address (to allow remote SSH management)

After a reboot, I was logged into a shell and was able to bring up the devices network details (including mac address).

Test the network connection

I was successful;y able to ping a remote server via wifi.

Pi Zero Network Update

Static IP Address

Todo: I set up my local ADSL router to give the Raspberry Pi a Static (known) IP every time it boots.

Update the Pi Software (after the network is up)

Now that I knew the network is working I was able to update the Pi software.

Pi Zero

I re-ran the update just in case. Now my Pi is all up to date.

Installed Misc Software

More on setting up a NGINX web server here.

Set the Pi Mouse (regular sized USB devices)

I had small micro USB to larger USB adapters (from eBay)  but could only use a keyboard or mouse at any one time. I Also had a micro USB to multiple micro USB and Ethernet adapter (from eBay) (SN: YS-LAN38) but it failed to work with NOOBS 2.0.0 (maybe it will work with Raspian latest)

I wanted to test regular devices in the case of an emergency.

Pi Zer UBS Adapter

And the mouse works (and the X desktop worked with 16MB assigned)

Help

40 Useful Raspberry Pi Commands.

Todo

Add information on adding and using a 5MP Raspberry Pi camera and setting this up as a https://fearby.com/article/raspberry-pi-2b-security-webcam/secuerity webcam.

 

Donate and make this blog better





Ask a question or recommend an article
[contact-form-7 404 "Not Found"]

v1.2 DRAFT: Added more images