Uncategorized

Finding software that needs updating on Windows using SUMo (Review)

May 30, 2020 by Simon

Recently I blogged about using Acronis True Image 2020 to backup and restore your Windows computer after a friend had to manually reinstall her Windows PC.

Her Windows PC died and she could not recover Windows with a System Restore snapshot or via the retailers systems image recovery process.

I use Back Blaze software to automatically backup my computer but I do not have an up to date collection of needed installation files in case I need to perform a Clean Installation of Windows 10.

I found SUMo in the process of assisting my friend.

SUMo to the Rescue

I found SUMo (Software Update Monitor) from KC Softwares that allows me to analyse my existing installed software and their versions, find and download new versions (installers or setup files) for my software.

I am using the Pro version of SUMo (buy here).

Tip: The trial version will inform you of an update but it will not show you links to obtain the updates easily.

SUMo Trial Limitations

The best feature of SUMo is I can find apps on my PC that need updating.

Also in one click (Right click) you will be directed to the applications official site (official site (or App page on SUMo)) and be able to download the app installer to the new version of the app.

Rigth click on an outdated app in SUMo to get the new aopo.

The Trial version is limited to informing you that an update is available.

SUMo telling your there is an update available

Also updating SUMo to the PRO version streamlines it’s own updates too. You can update SUMo in the PRO version with one click. 🙂

SUMo Pro Scan

After I opening SUMo PRO (I waited for it to find installed apps) then I clicked “Check” to scan all my apps for updates.

SUMo happily went through the list of my installed apps and gathered the applications versions (and checked online if newer versions existed).

SUMo found 239 installed applications on my PC, 188 were up to date, 53 needed minor (recent updates) and 2 needed major (old) updates.

Its awesome that in one click and I can see what software needs updating. In the process I was able to see what applications I don’t use (they were old versions) and I uninstalled them

SUMo Settings

I will update my software and to ensure my software does not get out of date I enabled “Load at startup” and “Check for new software at startup” (SUMo).

I did not enable “Scan for new software at startup” but seeing SUMo Start and update will remind me to check for updates if I have time.

I also enabled “Show file path” in the SUMo Application list so I can see where the programs are. I found multiple instances of Python installed all over my PC.

Getting Application Updates

I right clicked on each of my applications (that needed updating) and clicked “Get Update from…” then “.. from developer’s Site (Direct)“.

If you don’t want to right click you can click the “Get Update” button in the tool bar to get selected applications update.

This will open a web browser page to the software vendors website.

SUMo also has a page available with stats on the application’s known statistics (nice table of versions, beta or stable status and numbers of users).

Updating Software

I was able to right click “Get Update from…” – “.. from developer’s Site (Direct)” for each application in the list that needed an update. I was directed to the software vendors page for each software where I could download the app updates.

Within minutes I had dozens of vendor pages open with links to new versions of my outdated software.

Dozens of site pages with fresh app installs.

Within a short period I had amasses gigabytes of new application installs.

4.5GB of data of new application installs

I installed each of the new updates and then moved the new installs to my B:\Installs folder (in case I have to manually reinstall Windows in the future).

Checking Application Update Status

After I updated my out of date applications I then right clicked on the apps in SUMo and clicked Check to re check the status of the applications

SUMo Right Click Actions

Fyi: SUMo has a number of actions in the right click menu (on an installed Application).

Delete: Removes item from list. Will appear again on next scan or manually addition
Ignore: Blacklist : this Software/company pair will never appear again, whatever the version, file path, …
Skip: Items remains listed but a SPECIFIC version gets no longer notified as update for a given time (1 day / week / month) or forever.

(snip from here)

I had a few items I Deleted from the list or Ignored.

My Visual Studio (development) software was reporting an older version of Python (2.7). I updated Visual Studio to the latest version but Python 2.7 remained. I deleted Visual Studio from the list and was prompted (this is not deleting the application from the hard disk just from the generated list.

After a few hours updating apps

I whittled down the list of my applications that needed updating

I could sort applications in the list by OK status or apps that needed Minor or Major updates applied.

All Apps Updated

I can now say that all of my Applications are all up to date.

Export

I used the export feature in SUMo to build an excel file listing all my applications.

Fyi: SUMo exported this list as a “;” separated file, I opened the file in Sublime Text Editor and replaced “;” with TAB and then pasted this list back into Excel.

Conclusion

SUMo allowed me to update my installed applications and update my aged cache of application installs I was keeping in case I needed to manually rebuild Windows.

SUMo made this an absolute pleasure to do.

Pros

Price: 19.99 Euros (about $33.27 AUD) is a bargain.
Simplicity (it just works so well, why does Windows not do this)
Saves me time finding out of date apps and update files for an emergency and inevitable manual Windows reinstall.
My apps are more secure after updating them

Cons

I needed this in my life 10 years ago
Would be nice to have an inbuilt help file (instead of email support)

Try the free version here

v 1.0 Initial Post

Using Python to use buttons on the PiTFT Plus 320×240 TFT Touchscreen with a PiHole

May 9, 2020 by Simon

I recently setup a Raspberry PI as a DNS Sinkhole to block ads and Trackers on my home network and I love it. I paid about $120 for a Raspberry Pi LCD Screen and Case from Pakronics to go with a Raspberry Pi 3B+ I had laying around to Setup as a PiHole. This post will show how I made my Screen buttons work.

This post is not a sponsored advertisement, I like to log what I have done for documentation

Long story short, the Pi Hole is great and the LCD Screen and Case is working a treat and I needed to get the buttons on the LCD Panel and case working.

I love the stats that PiHole provides.

In only have Python 2.7 (and for a device on a private home network) and this is ok, given Python 2.7 is no longer supported.

How to check your Python version.

$ python --version
Python 2.7.16

I know Python 2.7 is a bit old, Python 2.7.16 was released in March 2019. I need to update my script to Python 3.

I created this Python Script to handle the button events

sudo nano /scripts/simonpihole_buttons.py

I added this code to the Python file that will handle the LCD button presses.

Tip: Make sure your commands work before you add them to the Python Script.


#!/usr/bin/python

import subprocess
import time
import RPi.GPIO as GPIO
import os

GPIO.setwarnings(False)

channel_list = [17, 22, 23, 27]

backlightOn = True
piHoleOn = True

def buttonPowerOff(channel):
    os.system("sudo shutdown -h now")

def buttonToggleBacklight(channel):
    global backlightOn
    if backlightOn:
        backlightOn = False
        backlight.start(0)
    else:
        backlightOn = True
        backlight.start(100)

def buttonDisablePiHole(channel):
    global piHoleOn
    if piHoleOn:
        piHoleOn = False
        os.system('pihole disable')
    else:
        piHoleOn = True
        os.system('pihole enable')

def buttonRestartDNS(channel):
    os.system('pihole restartdns')

GPIO.setmode(GPIO.BCM)
GPIO.setup(channel_list, GPIO.IN, pull_up_down=GPIO.PUD_UP)
GPIO.setup(18, GPIO.OUT)

backlight = GPIO.PWM(18, 1023)
backlight.start(100)

GPIO.add_event_detect(27, GPIO.FALLING, callback=buttonPowerOff, bouncetime=200)
GPIO.add_event_detect(22, GPIO.FALLING, callback=buttonToggleBacklight, bouncetime=200)
GPIO.add_event_detect(23, GPIO.FALLING, callback=buttonDisablePiHole, bouncetime=200)
GPIO.add_event_detect(17, GPIO.FALLING, callback=buttonRestartDNS, bouncetime=200)

while True:
    time.sleep(1)

GPIO.cleanup()

I tested the script and commands many many times and I can the python script to test the button functionality

cd /scripts
sudo python simonpihole_buttons.py

I decided to make the first button (from the top) power down the pi, then Disable the PiHiole Service then toggle the Raspberry Pi LCD (ON/OFF) and finally the last button restarts the DNS service.

I created this file to create the service for the Python script

sudo nano /etc/systemd/system/simonspibuttons.service

I added the following code to the service

[Unit]
Description=Simons pibuttons Script
After=multi-user.target

[Service]
User=pi
WorkingDirectory=/home/pi
ExecStart=/usr/bin/python -u /scripts/simonpihole_buttons.py

[Install]
WantedBy=multi-user.target

I ran this to start the button service

sudo systemctl enable simonspibuttons.service

If this does not work you can reboot your pi.

{“type”:”block”,”srcIndex”:24,”srcClientId”:”00b90020-9b1d-4d70-a9f9-62282db13e7f”,”srcRootClientId”:””}After the service was enabled I checked it’s status.

sudo systemctl status simonspibuttons.service

● pibuttons.service - pibuttons Padd PiHole Script
   Loaded: loaded (/etc/systemd/system/simonspibuttons.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2020-05-08 19:13:33 AEST; 21h ago
 Main PID: 787 (python)
    Tasks: 3 (limit: 2067)
   Memory: 6.4M
   CGroup: /system.slice/simonspibuttons.service
           └─787 /usr/bin/python -u /scripts/simonpihole_buttons.py

May 08 19:13:33 raspberrypihole systemd[1]: Started Simons pibuttons Script.

Cool the service has enabled and is working

Result

After a few weeks I am happy with the button order and functionality.

Shutting down from a button press (before a storm or a the kids bedtime is priceless). I did add some of my wife’s nail polish to add a different colour to each button.

I intend to add a flat heat pipe and external heatsink to the PiHole so there will be heaps of room for printed labels for each button. The Pi CPU doe’s not get super hot but it does get above 55c from time to time.

I love my PiHole

Troubleshooting

If you having troubles making the service run the following to force the service creation (but enter your service name)

cd /etc/systemd/system
sudo systemctl --force --full edit simonpibuttons.service

The Python service won’t boot, try

systemctl journalctl --boot --unit=simonibuttons.service

The Python script won’t run, try

chmod +X /scripts/simonpihole_buttons.py

v1.0

Setting up a Raspberry PI as a DNS Sinkhole to block ads and Trackers

April 15, 2020 by Simon

What is PiHole?

Snip from WikiPedia: “Pi-hole is a Linux network-level advertisement and Internet tracker blocking application which acts as a DNS sinkhole (and optionally a DHCP server), intended for use on a private network. It is designed for use on embedded devices with network capability, such as the Raspberry Pi, but it can be used on other machines running Linux and cloud implementations. Pi-hole has the ability to block traditional website advertisements as well as advertisements in unconventional places, such as smart TVs and mobile operating system advertisements.”

What is a Raspberry PI?

A Raspberry PI is an inexpensive (5V Volt, 2 Amp) ARM based computer that can run off the power from a USB cable.

Here is a photo of my Raspberry Pi 3B+ with an Adafruit LCD Screen

My Raspberry Pi has the following specifications

4 x 1.4GHz 64-bit (quad-core processor)
1GB LPDDR2 SDRAM
Dual-band wireless LAN
Bluetooth 4.2/BLE,
Faster Ethernet
Extended 40-pin GPIO header
Full-size HDMI 4 USB 2.0 ports
5V/2.5A DC power input

My screen has the following specifications (purchased from Pakronics)

3.5″ display with 480×320 16-bit colour pixels
Resistive touch overlay

I plugged in a full sized USB Keyboard, Mouse and HDMI cable.

SD Card Choice

Read my guide to download and write an Raspberry Pi Operating System to an SD card.

I would not put a cheap/slow MicroSD card in the Raspberry PI, aim for at least a UHS (1) or UHS (3) speed SD card for the best bang for buck.

fyi: I bought a new 32GB Samsung UHS 1 Ultra Micro SD card and it died after 12 hours of use. I replaced it with another 32GB No name brand CLASS 10 SD Card I had laying around.

Raspberry Setup

I download and saved the Raspian (Full) Operating System to a SD Card and inserted it into my Raspberry PI 3B+ (view the guide here on preparing an Operating System on a SD card).

I used the American 110-240V AC to 5.25V 2500ma DC power supply (with a US to AUS adaptor) that came with the Adafruit Screen. It had a Micro USB connection on one end.

It did not work though (I just had a flashing red light on the Raspberry Pi).

I had an Australian 240V AC to 5V 2500ma DC power supply to Micro USB. from a previous project and it worked (the Raspberry Pi Started up).

I also have a number of Moki brand 240V to USB (1A and 2.4A) adapters.

I will use the 2.4mA plug. I know my Adafruit screen uses 100mA so this will do.

I plugged the HDMI cable into my Monitor and set up the HDMI as a Picture in Picture output so I can see my Main 4K screen (Display Port) and the Raspberry Pi HDMI input at the same time.

Frst Raspberry Pi Boot

Mmmm my 4K screen with a 1080P HDMI picture in picture image (from the Raspberry Pi).

The Raspberry Pi booted fast and a welcome screen appeared

Apologies in advance, photos below are bad (I don’t have a HDMI capture card).

I clicked Next to setup the Raspberry PI

I set my timezone and language

I set a password

I skipped connecting to WiFi (I want pure Ethernet)

I was prompted to update the software

Setup is complete

I rebooted the Raspberry Pi

Second Boot

I changed further configuration by clicking the Raspberry Pi start button then Preferences then Raspberry PI Configuration

Screenshot of the Raspberry Pi menu showing Raspberry Pi Configuration

I changed the hostname to “raspberrypihole”, set Boot to CLI , Login as “pi“, and set Wait for network.

Update: After my Samsung SD card died I re setup my PI with a no name brand SD card and entered the name “raspberrypihole“

Under display I reviewed the display options

I enabled SSH, SPI and I2C.

Enable SSH, SPI and I2C features screenshot.

I increased the GPU memory to 132GB

Time for a Reboot

SSH Access

I do not want to leave a keyboard, mouse and screen connected once I finish setting it up so I setup a SSH connection to the Raspberry Pi.

TIP: Putty is a free program for SSH connections.

I SSH’ed (more information on SSH below) to the Raspberry Pi and ran these commands to update it’s software and firmware.

sudo apt-get update  && sudo apt-get upgrade

Output

[email protected]:~ $ sudo apt-get update  && sudo apt-get upgrade
Hit:1 http://archive.raspberrypi.org/debian buster InRelease
Hit:2 http://raspbian.raspberrypi.org/raspbian buster InRelease
Reading package lists... Done
Reading package lists... Done
Building dependency tree
Reading state information... Done
Calculating upgrade... Done
0 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.

The program “htop” is good for viewing system resources.

Now it’s time to look at the Adafruit screen and case.

I put the Raspberry PI in the Adafruit Case

I purchased this kit for the Raspberry Pi, the LCD screen just connects to the Raspberry Pi GPIO pins. It has a Adafruit LCD screen and a case for my Raspberry Pi 3B+

Photo of a Raspberry Pi 3B+, LCD screen and case

The screen just connects onto the GPIO pins

The LCD screen allows you to use pins below the screen.

GPIO pins documentation from https://www.raspberrypi.org/documentation/usage/gpio/

GPIO pin documentation from https://www.raspberrypi.org/documentation/usage/gpio/

A nice stack 🙂

Photo showing the LCD screen connected to the Pi

The case clips are hard to clip over the Raspberry Pi (Don’t force it or you might break for Pi)

Photo showing a case clip over the raspberry pi board

The case clip near the GPIO pins is on

Photo showing the case clip near the GPIO pins.

The clip near the power plug was off because the Raspberry Pi was not positioned correctly

Photo of the Raspberry PI off center of the lugs

After 30 minutes I carefully put the Raspberry Pi and LCD screen into the Case.

Photo showing the LCD screen in the case.

Side of the case with USB and Ethernet and USB porws exposed.

Photo of the Case exposing the USB and Ethernet port

HDMI, Power and Audio plugs are visible and lined up 🙂

Photo showing HDMI, Power and Audio plugs

The screen is visible through the case

The screen dips down on one side, I might have to prop it up (hot glue gun) a bit inside later

Photo showing the LCD screen dips to one side

SSH Connections to the Raspberry PI

I created a SSH connection to my Raspberry PI with Putty

I created a SSH connection to the Raspberry Pi and connected to it.

I ran the “ifconfig” command to get a list of all network interfaces.

I ran “ifconfig” to list all network interfaces.

I ran these commands to update my Raspberry PI Software

sudo apt-get update
sudo apt-get upgrade
sudo apt full-upgrade
sudo apt -y dist-upgrade

I updated the Pi Firmware too (this is dangerous, only update if you have issues).

sudo rpi-update

I rebooted and connected to the Raspberry Pi and ran this command to get the Ethernet and wireless mac address.

The first interface is my Ethernet adopter the second if the WiFi adaptor.

ifconfig |grep ether
ether b8:27:eb:d9:00:86 txqueuelen 1000
ether b8:27:eb:8c:55:d3 txqueuelen 1000

The first Mac address is my Ethernet address on The Raspberry PI and the second is WiFi.

I logged into my router (Telstra DJA0230) and clicked Advanced then Local Network. I could see my DHCP range was from 192.168.0.2 to 192.168.0.254, I shortened this to 192.168.0.2 to 192.168.0.200 (so I can set a static IP Address for the Raspberry PI) then I set a Static IP address for the Raspberry pi to 192.168.0.201.

I rebooted the Raspberry PI and checked the IP address

I logged into my Router (at https://192.168.0.1)

When my Samsung SD card died I had to re-setup a new SD card but the IP address came across as the mac address stayed the same (as it was the same hardware), I did however change the name of the Static IP hostname in my home router to match the new name “raspberrypihole” (not “pihole”)

I set a static IP for this Ethernet address and defined 192.168.0.201 as the IP address.

Setting Up PiHole on the RaspBerry PI

I SSH’ed to my Raspberry Pi (with the new IP address) and ran this command

Now its time to install Pi Hole onto My Raspberry Pi

curl -sSL https://install.pi-hole.net | bash

I received a Root user error

I read this guide on temporarily allowing root logins then rebooted my Pi and connected again as root. Running as root is not a good idea long term buy it works and my location is secure.

I logged in as root and checked my username

# whoami
root

I re-ran this command to restart the PiHole Setup

curl -sSL https://install.pi-hole.net | bash

Root user is found

Available packages are downloaded

Root user check ok, disk space ok, packages downloading.

Confirmation warning.

The pi is showing a confirm install screen

PiHole is free, consider donating.

I donated. Thanks

This will pay for itself in no time.

Static IP address is required.

I chose to just have PiHole work on Ethernet (and not Wifi)

I was prompted to set my upstream DNS provider.

I selected all default blacklist lists.

I allowed PiHole to use IPv4 and IPv6.

My IP and Gateway was displayed on the screen.

Final warning about setting a static IP address.

The PiHole IPv6 address is show

Install a web server (Yes)

I chose to log all DNS queries.

I oped to allow the viewing of all logged data. This is less secure but I can reduce this later.

The PiHole setup resumed and PiHole was downloaded from GitHub.

A progress bar displayed the install status.

A PiHole admin URL and Password was displayed (write this down)

I loaded the PiHole initial admin screen (http://192.168.0.201/admin/) and it was a bit empty.

I login and logged in.

I had better uninstall the nextdns.io (my blog post about NextDNS.io here) as the Pi will now be the main DNS blocking Sinkhole in our house.

The Raspberry Pi Pi Hole service was up and waiting for connections

On my Windows 10 PC I added the DNS server for the PiHole in IPV4 and IPV6.

I obtained the PiHole IPV4 and IPV6 addresses (1) PiHole Admin, 2) Pi Hole Settings Page, 3) IP Address)

I added the Pi Holes IPV4 IP address to my Windows 10 IP Settings.

I added the PIHole DNS to the IPV4 and IPV6 on my Windows 10 Ethernet adaptor

I added the Pi Holes IPV6 IP address to my Windows 10 IP Settings.

After 18 hours the Pi Hole Admin interface (at http://192.168.0.201/admin/index.php) was reporting stats.

I can view stats for Protocol and answered queries

Pie charts by protocol and answered queries

I can also see stats for permitted and blocked domains

Default Block Lists

I can also see the source blocked domains

Add 3rd party block lists

I added these block lists to my PiHole list of sites to block (Thanks Jol)

https://v.firebog.net/hosts/Easyprivacy.txt
https://v.firebog.net/hosts/Prigent-Ads.txt
https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
https://raw.githubusercontent.com/FadeMind/hosts.extras/master/add.2o7Net/hosts
https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
https://www.github.developerdan.com/hosts/lists/ads-and-tracking-extended.txt
https://hostfiles.frogeye.fr/firstparty-trackers-hosts.txt
https://hostfiles.frogeye.fr/multiparty-trackers-hosts.txt
https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/android-tracking.txt
https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/SmartTV.txt
https://raw.githubusercontent.com/Perflyst/PiHoleBlocklist/master/AmazonFireTV.txt
https://v.firebog.net/hosts/Airelle-trc.txt
https://bitbucket.org/ethanr/dns-blacklists/raw/8575c9f96e5b4a1308f2f12394abd86d0927a4a0/bad_lists/Mandiant_APT1_Report_Appendix_D.txt
https://gist.githubusercontent.com/anudeepND/adac7982307fec6ee23605e281a57f1a/raw/5b8582b906a9497624c3f3187a49ebc23a9cf2fb/Test.txt
https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-blocklist.txt
https://gitlab.com/quidsup/notrack-blocklists/raw/master/notrack-malware.txt
https://hosts-file.net/ad_servers.txt
https://hosts-file.net/emd.txt
https://hosts-file.net/exp.txt
https://hosts-file.net/grm.txt
https://hosts-file.net/psh.txt
https://isc.sans.edu/feeds/suspiciousdomains_Medium.txt
https://mirror1.malwaredomains.com/files/justdomains
https://mirror.cedia.org.ec/malwaredomains/immortal_domains.txt
http://someonewhocares.org/hosts/hosts
https://phishing.army/download/phishing_army_blocklist_extended.txt
https://ransomwaretracker.abuse.ch/downloads/CW_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/LY_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/RW_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/TC_C2_DOMBL.txt
https://ransomwaretracker.abuse.ch/downloads/TL_C2_DOMBL.txt
https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/spy.txt
https://raw.githubusercontent.com/crazy-max/WindowsSpyBlocker/master/data/hosts/win10/spy.txt
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.2o7Net/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Risk/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/add.Spam/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/data/KADhosts/hosts
https://raw.githubusercontent.com/StevenBlack/hosts/master/hosts
https://raw.githubusercontent.com/ZeroDot1/CoinBlockerLists/master/hosts
https://reddestdream.github.io/Projects/MinimalHosts/etc/MinimalHostsBlocker/minimalhosts
https://s3.amazonaws.com/lists.disconnect.me/simple_ad.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_malvertising.txt
https://s3.amazonaws.com/lists.disconnect.me/simple_tracking.txt
https://v.firebog.net/hosts/AdguardDNS.txt
https://v.firebog.net/hosts/Airelle-hrsk.txt
https://v.firebog.net/hosts/Easylist.txt
https://v.firebog.net/hosts/Easyprivacy.txt
https://v.firebog.net/hosts/Prigent-Ads.txt
https://v.firebog.net/hosts/Prigent-Malware.txt
https://v.firebog.net/hosts/Prigent-Phishing.txt
https://v.firebog.net/hosts/Shalla-mal.txt
https://v.firebog.net/hosts/static/SamsungSmart.txt
https://v.firebog.net/hosts/static/w3kbl.txt
https://www.malwaredomainlist.com/hostslist/hosts.txt
https://www.squidblacklist.org/downloads/dg-malicious.acl
http://sysctl.org/cameleon/hosts
https://zerodot1.gitlab.io/CoinBlockerLists/hosts
https://zeustracker.abuse.ch/blocklist.php?download=domainblocklist
http://www.joewein.net/dl/bl/dom-bl.txt
http://www.networksec.org/grabbho/block.txt

I updated all block lists

954,481 domains on my block list 🙂

I now have the PiHole blocking 945,481 domains

LCD Screen Setup

I followed thin guide to setup the screen.

I can this code from the pi (logged in as root)

cd ~
wget https://raw.githubusercontent.com/adafruit/Raspberry-Pi-Installer-Scripts/master/adafruit-pitft.sh
chmod +x adafruit-pitft.sh
sudo ./adafruit-pitft.sh

I edited /boot/config.txt and changed these values

framebuffer_width=320
framebuffer_height=240

I was prompted to choose a screen

Select configuration:
1. PiTFT 2.4", 2.8" or 3.2" resistive (240x320)
2. PiTFT 2.2" no touch (240x320)
3. PiTFT 2.8" capacitive touch (240x320)
4. PiTFT 3.5" resistive touch (320x480)
5. PiTFT Mini 1.3" or 1.54" display (240x240)
6. MiniPiTFT 1.14" display (240x135) - WARNING! CUTTING EDGE! WILL UPGRADE YOUR KERNEL TO LATEST
7. Quit without installing

SELECT 1-7:

I entered “3” for PiTFT 2.8″ capacitive touch (240×320)

I then was prompted to set the rotation of the screen

Select rotation:
1. 90 degrees (landscape)
2. 180 degrees (portait)
3. 270 degrees (landscape)
4. 0 degrees (portait)

SELECT 1-4:

I entered “3” for 270 degrees (landscape).

I was prompted to allow the console to appear on the screen

Would you like the console to appear on the PiTFT display? [y/n]
y

I rebooted

Reboot [y/n]
y

Installing PADD to display PiHole stats on the LCD

I followed this guide to install PADD (the software that displays the PiHole stats on the LCD screen)

cd ~
wget -N https://github.com/jpmck/PADD/files/4320681/padd.txt
rename padd.txt paddsimon.sh
chmod +x paddsimon.sh

Making PADD starts at boot

Edit this file

sudo nano  ~/.bashrc

and add the following to the end of the file

# Run PADD
# If we're on the PiTFT screen (ssh is xterm)
if [ "$TERM" == "linux" ] ; then
  while :
  do
    /root/paddsimon.sh
    sleep 0.2
  done
fi

I rebooted the PI.

Turning the screen OFF from the CLI

I created a bash script with this contents to turn the screen off

#!/bin/bash
gpio -g pwm 18 0

Turning the screen ON from the CLI

I created a bash script with this contents to turn the screen on

#!/bin/bash
gpio -g pwm 18 1023

How to Update the PiHole from the CLI

I ran the following command to update the PiHole block lists

pihole -g

Output…

  [i] Pi-hole blocking is enabled
  [i] Neutrino emissions detected...
  [✓] Pulling blocklist source list into range

  [i] Target: raw.githubusercontent.com (hosts)
  [✓] Status: Retrieval successful

  [i] Target: mirror1.malwaredomains.com (justdomains)
  [✓] Status: No changes detected

  [i] Target: sysctl.org (hosts)
  [✓] Status: No changes detected

  [i] Target: s3.amazonaws.com (simple_tracking.txt)
  [✓] Status: No changes detected

  [i] Target: s3.amazonaws.com (simple_ad.txt)
  [✓] Status: No changes detected

  [i] Target: hosts-file.net (ad_servers.txt)
  [✓] Status: No changes detected

  [i] Target: raw.githubusercontent.com (ytadblock.txt)
  [✓] Status: Retrieval successful

  [i] Target: v.firebog.net (Easyprivacy.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (Prigent-Ads.txt)
  [✓] Status: No changes detected

  [i] Target: gitlab.com (notrack-blocklist.txt)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (hosts)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (spy.txt)
  [✓] Status: Retrieval successful

  [i] Target: www.github.developerdan.com (ads-and-tracking-extended.txt)
  [✓] Status: Retrieval successful

  [i] Target: hostfiles.frogeye.fr (firstparty-trackers-hosts.txt)
  [✓] Status: Retrieval successful

  [i] Target: hostfiles.frogeye.fr (multiparty-trackers-hosts.txt)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (android-tracking.txt)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (SmartTV.txt)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (AmazonFireTV.txt)
  [✓] Status: Retrieval successful

  [i] Target: v.firebog.net (Airelle-trc.txt)
  [✓] Status: No changes detected

  [i] Target: bitbucket.org (Mandiant_APT1_Report_Appendix_D.txt)
  [✓] Status: No changes detected

  [i] Target: gist.githubusercontent.com (Test.txt)
  [✓] Status: Retrieval successful

  [i] Target: gitlab.com (notrack-malware.txt)
  [✓] Status: Retrieval successful

  [i] Target: hosts-file.net (emd.txt)
  [✓] Status: No changes detected

  [i] Target: hosts-file.net (exp.txt)
  [✓] Status: No changes detected

  [i] Target: hosts-file.net (grm.txt)
  [✓] Status: No changes detected

  [i] Target: hosts-file.net (psh.txt)
  [✓] Status: No changes detected

  [i] Target: isc.sans.edu (suspiciousdomains_Medium.txt)
  [✓] Status: Retrieval successful

  [i] Target: mirror.cedia.org.ec (immortal_domains.txt)
  [✓] Status: No changes detected

  [i] Target: someonewhocares.org (hosts)
  [✓] Status: No changes detected

  [i] Target: phishing.army (phishing_army_blocklist_extended.txt)
  [✓] Status: Retrieval successful

  [i] Target: ransomwaretracker.abuse.ch (CW_C2_DOMBL.txt)
  [✓] Status: Retrieval successful

  [i] Target: ransomwaretracker.abuse.ch (LY_C2_DOMBL.txt)
  [✓] Status: Retrieval successful

  [i] Target: ransomwaretracker.abuse.ch (RW_DOMBL.txt)
  [✓] Status: Retrieval successful

  [i] Target: ransomwaretracker.abuse.ch (TC_C2_DOMBL.txt)
  [✓] Status: Retrieval successful

  [i] Target: ransomwaretracker.abuse.ch (TL_C2_DOMBL.txt)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (spy.txt)
  [✗] Status: Not found
  [✗] List download failed: no cached list available

  [i] Target: raw.githubusercontent.com (hosts)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (hosts)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (hosts)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (hosts)
  [✓] Status: Retrieval successful

  [i] Target: raw.githubusercontent.com (hosts)
  [✗] Status: Not found
  [✗] List download failed: no cached list available

  [i] Target: reddestdream.github.io (minimalhosts)
  [✓] Status: No changes detected

  [i] Target: s3.amazonaws.com (simple_malvertising.txt)
  [✓] Status: Retrieval successful

  [i] Target: v.firebog.net (AdguardDNS.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (Airelle-hrsk.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (Easylist.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (Prigent-Malware.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (Prigent-Phishing.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (Shalla-mal.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (SamsungSmart.txt)
  [✓] Status: No changes detected

  [i] Target: v.firebog.net (w3kbl.txt)
  [✓] Status: No changes detected

  [i] Target: www.malwaredomainlist.com (hosts.txt)
  [✓] Status: No changes detected

  [i] Target: www.squidblacklist.org (dg-malicious.acl)
  [✗] Status: Connection Timed Out (Cloudflare)
  [✗] List download failed: no cached list available

  [i] Target: zerodot1.gitlab.io (hosts)
  [✓] Status: No changes detected

  [i] Target: zeustracker.abuse.ch (blocklist.php?download=domainblocklist)
  [✓] Status: Retrieval successful

  [i] Target: www.joewein.net (dom-bl.txt)
  [✓] Status: Retrieval successful

  [i] Target: www.networksec.org (block.txt)
  [✓] Status: Retrieval successful
  [i] Received empty file: using previously cached list

  [✓] Consolidating blocklists
  [✓] Extracting domains from blocklists
  [i] Number of domains being pulled in by gravity: 1178534
  [✓] Removing duplicate domains
  [i] Number of unique domains trapped in the Event Horizon: 954486
  [i] Number of whitelisted domains: 2
  [i] Number of blacklisted domains: 0
  [i] Number of regex filters: 0
  [✓] Parsing domains into hosts format
  [✓] Cleaning up stray matter

  [✓] Force-reloading DNS service
  [✓] DNS service is running
  [✓] Pi-hole blocking is Enabled

I can view all possible command line options by running

pihole /?

Output..

Usage: pihole [options]
Example: 'pihole -w -h'
Add '-h' after specific commands for more information on usage

Whitelist/Blacklist Options:
  -w, whitelist       Whitelist domain(s)
  -b, blacklist       Blacklist domain(s)
  --wild, wildcard     Wildcard blacklist domain(s)
  --regex, regex       Regex blacklist domains(s)
                        Add '-h' for more info on whitelist/blacklist usage

Debugging Options:
  -d, debug           Start a debugging session
                        Add '-a' to enable automated debugging
  -f, flush           Flush the Pi-hole log
  -r, reconfigure     Reconfigure or Repair Pi-hole subsystems
  -t, tail            View the live output of the Pi-hole log

Options:
  -a, admin           Web interface options
                        Add '-h' for more info on Web Interface usage
  -c, chronometer     Calculates stats and displays to an LCD
                        Add '-h' for more info on chronometer usage
  -g, updateGravity   Update the list of ad-serving domains
  -h, --help, help    Show this help dialog
  -l, logging         Specify whether the Pi-hole log should be used
                        Add '-h' for more info on logging usage
  -q, query           Query the adlists for a specified domain
                        Add '-h' for more info on query usage
  -up, updatePihole   Update Pi-hole subsystems
                        Add '--check-only' to exit script before update is perfo                                                     rmed.
  -v, version         Show installed versions of Pi-hole, Web Interface & FTL
                        Add '-h' for more info on version usage
  uninstall           Uninstall Pi-hole from your system
  status              Display the running status of Pi-hole subsystems
  enable              Enable Pi-hole subsystems
  disable             Disable Pi-hole subsystems
                        Add '-h' for more info on disable usage
  restartdns          Restart Pi-hole subsystems
  checkout            Switch Pi-hole subsystems to a different Github branch
                        Add '-h' for more info on checkout usage

After 1 Week

After 1 week stats were rolling into the PIHole.

40% of all traffic was being blocked.

I could see blocked and allowed domain calls

I can white list domains if they are blocked.

I white listed fearby.com and events.gfe.nvidia.com

Done

This is what it looks like done

PiHole stats on the LCD screen

View of the pi from the left hand side

View of the pi from the right hand side

Nice

After 2 month(s)

After 2 months the PiHole served 266,737 DNS requests. 95,268 requests were blocked (13,000 ad’s, tens of thousands of trackers.

Did it block Ad’s

Mostly yes. Not all advertisements are blocked but most are.

Some YouTube Advertisements seem to get through but I am seeing far less Advertisements in web pages

Troubleshooting

I needed to white list “events.gfe.nvidia.com” to allow my video card drivers to upodate.

Cooling

The Raspberry Pi is running cool at 47c (even though it is in a tight space).

I might add a heat pipe to it and have an external fan. I will thermal epoxy the hat pipe to the Pi CPU and run it outside to a external heat sink and fan.

eBay purchase for a fan, thermal epoxy and heatpipes.

I have many spare heat sinks laying around.

I will update when the part’s arrive.

Rotating the screen

I did exit my /boot/config.txt to rotate my LCD Screen orientation

Buttons

Read this guide to see how I setup a Python script to make my buttons work.

Do edit your /boot.config.txt to configure your screen rotation (if need be) and to check if the LCD screen is setup (by Adafruit)

framebuffer_width=240
framebuffer_height=320

# --- added by adafruit-pitft-helper Sun 8 Mar 2020 12:29:49 AM AEDT ---

dtparam=spi=on
dtparam=i2c1=on
dtparam=i2c_arm=on
dtoverlay=pitft28-capacitive,speed=64000000,fps=30
dtoverlay=pitft28-capacitive,rotate=270,touch-swapxy=true,touch-invy=true

# --- end adafruit-pitft-helper Sun 0 Mar 2020 12:29:49 AM AEDT ---

Handy Links

Handy Guide: https://learn.adafruit.com/pi-hole-ad-pitft-tft-detection-display/pitft-configuration to configure the LCD Screen

Schematics of the screen: https://learn.adafruit.com/assets/25555

Donate to PiHole: https://pi-hole.net/donate/

Raspberry Pi GPIO Pins: https://www.raspberrypi.org/documentation/usage/gpio/

v 1.2

Use your space computer resources to fight COVID-19

March 14, 2020 by Simon

Please read here ( https://foldingathome.org/2020/03/10/covid19-update/ ) for all the information on the [email protected] fight to simulate druggable protein targets for COVID-19.

Snip from https://foldingathome.org/about/

“[email protected] (FAH or [email protected]) is a distributed computing project for disease research that simulates protein folding, computational drug design, and other types of molecular dynamics. As of today, the project is using the idle resources of personal computers owned by volunteers from all over the world. Thousands of people contribute to the success of this project.”

The [email protected] GitHub project page is located here https://github.com/FoldingAtHome/coronavirus

Setup [email protected]

Warning running [email protected] on your completer all the time may cause the computer to over-heat, die (or have a shorter life). If you do not want this risk then don’t install [email protected] Use at your own risk.

Go to https://foldingathome.org/start-folding/ and download the

Open the installer and click Next

Read the Licence Agreement and click Next

Select Custom install then Next (for more information)

Choose Express of Custom Install Option Screenshot

Choose a folder to install to (I chose my fastest Hard Drive a PCIE-4 SSD) and click Next

Choose a folder to install to screenshot

Choose where to install the configuration data and click Next

Choose where to save the configuration data

Choose how to start the [email protected] client. Automatically as login, at boot time or manually. I chose Automatically at login.

I also ticked Enable [email protected] screen saver and click Install

Choose how to start the Folding@home client. Automatically as login, at boot time or manually.

[email protected] will install.

[email protected] has installed, click Finish

I chose to start the [email protected] app

[email protected] Screen Saver

The [email protected] screensaver displays the current scientific package simulation.

Windows System Tray icon

You can use the [email protected] system tray icon to start/stop any scientific work projects and review other settings.

Active Protein Viewer

Thew protein viewer will show the active scientific protein simulation.

Demo protein viewer.

Warning

The [email protected] app will consume a fair amount of processing power.

On my system I have processing power to use set to the default of “Medium” and it is working 11 of my 16 CPU threads heavily. I have an 8 core 16 thread AMD 3700X processor

My video card (a Nvidia Geforce RTX 2080) is about 45% busy.

Video card is showing 45% busy in Task Manager.

The [email protected] Advanced Control app allows you to reduce the CPU usage

Folding@home Advanced Control, Screenshot.

Web Control

By opening the [email protected] System Tray icon then clicking Web Control I can see a good overview of the projects being simulated and the status.

You can dedicated your system to run projects from a specific project (e.g Alzheimer’s or Cancer etc) but to contribute directly to Corona Virus at this time, leave the project as “Any” COVID-19 work is coming

Warning

It is advisable to check tour processors temperature core temp (I use Core Temp)

With all processor cores busy my processor is hitting 72c (warm but not hot). I might turn up the fans on my Processor cooler.

My video card is running about 51c, this is low because it is water cooler. I use GPU-Z to show my GPU temperatures.

You can configure the [email protected] client to slow down if need be, the following settings are

Light
- CPU is Folding at half speed, GPU Folding is off.
Medium
- CPU is Folding at three-quarter speed. GPU Folding is on. This is the default setting, and recommended for most users.
Full
- CPU is Folding at full speed. GPU Folding is on. This is the most productive setting. Will cause fans to run faster and may produce more heat.
While I’m working
- Folding is enabled to run at all times, based on the slider position.
Only when idle
- Folding starts only when the system has not been used for several minutes.

If temperatures are high you can lower the priority of the work completed.

I doubt my system will overheat outside of Summer.

Photo of my processor water cooler (radiator)

Photo of my video card water cooler (radiator)

Can I just work on COVID-19 simulations/projects (not Alzheimers, Cancer, Huntingtons or Cancer)?

No, not at this time as the interface does not allow this yet (it’s coming) but you can add a few expert settings to grab the beat projects if they are available. Most of the COVID projecds are in BETA.

Right click on the [email protected] SysTray Icon and click Advanced Control then click Configure button then click the Expert Tab, under Extra client options click Add and add these values.

Name: client-type

Value: bigadv

Click OK then Save.

It should look like this.

How does [email protected] work

[email protected] will use your CPU and or GPU in the background to do science.

I will let [email protected] work in the background while I work 🙂 I will keep and eye on the Temperatures and make sure my completer does not get too hot.

COVID-19 [email protected] Word Packages

I now have COVID-19 work packages processing on my computer

[email protected] Project Description
> Cause: covid

>CORONAVIRUS PROJECT

>Coronavirus SARS-CoV-2 (COVID-19 causing virus) protease – potential drug target

>These projects are part of [email protected]’s efforts to assist researchers around the world taking up the global fight against COVID-19. At the time of release, this is the latest news update: https://foldingathome.org/2020/03/10/covid19-update/

>These projects are CPU projects to simulate the main protease of the COVID-19, a possible drug target. These supplement high-priority GPU projects of the main protease as well as the COVID-19 receptor binding domains. We hope to have preliminary drug screening simulations on CPUs released in the next week.

>Please stay tuned to https://foldingathome.org/news/ for more current updates.

View all [email protected] projects

Does [email protected] work?

Yes

Happy folding.

Links

Read about other expert configuration values here

Here are some handy [email protected] links

v1.3 Now Receiving COVID-19 [email protected] projects

v1.2 Title Typo

v1.1 Updated Web Interface and Water cooling photos

v1.0 Initial Draft

Protecting your devices with nextdns.io a DNS based service that blocks malicious websites, trackers, ads, typo squatting domains, new or parked domains, TLD’s, mature YouTube content and comments and more

March 13, 2020 by Simon

I was going to setup a local (in my house) PiHole (with a Raspberry PI) that blocks internet trackers (DNS Sinkhole) and Advertisements (yes, like the ones on my website) because I don’t want my kids consuming a bucket loads of Advertisements online when they watch YouTube.

I am against online trackers and big data building a profiles on kids that are 6 and 10 years old. I have stopped using Facebook, Twitter and stopped using Google Analytics on this website.

I demo’ed to my son about the big data sucking up his data by looking at an IT retailer here in Australia for a random computer product then a few seconds later we looked as news sites (with Advertisements) in the UK and the US and to his surprise Advertisements for the randomly selected product in Australia was on his screen (sent from the other side of the world).

I am not against Content Creators making money from Advertisement revenue I am against the privacy issue. If you love consuming a Content Creators stuff then support them on their Merch store(s).

I don’t want my kids to accidentally fall victim to Malware, Cryptojacking, Phishing or spammy or known bad websites. I have a leading Antivirus products on their Computers but it is best not to put all of your eggs in one basket.

Enter https://nextdns.io (Free)

fyi: NextDNS.io is in BETA development and if you want rock a rock solid experience you may want to wait until until the beta period is over (Maybe March 2020. The only issue I have had is the NextDNS.io systray app sometimes does not open, I can work around this by starting and stopping the NextDNS.io service (“NextDNS DNS53 to DoH proxy“) in the Windows services app.

You can probably tell from the title of this post that NextDNS is a DNS based service that blocks malicious websites, trackers, ads, typo squatting domains, new or parked domains, TLD’s, mature YouTube content and comments and more. In my research for PiHole I found that nextdns.io was mentioned (How-to: Pi-Hole Plus DNSCrypt Setup on Raspberry Pi 4).

I logged into https://nextdns.io/ and was impressed by their slick interface.

Quick Setup Steps

Create a free account at https://my.nextdns.io/signup
Login to https://my.nextdns.io/ and click the Setup tab
Choose your platform that you want to setup nextdns onto (e.g Windows, Android etc)
Follow the setup prompts to access the installer and obtain the Configuration ID
Setup the app and enter the Configuration ID
Enable NextDNS

Then login to the analytics tab at https://my.nextdns.io/ to see the app in action

Read on for more technical information

https://my.nextdns.io/ Security Options (Tab)

On their account page they had these features

Threat Intelligence Feeds
Block domains known to distribute malware or launch phishing attacks and botnet command-and-control servers using a blend of the most reputable threat intelligence feeds—all updated in real-time.

Google Safe Browsing
Block malware and phishing domains using Google Safe Browsing—a technology that examines billions of URLs per day looking for unsafe websites. Unlike the version embedded in some browsers, this does not associate your public IP address to threats and does not allow bypassing the block.

Cryptojacking Protection
Prevent the unauthorized use of your devices to mine cryptocurrency.

DNS Rebinding Protection
Prevent attackers from taking control of your local devices through the Internet by automatically blocking DNS responses containing private IP addresses.

IDN Homograph Attacks Protection
Block domains that impersonate other domains by abusing the large character set made available with the arrival of Internationalized Domain Names (IDNs)—e.g. replacing the Latin letter “e” with the Cyrillic letter “е”.

Typosquatting Protection
Block domains registered by malicious actors that target users who incorrectly type a website address into their browser (e.g. gooogle.com instead of google.com).

Domain Generation Algorithms (DGAs) Protection
Block domains generated by Domain Generation Algorithms (DGAs) seen in various families of malware that can be used as rendezvous points with their command and control servers.

Block Newly Registered Domains (NRDs)
Block domains registered less than 30 days ago. Those domains are known to be favored by threat actors to launch malicious campaigns.

Block Parked Domains
Parked domains are single-page websites often laden with ads and devoid of any value. Parked domain monetization can sometimes get mixed up with suspicious practices and malicious content.

Top-Level Domains (TLDs) Blocking
Block all domains and subdomains belonging to specific TLDs (e.g “.ru”).

Block Child _______ Abuse Material (CSAM)
Block domains hosting child ______ abuse material with the help of Project Arachnid, operated by the Canadian Centre for Child Protection. No information is transmitted back to Project Arachnid when a domain is blocked.

https://my.nextdns.io/ Privacy Options (Tab)

Under their privacy tab they offer these features

NextDNS Recommended Ads & Trackers Blocklist
A comprehensive blocklist to block ads & trackers in all countries. This is the recommended starter blocklist.

> 78,795 entries • Updated 6 minutes ago

Block Disguised Third-Party Trackers
Automatically detect and block third-party trackers disguising themselves as first-party to circumvent recent browser’s privacy protections like ITP.

Allow Affiliate & Tracking Links (Disabled by default)

Allow affiliate & tracking domains common on deals websites, in emails or in search results. Those usually only get called after manually clicking on a link.

https://my.nextdns.io/ Parental Controls (Tab)

Under the Parental Controls tab these options are available

Safe Search
Filter out adult content on all major search engines, including images and videos. This will also block access to search engines not supporting this feature.

YouTube Restricted Mode
Filter out mature videos on YouTube and block embedded mature videos from being watched on other websites. This will also hide all comments.

Block Bypass Methods
Prevent or hinder the use of methods that can help bypass NextDNS filtering on the network. This includes VPNs, proxies, Tor-related software and encrypted DNS providers.

https://my.nextdns.io/ Blacklist Controls

You have the ability to blacklist (block) one or many domains. I decided to block a few sites from my kids eyes.

https://my.nextdns.io/ Whitelist Controls

You have the ability to whitelist (allow) one or many domains. I white;listed my site.

IMHO: The words Blacklist and Whitelist are a bit old, maybe they should use Allow Site/Deny Site?

How to Setup nextdns.io

Easy setup instructions are available for Android, iOS, Windows, macOS, Linux, Chrome OS, Firefox and Routers

Setup instructions for Android, iOS, Windows, macOS, Linux, Chrome OS, Firefox, Routers

I set these options

I disabled access to Mature websites along with Gambling, Piracy and Dating sites at my.nextdns.io

I enabled the Block Bypass Methods

I enabled YouTube Restriction Mode

Setting to enable YouTube restriction mode

I went to the Privacy Tab at https://my.nextdns.io and added all Blocklists (about 100 lists).

Tip: Don’t enable the “No Google (Completely block Google and its services)” as this will kill YouTube and my email (GSuite)

I am impressed by the number of available Blocklists and update frequency

Block ads & trackers using the most popular blocklists available—all updated in real-time.

Here is a list of the BLock lists availabe at the tile of this post.

NextDNS Recommended Ads & Trackers Blocklist
A comprehensive blocklist to block ads & trackers in all countries. This is the recommended starter blocklist.
78,795 entries • Updated 17 minutes ago

AdGuard Simplified Domain Names filter
A filter composed of several other filters (English filter, Social media filter, Spyware filter, Mobile Ads filter, EasyList and EasyPrivacy) and simplified specifically to be better compatible with DNS-level ad blocking.
github.com/AdguardTeam/AdguardSDNSFilter • 29,858 entries • Updated 13 hours ago

hpHosts (ATS / Ads & trackers)
hpHosts is a community managed and maintained hosts file that allows an additional layer of protection against access to ad, tracking and malicious websites.
hosts-file.net • 45,734 entries • Updated 2 days ago

Steven Black
Extending and consolidating hosts files from several well-curated sources like adaway.org, mvps.org, malwaredomainlist.com, someonewhocares.org, and potentially others.
github.com/StevenBlack/hosts • 51,977 entries • Updated a day ago

Disconnect (Tracking)
Free yourself from unwanted tracking. Enjoy a faster, safer internet.
disconnect.me • 34 entries • Updated 2 days ago

Disconnect (Ads)
Free yourself from unwanted tracking. Enjoy a faster, safer internet.
disconnect.me • 2,701 entries • Updated 2 days ago

CAMELEON
CAMELEON is a free system that helps Internet users or administrators to blocks web-adverts.
sysctl.org/cameleon • 20,568 entries • Updated 2 days ago

Anudeep’s Blacklist
A list of adserving and tracking sites maintained by me. This list will be updated frequently. github.com/anudeepND/blacklist • 38,251 entries • Updated 2 days ago

NSABlocklist
Block all known NSA / GCHQ / C.I.A. / F.B.I. spying servers. Originally based on 2007 published Wikileaks documents and includes my own modifications from 2008, 2012, 2014 and 2015.
github.com/CHEF-KOCH/NSABlocklist • 8,199 entries • Updated 17 minutes ago

EasyList
EasyList is the primary filter list that removes most adverts from international webpages, including unwanted frames, images and objects. It is the most popular list used by many ad blockers and forms the basis of over a dozen combination and supplementary filter lists.
easylist.to • 17,646 entries • Updated 17 minutes ago

Peter Lowe
Blocklist for use with hosts files to block ads.
pgl.yoyo.org/adservers • 3,283 entries • Updated 11 hours ago

notracking
Automatically updated, moderated and optimized list for blocking ads, trackers and other online garbage.

github.com/notracking/hosts-blocklists • 81,155 entries • Updated 7 hours ago

EasyPrivacy
EasyPrivacy is an optional supplementary filter list that completely removes all forms of tracking from the internet, including web bugs, tracking scripts and information collectors, thereby protecting your personal data.
easylist.to • 6,869 entries • Updated 17 minutes ago

AdGuard Mobile Ads filter
Filter that blocks ads on mobile devices. Contains all known mobile ad networks.
kb.adguard.com/general/adguard-ad-filters#mobile-ads-filter • 1,002 entries • Updated 19 hours ago

Fanboy’s Annoyance List
Fanboy’s Annoyance List blocks Social Media content, in-page pop-ups and other annoyances; thereby substantially decreasing web page loading times and uncluttering them.
easylist.to • 1,028 entries • Updated 17 minutes ago

StreamingAds
Streaming services ads sources.
github.com/FadeMind/hosts.extras • 57 entries • Updated 2 days ago

Disconnect (Malvertising)
Free yourself from unwanted tracking. Enjoy a faster, safer internet.
disconnect.me • 2,736 entries • Updated 2 days ago

Goodbye Ads
Specially Designed for Mobile Ad Protection.
github.com/jerryn70/GoodbyeAds • 76,536 entries • Updated 2 days ago

AdGuard Tracking Protection filter
The most comprehensive list of various online counters and web analytics tools. If you do not want your actions on the Internet be tracked, use this filter.
kb.adguard.com/general/adguard-ad-filters#tracking-protection-filter • 5,014 entries • Updated 18 hours ago

AdAway
Blocking mobile ad providers and some analytics providers.
github.com/AdAway/adaway.github.io • 12,176 entries • Updated 2 days ago

WindowsSpyBlocker (Spy)
Block spying and tracking on Windows systems.
github.com/crazy-max/WindowsSpyBlocker • 365 entries • Updated 2 days ago

MVPS HOSTS
Includes entries for most major parasites, hijackers and unwanted Adware/Spyware programs!
winhelp2002.mvps.org/hosts.htm • 10,476 entries • Updated 2 days ago

AdGuard Base filter
Filter that enables removing of the ads from websites with English content.
kb.adguard.com/general/adguard-ad-filters#base-filter • 19,667 entries • Updated an hour ago

someonewhocares.org (Dan Pollock)
Protects you from many types of spyware, reduces bandwidth use, blocks certain pop-up traps, prevents user tracking by way of “web bugs” embedded in spam, provides partial protection to IE from certain web-based exploits and blocks most advertising you would otherwise be subjected to on the internet.
someonewhocares.org/hosts • 14,404 entries • Updated 2 days ago

dbl.oisd.nl
Internet’s #1 domain blocklist. Blocks Ads, Mobile Ads, Phishing, Malvertising, Malware, Tracking, Telemetry, CryptoJacking, Analytics, Spyware, Ransomware, Exploid, Fraud, Abuse, Scam, Spam, Hijack, Misleading Marketing.
oisd.nl • 1,160,440 entries • Updated 5 hours ago

NoTrack Tracker Blocklist
Contains one of the largest compilation of sites associated with tracking your online activities and invading your privacy.
gitlab.com/quidsup/notrack-blocklists • 13,412 entries • Updated 2 days ago

antipopads
List of popads.net domains.
github.com/Yhonay/antipopads • 11,442 entries • Updated a day ago

1Hosts (Complete)
Protect your ‘data’ & eyeballs from being auctioned to the highest bidder.
forum.xda-developers.com/android/general/badmojr-one-host-file-to-block-t3713360 • 66,731 entries • Updated 2 days ago

AdGuard Social Media filter
If you do not like numerous «Like» and «Tweet» buttons on all the popular websites on the Internet, subscribe to this filter, and you will not see them anymore.
kb.adguard.com/general/adguard-ad-filters#social-media-filter • 55 entries • Updated 3 hours ago

squidblacklist.org (Ads)
Blocks advertisements and tracking.
www.squidblacklist.org • 552 entries • Updated a month ago

UncheckyAds
Windows installers ads sources.
unchecky.com • 10 entries • Updated 2 days ago

Fanboy’s Enhanced Tracking List
Fanboy’s Enhanced Tracking List blocks common tracking scripts such as Omniture, Webtrends, Foresee, Coremetrics, Google-Analytics, Touchclarity, ChannelIntelligence.
fanboy.co.nz • 151 entries • Updated 2 days ago

EasyList China
EasyList China is an affiliated filter list written by John and Li that specifically removes adverts on Chinese language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 5,818 entries • Updated 17 minutes ago

280blocker
280blocker adblock domain lists.
280blocker.net • 1,059 entries • Updated 2 days ago

Lightswitch05 – Ads & Tracking
A programmatically expanded list of hosts I’ve found to not be on other lists.
www.github.developerdan.com/hosts • 107,013 entries • Updated 21 hours ago

Shalla’s Blacklists (tracker)
Site keeping an eye on where you surf and what you do in a passive. Covers web bugs, counters and other tracking mechanism in web pages that do not interfere with the local computer yet collecting information about the surfing person for later analyis. Sites actively spying out the surfer by installing software or calling home sites are not covered with tracker but with -> spyware.
www.shallalist.de • 1,246 entries • Updated 2 days ago

Shalla’s Blacklists (adv)
All about advertising: This includes sites offering banners and banner creation as well as sites delivering banners to be shown in webpages. Advertising companies are listed, too.
www.shallalist.de • 14,275 entries • Updated 2 days ago

Energized Ultimate
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. Flagship Protection Pack from Energized Protection.
github.com/EnergizedProtection/block • 970,212 entries • Updated 16 hours ago

EasyList Germany
EasyList Germany is a filter list written by the EasyList authors MonztA, Famlam and Khrin that specifically removes adverts on German language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 595 entries • Updated a day ago

CHEF-KOCH’s HOSTS Spotify Ad-Filter List
Blocks all Spotify Ads, easy peasy lemon squeezy!
github.com/CHEF-KOCH/Spotify-Ad-free • 4,041 entries • Updated 2 days ago

add.2o7Net
2o7 Network tracking.
hostsfile.org/hosts.html • 1,286 entries • Updated 2 days ago

Personal Blocklist by WaLLy3K
Content added to this list has been manually verified, and is updated irregularly. firebog.net/about • 753 entries • Updated 2 days ago

EasyList Dutch
EasyList Dutch is an affiliated filter list written by the EasyList author Famlam that specifically removes adverts on Dutch language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 83 entries • Updated 17 minutes ago

Liste FR
Liste FR is an affiliated filter list written by Lian, Crits and smed79 that specifically removes adverts on French language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 5,411 entries • Updated 17 minutes ago

Energized Basic
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. An All-Rounder Balanced Protection Pack.
github.com/EnergizedProtection/block • 654,392 entries • Updated 16 hours ago

CertyficateIT
Polish ads filter.
github.com/MajkiIT/polish-ads-filter • 3,128 entries • Updated 8 hours ago

Energized Regional Extension
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. An Extension to Block Regional Annoyances.
github.com/EnergizedProtection/block • 64,320 entries • Updated 16 hours ago

Energized Spark
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. True Lite Blocking.
github.com/EnergizedProtection/block • 67,662 entries • Updated 16 hours ago

ABPindo
ABPindo is an affiliated filter list written by hermawan that specifically removes adverts on Indonesian language websites.
github.com/ABPindo/indonesianadblockrules • 594 entries • Updated 2 days ago

Energized Blu
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. A Mid Ranger Flagship Protection Pack.
github.com/EnergizedProtection/block • 330,817 entries • Updated 16 hours ago

EasyList Italy
EasyList Italy is a filter list written by the EasyList author Khrin that specifically removes adverts on Italian language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 204 entries • Updated 17 minutes ago

AdGuard Russian filter
Filter that enables removing of the ads from websites in Russian.
kb.adguard.com/general/adguard-ad-filters#russian-filter • 3,944 entries • Updated an hour ago

Energized Blu Go
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. A Lightweight Mid Ranger Protection Pack.
github.com/EnergizedProtection/block • 131,781 entries • Updated 16 hours ago

RU AdList
Russian supplement for EasyList.
forums.lanik.us/viewforum.php?f=102 • 7,887 entries • Updated 17 minutes ago

yhosts
AD hosts爱好群，群号:201973909；
github.com/vokins/yhosts • 8,940 entries • Updated 2 days ago

EasyList Hebrew
EasyList Hebrew is an affiliated filter list written by BsT that specifically removes adverts on Hebrew language websites.
github.com/easylist/EasyListHebrew • 129 entries • Updated 13 hours ago

EasyList Czech and Slovak
EasyList Czech and Slovak is an affiliated filter list written by tomasko126 that specifically removes adverts on Czech and Slovak language websites.
github.com/tomasko126/easylistczechandslovak • 83 entries • Updated 2 days ago

hostsVN
Hosts block ads of Vietnamese – Hosts chặn quảng cáo của người Việt.
github.com/bigdargon/hostsVN • 18,846 entries • Updated 2 hours ago

Lightswitch05 – Tracking Aggressive
A very aggressive block list for tracking, geo-targeting, & ads. This list will likely break functionality, so do not use it unless you are willing to maintain your own whitelist.
www.github.developerdan.com/hosts • 4,919 entries • Updated 2 days ago

Liste AR
Liste AR is an affiliated filter list written by smed79 and Crits that specifically removes adverts on Arabic language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 85 entries • Updated 17 minutes ago

Bulgarian list
Bulgarian list is an affiliated filter list written by Alex that specifically removes adverts on Bulgarian language websites.
easylist.to/pages/other-supplementary-filter-lists-and-easylist-variants.html • 10 entries • Updated 2 days ago

Latvian List
Latvian List is an affiliated filter list written by anonymous74100 that specifically removes adverts on Latvian language websites.
notabug.org/latvian-list/adblock-latvian • 53 entries • Updated 2 days ago

EasyList Lithuania
EasyList Lithuania is an affiliated filter list written by gymka that specifically removes adverts on Lithuanian language websites.
github.com/EasyList-Lithuania/easylist_lithuania • 19 entries • Updated 2 days ago

Energized Xtreme Extension
Strictly blocks advertisements, malwares, spams, statistics & trackers on both web browsing and applications. An Extreme Solution for Ultimate Protection.
github.com/EnergizedProtection/block • 32,635 entries • Updated 16 hours ago

hufilter
Block hungarian ads.
github.com/hufilter/hufilter • 149 entries • Updated 2 days ago

Finnish Easylist Addition
Finnish adblock list.
github.com/finnish-easylist-addition/finnish-easylist-addition • 66 entries • Updated 2 days ago

ABPVN List
The ABP advertising filter is built with the mission of improving the browsing experience for users and for the Vietnamese.
abpvn.com • 19,490 entries • Updated 2 hours ago

Frellwit’s Swedish Hosts File
Reduce your exposure to ads, tracking, scams & badware, and occasionally some annoyances on (mostly) Swedish websites.
github.com/lassekongo83/Frellwits-filter-lists • 669 entries • Updated 2 days ago

1Hosts (Pro)
Protect your ‘data’ & eyeballs from being auctioned to the highest bidder.
forum.xda-developers.com/android/general/badmojr-one-host-file-to-block-t3713360 • 215,683 entries • Updated 2 days ago

bkrucarci turk-adlist
Ad servers list to block ads on Turkish websites.
github.com/bkrucarci/turk-adlist • 847 entries • Updated 2 days ago

AdAway Blocking Hosts File for Japan
AdAwayで使用可能な、日本環境用広告除去用hostsを公開します。日本環境用に特化しています。
logroid.github.io/adaway-hosts • 31,809 entries • Updated 2 days ago

Goodbye Ads Ultra
Specially Designed for Mobile Ad Protection. Premium protection.
github.com/jerryn70/GoodbyeAds • 459,196 entries • Updated 2 days ago

ad-wars
只是 ad-wars 的帮助文档
github.com/jdlingyu/ad-wars • 1,548 entries • Updated 2 days ago

1Hosts (mini)
Protect your ‘data’ & eyeballs from being auctioned to the highest bidder.
forum.xda-developers.com/android/general/badmojr-one-host-file-to-block-t3713360 • 53,512 entries • Updated 2 days ago

YousList
Block filter for advertisements, mainly on Korean sites.
github.com/yous/YousList • 145 entries • Updated 5 hours ago

No Google Completely block Google and its services. github.com/nickspaargaren/no-google • 302 entries • Updated 2 days ago

That is a lot of protection

Nextdns.io Android Setup

I setup nextdns.io on my Android 10 Device

I installed the official NextDNS app from the Play Store (https://play.google.com/store/apps/details?id=io.nextdns.NextDNS)

After the app was installed I entered my specified Configuration ID (tip: don’t copy the space on the end of the text) into the In the NextDNS app.

Now I can enable or disable the NextDNS.io protections.

Next DNS App on Androiud (Simple Connect/Disconnect Button)

Options are available in the app but are limited. If you need to change options go to https://my.nextdns.io/

I enabled NextDNS then I tried accessing a mature website.

Access to the site was blocked.

I tried bypassing the invalid HTTP’s certificate.

This too was blocked also.

I opened YouTube and some advertisements were gone but not all (Hello PewDiePie, nice Merch)

Comments were disabled, Nice. Restricted mode is enabled, this is perfect for my kids (PewDiePie Fans).

I am not against Content Creators making money from Advertisement revenue I am against the privacy issue. If you love consuming a Content Creators stuff then support them on their Merch store(s).

Windows 10 Setup

Installing on Windows is easy, login to https://my.nextdns.io/ and click on the Windows Button

1. Install the official NextDNS app → https://nextdns.io/download/windows/stable.

Agree to the licence agreement

Choose an Install location

Installing TAP Device (looks like it is from https://openvpn.net/)

You need to install the device.

2. After installing, right-click on NextDNS icon in the Systray then open the Settings. Set your supplied ###### as Configuration ID found at

3. Right-click on NextDNS icon in the Systray, then click on Enable.

You can enable or disable the NextDNS service from the Windows system tray

You can also Start and Stop the service from the Windows services app

Blocking bad sites , YouTube comments etc all work on Windows as they do on Android.

Nextdns.io iPad Setup

If I can get my daughters iPad from her hands I will show the steps. Basically the same as android but from the Google Play store.

1. Install our official app from the App Store → https://apps.apple.com/app/nextdns/id1463342498

2. Open the app then go to Settings and toggle “Use Custom Configuration”. Enter your Configuration ID

3. Enable NextDNS in the NextDNS iOS app

Blocking bad sites , YouTube comments etc all work on iOS as they do on Windows and Android.

Analyics (the nest feature)

The best part of NextDNS.io is the superb analytics available. Because they are a DNS server they can track incoming request(s) from all of my connected computes.

In 2 days my home network made 22,247 queries to the internet, 5684 requests were blocked, I could see the top accessed websites (antivirus and kids games) and a map of where the requests were going.

If the Analytics was not enough I could see all the requests logs.

I filtered all blocked web traffic.

The internet is a dumpster fire with all this tracking.

Pricing

Snip from here. “Pricing is completely free during the beta, then free up until about 300,000 DNS queries/month — $1.99/month for unlimited queries. If you decide to stay on the free plan, NextDNS will simply behave like a classic public resolver after reaching the 300,000 queries limit.”

Conclusion

Don’t surf the web without protection. Every parent should install this on their kids machines.

I can’t wait for this product to leave beta, I wan’t this service in my house.

I have not been paid to promote this product.

Version: v1.2

v1.2 Quick Setup Guide

v1.1 Updated Conclusion and Pricing.

v1.0 Initial Draft

How to use the Raspberry Pi Imager to download an Operating System to an SD card ready to use with your Raspberry Pi

March 9, 2020 by Simon

This is quick post to show how you can choose and write an Operating System to a SD card for use on a Raspberry PI computer.

Raspberry PI

A Raspberry Pi is a wonderful 5V mini computer that you can use in IoT projects, Programming, Retro Gaming or as a desktop replacement.

View all Raspberry Pi computers here: https://www.raspberrypi.org/products/

All Raspberry Pi Boards at https://www.raspberrypi.org/products/

The Raspberry Pi has an official Operating system called Raspian.

Many other 3rd party Operations system exist

Raspian comes with a Windows like desktop that you can use.

You can even install a Retro Pi image to allow you to play old arcade and Console games in an emulator.

Raspberry Pi Zero W

For this post I will be using the older Raspberry Pi Zero W to setup an operating system.

I have a few Raspberry PI Zero W’s (v1.1) laying around

Closer look at one of them

Raspberry Pi Zero W with all plugs plugged in (angle 2)

It’s only has 1x CPU Core and 512MB of memory. It had Micro USM keyboard and power in plugs.

I have added a small eBay heat sink to the CPU to help limit thermal throttling when the CPU is working hard.

The good thing about the Raspberry Pi is you can add a fan from the supplied 5V pin. Read more about available pin’s here: https://raspberrypi.stackexchange.com/questions/83610/gpio-pinout-orientation-raspberypi-zero-w or here https://www.raspberrypi.org/documentation/usage/gpio/

Obviously I would mount it better than this when I turn it on.

You can buy inexpensive drop on references boards to help remind ytou of each pin.

The reference board just pops over the pins

Please visit this page to compare all Raspberry Pi Model Specifications: https://en.wikipedia.org/wiki/Raspberry_Pi#Specifications

SD Card (Required)

The Raspberry Pi does not have a Hard Drive (Magnetic, SSD or NVME) and you will need a Micro SD card. An operating system will need to be written to the SD card and inserted into the MicroSD slot on the Raspberry Pi.

I would not put a cheap/slow MicroSD card in the Raspberry PI, aim for a UHS (1) or UHS (3) speed SD card for best bang for buck.

SD card speeds https://www.sdcard.org/developers/overview/speed_class/

Make sure you SD card is not write protected (if it is inserted into an old SD card adapter). My computer does not have a MicroSD card slot.

I have 2x SD card adopters that my MicroSD card inserts into. Then the SD card inserts into my PC.

Make sure the write lock is not on.

SD Image Writing Software

I want to setup a new Pi for use as a PiHole Ad Blocker on my home network.

I can see the PiHole software needs Rasbian Operating System on the Raspberry Pi (ARM CPU)

Now that I know what Operating System I need to install I need to grab the SD Imager software from https://www.raspberrypi.org/downloads/

I downloaded the software and open it to install it.

I clicked Install

I clicked Next

After the software was installed I clicked Finish to open the software

In the background I formatted (as FAT32) my 32GB MicroSD Card (that was in the SD card adapter).

I will not do a quick format so every sector on the card is tested and formatted.

This may take 30 minutes to format buy it is worth while. I don’t want a bad SD card making its way to the Raspberry PI.

Format comlplete

Open the Raspberry Pi Imager program that you installed earlier.

I clicked “CHOOSE OS“

I can choose an operating system to download and write to the SD card

I chose “Raspian (other)” then “Raspian Full“

Raspian is now the selected operating system

I clicked “CHOOSE SD CARD” and selected my 32GB SD card.

The Operating System and SD Card have been selected

I clicked “WRITE” and within 5 seconds noticed my WiFi was download the Operating System, Nice.

Raspian Full Downloading via WIFI and writing to the SD card.

The write process will take at least 30 minutes (depending on your internet speed).

When it’s complete a Verify will be performed and hopefully will have a SD card ready for use in a Raspberry Pi computer.

The write completed and files are on the SD card.

I can now use this SD card in my Raspberry Pi and boot from it to setup the Operating System.

I will prepare 3x MicroSD cards I have ready for My PiHole adventures. Blog post coming.

Adafruit BME680 (Adafruit BME680 – Temperature, Humidity, Pressure and Gas Sensor) on a Wemos D1 Mini (ESP8266)

March 8, 2020 by Simon

Here is a short post to help document communicating a Adafruit BME680 (Adafruit BME680 – Temperature, Humidity, Pressure and Gas Sensor) on a Wemos D1 Mini (ESP8266).

Yes this is normally a Security/Server blog but I needed a break (and an excuse to teach my 10 year old son how to code). I have promised my son I will make an IoT Easter Hat for school this year. Why not have an 80Mhz ESP8266 and a few sensors in his hat.

A few years back before I got into Server’s and Security I was developing a weather related mobile app IoT device and . n have always had a hardware and coding bug.

I have played with Arduino’s in the past but I wanted a micro controller with WiFi, enter the ESP8266

WeMos Mini D1

Maybe it is time to upgrade to the ESP8266 with WiFi

I was temped to order a cheaper Wemos D1 Mini from China (eBay) but delivery is usually 2 months.

Ebay listing for WeMos D1 Mini (ESP8266) from CHina for $4.27 AUD

I ordered a Wemos D1 Mini from Australia (eBay) but it did not turn up after 10 days (6 days after the estimated delivery), I got a refund from the seller.

ebay listing for a Wemos D1 Mini (ESP8266) from Australia for $7.99 AUD

I ended up visiting my new Jaycar store and purchased a WeMos D1 Mini.

Wemos D1 Mini from Jaycar.com.au ion Australia for $24.95

$24.95 is a lot but they have it in stock (and are open on Sundays).

#win #win

Soldering the Wemos D1 Mini

This is what it looked for before I soldered the pins on.

Picture of a Wemos D1 MIni with no legs soldered on

I decided to solder the board this way.

Picture of a Wemos D1 MIni with no legs soldered on (flipped)

It had been a while since I soldered (was previously Leaded solder).

I got some Lead Free solder from Jaycar and tested some soldering (lead free is harder than leaded solder).

I got some new Flux from Jaycar as the flux I had was old and dry.

Picture of Rosion Flux from jaycar.com.au

I practices some soldering again. Lead free solder needs good flux and a higher solder temperature.

Even though my short sight is an issue (I am getting old) solder joints seem ok (for Lead Free). New flux did the trick. Thank’s JayCar

Picture of 3x soldered points, looks good enough for me.

(Now I need to get some reading glasses for close up activity)

Sensors

I had a few DHT11 and DHT22 sensors laying around but they don’t measure barometric pressure so time to upgrade.

I ordered a BME280 (GYBEMP) from eBay but it did not like the Arduino IDE and I could not get any data from the device.

Picture of a 4 pin BMP280 sensor from eBay (GYBMEP)

I looked for tutorials on YouTube and the newer 6 pin BMP280 seemed to be the device to get (Smarter Home Things).

Picture of a 6 pin BMP280 sensor from eBay

This sensor was no good either, I could not get any data from it. Libraries were installed too???

From now on I will only use Adafruit sensors as they are well supported libraries and they have fritzing compatible parts (https://github.com/adafruit/Fritzing-Library)

I purchased a Adafruit BME680 and an Adafruit Analog UV Sensor.

ESP8266 Board (Arduino IDE)

You will need to add ESP8266 Boards added to the Arduino IDE (read more here: https://github.com/esp8266/Arduino).

Open the Ardiono IDE and click, File then Preferences and add this URL to the Additional Boards Manger URLs.

https://arduino.esp8266.com/stable/package_esp8266com_index.json

Re-open your Arduino IDE and click Tools, Boards, Boards Manager and add the “ep8266” boards

Now I can choose ESP8266 Boards in the Arduino IDE

I have a WeMos D1 Mini (ESP8266) so I will choose it.

Adafruit BME8266 Arduino IDE Libraries

Now I need to add the Adafruit BME680 library to the Arduino IDE.

I added the Adafruit BME680 librarary to Android IDE (I restarted the IDE)

Adafruit BME680 libraries added to Android Studio

In the Android IDE I clicked on File, New then Adafruit BME680 Library then eme680test

Picture of the New Adafruit BME680 project menu

Code

This is the code in the sketch

  /***************************************************************************
  This is a library for the BME680 gas, humidity, temperature & pressure sensor

  Designed specifically to work with the Adafruit BME680 Breakout
  ----> http://www.adafruit.com/products/3660

  These sensors use I2C or SPI to communicate, 2 or 4 pins are required
  to interface.

  Adafruit invests time and resources providing this open source code,
  please support Adafruit and open-source hardware by purchasing products
  from Adafruit!

  Written by Limor Fried & Kevin Townsend for Adafruit Industries.
  BSD license, all text above must be included in any redistribution
 ***************************************************************************/

#include <Wire.h>
#include <SPI.h>
#include <Adafruit_Sensor.h>
#include "Adafruit_BME680.h"

#define BME_SCK 13
#define BME_MISO 12
#define BME_MOSI 11
#define BME_CS 10

#define SEALEVELPRESSURE_HPA (1013.25)

Adafruit_BME680 bme; // I2C
//Adafruit_BME680 bme(BME_CS); // hardware SPI
//Adafruit_BME680 bme(BME_CS, BME_MOSI, BME_MISO,  BME_SCK);

void setup() {
  Serial.begin(9600);
  while (!Serial);
  Serial.println(F("BME680 test"));

  if (!bme.begin()) {
    Serial.println("Could not find a valid BME680 sensor, check wiring!");
    while (1);
  }

  // Set up oversampling and filter initialization
  bme.setTemperatureOversampling(BME680_OS_8X);
  bme.setHumidityOversampling(BME680_OS_2X);
  bme.setPressureOversampling(BME680_OS_4X);
  bme.setIIRFilterSize(BME680_FILTER_SIZE_3);
  bme.setGasHeater(320, 150); // 320*C for 150 ms
}

void loop() {
  if (! bme.performReading()) {
    Serial.println("Failed to perform reading :(");
    return;
  }
  Serial.print("Temperature = ");
  Serial.print(bme.temperature);
  Serial.println(" *C");

  Serial.print("Pressure = ");
  Serial.print(bme.pressure / 100.0);
  Serial.println(" hPa");

  Serial.print("Humidity = ");
  Serial.print(bme.humidity);
  Serial.println(" %");

  Serial.print("Gas = ");
  Serial.print(bme.gas_resistance / 1000.0);
  Serial.println(" KOhms");

  Serial.print("Approx. Altitude = ");
  Serial.print(bme.readAltitude(SEALEVELPRESSURE_HPA));
  Serial.println(" m");

  Serial.println();
  delay(2000);
}

Connecting the WeMos D1 Mini to the Adafruit BME680

This is the wiring guide. My wire colours are in the brackets

Wemos D1 Mini	Adafruit BME680
3V3 (RED)	3V
Ground (BLACK)	Ground
D1 (BLUE)	SCK
D2 (YELLOW)	SDI

Schematic of the Adafruit BME680 and the Wemos D1 Mini

I used the Fritzing program to make the above circuit diagram.

Adafruit make available Fritzing compatible parts (https://github.com/adafruit/Fritzing-Library) for their products.

Here is the Fritzing Sketch (you may need to add “Adafruit BME680.fzpz” from the Adafruit parts library

Full credit for this wiring goes to esp8266learing.com for the original diagram (I redid it to make the labels on the BME680 visible (OCD))

I used the Adafruit parts library for Fritzing.

Adafruit parts for Fritrzing (visual) circuit documenting app (https://github.com/adafruit/Fritzing-Library)

Wemos D1 Mini (JayCar) Driver

I had to install the CP210x USB to UART Bridge VCP Drivers from Silicon Labs (CP210x_Windows_Drivers.zip) tpo get the JayCar WeMos D1 Mini working

The supplied “setup.exe” program did not work for me (it kept crashing) so I added the driver by manually to Windows.

Add Hardware

Click Start then type Run and press Enter.

Type “hdwwiz.exe” and click OK

The Add Hardware Wizard will show, click Next

Click Next (Show All Devices is selected)

Click Have Disk

I selected the “slabvcp.inf” from the extracted “CP210x_Windows_Drivers.zip“

I clicked Open then OK, I selected “Silicon Labs CP210x USB to UART Bridge” and clicked Next

The driver installed.

I rebooted (to be sure) then checked in device manager for the device.

Device manger showing the Silicon Labs CP210x USB to UART Bridge is setup as port 10

I can see the “Silicon Labs CP210x USB to UART Bridge” is setup as port 10

Wiring

Make the wiring as pictured below.

Here is the Fritzing Sketch (you may need to add “Adafruit BME680.fzpz” from the Adafruit parts library

Close enough

Wemos D1 Mini and the Adafruit BME680 wired up.

I doubled checked the wiring.

Wemos D1 Mini and the Adafruit BME680 wired up. (diffrent angle)

I plugged in the USB cable.

Wait until you hear the add new hardware sound (if you have sounds in Windows)

Arduino IDE Setup

In the Arduino IDE make sure you choose the same COM port as the previously installed UART Bridge and ensure “WeMos D1 R1” is selected as board type and check that the programmer is “AVRISP mkII“

Arduino IDE board and COM port selection

I clicked Upload Sketch in the Arduino IDE

I can now get Temperature, Pressure, Humidity and Gas readings over 3 wires 🙂

Conclusion

Don’t use cheap sensors or ESP8266 micro controllers from eBay.
Use Lead free solder (practice soldering).
Use fresh Flux (not old stuff)
JayCar is convenient

From now on I will use Adafruit sensors. It’s a shame Jaycar don’t have Adafruit sensors.

Now I need to output the information to a screen.

Maybe I connect this to https://blynk.io/

Maybe create a local web server?

Thanks for reading.

v1.3 New Featured Image

v1.1 Moved Heading.

v1.0 Initial Draft

Renewing a Let’s Encrypt HTTPS certificate on a back end web server with DNS Verification/Cloudflare API and acme.sh ACME Client on Debian

March 4, 2020 by Simon

Background

I have a Debian server hosted on UpCloud (because they are fast) with a Nginx Web Server, My Domain name is purchased through https://porkbun.com, DNS and SSL certificates are managed by https://cloudflare.com

Cloudflare are cool, for a number of reasons

Always Use HTTPS
Redirect all requests with scheme “http” to “https”. This applies to all http requests to the zone. — Always use HTTPS

Minimum TLS Version
Only allow HTTPS connections from visitors that support the selected TLS protocol version or newer. — Minimum TLS Version

Opportunistic Encryption
Opportunistic Encryption allows browsers to benefit from the improved performance of HTTP/2 by letting them know that your site is available over an encrypted connection. Browsers will continue to show “http” in the address bar, not “https”.TLS 1.3
Enable the latest version of the TLS protocol for improved security and performance.Automatic HTTPS Rewrites
Automatic HTTPS Rewrites helps fix mixed content by changing “http” to “https” — Opportunistic Encryption, TLS 1.3 and Automatic HTTPS Rewrites

Certificate Transparency Monitoring
Receive an email when a Certificate Authority issues a certificate for your domain. — Certificate Transparency Monitoring

Requests Through Cloudflare (Unique Visitors, Ttoal Requests, Percent Cached and Data Cached) — First class website caching and analytics.

Cloudflare handle my HTTPS certificate that the public receive. This costs me $5 a month and I don’t need to manage renewals.

https://dev.ssllabs.com/ reports that the site is Certificate is OK.

Cloudflare do allow you to create a free Cloudflare certificate to use on your website but I am happy with Lets Encrypt.

Origin Certificates
Generate a free TLS certificate signed by Cloudflare to install on your origin server.Origin Certificates are only valid for encryption between Cloudflare and your origin server.

Maybe one day I will try out the Cloudflare certificate on my website behind the scenes

Origin Certificate Installation
Follow the steps below to generate and install a certificate on your origin server.

I awoke today with news that Let’s Encrypt to revoke 3 million certificates on March 4 due to software bug. I will need to update my Lets Encrypt certificate on my server but the catch is because Cloudflare handle the HTTS certificate any attempt to renew my HTTPS certificate will be blocked by Cloudflare.

Acme.sh

This is where a handy Open Source script called (acme.sh) on Github comes in, it will renew my Let’s Encrypt Certificate from the command line and verify my server with Let’s Encrypt via DNS records. To do this it will use the Cloudflare API and create a DNS records to verify my server with Let’s Encrypt. Nice.

Go to https://github.com/acmesh-official/acme.sh and read how you can install acme.sh

e.g

curl https://get.acme.sh | sh

wget -O -  https://get.acme.sh | sh

git clone https://github.com/acmesh-official/acme.sh.git
cd ./acme.sh
./acme.sh --install

Now you can login to Cloudflare and generate an API Token

Once you have a Cloudflare API Token you can edit the acme.sh file

nano acme.sh

Add the following to the top of you acme.sh after the “VER=” line.


CF_Key="#########your#cloudflare-api-key##########"
CF_Email="[email protected]"
CF_Api="https://api.cloudflare.com/client/v4"

Review these values


_SUB_FOLDER_NOTIFY="notify"
_SUB_FOLDER_DNSAPI="dnsapi"
_SUB_FOLDER_DEPLOY="deploy"

VTYPE_HTTP="http-01"
VTYPE_DNS="dns-01"
VTYPE_ALPN="tls-alpn-01"

LOCAL_ANY_ADDRESS="0.0.0.0"

DEFAULT_RENEW=60

DEFAULT_DNS_SLEEP=120

NGINX="nginx:"

You can renew a Certificate via the Cloudflare API by running this command

sudo /root/.acme.sh/acme.sh --cron --home /root/.acme.sh --force

The output of the command will look like this

sudo bash /parentfolder/.acme.sh/acme.sh --cron --home /parentfolder/.acme.sh --force
[Wed Mar  4 18:20:59 AEDT 2020] ===Starting cron===
[Wed Mar  4 18:20:59 AEDT 2020] Renew: 'fearby.com'
[Wed Mar  4 18:20:59 AEDT 2020] Multi domain='DNS:fearby.com,DNS:www.fearby.com'
[Wed Mar  4 18:20:59 AEDT 2020] Getting domain auth token for each domain
[Wed Mar  4 18:21:00 AEDT 2020] Getting webroot for domain='fearby.com'
[Wed Mar  4 18:21:00 AEDT 2020] Getting webroot for domain='www.fearby.com'
[Wed Mar  4 18:21:00 AEDT 2020] fearby.com is already verified, skip dns-01.
[Wed Mar  4 18:21:00 AEDT 2020] www.fearby.com is already verified, skip dns-01.
[Wed Mar  4 18:21:00 AEDT 2020] Verify finished, start to sign.
[Wed Mar  4 18:21:00 AEDT 2020] Lets finalize the order, Le_OrderFinalize: https://acme-v02.api.letsencrypt.org/acme/finalize/###############/############
[Wed Mar  4 18:21:02 AEDT 2020] Download cert, Le_LinkCert: https://acme-v02.api.letsencrypt.org/acme/cert/############################
[Wed Mar  4 18:21:02 AEDT 2020] Cert success.
-----BEGIN CERTIFICATE-----
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
**************************************************************
-----END CERTIFICATE-----
[Wed Mar  4 18:21:02 AEDT 2020] Your cert is in  /parentfolder/.acme.sh/fearby.com/fearby.com.cer
[Wed Mar  4 18:21:02 AEDT 2020] Your cert key is in  /parentfolder/.acme.sh/fearby.com/fearby.com.key
[Wed Mar  4 18:21:02 AEDT 2020] The intermediate CA cert is in  /parentfolder/.acme.sh/fearby.com/ca.cer
[Wed Mar  4 18:21:02 AEDT 2020] And the full chain certs is there:  /parentfolder/.acme.sh/fearby.com/fullchain.cer
[Wed Mar  4 18:21:02 AEDT 2020] ===End cron===

Open the Crontab

crontab -e

Add this line to your crontab and save it.

0 0 1 */2 * /theparentfolder/.acme.sh/acme.sh --cron --home /root/.acme.sh --force

Conclusion

I can now update my websites behind the scenes HTTPS certificate that Cloudflare will use via the command line.

Monitoring cronjobs on Linux

Troubleshooting

Read the acme.sh documentation here: https://github.com/acmesh-official/acme.sh

Backing up files to a Backblaze B2 Cloud Bucket with Duplicati

June 6, 2019 by Simon

This guide will show how you can automatically backup files to a Backblaze B2 clouds bucket with the Duplicati software on Windows and Linux.

I have blogged about how you can use the Backblaze personal backup program ( here, Use my link and get your first month of Backblaze backups free). The personal application from Backblaze is cool but what if you wanted to store 10GB for free (the first 10GB is free) or setup different buckets for different computers or applications.

Welcome to Backblaze B2 Cloud Storage.

In Australia B2 cloud costs (+GST).

First 10GB is free.
$0.005c per GB stored per month.
To download, 1x GB costs $0.01c.

Check our my other related posts

This works alongside the private Backblaze backup mentioned here.

Screenshot: Duplicati Downalods page https://www.duplicati.com/download

Backblaze B2 Pricing

How does Backblaze B2 Compare that to other cloud data storage provides.

Screenshot of https://www.backblaze.com/b2/cloud-storage.html

Comparison chart.

Provider	Storage ($/GB/Month)	Download ($/GB)
Backblaze	$0.005	$0.01
Amazon S3	$0.021	$0.05+
Microsoft Azure	$0.018+	$0.05+
Google cloud	$0.020	$0.08

Vendor cost breakdown (source).

Read here to view the official page on pricing.

Snip:

Storage:

The first 10 GB of total storage is free, and is $0.005 per GB per month for additional storage beyond 10 GB.   

Download:

The first 1 GB of downloads per day are free.  The price to download stored data costs $0.01 per GB beyond the free daily 1 GB. 

Transactions:

A 'transaction' refers to any time an api call is made with your account, such as listing the contents of your bucket or downloading a file.   B2 will charge for some transactions however, for the average user, these charges are largely negligible unless you are making an extremely high volume of api calls.

Class A transactions are free.

The first 2,500 Class B transactions are free each day. Additional Class B API calls are charged at $0.004 per 10,000 calls.  

The first 2,500 Class C transactions are free each day. Additional Class C API calls are charged is $0.004 per 1,000 calls.

Vendor Relability

https://downdetector.com is a great site for viewing is a vendor is down or has issues, lets compare, Backblaze with Amazon, Azure and Google Cloud

Downdetector.com: Backblaze
Downdetector.com: Azure
Downdetector.com: Amazon
Downdetector.com: Google Cloud

Backblaze have has far fewer downtime issues during the great outage of May/June 2019 compared to its competitors. What I like is Backblaze tell us how they do it and tell us what hard drives are good or bad.

Creating a Backblaze B2 Account

You can have a sneak peek at the personal Backblaze backup signup steps in this guide (why not take up a free 1 month back free by clicking this link when you create an account).

From your account page click billing and add your account details

Screenshot of https://secure.backblaze.com/billing.htm?billing_page=b2

Read the B2 Strater Guide: https://www.backblaze.com/b2/docs/

Also review your review the following under ‘My Settings‘

Phone Numer (mobile) for usage alerts.
Two Factor Authorization for extra security (don’t forget to çopy your backup codes)

Setup Billing Alerts (just in case)

In the Backblaze portal, under ‘Caps and Alerts‘ you should set..

Alerts to your mobile phone (for Daily storage caps,bandwidth caps and class B/C caps)
Under ‘Daily Storage Caps’ click ‘edit’ and set monetary alerts (you can increase alert limits later)

Click ‘Edit Caps’ and set daily limits on all categories

I set alert limits of 10c (I can increase this later)

Daily 10c alerts should see is costs creep above $3 a month.

Screenshot of cap alerts, I set 10c warnings on everyitng

Create a cloud bucket to store files

Go to https://secure.backblaze.com/b2_buckets.htm and click ‘Create Bucket’

Screenshot of https://secure.backblaze.com/b2_buckets.htm (create bucket button)

You can create upto 100 buckets per account.

I created a descriptive bucket name that reminded me what server, drive and folder this was used to back up.

"#######-drive-backup-folder" (####### was the name of my computer)

Screenshot of bucket being created an named.

I created a number of buckets (all private).

The Backblaze portal lists all of my new storage buckets.

Bucket Storage (Lifestyle Settings)

You can click the ‘Lifecycle Settings‘ button and set the desired buckets file retention policy (overwrite or keep all versions of a file etc).

Read the official Lifestyle settings faq here.

The default is to keep all versions of a file. Choose a setting that is required for each bucket(for me ‘Keep only the latest version of the file’ is best and lowers potential cost blowouts. Having all versions of a file may be handy for a Documents folder.

Screenshot of Lifestyle settings at https://secure.backblaze.com/b2_buckets.htm

You could opt to keep all versions of a file (forever) but that would be expensive.

You tell Backblaze to delete older versions of a file after xx days (this is a good option)

You can also setup custom lifecycle rules to hide a file with a certain prefix after xx days then delete after xx days.

Create a Master Application Key for all buckets

You will need to create a Master Application Key that will grant you full access (Permissions: listKeys, writeKeys, deleteKeys, listBuckets, writeBuckets, deleteBuckets, listFiles, readFiles, shareFiles, writeFiles, deleteFiles ) to each bucket.

Snip from: https://secure.backblaze.com/app_keys.htm

Application keys are used as a pair: Key ID and Application Key. This allows B2 to communicate securely with different devices or apps. Once you generate your Master Application Key, this key has full capabilities. Create your own Application Keys to limit features like read/write. Learn more.

Do write down this “KeyID” and “applicationKey”

Screenshot of https://secure.backblaze.com/app_keys.htm (generated "KeyID" and "applicationKey")

Don’t lose the Master Application Key.

Create an App Key for each bucket

A master key is like an administrator account, its great for complete access but you should create a separate key per bucket for better security.

Back under the App Keys you can click “Add a New Application Key” for each bucket.

Screenshot of Add a new application key button

Read the official guide on Application Keys here

Screenshot of add application key (add name, option to choose a bucket and assign read/write acess

Do Set..

“Name of Key” (similar to your bucket (e.g “my-bucket-name-key”))
“Allow access to Bucket(s)” (choose a bucket to grant access to).
“Type of Access” (Read and Write, Read Only, Write Only)

Optional..

Set a “File name prefix” (snip from here)

"The file name prefix of files the download authorization token will allow b2_download_file_by_name to access. For example, if you have a private bucket named "photos" and generate a download authorization token for the fileNamePrefix "pets/" you will be able to use the download authorization token to access: https://f345.backblazeb2.com/file/photos/pets/kitten.jpg but not: https://f345.backblazeb2.com/file/photos/vacation.jpg."

Set a “Duration” (snip from here)

"The number of seconds before the authorization token will expire. The minimum value is 1 second. The maximum value is 604800 which is one week in seconds."

I now has 1x Master Application key and 4x Application Keys for my 4x Buckets

Screenshot of https://secure.backblaze.com/app_keys.htm (1 master key and 4x app keys)

TIP: Make sure you save the keys, if you did not grab the keys delete the keys and buckets (if there is not data and try again). If you don;t have the keys you are locked out of the buckets

OK, Let’s fill those buckets with data with Duplicati

Go to https://www.duplicati.com/ and familiarise yourself with the site

Setup Duplicati on Windows (10)

Go to https://www.duplicati.com/download and download the latest Windows client. At the time of writing Duplicati 2.0.4.5 beta was the latest version. Thew instal was 1MB (the download serve was not that fast)

Warning: Beta software may have bugs, feel free to check out the Github page for Issues etc.

Open the install and Click ‘Next‘

Duplicati install wizard start, click Next

Agree to the ‘Licence Agreement‘ by clicking ‘Next‘

Select Duplicati components to install and click ‘Next‘

Tick ‘Launch Duplicati‘ an click ‘Finish‘

Duplicati has finished installing screenshot.

Using Duplicati Software

Upon first open you will be prompted to set a password (do this even if you are the only user of thew computer)

Set a strong and unique password

Screenshot of setting a duplicati password

Click ‘Ok‘

When the page reloads it will report it is not logged in.

Screenshot of user is not logged in error.

Enter your password

The main screen of Duplicati reports that there are no scheduled set.

Screenshot no scheduled tasks screenshot.

OK, lets create a backup schedule.

Creating a Backup on Windows

If your screen is small you will need to click the menu at the top right and click ‘Add Backup’, if your screen is large there will be an ‘Add Backup’ menu on the left hand side of the screen.

Select ‘Configure a new backup‘ and click ‘Next‘

You will be able to name and describe the backup schedule name and be able to set up an encryption password.

Then click ‘Next‘

Screenshot of create backup schedule with name, description and encryption password

Under ‘Storage Type‘ choose ‘B2 Cloud Storage‘

When you choosing ‘B2 Cloud Storage‘ you will be directed to the screen below where you stecify..

B2 Cloud bucket name
Path in the bucket to store the files (leave blank)
Enter your Backblaze Account ID (e.g. ##########################)
Enter the Buckey ApplicationKey (e.g. ###############################)

Click ‘Next‘

I entered my credentials

Click ‘Test Connection‘, if you entered everything OK you should see ‘Connection Worked‘

Click ‘Next‘

Now you can select the folders to backup on your local computer

Click ‘Next‘

Now you can choose a schedule to backup

Screenshot choose time and days to run the backup.

Click ‘Next‘

Now you can set the remote backup size and backup retention mode.

Read the official guide on setting the volume size here

Increase the ‘Remote Volume Size’ if you are worried about consuming your daily API quota in Backblaze (this may happen if your upload speed is fast and your backup is very large).
Lower the ‘Remote Volume Size’ size if backups are small or your upload speed is low.

Screenshot of remote volume size and backup retention

Choose a desirable backup retention mode. Duplicati only offers full backups (not incremental or differential backups). This will be an issue if your backups are large.

Keep all backups (“Nothing will be deleted. The backup size will grow with each change.“)
Delete backups that are older than (“If at least one newer backup is found, all backups older than this date are deleted.“)
Keep a specific number of backups (“Once there are more backups than the specified number, the oldest backups are deleted.“)
Smart Backup Retention (“Over time backups will be deleted automatically. There will remain one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup.“)
Custom backup retention (“Enter a retention strategy manually. Placeholders are D/W/Y for days/weeks/years and U for unlimited. The syntax is: 7D:1D,4W:1W,36M:1M. This example keeps one backup for each of the next 7 days, one for each of the next 4 weeks, and one for each of the next 36 months. This can also be written as 1W:1D,1M:1W,3Y:1M.“)

Each storage option has pro’s and cons with potential cost implications. I would not set 300GB to “Keep all backups” and backup daily. That would be expensive.

Screenshot of the 5 backup retention modes

Click ‘Save‘

TIP: Don”t forget to save your encryption key if you set one

Screenshot of a duplicati warning saying have we saved our encryption ley

Now you will see the backup schedule on your Duplicati screen.

You can run, edit or delete the backup status here.

Screenshot of the backup schedule on the home screen.

I clicked ‘Run‘ and the backup was under way.

A progress bar updates as the backup happens.

A slight delay happens at the end of the backup.

Screenshot of backup waiting to finish screen.

I logged into the Backblaze portal to see how many files and usage credits were used.

Note it seems?

When the files are backed up a verification processes happens.

Now the backup has completed.

Screenshot of the backup completed screen

I waited 2 minutes and finally Backblaze updated it’s status, Duplicati used 3 files and 87MB

I logged into the view the Duplicati files that were uploaded and I could see the three files were there and they were encrypted.

Screenshot of B2 cloud encrypted Duplicati files.

Awesome

Restoring a Backup on Windows

Having a backup is useless you can restore the files (especially when they are encrypted).

Lets restore a few files and see if it works.

I opened Duplicati and clicked ‘Restore‘

I clicked the bottom option that listed my bucket name/backup name and clicked ‘Next‘

Before I ran the restore I deleted some location files that were just backed up.

Duplicati now asked me to choose files to restore.

Screenshot restore files treeview with selections

I chose the original location to restore to and clicked ‘Restore‘

Duplicati took a few minutes to restore the files.

The restore was a success.

Nice

Backblaze Costs?

Beware, make sure you calculate the costs to upload and download files.

Do set alarms on cap’s limits and review them often.

Second Backup Schedule on Windows

I quickly set a second backup schedule of a different folder to a different bucket.

This backup was 300GB so I will only backup once a month

The backup is listed on the front screen of Duplicati.

Do create multiple buckets and set multiple backup schedules to break up the backups into smaller chunks.

Screenshot of multiple backup schedules.

Setup Duplicati on Linux (Debian or Ubuntu)

Lets setup B2 Cloud Storage backup on Linux.

I visited the Duplicati forums here. Duplicati informed me that the Linux client needs mono installed

Duplicati depends on other software. For Windows, Microsoft .NET Framework 4.5 or higher needs to be installed. Linux and Mac OS X require Mono to be installed.

Mono is a Cross Platform open source .NET that runs on Linux (read here)

On Ubuntu 1.08 I installed mono with these commands

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/ubuntu stable-bionic main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

On Debian 9.9 I installed Mono with these commands (from here)

sudo apt install apt-transport-https dirmngr gnupg ca-certificates
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF
echo "deb https://download.mono-project.com/repo/debian stable-stretch main" | sudo tee /etc/apt/sources.list.d/mono-official-stable.list
sudo apt update
sudo apt install mono-devel gtk-sharp2
sudo apt-get install libmono-2.0-1

I rebooted the serves and the CPU and memory went crazy after I installed Mono. My Nixstats Service reports were going off telling me my servers CPU was busy.

If you don’s have reports on Linux server CPU/MEM/Disk reports coming to you via Email and Telegram then read this.

I felt sorry for my web and database server server(s) so I rebooted them, wow, My website came back up in about 10 seconds thanks to UpClouds fast Max IOPS disks.

Installing Duplicati on Linux

I visited the download Duplicati page here and copied the Ubuntu and Debian download URL.

At the time of writing it was..

https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I downloaded the deb package

cd /utils/duplicati
wget https://updates.duplicati.com/beta/duplicati_2.0.4.5-1_all.deb

I then ran this command to install the package

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

I received this error even thpogh libmono-2.0-1 was installed

dpkg: dependency problems prevent configuration of duplicati:
duplicati depends on libmono-2.0-1; however:
Package libmono-2.0-1 is not installed.

This forum post helped me, I ran

sudo apt install -f

Output

sudo apt install -f
Reading package lists... Done
Building dependency tree
Reading state information... Done
Correcting dependencies... Done
The following additional packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
Suggested packages:
  indicator-application
The following NEW packages will be installed:
  libappindicator0.1-cil libappindicator1 libdbusmenu-glib4 libdbusmenu-gtk4
  libindicator7 libmono-2.0-1
0 upgraded, 6 newly installed, 0 to remove and 7 not upgraded.
1 not fully installed or removed.
Need to get 143 kB of archives.
After this operation, 541 kB of additional disk space will be used.
Do you want to continue? [Y/n]

And I ran the following

sudo apt dist-upgrade --fix-missing

I now ran the following to install the duplicati package.

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

It worked on Ubuntu 18.08 bt not Debian 9.9

Debian produced the following error

sudo dpkg -i duplicati_2.0.4.5-1_all.deb                (Reading database ... 70082 files and directories currently installed.)
Preparing to unpack duplicati_2.0.4.5-1_all.deb ...
Unpacking duplicati (2.0.4.5-1) over (2.0.4.5-1) ...
dpkg: dependency problems prevent configuration of duplicati:
 duplicati depends on libappindicator0.1-cil | libappindicator3-0.1-cil; however:
  Package libappindicator0.1-cil is not installed.
  Package libappindicator3-0.1-cil is not installed.

dpkg: error processing package duplicati (--install):
 dependency problems - leaving unconfigured
Processing triggers for mime-support (3.60) ...
Errors were encountered while processing:
 duplicati

On Debian I installed the stretch version of the package

sudo apt-get install libappindicator1

Then in Debian I fixed broken packages

sudo apt --fix-broken install

This I was able to install Duplicati on Debian

sudo dpkg -i duplicati_2.0.4.5-1_all.deb

Running Duplicati CLI

I can now Run Duplicati CLI (it works)

duplicati-cli

See duplicati.commandline.exe help <topic> for more information.
  General: example, changelog
  Commands: backup, find, restore, delete, compact, test, compare, purge, vacuum
  Repair: repair, affected, list-broken-files, purge-broken-files
  Debug: debug, logging, create-report, test-filters, system-info, send-mail
  Targets: aftp, amzcd, azure, b2, box, cloudfiles, dropbox, ftp, file,
  googledrive, gcs, hubic, jottacloud, mega, msgroup, onedrive, onedrivev2,
  sharepoint, openstack, rclone, s3, ssh, od4b, mssp, sia, tahoe, webdav
  Modules: aes, gpg, zip, 7z, console-password-input, mssql-options,
  hyperv-options, http-options, sendhttp, sendmail, runscript, sendxmpp,
  check-mono-ssl
  Formats: date, time, size, encryption, compression
  Advanced: mail, advanced, returncodes, filter, filter-groups, <option>

http://www.duplicati.com/              Version:  - 2.0.4.5_beta_2018-11-28

TIP: Don’t run “duplicati” that’s the desktop verison.

Creating a Duplicati Backup on Linux

I read this page and this page to get an understanding of how to back up from the command line. I was not sure if I needed to run a service or run the CLI from a bach script so I asked on the forums here.

Within minutes I had this reply

kenkendk
1h
Hi @feartec, welcome to the forum!

The CLI and Web-UI are not the same, and operate independently.

If you use the CLI, you need to use cron or something similar to run the backups periodically. You can use duplicati-cli help to get started. You do not need the server running for this.

For the web-UI we do not currently have any tools for manipulating the backups other than the main WebUI itself. You need to somehow forward/tunnel access to the server, and then access the WebUI from a local browser.

Backing up a folder with Duplicati to a Backblaze B2 Bucket

I created this folder structure

/test/
/test/testfile.txt < hello
/test/subfolder/
/test/subfolder/file.txt < world

I ran this command to back up the folder structure above to backblaze (official guide). See advanced switches.

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=**********

TIP: read more about “dblock” size here

Output

Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  4 files need to be examined (8 bytes)
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1021 bytes) ...
  Downloading file (1.06 KB) ...
  Downloading file (2.00 KB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:06
  Remote files: 3
  Remote size: 4.05 KB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 0
  Files deleted: 0
  Files changed: 0
  Data uploaded: 0 bytes
  Data downloaded: 4.05 KB
Backup completed successfully!

I logged into Backblaze to check the backups

I created a second test file and added 2 test files

mkdir /test2
cd /test2
wget http://ipv4.download.thinkbroadband.com/10MB.zip
wget http://ipv4.download.thinkbroadband.com/5MB.zip

ls -al
total 15368
...
-rw-r--r--  1 user user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

I ran the backup with the second folder in the command line

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=********** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/07/2019 23:05:58
Checking remote backup ...
  Listing remote folder ...
Scanning local files ...
  7 files need to be examined (15.00 MB)
  Uploading file (15.03 MB) ...
  Uploading file (11.43 KB) ...
  Uploading file (1.29 KB) ...
Checking remote backup ...
  Listing remote folder ...
Verifying remote backup ...
Remote backup verification completed
  Downloading file (1.29 KB) ...
  2 files need to be examined (8 bytes)
  Downloading file (11.43 KB) ...
  Downloading file (15.03 MB) ...
  0 files need to be examined (0 bytes)
  Duration of backup: 00:00:10
  Remote files: 6
  Remote size: 15.05 MB
  Total remote quota: 0 bytes
  Available remote quota: 0 bytes
  Files added: 2
  Files deleted: 0
  Files changed: 0
  Data uploaded: 15.05 MB
  Data downloaded: 15.05 MB

Success

Restoring a Duplicati Backup on Linux

O’k lets delete the local “/test2/” folder then restore it

rm -R /test2

Now lets restore it (official guide)

/usr/bin/duplicati-cli restore "b2://bucket-name-goes-here/test" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --passphrase=************* --overwrite=true --restore-path="/test2"

Output

Restore started at 06/07/2019 23:21:51
Checking remote backup ...
  Listing remote folder ...
Checking existing target files ...
  2 files need to be restored (15.00 MB)
Scanning local files for needed data ...
  Downloading file (15.03 MB) ...
  0 files need to be restored (0 bytes)
Verifying restored files ...
Restored 0 (0 bytes) files to /test2
Duration of restore: 00:00:05

Did it work

cd /test2
ls -al
total 15368
...
-rw-r--r--  1 use user 10485760 Jun  3  2008 10MB.zip
-rw-r--r--  1 user user 5242880 Jun  3  2008 5MB.zip

Yes it worked

Backup 2 folders an emailing the results

I use Google GSuite for email and have an app password setup just for duplicati. Here is the official guide for sending an email after a backup operation

I downloaded a 100MB file

cd /test2
wget http://ipv4.download.thinkbroadband.com/100MB.zip

I ran this command to backup an emailthe progress

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=***************** [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Output

Backup started at 06/08/2019 00:02:28
Checking remote backup …
Listing remote folder …
Scanning local files …
8 files need to be examined (115.00 MB)
7 files need to be examined (15.00 MB)
Uploading file (49.99 MB) …
Uploading file (18.01 KB) …
Uploading file (49.99 MB) …
Uploading file (17.97 KB) …
Uploading file (233.33 KB) …
Uploading file (33.17 KB) …
Uploading file (1.42 KB) …
Compacting remote backup …
Checking remote backup …
Listing remote folder …
Verifying remote backup …
Remote backup verification completed
Downloading file (1.42 KB) …
4 files need to be examined (15.00 MB)
Downloading file (18.01 KB) …
Downloading file (49.99 MB) …
0 files need to be examined (0 bytes)
Duration of backup: 00:00:15
Remote files: 13
Remote size: 115.33 MB
Total remote quota: 0 bytes
Available remote quota: 0 bytes
Files added: 1
Files deleted: 0
Files changed: 0
Data uploaded: 100.28 MB
Data downloaded: 50.01 MB
Backup completed successfully!

Screenshot of and email from Duplicati software

Backup Types and Versions

Read up here on backup retention policies here

TIP: Use the desktop Duplicati software to generate a backup then use the export to command line option to get the command line parameters to use in the command line version.

Screenshot of the desktop version of Duplicati and the export to command line link

After you click Commandline you can view each separate command liene option and parameter.

This was handy for finding this backup retention parameter

--retention-policy="1W:1D,4W:1W,12M:1M

“This will give me “one backup for each of the last 7 days, each of the last 4 weeks, each of the last 12 months. There will always be at least one remaining backup. “

Setup Auto Backup Cron Scripts

Make the script executable

chmod +x /scripts/backupnow.sh

Add the following

#!/bin/bash

/usr/bin/duplicati-cli backup "b2://bucket-name-goes-here/test" "/test/" "/test2/" --auth-username="#########keyID#########" --auth-password="###########applicationKey#########" --compression-module="zip" --dblock-size="50mb" --passphrase=123 [email protected] --send-mail-subject="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-body="Duplicati Backup on yourserver.com - Src: %LOCALPATH% , Dest: %REMOTEURL% (%RESULT%)" --send-mail-url="tls://smtp.gmail.com:587" [email protected] --send-mail-password=***************** --retention-policy="1W:1D,4W:1W,12M:1M"

Now you can add the script to your crontab by running the following.

crontab -e

Make sure you have the following in your crontab

SHELL=/bin/bash

Add a cron entry to run the bash script

#Backup Database
0 22 * * * /bin/bash /scripts/backupnow.sh

Use https://crontab.guru/ to check the time you want to run thew script

Links

Version

v1.1 Fixed typo (”downalod” thanks to @NicolasWolf for reporting.)

v1.0 Initial Post (added retention polity and Linux crontabs

v0.95 Backup, Restore and Send Email

v0.91 Added Duplicati service info

v0.9 Working Copy

I am moving away from Apple hardware

February 3, 2019 by Simon

My Late 2012 Mac Book Pro Retina laptop is all but dead, it has many dead pixels and because of the poor cooling and is NOT a joy to use anymore. It does not “JUST WORK” and personally, I do not think “thinner” laptops can handle Australian summers as its hardware cooling it inadequate above 40c air temperatures.

My laptop processor would spend more time thermal throttling (at 104c) in Web Browsers and text editors that at normal speeds. Opening up productivity apps like Photoshop or Premiere Pro would send the laptop into meltdown.

Frequent high temps were common.

Attempted Fixes

Warning Disclaimer: My laptop is out of warranty and I know my way around the inside of a computer hardware without zapping it. Do not attempt to open your laptop unless you know what you are doing, have backed up your data and are prepared to brick your computer.

I removed dust from inside the laptop.
I tried to only use the laptop refrigerative air conditioning
I replaced the thermal paste on the CPU and GPU (3 times)
I reinstalled OSX Mojave and reset the SMC and PRAM multiple times.
I ran the fans at 100% (see post here), The fans were operating at full capacity and were not broken.

The stock thermal paste was crusty after 5 years. The plastic CPU/GPU cover was visibly cooked.

I ordered some new Thermal grizzly thermal paste, I had some older silicone paste on hand just in case.

After many reapplications of the Thermal Grizzly, the older silicone paste seemed to work the better???

After a few months, all of the fixes above did not seem to work. OSX Mojave would spin up the CPU and GPU into a frenzy overloading the single heat pipe within minutes.

Time to try some more drastic cooling modifications?

I tried improving the efficiency of the single (copper) heat pipe that is shared between an Intel i7 2.6 GHz and an Nvidia Video Card by removing the black paint by stripping the paint with acetone.

I manually removed paint from in between the heat sink fins with a LED to reveal the metal.

I reinstalled the heat pipe with high hopes? That looks nice 🙂

I removed the old thermal paste and added new paste. First I tried Thermal Grizzly Cryonaut. I re-applied the paste three separate times as each application was not that much better than the old crusty stock paste from Apple. Did I have a bad batch of Thermal Grizzly?, It seemed thick and not very viscous. I ended up using an old tube of silicone paste (the white stuff) as my Arctic Silver was too old to try and I did not want to order more.

With the silicone paste applied and the paint removed temperatures were about 15c lower at max, I still had frequent thermal throttling but at least I had a reserve buffer.

This was all before the Aussie Heatwaves and high temperatures soon returned.

Is there still room for improvement?

How heat pipes work

Heat pipes have an evaporating (hot part) and condensing zone (cool part) on the heat pipe. I noticed Apple’s “stock” condensing fins were small, would improving this zone help?. Time to improve the condensers zones by adding larger copper heat sinks to the bare side of the heat pipe.

I purchased a few copper Xeon/Sun server sized heat sinks and thermal epoxied them to the condensing end of the heat pipe. Yes, they would protrude out the bottom of the case but #Meh. I can fix that by extending the base of the laptop down and making it thicker (old school style).

The server heat sinks arrived

I cut the heat sinks in half.

I packed the fins with paper before cutting to ensure the cut did not damage the fins.

After cutting, I wiped the copper heat sinks with vinegar to restore the surface to a nice copper shine.

I tested the heat sink idea with silicone paste first

Temps were 25c lower, Now it’s time to use Arctic Silver Thermal Epoxy

I applied the Thermal Epoxy to the heat pipe (I temporary had foil strips above the fans so I did not block them while the epoxy dried.

I then stuck the heat sink’s to the heat pipe (with Arctic Silver Thermal Epoxy).

I toyed with a clear case but decided against it for static electricity and stability reasons.

I purchased a second Mac Book base for so I could cut holes for the heat sinks to protrude and use the original base to hide the modification.

I made a 30 mm base wall to so I could use it as a wall between the laptop base and the new 30mm lower base.

I added some 5-volt and 12 -volt fans inside the new extended 30mm base.

Finished Product

A normal looking Mac Book except for the 30mm lower base and internal 5V or 12V fans.

External power plugs on the left side, I will add lights at a later stage.

Are the temps lower?

Videos

Video: Mac Book Pro cooling mod, I can now watch 1080p videos without maxing the CPU

Video: Mac Book Pro cooling mod with external powered 5v or 12v fans

Conclusion

50c lower temps are nicer at idle but in Premiere Pro (exporting video) the laptop was still thermal throttling like mad and temps were terrible (100+). Lets not get started when I start some development VM’s

Conclusion 2 weeks later

This is still not a joy to use. I don’t think I have the right to expect a 5-year-old laptop to keep up running a CPU/GPU intensive OS and applications.

Time to buy a new computer, Apple still makes thin and over heating laptops by the looks of it?

Maybe I need to buy a fridge to stick a computer in a fridge to use these days?

YouTube users indicate Apple has a problem with heat.

What computer do I get next?

Not an Apple made one. I will be moving back to Windows for local development and Linux on servers

Dell Alienware has many heat pipes.

Acer Predator 500

I read a few reviews (e.g this one from Ultra book reviews) and Acer have good cooling.

MSI GT Series laptops look the best if cooling is important.

Or should I build a custom desktop with way more cores

CPU: Threadripper 2950X 16C 32T

SSD: M.2 SSD: Samsung 970 PRO 512GB
MOBO: Asus Zenith Extreme
Power: Corsair RM1000x 1000W
MEM: Quad 3600 Mhz
GPU: AMD Radeon VII Navi 3980

Thanks for reading.

v1.3 Added videos

v1.2 Updated alt tag descriptions

v1.1 Added “I will be moving back to Windows for local development and Linux on servers”

1.0 Initial Draft